So the talk today is very particular about mission,
really? Investors setting. And this time, the
focus on women's health evaluation as well as some cases
you get, uh, you ask me questions.
Um, So here's the outline A lot. So
we introduced the problem, then explain to you find
out to develop a few years and also held a
find operation which turns out not so straightforward and benefit
compared to from then, show some experiment themselves and
then complete the call. And at the end of
your time to talk about some research on the 24th
. So So this is a graph that shows the
direction between uh huh uhh! You sure? Okay
, this is a confirmation like general age, and
then he comes to the absolute, and that one
has observed strategy that has to decide what form,
use or ranking results to the user and return.
You can provide it back in the fall of the
click revenue. Um, and then all of this
in this kind of direction, uh, we call
this provision of context, um, and then relations
here, Uh, and then the strategy by called
policy and then all the feedback by user is called
This is called action and be like, totally Ward
. And this whole process to go, which I
view, is to maximize the total reward by optimizing
the policy here. So you can imagine it's optimizing
happy to maximize the 6 to 8 and the current
users. So let me give you a more complete
example on the front page. So eventually quotas on
a particular model here called a model a closer look
. Uh, so the module that recommendations are going
to use you can see here, which is a
small number of articles, articles are handpicked by human
, and, uh and then there's it's okay for
that has a big image. So that's the highlighted
articles that we recommend. This is the most problem
, one of the most common positions, and therefore
, we really want to amend the most interesting items
. So, um, so informal. Yes.
I want to interest you survive, explain useful articles
when you press on and leader former if we want
to maximize the number text for the number of effective
rates of ST uh, the many challenges here,
one of them is that the content for several articles
it's changing all the time, so the editors may
actually want to fool or remove forward, because we
try to keep the comments, complicate. Um,
And then there's also the smart services is where,
uh, users, uh, we have a huge
number of searches today, but you're certain movies is
small. So to transfer the knowledge of one another
so that I can use your information to identify your
interest from running a personalized organization, Okay. And
most important, this office is also the impact.
Very prized. Addition, exploration. So let me
elaborate on that. Uh, so again, um
, So I want to recommend the use of,
uh, trying to maximize the question. So the
observation and this process that we can only obtain intact
, but we recommend Well, I know that we
don't recommend you don't see it. So therefore,
don't see whether the use of the artist So this
is partially both. So we want to recommend this
is what we want to recommend to the user.
To do that, we need to estimate each article
trip to raise a great, but we don't know
the rate beforehand. So therefore we need sometimes when
the article is new to the school to mention related
trade for the user to see whether you like it
. So there's a There's a trade off between this
process. Wanted to utilize the knowledge we have data
from the data to do what you want. The
other is to do exploration, collect information from two
. So there's a trade off between these two conflicting
goals. And the problem is how to do that
Turn off dynamic pool or when you try to continue
interest as well. So let me give you an
example, that's where. Sufficient inspiration. So let's
and you. It's too bad machines, machines there
so usually play slot machine expectations. This money.
But, uh, you are nice. You can
earn money right here in the swamp and ability to
maximize them. You're not want one of these machines
. You don't know which one is higher, attractive
rewards for money. So you need to try one
of each one of them that that converts to one
of the things that have a high reward. So
let's say you try first. Want to give$5
and try to 2nd 10 1st, 1570 earlier and
five right, So at this point, I think
that the first machine gives you$5 on average and
the second machine doesn't do anything. So you think
that's okay? Maybe the first machine better than China
. So this is good. I looks good.
However, I think we only see this machine is
the only three data points for the second machine,
so the estimate here can actually be expected. So
let's say it turns out that the first machine$5
per round but the and actually having$100 a lot
of time. So now you have$25 per round
. So it's just unlucky to miss all the$100
around here because you don't China. So it's Friday
so you can see$100 a year and therefore comparing
to the person a draft of$20 per now.
So this is an example, as sufficient exploration of
the second issue give you an estimate of the average
payoff afternoon money to learn, and then we converse
this off first, so even formulate this problem kind
of a record product into something called contextual problem.
There's many names in the literature, so I think
this one, because emphasize the better make sure and
also the contextual nature. So let me explain what
it is. Um, so here we have,
uh, so it's a terrific process. Um,
here we have a candidate which will cost a arm
so you can meditate on the front page module we
have candidates are going to recommend to the user.
Um and then also So this is the teacher asked
. So you're very small, which is, like
a general education and personal interests, etcetera. And
then the decision maker you can choose one of the
actions from this content is displayed to the user and
in return, receive a numerical rewarding, uh,
American from the user. Um, so And then
, after observing this report, you can use this
signal to update our policy strategy and this process with
over time, you want to maximize the summer rewards
before this process. Let's say, one of the
key steps you want to maximize, uh, summer
rewards this key round. Okay. Um so so
technology can then calculate. And the key is the
use of features, uh, is obtained with Spain
on growth. Thank you, sir, And reported
correspondent uh, corresponding. Pointing to care about you
want to maximize the number of then it's natural that
are 10 that is expected by our the great CPR
. So it turns out that this form of this
foundation is one another capturing more patients on the Internet
. So let me see the example of a lot
of money in the street. So this is one
of the Chinese yahoo answers. So you can post
questions. Has questions this one, uh, computers
, they have computers, great computers. And then
these electrons are also advertising. Uh, usually it's
interesting. Uh, then so here in this example
, you can take a traveled a candidate who set
of boxes set up I can display a small region
and then access to the feature of the users in
general, every information in this article, and then
they have to stay here and our state. And
then there's a possible nature where for all other at
that stage is what you don't see the, um
So, for example, is ranking I contractual value
here and see a list of rankings out. Um
, so usually so in this case, uh,
a set of rankings, U. S T is
pretty documents. Are you certain you want to be
personalized. Um, 80 is the ranking of the
breakfast. And are we can define various ways to
find one when the session is a zero. Just
, um So, um yeah, so all I
can. So this, for example, show,
uh, that connection that is, uh, this
morning for many critical applications on the internet. And
it's related to some other, uh, other areas
in the surety of science and statistics. So one
thing you might want to, uh, I wonder
if the connection information that you were collaborative filtering both
of which Parabellum recommending, um, in those organisms
usually, um, they care. They assume that
the established set of articles study seven movies for you
recommend to do so. There's, uh there's no
dynamic content food, So everything that has been a
long time, So you have some data for them
, so you don't need to remind exploration exploitation.
You just need to speak the dataset train part of
the best part train the first test to evaluate the
test. So it's not even our inspiration. Um
, there's also foreign personally, very building is at
the top, so it's more generally that it is
a special case but it has to tackle the Temple
Islamic model. But hopefully it's not a very soon
in a kind of applications that explained just now,
Um, So there's all the traditional, more competitive
, Uh, they don't consider contextual information, so
you can, uh, personalized recommendation. So So
, actually, give you are last report. So
this is an introduction to get kind of problem,
too. So this is, uh, inspection of
the algorithms. Uh, so, uh, this
is the first time that the organization which explain more
later, um, so one way to do exploration
is trying things. So, uh, example,
Uh, So let's say you have three articles here
, and each has its own rates that you don't
know these numbers. Um, And then by using
the locking system, you can estimate the CR use
are just numbers first, uh, so that's a
reasonable estimate of CR. But then you know that
these issues are not accurate. So to collect data
for our refined estimate review so that the better you
can allow us to make the decisions in the future
. So one way call it really is to choose
article that has the highest estimate with high probability one
month and so on. And then a small probability
and so on exploring tomorrow. So this is a
very strange idea. Just assign probabilities to explore something
. Okay, Uh, this exploration is unguided.
You explore. The article is random. So as
you can imagine, it's not the most efficient rate
. So another strategy called UCB one is to choose
articles according to the index computer. This way,
So new is again, you estimate click rate of
the way. And then this, uh, wrap
up here as well and a s okay. It's
a number of times this article has been shown to
music. So at the beginning, when the new
system is very small, that means that we don't
have enough data to to build a good estimate for
that. So this term is paid. And then
, in other words, it encourages the system to
Chinese action. So you can think of that as
exploration boss term approach exploration, Um, and all
the time. And then something becomes watching. This
has made solution s essentially zero, and essentially using
this estimate, it's already very so. These are
the two typical challenges and traditional film problems that do
not consider contact So it means that the assumption is
that the critical made in the article does not depend
on user information. Doesn't become a certification organization,
which is not a reasonable assumption right there with the
different information so consequences that are considering the pictures.
And there's no way to do personalization. So now
so have been candidate or other fraternities were doing personalization
of contextual more. Combat it in the refrigerator back
in 2002. Uh, so, like these people
, as long as the sample, uh, strong
direct guarantees, but conditionally intractable in general, there's
also probably need 2008 to really explain the previous like
you randomize exploration, they do not motivation. Okay
, so this is what I'm going to talk about
. This more compact geometric models that both complications application
and the fact is optimizing maximizing total work with like
so the rest of this session will focus on three
parts. First, I'll explain a generalization of the
you see the strategy for the animals and then extended
to generalize. And then, uh, talk about
randomize our, uh, conscious. Yeah. Yeah
. So let me start with the You see the
strategy for the animals. Uh, so Let's say
so. Here's your moment. Assumption is that these
are the context before the user. Right? Uh
, expectedly. What is? It means that,
uh, that can be estimated by the operation of
this feature. And here take a coefficient corresponding to
that location. But these are not, so you
need to estimated from labor. And now close everything
up. They in the form of making the information
that we have many years of age and a spacious
exhibition and the corresponding let's see that are reversed.
So, in order to estimate favor a very straightforward
to apply a little question for this rich regression.
Mhm. Here, uh, this system statistics,
Um, now. So once you have an explainer
of data and reduce that to estimate when you count
estimated, uh, hold W is estimated close enough
to shoot estimates that, uh, so far so
good. But this grace only gives you a point
estimate in the sense that we give you a number
and tell how confident is this estimation? So we
need to qualify the uncertainty so that we know I'm
sure about the City of article so committed for exploration
, but article I don't have enough data I'm very
unconfident about my in my estimate that we need more
exploration. So we need to qualify this 17 so
that when they do it is to divide, uh
, to use this, uh, quality. We
can show that with high probability, our dictionary So
that the left hand side here is the prediction area
. So this is my question estimate. This is
true ground truth that I don't know, the absolute
differences Prediction Error is founded by the square of this
guy. Modified a super friend to constant. Uh
, So, um, so this time sometimes measures
how similar the new user X is two previous usage
. So it is very close to the previous certain
a Then, um Then the term is small,
in other words, that we have a group of
made this repression, so confidence level is small,
so there's no people need for exploration between, uh
Okay, so, uh huh, for high priority
, but then how we're going to use it.
So we have called in the city or, uh
, this model. So essentially, when you have
a user, then always chooses and on maximizes so
remember no context, you see, you know,
remember this for The first time of both about exploitation
is a point estimate of the, uh, a
second term is a communist involved how a certain in
your estimation and then the other is to the ground
, and I combined two algorithm gives a trade off
between the first part exploitation and exploration. Uh,
the album should have mentioned this job is for you
, C b. Because it changes on according to
offer confidence down nature. So I should mention that
, uh, the same government. And then it's
similar to what we want when we have more and
more simple in the data set and this time it
becomes small. And then you have an estimate for
the first time and essentially doing more exploitation rather than
exploration related algorithm back in 2000 and two that,
uh but it works in a more complicated way.
So how How so Anymore we started doing this work
with many, uh, Chinese. We can follow
, however, is not always perfect. So,
for instance, in the way that we want to
maximize without, it made the criminal rate a lot
of which is probably so it has to be between
01 but this model here. It's kind of case
that this estimating people just make it doesn't make any
sense, right? So So it's part of the
leading model is not always a supermodel for all applications
provide Everyone is like place generalizing enormous monusco. So
So in general as well as we make the assumption
that expected rewarding even the use of future because it's
the linear, uh, combination of peace of features
. Uh, followed by a fine investment company here
at the university. So there are two, uh
, journalists bringing all those ones power regression where,
uh, the city are tricky ways. Is this
, uh, have statements in many cases, uh
, combination here and then take an exponential and do
this. So after this is washing effect, this
number the brain dysfunction too. So it naturally also
for chronic model with, uh, being here the
cumulative distribution function of distribution. So again, a
function. Unfortunately, when you work with journalists,
many models you can always get those solutions like a
million models. So what we can do here is
to do a presentation. So even you want to
get a point estimate with journalist building models. We
can do this regression so instead of using a combination
to get the point, given the data so that
give us the optimal point estimates. And then we
can do something similar to derive confidence in the role
of the estimation. And then and then the energy
combined, too, can do something like that you
simply by choosing on the the highest wind plus compensation
for doing exploration. So here again, all the
stations in the first case I'm in here, which
is important in the second case of logistic and product
. And so the bones nails a little mysterious boarding
a technique side all the same to travel with the
loss of the being fixed up. It's more comfortable
, so I'm going to show that you don't hear
the ideas of person to indicate, and later we'll
show you some show. Some comparisons showed journalists better
than, uh rather than America. So so other
confident found exploration has been very popular in the literature
for a long time. People know that it's very
topic you can you can put a lot of interesting
here, uh, show that very quickly to the
solution. Um, and then, um and then
, uh, there are also indications of this kind
of exploration. So, uh, so the first
thing is that exploration and too much into the exploration
by adding exploration, exploration, bonus to index.
Uh, the economic can easily explored all the potential
piece part of the French stage. Explore all the
friends. Uh, this could be inefficient. Uh
, then it especially when you have high large,
that helps you. This part of the primary is
hopefully helping out parts that used to be still exploding
. Providence is uncertain. So you do want to
use any prior knowledge you have to hurt into performances
. The algorithm. Uh, the second thing is
that exploration using instinctual next so you can see is
the mystic out that always chooses the same home with
that has the maximum mean plus confidence. And,
uh, so there's a problem with the rewards of
the right. So you got to bring, since
we don't really get to use, these are needed
to. So we should the pace and user user
make looking to pay for three seconds and place,
and due to the infrastructure of their expectations, that
doesn't go back to the back and updates immediately,
so they usually some kind of delay and considering the
amount of time to have that few seconds or minutes
of delay correspond to thousands, tens of thousands of
users or maybe more. So in that way there's
a delay of 10,001 million steps, so you can
only get New York one million steps, uh,
in the past. So this is bad for eternity
. Determination, strategies because this black healthcare is always
use the same over and over again. The same
symptoms. Um, and that may not be the
best thing to do because you could have randomized strategies
so that you can explore different things in this back
to, uh, the first thing is that writing
a comic style is not always easy to use it
, so we can do that exactly. Arena Nigeria
approximately for generalized linear models after a bit of a
bit more work, but for other models, more
difficult. So here I'm going to describe a socialistic
concepts. In fact, three. Um so,
yeah, it's called probability. Metric, um,
so even the user X takes them on a case
. An article accord, a probability that is the
probability that this article, so it's the algorithm thinks
that Apple is 90% of the property. Chances are
so is optimal for most interesting for this current user
. Then, uh, we think properly. So
that's the idea. Uh um So, first of
all, you know, this is a randomized strategy
, so it's more of us to work the legend
you finally can. Delay is different. You can
do exploration, different users. And at the end
of the financial aid, you can have people.
You can have data for different sort of users and
that that's the whole, um and then, more
importantly, is straightforward information. So So, for
instance, in, uh, you have a logistic
model role model, you can just have a posterior
institution of gravity trying to work and posterior computer by
any standard based in the tradition of the data product
. So you can maintain posterior here and then and
then when the user accounts you want to recommend an
article, you just draw 11 parameter from this posterior
so each age or randomized to a and then to
choose out According to this random, uh, Grant
Baker. So efforts were model model logistic models,
so you can see because this market to join from
this possibility this process can be targeted. You can
show that this problem actually satisfied the probability that in
principle that I thought has 90% of China's, ultimately
you will be chosen probably 10%. Uh, this
part of Italy combined with many other models, financial
models and models because I think it's more you don't
have to devise new forms or anything. So this
is, uh this is useful in practice. Okay
, So I have really, uh, kind of
algorithm in general thing. And so you're like,
wonder how can probably whether different so your machine,
you do care about finding a massive, rigorous way
to measure the performance and the speed of an algorithm
. In this case, that's about it. The
normal, uh, metric is called agreement. So
here, uh, this is the expectation of the
summer rewards. Uh, so this is an expectation
of some reward of, uh, article algorithm,
assuming that the horrible knows the parameters of this problem
. So this is the best possible highest possible result
. Actually, you can hope for assuming we loaded
it. But on the other hand, you have
an algorithm that does not know the data beforehand.
So thousands adaptive and try to take a and this
is the sequence chooses for the user. So so
that we have an expectation of the total reward.
And the difference between them is called. Uh,
it's always nominated and the strength here, uh,
And then, uh so, uh and then regret
regret. It's not really it's suffering. Then you
can say that the album learn because it's suffering,
then divide users and have less than to regret.
So the first regret, if you decrease the zero
, this other small, smaller one. So the
personal regret decrease to zero all the time. In
other words, it is that we're running algorithms for
a long time. Then the algorithm will converge to
the point that you don't know what point. So
whenever this is how you learn and the album and
the algorithm learns that when, um, other small
a small outbreaks. But the bastard this average increases
to zero. So the algorithm suburbs. So this
is a pragmatic, uh, there's a mathematical metric
to measure the performance of the algorithm down. So
, for instance, the in your face and show
that, uh, the things that regret growth on
all of us were kpp case A number of articles
system the the features of the, uh, step
. So So let me give you that. How
these numbers mean, So that means that the actual
decreases the optimal solution at the rate of just turned
by key. First step, regret this, um
, So when you have 20 articles and 15 features
and 10 million is just a small community, china
, and then you can see that kind of a
small, uh, so quickly go to the ultimate
solution. Uh, that is a very good practice
, and generalized model is still open. Uh,
there's a valiant general model last year. Uh,
people should regret that study a slightly different, Uh
, this is not comparable and constant something, uh
, I think is a whole algorithm that, uh
, there's not been much attention to the Microsoft,
uh, exploration strategy to be particularly helpful in practice
. Um, so now people are starting to analysis
. Um, so So just, uh, last
week or three candidates. So something, uh,
I don't have to get the best possible way.
Uh, so the two days ago, Uh,
okay, So I hope so far. Show you
a lot of problems Were interested in solving, uh
, typical and algorithms that we can reason in solving
problems. And now we switch to the, uh
, evaluation problem. Um, so So the goal
here is that you have an algorithm of natural bank
problem, and you should become either come to the
side of the distribution. Um, and then the
algorithm is adapted, meaning that the learn from past
data and then for dessert, nature. Even then
you can define the value of the average percentage would
collapse Hope. So this is, uh, compared
to regret the same set of, uh, aggressive
difference between, uh, we can actually measure.
We cannot measure the optimal because we don't have a
conservation. That's why you're not fishing for religious emergency
. Uh, look at the financial quantity and then
for study, uh, study How high? It
doesn't learn something policy doesn't learn from when the savings
accounts always recommend the market. So so now can
this vulgar the first one can achieve. So these
are these two numbers are the numbers trying to estimate
prevented the way? Uh, the question is whether
we can do it online. So that was a
lot of data from QuickBooks. We don't want to
do when we have the algorithm. We don't want
to be the algorithm real system. It's always depend
a lot of time. It takes money. And
so immigration utilize data to evaluate a new algorithm in
a lot of benefits. That way uh, even
, uh, testing system. In a way,
this reserve experiences and it's very common in a machine
power, like we have a lot of benchmark.
It is that organizes input, use of nature versus
label finding classification zero in this case. So it's
a different training part of the test part. I
think we have done a very uh huh. But
interactive machine learning problem is not so straightforward. So
, for instance, better in the morning on benefits
, especially in this for the contact use of interest
we have recommended on and also the use of my
article for that, uh, so when you use
this data, this historical data to develop a new
outbreak, then you don't have the rewards technical where
the algorithm recommends and then on the law, because
we only see we work on the only see when
you press on the on the right Now for other
on this state. Uh, therefore, there's a
this is what we call a national level nature.
We don't see that we work for honest, not
in the law. Uh, for this reason,
A straightforward way to use historical data as follows.
Um, so let's say you have a lot of
historical data this morning. You interact and work and
whatever statistical machine learning techniques to estimate this, we
work at work. So you can say to have
a function that stimulates what users in practice. And
hopefully, if you can get this right, then
you can use this simulator to estimate how well the
algorithm. Unfortunately, um, the stuff here is
very difficult. All bias into more consumption is such
a uh and then the second step here, evaporation
to be unreliable. So that gives you a lot
of pain. When you use this kind of data
violation, you say, let's say a 10% so
that's a good number. But since these numbers are
undeniable, so because my vision in 10% 1% or
20%. So it's not good for my evaluation purposes
. And in contrast, uh, solution that we
and that is a kind of modern step, which
is make a procedure simple and most importantly, we
try to imagine a method that some unbiased. So
it's reliable. So this is the so here's how
we're going to do it again. So this is
the quantity which way? Simplicity. I just showed
the, uh, case, uh, in case
you have done the same way better. Here we
have an algorithm is a key requirement in the data
collection is that we have, uh, in this
war, all this age, you have to be
chosen on the revenue. So this young piece that
all our candidates help chances state. So no one
, no one will be starting from 2016. Um
, And then, if that's the function, both
then when we so then we can go to the
data one by one. So you can look at
the data and then the resource to the way that
they were going to recommend a two. So we
have here is recommended articles, and then we can
compare this a hash to the article in the paper
. And when when these two articles are the same
, then we call it a match, and then
we have to work. Single article was the algorithm
to allow you to learn more, and it was
not a match. Then we just simply ignore that
step. So we use that point of the data
. And then finally, we add up all the
rewards in the masculine and modern Friday by. So
this icy indicator function, which is one when the
equal match zero when there is no match. So
since the arms in this past in this data is
chosen by uniform, that random and therefore the probability
that you see on that it's one Okay, uh
, network. And you see okay here, which
is to normalize something. But this is the size
of data. So they can. So that knowledge
or producer Okay, um so yes, probably.
Uh, so this method is satisfied with what we
wanted to be. So the first one is that
this number is estimated so which means that if you
use the estimated on the algorithm and on average,
keeping the same number because you run the algorithm real
system. So this is nice because you don't have
to run the system expensive way in the system.
But again, you can get reliable and seven,
it's about estimation there. So you can You can
also show that when you have more and more data
than the error estimation, error goes away. Decreases
to zero rate of this. So again, case
number, arms and no signs of abating. So
you have a very large out then this term is
small, and that's what estimation areas is very small
. So the second theory makes an idea how confident
to estimation is. So now I can say my
CEO, that with this deal algorithm, I can
improve the system by 10% plus minus 1% this country
. Uh, so, yeah, so let me
show you something, case knowledge. Is that how
so? We have a lot more in the recent
, uh, recommendations. Um, we have to
evaluate, start much power with she recommends producers,
and then we have a data violation. So this
data satisfied climate that, uh, the article recommended
this. Uh huh. And then we use this
data to do online valuation for these policies of this
problem. So now we have two numbers. The
first is that because Iran is also in the campaign
. So we have metrics to measure what the average
trip to a sexual using Rivera just a counter process
experiencing on the other hand, we have outlined valuation
metrics using methodology scratch now. And the question is
whether the clothes, the clothes, that means the
outline valuation that is reliable French. So let me
show you some numbers. So the first part is
the water of the online click through rate of an
article versus the online estimated to sit down. So
every point respond to one of the articles a 10
day period so you can see the articles here.
Um and then I feel if the offline estimated a
that is a the same as online PR, then
every point right on this one x line, but
later on a finite. And there's some noises.
So you don't actually see, uh, in perfect
alignment agency that they're very close to this while you
explain meaning that this estimated to be in compliance so
you can give you a reliant estimated probably about running
. So here's another bottle that shows that compares.
The baby's getting on okay, offices and they that's
a mandate, period. So the green walk the
red one is the online and the background here,
the Alibaba CEO and each one responded overall city of
the whole day, you can see that there's a
strong correlation between these two curves. Okay, so
it's also equally on the table on four applicator.
The second question how the estimation ever of the CPR
carries zero and you have more and more data.
So we control the size of the U. S
. And look at the estimation of the sciences.
And, uh, I mentioned that they are mobilized
because business sensitivity We are not allowed to view the
real numbers. So my mother, all these numbers
by constant constant therefore so these numbers are absolutely You
can see that this estimation error is not allowed to
feel that destination to always, so you can actually
see it back. So there are many extensions of
the considerations of them. There are improvements for when
you don't have access to the data into something and
you can see the last estimate or projection to do
more fancy stuff data. So you see that interested
? I can explain one online. Okay. So
far, retail. So in this online operation,
such part where we have shown is that a biased
or reliable have to evaluate the most of them are
driven from historical data without without having to be a
real system. So you can think of why not
to, uh, datasets in supplies, for instance
, or consultation for, uh, interesting. So
this is the first benchmark it's just released by that
philosophy, interested in this kind of problem down.
Uh, And the second version is coming for competition
. Probably fostered by hostile. And also, uh
, also, I send out workshop. Okay,
So let me show you some experiments. Um,
how well we work. And so I have.
So you also describe how to do evaluations. So
this section is about how to find a way to
evaluate the, uh So we have, uh,
about 20 upwards her. Uh huh. And then
you dimensionality of use of features, which over 100
features by critical important analysis, uh, make the
future. Uh, And then, uh, so
remember, there's a model. There's an updated because
the system, uh, system infrastructure constraints. So
we also stimulate that fact. Particularly updated model every
five minutes. So finally use use the protected by
the last five minutes. Model used to model to
the next. Uh, hi. Would you like
to do a model here? They use the model
to use the existing time period and a model for
the next because it's great. And then the main
metric there'll be compared to show the next few slides
is the overall novel CPR each algorithm, uh,
a pocket. So, uh, so that's where
you have, Uh, yeah, but a bucket
here that you can run your content outward and then
in asserting bucket you can use the model is used
for users in this pocket by trying on all the
exploration components here. So when it usually comes here
, uh, it's been a point inside of the
user. All this in your pocket, Uh,
this story, bucket and bucket. And, uh
, welcome to the user that you also an exploration
here. Users also this pocket, then, uh
, Children without doing so. Because the value of
this started back here that are most important measures How
? Well, how well the model converts to run
. Yeah. So the first one, the first
comparison between these models and generally models. So everyone's
related models seeking a 31 structure store equation. And
this is the$20 1st, the left hand side
, actually. Really? So I found is the
probability that you choose depending on the exploration, the
right one is You see, we control the Crown
Corporation where you can find me and conference center.
So basically, offer is more confidence in the okay
, So more about more inspiration. And then even
so you can see that all the usually all the
food or the green ground curved bars are much higher
than the blue one. Meaning that, uh,
generalized linear models that provide the model than Windows because
the country, the refinery, worst signals and problem
And also the second observation is that compared to the
left hand side, right hand side. You see
, the BCB exploration is usually more efficient when you
have the right friends often here, which is also
consistent with previous work that shows you cities in general
more attractive exploration strategy than have strong really driving exploration
A second. A lot of you explain quickly is
the comparison between Thompson something and you see so many
algorithms here. I'm going to No, most of
them are focusing on Thompson something to here. And
they used to be one which is the agreement here
. So x delay minutes. So that way you
want it still slightly better and that way. But
as you increase the way that I think the comment
something is competitive uniform on this for me and more
importantly, within 60 minutes when the randomizes is more
robust doesn't seem to be affected by the way.
But in contrast to a so uh included in the
first part. So I showed them how to use
potential benefits principle to, uh, a lot of
critical notifications like this recommendation, ranking of computational advertisement
also show you how to use the starting point evaluation
without implementing something real system Also show you encouraging results
in communications is English recommendation on you later and particularly
I want to highlight the practices using the exploration exploration
future many interesting what to do offline violation many ways
to use non flavor. And also, when you
have prior knowledge devised a much better talk something along
that line by prior model, you can use prior
knowledge that way. Many other ways to find,
also in many various abandoned will be rectified. Reality
. Uh, so, uh okay, so that's
the, uh, the first part of a national
ban it So we need a couple more minutes on
research, so I have a background in the most
learning. Uh, so unfortunately, uh, it's
, uh, much money problem that optimized strategies and
the sequential decision making important. So here's one example
that are working towards and, uh, in China
, so it's called a dire uh, it's a
system that calls the young system that says, there
you can hold it. We can get out of
China and valuable transfer the call to the person that
you want. So I get that you want to
call someone here each and then and then you say
something like that, Max Peter Johnson go home.
And then this is a sound signal and then the
speech recognition techniques, and then transfer to some computer
recommended representation to take antiviral that depending on the signal
and decide whether you understand the question well, even
understanding and transfer the call to jump it was not
sure. Then you can Can you confirm that you
really want you really want to join me and then
correct question. Okay, so that repeat. So
in this kind of process, the notion of state
there's notion of actions and also, uh and then
we want to design a dialogue by these two so
that the system can succeed in, uh, still
conversation as possible. So if you do this,
we can define report function of minus one response.
So response here, then is its success again,
minus 20. So I maximize the reward system and
devilish behavior to optimize by the objective. So cute
, teacher that that report with defined world cultures systems
do left trade reports seem to have a lot of
problems. Control problems gain, not computing Cuban introduction
patients. Um, so let's see that usually,
uh, position processes. Uh, so in my
dissertation work on efficient exploration and algorithms for solving reinforcement
, learning a market position process, that the idea
is distinguished, non process is unimportant dynamics. And
then So if I connect that patient and I know
where I should expose at this stage, if I
know where certain about dynamics and they can do exploitation
so that it would be useful for doing exploration in
reinforcement rain and particularly proposed a simplified the process of
faith, uh, most what he knows, And
from within, you can devise a principal algorithm called
big formats that utilizes this, uh, principal during
exploration and then unify close to many of the existing
Nothing. You want various kinds of various kinds of
reinforcement problems, So it's not like a slight.
So in the first part, I talked about that
problem capture Internet application. And then, uh so
, uh, deep reinforcement, learning. And two
minutes that captured many sequences, decision making problems,
biologics and CI. Uh, so you can see
that the reinforcement camps a lot of, uh,
my research focuses in on exploration, expectation, tradeoff
, evaluation and also working on the selection and additional
confirmation of convergence. Trade analysis. Uh, unfortunately
, you also time right after learning, learning.
So, yeah. So that's the end of my
talk and I depression That our thanks. Yeah.