Unit 9 04 MDP Grid World knowitvideos English Japanese

Unit 9 04 MDP Grid WorldYouTube↗ IA↗(long shot)
Howto & Style 12.5K views 28 likes 2 dislikes 01 Nov 2011

knowitvideos 11.8K subscribers N/A YouTube↗

Click to Play Video

[Narrator] Let me give a much simpler example often called grid world for MDPs,

グリッドワールドと呼ばれるシンプルな例でマルコフ決定過程を説明します

and I'll be using this insanely simple example over here throughout this class.

講義では一貫してこの非常にシンプルな例を用います

Let's assume we have a starting state over here, and there's 2 goal states who

この開始状態から始めて 2つの終了状態があります

are often called absorbing states with very different reward or payout.

これらは“吸収状態”と呼ばれ報酬や費用が周囲とはまったく異なります

Plus 100 for the state over here, minus 100 for the state over here,

この状態には＋100がこの状態には－100が設定されています

and our agent is able to move about the environment,

エージェントはこの環境で移動ができます

and when it reaches one of those 2 states, the game is over and the task is done.

エージェントがどちらかの吸収状態に到達すればゲームは終了です

Obviously the top state is much more attractive than the bottom state with minus 100.

上の状態は明らかに下よりも魅力的です

Now to turn this into an MDP, let's assume

マルコフ決定過程に話を戻しましょう

actions are somewhat stochastic.

行為は確率論的であるとします

So suppose we had a grid cell, and we attempt to go north.

つまりあるグリッドセルから北に行こうとした時

The deterministic agent would always succeed to go to the north square if it's available,

決定論的なエージェントの場合状況が許せば常に北に移動しますが

but let's assume that we only have an 80% chance to make it to the cell in the north.

今回は80％の確率でしか北に移動できないとします

If there's no cell at all, there's a wall like over here, we assume with 80% chance, we just bounce back to the same cell,

もし北にセルがなければ壁に跳ね返されて 80％の確率で同じセルにとどまります

but with 10% chance, we instead go left.

しかし10％の確率で左に移動し

Another 10% chance, we go right.

残り10％の確率で右に移動します

So if an agent is over here and wishes to go north,

エージェントがここから北に行こうとすると

then with 80% chance, it finds itself over here, 10% over here, 10% over here.

80％の確率で上へ 10％で左へ10％で右へ移動します

If it goes north from here, because there's no north cell,

ここから北に移動しようとした時は北にセルがないので

it'll bounce back with 80% probability, 10% left, 10% right.

80％の確率で跳ね返され 10％で左へ10％で右へ移動します

In a cell like this one over here, it'll bounce back with 90% probability,

もしこのセルにいたならば90％で跳ね返されます

80 from the top and 10 from the left, but it still has a 10% chance of going right.

80％は上から10％は左から跳ね返され残りの10％は右に移動します

This is a stochastic state transition which we can equally define for actions, south, west and east, and

この確率論的な状態遷移はどの方角への移動に対しても同様に定義できます

now we can see a situation like this conventional planning is insufficient.

このような状況においては通常のプランニングは不十分です

So for example if you're plan a sequence of actions starting over here,

例えば一連の行為をここから開始して

you might go north, north, east, east, east to reach our plus 100 absorbing or final state,

北、北、東、東、東と進んで＋100の吸収状態に到達するとします

but with this state transition model over here, even with the first step it might happen with 10% chance do you find yourselves over here,

しかしこのモデルでは最初のステップでさえ 10％の確率で右に移動してしまい

in which case conventional planning would not give us an answer.

通常のプランニングでは解決できません

So we wish to have a planning method that provides an answer no matter where we are and that's called a policy.

そこで“ポリシー”と呼ばれる新しいプランニング手法を導入します

A policy assigns actions to any state.

ポリシーは各状態に行為を割り当てます

So for example a policy might look as follows:

例えば次のようになります

for this state, we wish to go north, north, east, east, east,

この状態には北、次は北、東、東、東と割り当てます

but for this state over here, we wish to go north, maybe east over here, and maybe west over here.

この状態には北、この状態には西、これには東を割り当てます

So each state, except for the absorbing states, we have to define an action to define a policy.

吸収状態を除く各状態に対し行為を1つ定めることでポリシーを決めます

The planning problem we have becomes one of finding the optimal policy

プランニング問題は最適なポリシーを見つける問題になりました

Similar videos:

Unit 11 07.mp4 Markov Chain Question 2.mp4

8.9K views 9 likes 0 dislikes 11 Nov 2011100% match

Unit 2, Topic 13, Uniform Cost Search

35.8K views 15 likes 28 dislikes 05 Oct 2011100% match

CEOS Chatter
Fernando Garcia
1.3K views 2 likes 1 dislikes 14 Mar 2012100% match

Unit 9 35 POMDP

8.2K views 39 likes 0 dislikes 01 Nov 2011100% match

Unit 2, Topic 23, A-Star Search

138K views 392 likes 60 dislikes 05 Oct 2011100% match

Mr. Peabody & Sherman
TheCaptionedInternet
43.6K views 74 likes 9 dislikes 03 Mar 2014100% match

Instrucciones para llorar, de Julio Cortázar
Inés Andrés
5.9K views 10 likes 0 dislikes 10 Jan 2012100% match

Unit 3 8 More Complex Bayes Networks

13K views 26 likes 3 dislikes 08 Oct 2011100% match

Unit 2, Topic 7, Graph Search Answer

23.9K views 11 likes 7 dislikes 05 Oct 2011100% match

Unit 2, Topic 26, A-Star Fourth Question

28.6K views 17 likes 7 dislikes 05 Oct 2011100% match

America By The Numbers with Maria Hinojosa: Clarkston Georgia
Maria Hinojosa
28.6K views 102 likes 29 dislikes 24 Sep 2012100% match

Transformer la peur, l'angoisse, la colère, la haine - 5'
Selim Aïssel
11.5K views 83 likes 4 dislikes 31 Jan 2013100% match

PROBLEMAS DE APRENDIZAJE
Kuenta Libros
36.9K views 103 likes 4 dislikes 15 Sep 2011100% match

Recomendación kuentalibros - pizarra digital y niños con discapacidad auditiva
anitamc20
933 views 1 likes 0 dislikes 21 Dec 2012100% match

Comprendre la voie du coeur
Selim Aïssel
8.5K views 88 likes 2 dislikes 31 Jan 2013100% match

Unit 11 26.mp4 Particle Filter Algorithm.mp4

15.9K views 61 likes 5 dislikes 11 Nov 2011100% match

حقوقية امريكية تفضح جون برينان و الامن يقوم باعتقالها
عين اليمن
13.8K views 120 likes 5 dislikes 08 May 2012100% match

(ENG/CHI SUB) 140216 SM True Thailand Facebook EXO
Baby Xiumin
783 views 10 likes 0 dislikes 16 Feb 2014100% match

Unit 1w, 15 Congratulations

13.6K views 50 likes 1 dislikes 05 Oct 2011100% match

El gran Farol
canaloie24
8.4K views 3 likes 0 dislikes 24 Nov 2011100% match

Unit 4, 2 Enumeration

22.5K views 67 likes 10 dislikes 06 Oct 2011100% match

Unit 4, 4 Variable Elimination

31.1K views 105 likes 14 dislikes 06 Oct 2011100% match

Unit 2, Topic 20, Storage Requirements, Completeness

18.1K views 19 likes 3 dislikes 05 Oct 2011100% match

Efecto Desplazamiento (Crowding_out effect)
canaloie24
10.2K views 21 likes 1 dislikes 24 Nov 2011100% match

Unit 1w, 8 Robotic Car

29.5K views 18 likes 3 dislikes 05 Oct 2011100% match

La perte de contact avec le monde spirituel
Selim Aïssel
9.5K views 74 likes 2 dislikes 31 Jan 2013100% match

TED & TEDx explained
Kyo young Chu
1.1K views 7 likes 0 dislikes 02 Apr 2012100% match

Unit 5 48 Perceptron

49.1K views 139 likes 6 dislikes 17 Oct 2011100% match

0 views 0 likes 0 dislikes 07 Jun 2026100% match

Unit 1w, 4 Terminology

75.1K views 114 likes 4 dislikes 05 Oct 201143% match

Settings Privacy About Contact Donate