AI "Stop Button" Problem - Computerphile Computerphile English English (auto-generated)

AI "Stop Button" Problem - ComputerphileYouTube↗ IA↗(long shot)
Education 1.1M views 28.6K likes 1.1K dislikes 03 Mar 2017 Show Contributors 4

Computerphile 2.6M subscribers N/A YouTube↗

Contributed subtitles

English NOMAN AKRAM

Siddhant Parekh

Click to Play Video

In almost any situation being given a new utility function,

in almost any situation being given a

Is gonna rate very low on your current utility function. So that's a problem.

new utility function is going to write very low on your current utility function so that's a problem you want if

if you want to build something that you can teach, that means you want to be able to change its utility function.

you want to build something that you can teach that means you want to be able to

And you don't want it to fight you.

change its utility function and you don't want it to fight you so this has

So this has been formalized, as this property that we want early AGI to have called 'corrigibility'

been formalized as this property that we want early AGI to have called courage

That is to say it is open to be corrected.

ability that is to say it's open to be corrected it understands that it's not

It understands that it's not complete, that the utility function that it's running is not the be all and end all.

complete that the utility function that's running is not the be-all and end-all so let's say for example you've

So let's say for example you've got your AGI. It's not a super intelligence, it's just you know, perhaps around human level intelligence.

got you've got your AGI it's not a superintelligence it's just you know perhaps around human level intelligence

And it's in a robot in your lap.

and it's in a robot in your lab and

And you're testing it. But you... saw a Youtube video once that said maybe this is dangerous.

you're testing it but you saw a youtube video once that said that maybe this is

So you thought, okay well we'll put a big red stop button next to it.

dangerous so you've thought ok well we'll put a big red stop button next to

This is the standard approach to safety with machines. Most robots in industry and elsewhere will have

it this is the standard approach to safety with machines most robots in industry and elsewhere will have a big

a big red stop button on them. Oh yeah

red stop button on them oh yeah

I happen to have a button, of appropriate type. so

wonderful perfect yeah so we have we

and we have we have that. So right... there you go.

have hearts are right they go right so

Alright, so, so if only HAL would've been fitted with said stop button.

so if only I would have been fitted with yeah sorry hey if I stop can't do that

Can't do that Dave, uh yes I can.

Dave yes yes I can

except yeah, probably not.

except yeah no probably not I know that

I know that you and Frank were planning to disconnect me.

you and Frank were planning to disconnect me

And I'm afraid that's something I cannot allow to happen.

and I'm afraid that's something I cannot

That was an incorrigible design.

allow to happen how those incorrigible

Is this the point we're making? Kind of.

design is this the point where kind of

You've got your big stop button,

makes you kind of you've got your big

because you want to be safe. You understand it's dangerous.

stop button because you want to be safe you understand you know okay is

And the idea is if the AI starts to do anything

dangerous and the idea is if the AI

that maybe you don't want it to do,

starts to do anything that maybe you

you'll smack the button, and if the button's like, mounted on its chest, something like that, ya know.

don't want it to do you'll smack the button makes like a mountain I suggest something like that

So you create the thing, you set it up with a goal, and it's the same basic type of machine as the stamp collector, but less powerful.

you know so you create the thing you set it up with a gold and it's the same basic type of machine as the stamp collector but less powerful in the sense

In the sense it has a goal, a thing that it's trying to maximize,

it has a goal a thing that it's trying

and in this case, you know, it's in a little robot body

to maximize and in this case you know

so they can tootle around your lab and do things.

it's in a little robot body so that it can tootle around your lab and do things

So you, you want it to get you a cup of tea just as a test, right?

so you you you want it to get you a cup

So you set it up with this goal you manage to specify in

of tea just as a test right so you set it up with this goal you managed to

the bot's like in the AI's ontology

specify in the bots like in the a is an

what a cup of tea is and that you

ontology what a cup of tea years and

want one to be in front of you.

that you want one to be in front of you

You switch it on and it looks around, gathers

you switch it on and it you know looks

data and it says oh yeah there's a

around gathers data and it says oh yeah

kitchen over there it's got a kettle

there's a kitchen over there it's got a

and it's got teabags and this is like

kettle and it's got tea bags and this is

the easiest way for me to fulfill this

like the easiest way for me to fulfill

goal with the body i have now and

this goal with the body I have now and

everything setup is to go over there and

everything set up is to go over there

make a cup of tea. So far we're doing

and make a cup of tea so far we're doing

very well right. So it starts driving over but then oh no you forgot it's bring

very well right so it starts driving over but then oh no you forgot it's bringing your

your adorable baby to the lab day or

adorable baby to the lab day or

something and there's a kid

something and then and there's a kid in

in the way. Your utility function only

the way your utility function only cares

cares about tea right. So it's not going

about tea right so it's not going to

to avoid hitting the baby. So you rush

avoid hitting the baby so you rush over

over there to hit the button

there to hit the button obviously as you

obviously as you built it in and what

built it in and what happens of course

happens of course is that the robot will

is that the robot will not allow you to

not allow you to hit that button because it

hit that button because it wants to get

wants to get you a cup of tea and if you

you a cup of tea

hit the button it won't get you any tea so

and if you hit the button it won't get

this is a bad outcome so it's going to

you any tea so this is a bad outcome so

try and prevent you in any way possible

it's gonna try and prevent you in any

from shutting it down that's a problem plausibly it fights you

way possible from shutting it down that's a problem plausibly it fights you off crushes the

off crushes the baby and then carries on

baby and then carries on and makes you a

and makes you a cup of tea and the fact that

cup of tea and the fact that this button

this button is supposed to turn it off is not in your utility function that you

is supposed to turn it off is not in

gave it so obviously it's going to fight you ok that was a bad design right assuming

your utility function that you gave it so obviously it's gonna fight you okay that was a bad design right assuming

you're still working on the project

you're still working on the project

after the terrible accident and you have

after the terrible accident you have

another go try to improve things right

another guy trying to improve things

and rather than read any AI safety

right and rather than read any AI safety

research what you do is just come up

research what you do is just come up

with the first thing that pops into your

with the first thing that pops into your

head and you say ok let's add in some reward for the button so that because what it's looking

head then you say okay let's add in some reward for the button so that because

at right now is it says button gets

what it's looking at right now is it

hit i get 0 reward. Button doesn't get

says button gets hit I get zero reward

hit if I manage to stop them then I get

button doesn't get hit you know I managed to stop them then I get the cup

the cup of tea i get like maximum reward

if you give some sort of compensation

I get like maximum reward if you give

for the button being hit maybe it won't

some sort of compensation for the button

mind you hitting the button. If you give it

being hit maybe it won't mind you

less reward for the button being hit

hitting the button if you give it less

than for getting tea, it will still fight you

reward for the button being hit than for getting tea it will still fight you

cause it will go well I could get five reward

because it'll go well I could get five

for accepting your hitting the button

reward for accepting you hitting the

but I could get 10 for getting the tea

button but I could get ten for getting

so I'm still gonna fight you.

the tea so I'm still gonna fight you the

The button being hit has to be just as

button being hit has to be just as good

good as getting the tea so you give it the

as getting the tea so you give it the

same value right so so now you've got a

same value right so so now you've got a

new Version 2. You turn it on

new version too

and what it does immediately is shut itself down because that's so much

you turn it on and what it does immediately is shut itself down because

quicker and easier than going and

that's so much quicker and easier than

getting the tea and gives exactly the

going in getting the tea and gives

exactly the same reward why would it not

Why would it not just immediately shut

just immediately shut itself down so

itself down? So you've accidentally

you've accidentally made like a

made like a dramatically suicidal robot.

dramatically suicidal robot we're kind

We're kind of back to the Three Laws of

of back to the Three Laws of Robotics

Robotics again but could you presumably

again but could you presumably say well

say well the robot is not allowed to to

the robots not allowed to shut itself

shut itself down or something like that? Right yeah so. It's still a reward if it does

down or something like right yes so it's

so it's not gonna fire you but it's not

still a reward if it does so it's not

allowed to do itself i'm guessing that's

gonna fight you but it's not allowed to do it itself I'm guessing that's a

a possibility. The robot cares about

possibility the robot cares about things

things in the real world and the button

in the real world and the button whether

whether it's pushed or not has a big

it's pushed or not has a big effect on

effect on the things that the robot cares

the things that the robot cares about

about which means that the robot cares

which means that the robot cares about

about the button so either it's trying

the button so either it's trying to stop

to stop you from pressing the button or

it's trying to press the button. Either

pressing the button or it's trying to press the button either way that's not

way that's not what you want so suppose then you don't mount the

what you want so suppose then you don't

button on its chest where it can hit it

mount the button on its chest where it

right. You put the button somewhere only

can hit it right you put the button

you can hit it somehow you managed to

somewhere only you can hit it somehow

completely isolate the bot from the

you manage to completely isolate the box

button. It's a fingerprint scanner or something?

from the button it's a fingerprint

It's yeah it's something that it can't do

scanner or something it's yeah it's

itself right. It still either wants to

something that it can't do itself right it's still either wants to press the

press the button

button or wants to button not to be

or wants the button not to be pressed but now the only path is through you

pressed but now the only path is through

so it's now incentivised to manipulate

you so it's now incentivized to

you to deceive you to lie to you in some

manipulate you to deceive you to lie to

way either to cause you to not press it to

you in some way either to cause you to

cause you to press it if you somehow

not press it or to cause you to press it

managed to persuade it it's not allowed to

if you somehow manage to persuade it

interact with the button itself so in the case where it wants the button not

it's not allowed to interact with the button itself so in the case where it

to be pressed it has a reason to deceive

wants the button not to be pressed it has a reason to deceive you so in this

you so in this example it would steer

example it would steer around the baby

around the baby it would avoid the baby

it would avoid the baby assuming it has

assuming it has basic level of intelligence to know you don't want it

basic level of intelligence to know you

to hit the baby and moreover if you

don't want it to hit the baby and

realize your mistake you will hit the

moreover if you realize your mistake you

will hit the button shut it down and

shut it down and change it which it doesn't want so it will behave as though

change it which it doesn't want so it

this is a there's a term which i

will behave as though this is a Turk there's there's a term which I like to

like to use here i'm not sure if I

use here I'm not sure if I should just

should just use it because it's my own which is that it will Volkswagen you.

use it because it's my own which is that

That it will set itself up so

it will fall swag on you that it will

that it it behaves properly in the

set itself up so that it it behaves

testing environment despite the fact

properly in the testing environment

that it actually shouldn't pass the

despite the fact that it's actually

test. You're testing it to see if it

shouldn't pass the test you you're

avoids running over babies. So

testing it to see if it avoids running over baby's butt so you're trying to see

you're trying to see does it care about harming humans but actually all it

does it care about her harming humans but actually all it cares about is you

cares about is you not hitting the button and so then you might end up in a

not hitting the button and so then you

situation in which you become convinced

might end up in a situation in which you

that the thing is perfectly safe because

become convinced that the thing is

it keeps passing all of your moral

perfectly safe because it keeps passing

puzzles and test that you give it but

all of your moral puzzles and tests that

actually it's just has enough of an

you give it but actually it's just has

understanding of your psychology to tell

enough of an understanding of your

what it thinks you've put in and what it

psychology to tell what it thinks you've put in and what it

wants of what you want rather so it will

wants of what you want rather so it will

for as long as that button is able to

for as long as that button is able to

shut it down it will continue to attempt

shut it down it will continue to attempt

to deceive you until it's in a

to deceive you until it's in a situation

situation in which it believes it can't

in which it believes it can't be shut

be shut down and then you have a problem

down and then you have a problem on the

on the other side if you accidentally

other side if you accidentally make

make something that wants the button to

something that wants the button to be

be hit but is unable to it is going to

hit but is unable to it is going to try

try and manipulate you into pressing the

and manipulate you into pressing the

button so if you create your thing you

button so if you create your thing you

say okay make me a cup of tea if the button is

say okay make me a cup of tea if the

hit shut yourself down

button is hit shut yourself down you

you aren't allowed to shut yourself down

aren't allowed to shut yourself down you

you aren't allowed to hit the button but if

aren't allowed to hit the button but if

the button is hit the reward you get is

the button is hit the reward you get is

just as good at getting the tea so that

just as good as getting the tea so that

you don't have a preference this machine

you don't have a preference this machine

really wants to hit it's own button because

really wants to hit its own button

that's as good as getting the tea so

because that's as good as getting the

what it's likely to do probably is just take a

tea so what it's likely to do probably

swing at you or something just

is just take a swing at you or something

immediately because if it can quickly

just immediately because if it can

persuade you to hit the button if

quickly persuade you to hit the button

scaring you into hitting the button is

if scaring you into hitting the button

easier than getting the tea it will just

is easier than getting the tea it'll

do that instead which is a really kind

just do that instead which is a really

of unexpected outcome that you've made

kind of unexpected outcome that you've

this thing with perfectly reasonable

made this thing with perfectly

sounding rewards and what it does

reasonable sounding rewards and what it

immediately is try to terrify you

does immediately is try to terrify you

it reminds me of the proverbial carrots

it reminds me of the proverbial

and a stick this this is almost like this

character mistake this this is it almost

is the stick and actually we need to

like this is the stick and actually we

find what the carrot is would that be a

need to find what the characters would

fair thing to say

that be a fair thing to say yeah yeah

yeah yeah you want it to actually want it it's interesting because it has

you want it to actually want you it's

to not care about whether the button is

interesting because it has to not care about whether the button is pressed

pressed right because it has to take

right because it has to take no steps to

no steps to try and cause the button to

try and cause the button to be pressed

be pressed and take no steps to try and

and take no steps to try and prevent the

prevent the button from being pressed

button from being pressed but

but nonetheless really care that the

nonetheless really care that the button

button exists so one thing that you

exists so one thing that you can do

can do something slightly more sensible is you define the utility function such

something slightly more sensible is you define the utility function such that

that the whole part of of what it's

the whole part of of what it's really

really trying to achieve in the world

trying to achieve in the world and the

and the part about paying attention to

part about paying attention to the

the button being pressed and turning

button being pressed

itself off and it sets it up so that

turning itself off and it sets it up so

those it adds an adjustment tab so that

that those it adds an adjustment term so

those are always exactly equal however

that those are always exactly equal

however much value it would get from

however however much value it would get

either it being pressed it not being

from either it being pressed during not

pressed it normalizes those so that it's

being pressed it normalizes those so

always completely indifferent to whether

that it's always completely indifferent

the button is being pressed. It just doesn't

to where from the button is being

pressed he just doesn't care so that way

so that way it will never try and hit

it will never try and hit the button on

the button on its own it will never try

its own it will never try and prevent

and prevent you from hitting the button

you from hitting the button that's the

that's the idea that's a fairly sensible

idea that's a fairly sensible approach

approach it has some of its own problem feels like a really complicated thing to

it has some of its own problems it

evaluate to be honest

sounds like a really complicated thing

yeah firstly yeah it is kind of tricky

to evaluate to be honest yeah

and you have to get it right but that's

firstly yeah it is kind of tricky and

always the always part of this but one

you have to get it right but that's always always part of this but one thing

thing that's interesting about that is

that's interesting about that is that it

that it isn't what's called subagent stable so this is something that is a

isn't what's called sub-agent stable so this is something that is a desirable

desirable property and it's part of

property and it's part of courage

corrigibility in fact which is that the if there's some property that the agent has you want it to if it makes

ability in fact which is that the if there's some property that the agent has you want it to if it makes any new

any new agents that they will have this

agents that they will have this property

property as well so you get some of that

as well so you get some of that for

for nothing like in the same way that

nothing like in the same way that you

you don't want to take a pill that will

don't want to take a pill that will make

make you want to kill your kids

you want to kill your kids you also

you also don't want to make a clone of

don't want to make a clone of yourself

yourself that wants to kill your kids right if

that one's secure kids right if you're

you're making another instance of

making another instance of yourself or

yourself or you're creating some agent

you're creating some agent you want it

you want it to want what you want this

to want what you want this is like the

is like the usual classic disney way out

usual classic Disney way out of a

of a problem for a baddie character where

problem for a baddie character where

they go I'm not allowed to do this but that

they go I'm not allowed to do this but

doesn't mean I can't get this person to

that doesn't mean I can't get this

person to do this for me I need a young

I need a young pair of legs and a strong

man names destroyed

back to go in after it

going after it exactly so in general

exactly so in general when an AI designs its successor it will design it with the same utility function as it has for

when an AI designs its successor it will design it with the same utility function

the same reason but unlike with some

as it has for the same reason but unlike

safety measures where the AI will be

with some safety measures where the AI

explicitly motivated to remove the

will be explicitly motivated to remove

safety measures if it can by making new

the safety measures if it can by making

agents which doesn't follow them in this

new agents which don't follow them in

case it doesn't care so you might think

this case it doesn't care so you might

well you're okay then it doesn't care it

think what you're okay then it doesn't

won't remove the safety precautions but

care it won't remove the safety

it also won't to put in any effort to

precautions but it also won't put in any

keep them so if it's making a new

effort to keep them so if it's making a

version and it would take a couple of

new version and it would take a couple

extra seconds to put in

of extra seconds to put in oh and also

oh and also respond appropriately to the

respond appropriately to the button and

button and it could do anything else

it could do anything else with that time

with that time or those resources it

or those resources he just won't do it

just won't do it because it looks at the

because it looks at the utilities and

utilities and says if the button is pressed

says if the button is pressed I get ex

i get x utility if the button isn't

utility if the button is impressed I get

pressed i get exactly the same utility

exactly the same utility this bit of

this bit of code does nothing there's no

code does nothing there's no point in it

point in it right this is going to be optimizing itself or optimizing its

so it's gonna be optimizing itself or

successor in a way that

optimizing its successor in a way that

yeah it think is being

yeah it thinks is is being streamlining

streamlining potentially but actually

potentially but actually right missing

is missing the point of and that's

the point of and that's kind of

kind of unavoidable if you give if you

unavoidable if you give if you have a

have a situation in which it wants to

situation in which it wants to preserve

preserve the button so that it can be

the button so that it can be pressed

pressed because it wants that then you've

because it wants that then you've got

made something that wants the button to

made something that wants the button to

be pressed and will try and manipulate you if

be pressed and will try and manipulate you if you if you do it the other way

you if you do it the other way you know

you know whichever way you do it you

whichever way you do it you have

have problems another thing that people

problems another thing that people have thought you know as a suggestion of something

have thought you know as a suggestion or

you might try in this situation is to

something you might try in this

just have the robot have the AI not know

situation is to just have the robot have

about the button. Oh so keep it a secret

the AI not know about the button oh so

right you have it hidden somewhere you

keep it a secret right you have it

know the the machine is being observed

hidden somewhere you know the machine is

from somewhere else and they have the

being observed from somewhere else and

button and it's outside of their control

they have the button and it's so it's

and they don't even know the button exists

outside of their control and they don't

so they're not going to be motivated to

even know the button exists so they're

try and deceive you because they're not

not going to be motivated to try and

aware that you have the capacity to

deceive you because they're not aware that you have the capacity to shut them

shut them down and change them. There are

down and change them there are a few

a few different variants of this but

different variants on this but at the

at the core of the problem there you've

core of the problem there you've got the

got the fact that you're building a

fact that you're building a

general-purpose machine to learn and

general-purpose machine to learn and

reason about the real world of uncertain

reason about the real world of uncertain

abilities I suppose I mean even a very simple

abilities I suppose I mean even very simple optimization systems

optimization systems consistently surprise people and so the idea that you

consistently surprise people and so the

could reliably keep such a thing a

idea that you could reliably keep such a

secret indefinitely and it would never

thing a secret indefinitely and it would

put two and two together in fact while

never put two and two together the fact

you're while you're improving it and

while you're while you're improving it

training it's going to be learning about

and training it it's going to be

human psychology because that's

learning about human psychology because

important for the final goal. It needs to

that's important for the final goal it

understand humans in order to understand

needs to understand humans in order to

what humans want in order to do what humans

understand what humans want in order to

want so the idea that it's never

do what humans want so the idea that

going to think oh interesting these guys

it's never going to think oh interesting

you know they don't seem to trust me i'm

these guys you know they don't seem to

aware there is this concept of an off

trust me I'm aware there is this concept

button and it's weird that they haven't

of an off button and it's weird that

given me one so maybe they have actually

they haven't given me one so maybe they

and just not told me about it that kind

have actually and just not told me about

of hypothesis shouldn't take too long to

it that kind of hypothesis shouldn't take

occur and of course if you've designed

too long to occur and of course if

the thing in an incorrigible way it

you've designed the thing in an incorrigible way it won't let you know

won't let you know that it's figured out

that it's figured out that you have an

that you have an off button and will just

off button and we'll just deceive you as

deceive you as it always had so that's the

it always had so that's the kind of

kind of thing which is which could work

thing which is which could work but it's

but it's not really a reliable solution

not really a reliable solution the other

the other thing that people sometimes

thing that people sometimes suggest and

suggest and this happens this happens in

this happens this happens in

conversation sometimes is that people

conversations sometimes is that people

keep applying patches you have a you

keep applying patches you have a you

have a bad idea for a way to do this and

have a bad idea for a way to do this and

then somebody points out the way that

then somebody points out the way that

would go wrong and then rather than

would go wrong and then rather than

realise the big core approach is flawed

realize the big core approach is flawed

you apply a patch you say oh well we'll

you apply a patch you say oh well we'll

also add a negative term for doing that

also add a negative term for doing that

and then also for doing that you know

and then also for doing that

The spaghetti code ensues yeah yeah and

you know those spaghetti code ensues

what's more you're then in a situation

yeah yeah and what's more you're then in

in which you've got this system that you

a situation in which you've got this

believe you've patched every possible

system that you believe you've patched

way it's it's kind of um you haven't proved

every possible way it's it's kind of you

it's safe you've just proved that you

haven't proved it safe you've just

can't figure out how it's dangerous but what are the chances that you've

proved that you can't figure out how it's dangerous but what are the chances

genuinely thought of every possibility

that you've genuinely thought of every

ideally we really want to be able to

possibility ideally we really want to be

formally prove that the system has these

able to formally prove that the system

properties you don't want a system in

has these properties you don't want a

which you've blocked off loads of

system in which you've blocked off loads

specific actions that the AI can do

and you're just relying on it it's

actions that the AI can do and you're just relying on it it's it's like

like running a complicated search trying

running a complicated search trying to

to figure out a way to screw you over

figure out a way to screw you over and

and you're pretty sure you've blocked

you're pretty sure you've blocked off

off all the angles. You've kind of failed

all the angles you've kind of failed

before you've begun there that your code

before you've begun there that your your

is running this extensive search that

code is is running this extensive search

you just hope fails and if it finds any

that you just hope fails and if it finds

way to do it will jump on that

any way to do it it will jump on that

opportunity it's not it's not a good way of going

opportunity it's not it's not a good way

about things the other point about this

of going about things the other point

is that the button is a toy

about this is that there the button is a

problem it's a simplification that's

toy problem it's a simplification that's

useful for thought experiments because

useful for thought experiment because it

it lets you express your you know it lets you

lets you express your you know lets you

formalize things quite well you only

formalize things quite well you only

have two possible outcomes you hit

have two possible outcomes you hit the

the button or you don't hit the button but in

button or you don't hit the button but

fact with corrigibility what we want is a

in fact with courage ability what we

more complex range of behaviors we

want is a more complex range of

want it to actually assist the

behaviors we want it to actually assist

programmers in its own development

the programmers in its own development

because if it has you know it has some

but if it has you know it has some

understanding of its own operation you

understanding of its own operation you

want it to be able to actually point out

want it to be able to actually point out

your mistakes to you or seek out new

your mistakes to you or seek out new

information perhaps if you say something

information perhaps if you say something

ambiguous rather than just assuming to

ambiguous rather than just assuming to

say well do you mean this or do you

say well do you mean this or do you mean

mean this or if you if you believe that

this or if you if you believe that

you've been programmed poorly to

you've been programmed poorly to

actually draw the programmer's attention to

actually draw the programmers attention

what may be the mistake rather than like

to what may be the mistake rather than

quietly storing that away for any time

like quietly storing that away for any

that they might try and press this button on you

time that they might try and press this

button on you you know

likewise wanting to maintain and repair

likewise wanting to maintain and repair

the safety systems and so on these are these are more complicated behaviours than

the safety systems and so on these are these are more complicated behaviors

just not stopping you from pressing the

than just not stopping you from pressing

button and not trying to manipulate you

the button and not trying to manipulate

into not pressing the button so there are

you into not pressing the button so

some things that might work as solutions

there are some things that might work as

for this specific case but you would

solutions for this specific case but you

hope that a really good solution to the

would hope that a really good solution

off button problem would if you run it in a more

to the off button problem would if you

complicated scenario also produce these

run it in a more complicated scenario

good more complicated behaviours in

also produce these good more complicated

that situation so that's part of why

behaviors in that situation so that's

there are some things that maybe are

part of why there are some things that

solutions to this problem but they're

may be our solutions to this problem but

not they're not they're only solutions

they're not they're not there

to this specific instance of the problem

solutions to this specific instance of

rather than the general issue we're trying

the problem rather than the general

to deal with right now we have a few different proposals for ways to ways to create a AGI with these properties

issue are trying to deal with right now we have a few different proposals for ways to ways to create AGI with these properties but none of them are without

but none of them are without problems

none of them seems to perfectly solve

none of them seems to perfectly solve

all of these properties in a way that we

all of these properties in a way that we

can be really confident of. So this is

can be really confident of so this is

considered an open problem so I kind of

considered an open problem so I kind of

like this as a place to go from the

like this as a place to go from the

previous thing because it gives I think

previous thing because it gives I think

it gives people a feel for where we are

it gives people a feel for where we are

the types of problems that it seems like

the types of problems that it seems like

the simplest thing in the world right

the simplest thing in the world right

you've got a robot with a button

you've got a robot with a button how do

how do you make it not stop you from

you make it not stop you from hitting

hitting the button but also not try and

the button but also not try and persuade

persuade you to hit the button

you to hit the button that should be

that should be easy and doesn't seem like it is

easy and doesn't seem like it is so

so utility function is what the AI

utility function is what the AI cares

cares about so the the stamp collecting

about so the stamp collecting device its

device its utility function was just how

utility function was just how many

many stamps in a year

stamps in a year this is kind of like

this is kind of like its measure is it

it's measure is it yeah it's what it's

yeah it's what it it's the thing that

it's the thing that it's trying to

it's trying to optimize

Similar videos:

General AI Won't Want You To Fix its Code - Computerphile
Computerphile
359.6K views 10K likes 177 dislikes 28 Feb 201787% match

Unicorn AI - Computerphile
Computerphile
339.5K views 7.8K likes 189 dislikes 04 Jul 201931% match

AI Self Improvement - Computerphile
Computerphile
386.9K views 10K likes 111 dislikes 24 Jul 201529% match

Generative Adversarial Networks (GANs) - Computerphile
Computerphile
449.4K views 13.2K likes 226 dislikes 25 Oct 201729% match

50 Years of Computer Science: Professor Brailsford Q&A - Computerphile
Computerphile
81.3K views 3K likes 17 dislikes 22 Apr 202028% match

P vs NP on TV - Computerphile
Computerphile
527.4K views 7.8K likes 537 dislikes 08 Feb 201427% match

Inside the CPU - Computerphile
Computerphile
280.2K views 6.2K likes 94 dislikes 22 Mar 201726% match

Where did Bytes Come From? - Computerphile
Computerphile
195.3K views 6.1K likes 60 dislikes 01 May 201826% match

ALGOL 60 at 60 - Computerphile
Computerphile
73.7K views 3.1K likes 20 dislikes 04 Jun 202026% match

PDF, What is it FOR? - Computerphile
Computerphile
409K views 12K likes 144 dislikes 16 Sep 201625% match

Why C is so Influential - Computerphile
Computerphile
1.6M views 22.5K likes 1.6K dislikes 25 Aug 201725% match

Resizing Images - Computerphile
Computerphile
360.9K views 8.5K likes 115 dislikes 02 Sep 201624% match

Lambda Calculus - Computerphile
Computerphile
803.9K views 19.5K likes 567 dislikes 27 Jan 201724% match

Computing Limit - Computerphile
Computerphile
358.4K views 11.3K likes 443 dislikes 03 Nov 201724% match

MegaProcessor - Computerphile
Computerphile
360.9K views 15.5K likes 101 dislikes 02 May 201724% match

Where GREP Came From - Computerphile
Computerphile
644.1K views 23.4K likes 152 dislikes 06 Jul 201824% match

CNN: Convolutional Neural Networks Explained - Computerphile
Computerphile
718K views 11.2K likes 142 dislikes 20 May 201624% match

Bitcoin Problems - Computerphile
Computerphile
307.7K views 5.3K likes 599 dislikes 12 Feb 201624% match

Deadly Truth of General AI? - Computerphile
Computerphile
817.6K views 18.7K likes 358 dislikes 17 Jun 201524% match

Holy Grail of AI (Artificial Intelligence) - Computerphile
Computerphile
478.7K views 8.2K likes 109 dislikes 01 May 201524% match

Physics of Computer Chips - Computerphile
Computerphile
552.2K views 13.7K likes 207 dislikes 19 Aug 201624% match

Password Cracking - Computerphile
Computerphile
3M views 60.3K likes 1K dislikes 13 Jul 201623% match

Parsing Explained - Computerphile
Computerphile
134.2K views 3.6K likes 45 dislikes 13 Nov 201923% match

Deep Learning - Computerphile
Computerphile
392.2K views 6.1K likes 280 dislikes 19 Apr 201623% match

Turing & The Halting Problem - Computerphile
Computerphile
684.3K views 11.9K likes 461 dislikes 21 Aug 201423% match

How to Choose a Password - Computerphile
Computerphile
1M views 21.1K likes 235 dislikes 20 Jul 201622% match

The Problem with Time & Timezones - Computerphile
Computerphile
2.7M views 95.1K likes 893 dislikes 30 Dec 201322% match

Inside a Neural Network - Computerphile
Computerphile
347.3K views 6.4K likes 67 dislikes 30 Jun 201622% match

Hill Climbing Algorithm & Artificial Intelligence - Computerphile
Computerphile
355K views 7.7K likes 165 dislikes 09 Sep 201422% match

The Joys of Updating & Upgrading - Computerphile
Computerphile
95.9K views 4.1K likes 69 dislikes 06 May 202022% match

Settings Privacy About Contact Donate