11. Introduction to Machine Learning MIT OpenCourseWare English English (auto-generated)

11. Introduction to Machine LearningYouTube↗ IA↗(long shot)
Education 1.1M views 15.5K likes 298 dislikes 19 May 2017

MIT OpenCourseWare 6.3M subscribers United States YouTube↗

Click to Play Video

The following content is provided under a Creative

the following content is provided under

Commons license.

a Creative Commons license your support

Your support will help MIT OpenCourseWare

will help MIT OpenCourseWare continue to

continue to offer high quality educational resources for free.

offer high quality educational resources for free

To make a donation or to view additional materials

to make a donation or to view additional

from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

materials from hundreds of MIT courses visit MIT opencourseware at ocw.mit.edu

ERIC GRIMSON: OK. Welcome back.

okay welcome back you know it's that

You know, it's that time a term when we're all kind of doing this.

time a term when we're all kind of doing

So let me see if I can get a few smiles by simply noting to you

this so let me see if I can get a few smiles by simply noting to you that two

that two weeks from today is the last class.

weeks from today is the last class

Should be worth at least a little bit of a smile, right?

should be worth at least a little bit of

Professor Guttag is smiling.

a smile right professor Guttag is

He likes that idea.

smiling he likes that idea you're almost

You're almost there.

What are we doing for the last couple of lectures?

what are we doing for the last couple of

We're talking about linear regression.

lectures we're talking about linear

And I just want to remind you, this

regression and I just want to remind you

was the idea of I have some experimental data.

this was the idea of I have some experimental data case of a spring where

Case of a spring where I put different weights on measure

I put different weights on a measure

displacements and regression was giving

And regression was giving us a way of deducing a model

us a way of deducing a model to fit that

to fit that data. And In some cases it was easy.

data in some cases it was easy we knew

We knew, for example, it was going to be a linear model.

for example it was going to be a linear

We found the best line that would fit that data.

model we found the best line that would

In some cases, we said we could use validation

fit data in some cases we said we could use validation to actually let us

to actually let us explore to find the best model that

explore to find the best model it would

would fit it, whether a linear, a quadratic, a cubic,

fit it whether it's a linear or cubic or sorry linear a quadratic ax cubed some

some higher order thing.

higher order thing so we be using that

So we'll be using that to deduce something about a model.

to deduce something about a model that's

That's a nice segue into the topic for the next three

a nice segue into the topic for the next

lectures, the last big topic of the class,

three lectures the last big topic of the

which is machine learning.

class which is machine learning and I'm

And I'm going to argue, you can debate whether that's actually

going to argue you can debate whether

an example of learning.

that's actually an example of learning

But it has many of the elements that we

but it has many of the elements that we

want to talk about when we talk about machine learning.

want to talk about when we talk about machine learning so as always there's a

So as always, there's a reading assignment.

reading assignment chapter 22 of the

Chapter 22 of the book gives you a good start on this,

book gives you a good start on this and

and it will follow up with other pieces.

it will follow up with other pieces and

And I want to start by basically outlining

I want to start by basically outlining

what we're going to do.

what we're going to do and I'm going to

And I'm going to begin by saying,

begin by saying as I'm sure you're aware

as I'm sure you're aware, this is a huge topic.

this is a huge topic I've listed just

I've listed just five subjects in course six

five subjects in course six that all

that all focus on machine learning.

focus on machine learning and that

And that doesn't include other subjects

doesn't include other subjects we're

where learning is a central part.

learning as a central part so natural

So natural language processing, computational biology,

language processing computational

computer vision robotics all rely today,

biology computer vision robotics all rely today heavily on machine learning

heavily on machine learning.

and you'll see those in those subjects

And you'll see those in those subjects as well.

as well so we're not going to compress

So we're not going to compress five subjects into three lectures.

five subjects into three lectures but

But what we are going to do is give you the introduction.

what we are going to do is give you the

We're going to start by talking about the basic concepts

introduction we're going to start by talking about the basic concepts of

of machine learning.

machine learning the idea of having

The idea of having examples, and how do you talk about features

samples and how do you talk about

representing those examples, how do

features representing those examples how

you measure distances between them,

do you measure distances between them

and use the notion of distance to try

and use the notion of distance to try

and group similar things together as a way

and group similar things together as a

of doing machine learning.

way of doing machine learning and we're

And we're going to look, as a consequence,

going to look as a consequence of two

of two different standard ways of doing learning.

different standard ways of doing learning one we call classification

One, we call classification methods.

methods example we're going to see there

Example we're going to see, there is something called "k nearest neighbor"

something called K nearest neighbor in

and the second class, called clustering methods.

the second class called clustering

Classification works well when I have what

methods classification works well when I

we would call labeled data.

have what we would call label data I

I know labels on my examples, and I'm

know labels on my examples and I'm going

going to use that to try and define classes

to use that to try and define classes

that I can learn, and clustering working well,

that I can learn and clustering working

when I don't have labeled data.

well when I don't have label data and

And we'll see what that means in a couple of minutes.

we'll see what that means in a couple of

But we're going to give you an early view of this.

minutes but we're going to give you an early view of this unless professor

Unless Professor Guttag changes his mind,

Guttag changes his mind we're probably

we're probably not going to show you the current really

not going to show you the current really

sophisticated machine learning methods

sophisticated machine learning methods

like convolutional neural nets or deep learning,

like convolutional neural nets or deep

things you'll read about in the news.

learning things you'll read about in the

But you're going to get a sense of what's

news but you're going to get a sense of

behind those, by looking at what we do when we

what's behind those by looking at what

talk about learning algorithms.

we do when we talk about learning algorithms before I do it I want to

Before I do it, I want to point out to you

point out to you just how prevalent this

just how prevalent this is. And I'm going to admit with my gray hair,

is and I'm going to admit with my gray

I started working in AI in 1975 when machine learning was

hair I started working in AI in 1975

a pretty simple thing to do.

when machine learning was a pretty

And it's been fascinating to watch

simple thing to do and it's been

over 40 years, the change.

fascinating to watch over 40 years the

And if you think about it, just think about where you see it.

change and if you think about it just think about where you see it alphago

AlphaGo, machine learning based system from Google that beat

machine learning based system from

a world-class level Go player.

Google the beat a world-class level go

Chess has already been conquered by computers for a while.

player chess has already been computed

Go now belongs to computers.

for a while go now belongs to computers

Best Go players in the world are computers.

best go players in the world or computers I'm sure many of you use

I'm sure many of you use Netflix.

Netflix any recommendation system

Any recommendation system, Netflix, Amazon, pick your favorite, uses a machine learning algorithm

Netflix Amazon pick your favorite uses a

to suggest things for you.

machine learning algorithm to suggest

And in fact, you've probably seen it on Google, right?

things for you in fact you've probably seen it on Google right the ads that pop

The ads that pop up on Google are

up on Google are coming from a machine

coming from a machine learning algorithm that's

learning algorithm that's looking at

looking at your preferences. Scary thought.

your preferences scary thought drug

Drug discovery, character recognition-- the post office

discovery character recognition the post

does character recognition of handwritten characters using

office does character recognition of

a machine learning algorithm and a computer vision system

handwritten characters using a machine learning algorithm in a computer vision

system behind it you probably don't know

You probably don't know this company.

this company it's actually an MIT

It's actually an MIT spin-off called Two Sigma,

spin-off called to Sigma it's a hedge

it's a hedge fund in New York. They heavily use AI and machine learning techniques.

fund in New York they heavily use AI a

And two years ago, their fund returned a 56% return.

machine learning techniques in years ago their fund returned a 56% return I wish I'd invested in the fund I

I wish I'd invested in the fund.

don't have the kinds of millions you

I don't have the kinds of millions you need, but that's an impressive return.

need but that's an impressive return 56%

56% return on your money in one year.

return on your money in one year last

Last year they didn't do quite as well,

year they didn't do quite as well but

but they do extremely well using machine learning techniques.

they do extremely well using machine

learning techniques Siri

Another great MIT company called Mobileye

another great MIT company called mobile

that does computer vision systems with a heavy machine

I that does computer vision systems with

learning component that is used in assistive driving

a heavy machine learning component that is used in assistive driving and will be

and will be used in completely autonomous driving.

used in completely autonomous driving it

It will do things like kick in your brakes

will do things like kick in your brakes

if you're closing too fast on the car in front of you,

if your closing too fast on the car in

which is going to be really bad for me

front of you which is going to be really

because I drive like a Bostonian.

bad for me because I Drive like a like a

And it would be kicking in constantly. Face recognition.

Bostonian and it would be kicking in constantly face recognition Facebook

Facebook uses this, many other systems

uses this many other systems do to

do to both detect and recognize faces.

recognize both detect and recognize

IBM Watson-- cancer diagnosis.

faces IBM Watson cancer diagnosis these

These are all just examples of machine

are all just examples of machine

learning being used everywhere.

learning being used everywhere and it

And it really is. I've only picked nine.

really is I've only picked nine so what

is it I'm going to make an obnoxious

I'm going to make an obnoxious statement. You're now used to that.

statement you're now used to that I'm

I'm going to claim that you could

going to claim that you know you could

argue that almost every computer program learns something.

argue that almost every computer program learns something but the level of

But the level of learning really varies a lot.

learning really varies a lot so if you

So if you think back to the first lecture in 60001,

think back to the first lecture in six triple oh one we showed you Newton's

we showed you Newton's method for computing square roots.

method for computing square roots and

And you could argue, you'd have to stretch it,

you could argue you have to stretch it

but you could argue that that method learns

but you could argue that that method

something about how to compute square roots.

learn something about how to compute

In fact, you could generalize it to roots of any order power.

square roots in fact you could generalize it to roots of any order

But it really didn't learn.

power but it really didn't learn I

I really had to program it.

really had to program it and I think

All right. Think about last week when we talked about linear regression.

about last week when we talked about

Now it starts to feel a little bit more

linear regression that starts to feel a

like a learning algorithm.

little bit more like a learning

Because what did we do?

algorithm because what did we do we gave

We gave you a set of data points,

you a set of data points mass

mass displacement data points.

displacement data points and then we

And then we showed you how the computer could essentially

showed you how the computer could

fit a curve to that data point.

essentially fit a curve to that data

And it was, in some sense, learning a model for that data

point and it was in some sense learning a model for that data that it could then

that it could then use to predict behavior.

use to predict behavior in other

In other situations.

situations and that's getting closer to

And that's getting closer to what

what we would like when we think about a

we would like when we think about a machine

machine learning algorithm we'd like to

learning algorithm. We'd like to have program that can learn from experience,

have program that can learn from experience something that it can

something that it can then use to deduce new facts.

then use to deduce new facts now it's

Now it's been a problem in AI for a very long time.

been a problem in AI for a very long

And I love this quote.

time and I'd love this quote as from a

It's from a gentleman named Art Samuel.

gentleman named art Samuel 1959 is the

1959 is the quote in which he says,

quote in which he says his definition of

his definition of machine learning

machine learning it's the field of study

is the field of study that gives computers

that gives computers the ability to

the ability to learn without being explicitly programmed.

learn without being explicitly programmed and I think many people would

And I think many people would argue, he wrote the first such program.

argue he wrote the first such program it

It learned from experience.

learned from experience in his case it

In his case, it played checkers.

played checkers kind of shows you how

Kind of shows you how the field has progressed.

the field is progressed at we started

But we started with checkers, we got to chess, we now do Go.

with checkers we got the chess we now do

But it played checkers.

go but it played checkers it beat

It beat national level players, most importantly,

national level players most importantly

it learned to improve its methods

it learned to improve its methods by

by watching how it did in games and then inferring something

watching how it did in games and then

to change what it thought about as it did that.

inferring something to change what it thought about as it did that Samuel did

Samuel did a bunch of other things.

a bunch of other things I've just

I just highlighted one.

highlighted one you may see in a

You may see in a follow on course,

follow-on course he invented what's

he invented what's called Alpha-Beta Pruning, which

called alpha-beta pruning which is a

is a really useful technique for doing search.

really useful technique for doing search

But the idea is, how can we have the computer learn

but the idea is how can we have the computer learn without being explicitly

without being explicitly programmed?

programmed in one way to think about

And one way to think about this is

this is to think about the difference

to think about the difference between how we would normally

between how we would normally program

program and what we would like from a machine learning

and what we would like from a scene

learning algorithm normal programming I

Normal programming, I know you're not convinced

know you're not convinced there's such a

there's such a thing as normal programming,

thing as normal programming but if you

but if you think of traditional programming,

think of traditional programming what's

what's the process? I write a program that I input to the computer

the process I write a program that I input to the computer so that it can

so that it can then take data and produce

then take data and produce some

some appropriate output.

appropriate output and the square root

And the square root finder really sits there, right?

finder really sits there right I wrote

I wrote code for using Newton method to find a square root,

code for using Newton method to find a

and then it gave me the process of given any number,

square root and then it gave me the problem or the process of given any

I'll give you the square root.

number I'll give you the square root but

But if you think about what we did last time,

if you think about what we did last time

it was a little different.

it was a little different and in fact in

And in fact, in a machine learning approach,

a machine learning approach the idea is

the idea is that I'm going to give the computer output.

that I'm going to give the computer output I'm going to give it examples of

I'm going to give it examples of what I want the program to do,

what I want the program to do labels on

labels on data, characterizations

data characterizations of different

of different classes of things.

classes of things and what I want the

And what I want the computer to do

computer to do is given that

is, given that characterization of output and data,

characterization of output and data I

I wanted that machine learning algorithm

want it that machine learning algorithm

to actually produce for me a program,

to actually produce for me a program a

a program that I can then use to infer

program that I can then use

new information about things.

to infer new information about things

And that creates, if you like, a really nice loop

and that creates if you like a really nice loop where I can have the machine

where I can have the machine learning algorithm

learning algorithm learn the program

learn the program which I can then use

which I can then use to solve some other

to solve some other problem.

problem that would be really great if we

That would be really great if we could do it.

could do it and as I suggested that

And as I suggested, that curve-fitting algorithm

curve fitting algorithm is a simple

is a simple version of that.

version of that it learned a model for

It learned a model for the data, which I could then

the data which I could then use to label

use to label any other instances of the data

any other instances of the date or

or predict what I would see in terms of spring displacement

predict what I would see in terms of

as I changed the masses.

spring displacement as I changed the masses so that's the kind of idea we're

So that's the kind of idea we're going to explore.

going to explore if we want to learn

If we want to learn things, we could also

things we could also ask so how do you

ask, so how do you learn? And how should a computer learn?

learn and how should a computer learn

Well, for you as a human, there are a couple of possibilities.

well for you as a human there are a

This is the boring one.

couple of possibilities this is the

This is the old style way of doing it, right?

boring one this is the old-style way of

Memorize facts.

doing your right memorize facts memorize

Memorize as many facts as you can and hope that we ask you

as many facts as you can and hope that

on the final exam instances of those facts,

we ask you on the final exam instances

as opposed to some other facts you haven't memorized.

of those facts as opposed to some other facts you haven't memorized this is if

This is, if you think way back to the first lecture,

you think way back to the first lecture

an example of declarative knowledge, statements of truth.

an example of declarative knowledge statements of truth memorize as many as

Memorize as many as you can.

you can have Wikipedia in your back

Have Wikipedia in your back pocket. Better way to learn is to be able to infer, to deduce

pocket a better way to learn is to be able to infer to deduce new information

new information from old.

from old and if you think about this

And if you think about this, this

this gets closer to what we called

gets closer to what we called imperative knowledge--

imperative knowledge ways to deduce new

ways to deduce new things. Now, in the first cases, we built

things now in the first cases we built

that in when we wrote that program to do square roots.

that in when we wrote that program to do

But what we'd like in a learning algorithm

square roots but what we like in a

is to have much more like that generalization idea.

learning algorithm is to have much more like that generalization idea we're

We're interested in extending our capabilities

interested in extending our capabilities

to write programs that can infer useful information

to write programs that can infer useful

from implicit patterns in the data.

information from implicit patterns in

So not something explicitly built

the data so not something explicitly

like that comparison of weights and displacements,

built-in like that comparison of weights

but actually implicit patterns in the data,

and displacements but actually implicit

and have the algorithm figure out what those patterns are,

patterns in the data and have the algorithm figure out what those patterns

and use those to generate a program you

are and use those to generate a program

can use to infer new data about objects,

you can use to infer new data about

about string displacements, whatever

objects about spring displacements

it is you're trying to do.

whatever it is you're trying to do ok so

OK. So the idea then, the basic paradigm

the idea then the basic paradigm that

that we're going to see, is we're

we're going to see is we're going to

going to give the system some training

give system some training data some

data, some observations.

survey shion's we did that last time

We did that last time with just the spring displacements.

with just the spring displacements we're

We're going to then try and have a way

going to then try and have the way to

to figure out, how do we write code, how do we

figure out how do we write code how do

write a program, a system that will infer something

we write a program a system that will

about the process that generated the data? And then from that, we want to be

infer something about the process that generated the data and then from that we

able to use that to make predictions about things

want to be able to use that to make

we haven't seen before.

predictions about things we haven't seen before so again I want to drive home

So again, I want to drive home this point. If you think about it, the spring example fit that model.

this point if you think about it the spring example fit that model I gave you

I gave you a set of data, spatial deviations

a set of data spatial deviations

relative to mass displacements.

relative to massive places for different

For different masses, how far did the spring move?

masses how far did the spring move

I then inferred something about the underlying process.

I then inferred something about the underlying process the first case I said

In the first case, I said I know it's linear,

I know it's linear but let me figure out

but let me figure out what the actual linear equation is.

what the actual linear equation is

What's the spring constant associated with it?

what's the spring constant associated

And based on that result, I got a piece of code

with it and based on that result I got a

I could use to predict new displacements.

piece of code I could use to predict new

So it's got all of those elements, training data,

displacements so it's got all of those elements training data into inference

an inference engine, and then the ability

engine and then the ability to use that

to use that to make new predictions.

to make new predictions but that's a

But that's a very simple kind of learning setting.

very simple kind of learning setting so

So the more common one is one I'm

the more common one is what I'm going to

going to use as an example, which

use as an example which is when I give

is, when I give you a set of examples,

you a set of examples those examples

those examples have some data associated with them,

have some data associated with them some

some features and some labels.

features and some labels that vary each

For each example, I might say this

example I might say this is a particular

is a particular kind of thing.

kind of thing this other one's another

This other one is another kind of thing.

kind of thing and what I want to do is

And what I want to do is figure out

figure out how to do inference on

how to do inference on labeling new things.

labeling new things so it's not just

So it's not just, what's the displacement of the mass,

what's the displacement of the mass it's

it's actually a label.

actually a label and I'm going to use

And I'm going to use one of my favorite examples.

one of my favorite examples I'm a big

I'm a big New England Patriots fan,

New England Patriots fan if you're not

if you're not, my apologies.

my apologies but I'm going to use

But I'm going to use football players.

football players so I'm going to show

So I'm going to show you in a second,

you in a second I'm going to give you a

I'm going to give you a set of examples of football players.

set of examples of football players the

The label is the position they play.

label is the position they play in the

And the data, well, it could be lots of things.

data well it could be lots of things

We're going to use height and weight.

we're going to use height and weight but

But what we want to do is then see

what we want to do is then see if how

how would we come up with a way of characterizing

would we come up with the way of characterizing the implicit pattern of

the implicit pattern of how does weight and height predict

how does weight and height predict the

the kind of position this player could play.

kind of position this player could play

And then come up with an algorithm

and then come when you come up with an

that will predict the position of new players.

algorithm will predict the position of

We'll do the draft for next year.

new players we do the draft for next

Where do we want them to play?

year where do we want them to play

That's the paradigm.

that's the paradigm set of observations

Set of observations, potentially labeled, potentially not.

potentially they will potentially not

Think about how do we do inference to find a model.

think about how do we do inference to

And then how do we use that model to make predictions.

find a model and then how do we use that model to make predictions what we're

What we're going to see, and we're

going to see and we're going to see

going to see multiple examples today,

multiple examples today is that that

is that that learning can be done in one of two very broad ways.

learning can be done in one of two very broad ways the first one is called

The first one is called supervised learning.

supervised learning in that case for

And in that case, for every new example

every new example I give you as part of

I give you as part of the training data, I have a label on it.

the training data I have a label on it I

I know the kind of thing it is.

know the kind of thing it is and what

And what I'm going to do is look for how

I'm going to do is look for how do I

do I find a rule that would predict the label associated

find a rule that would predict the label

with unseen input based on those examples.

associated with unseen input based on

It's supervised because I know what the labeling is.

those examples it's supervised because I know what the labeling is second kind if

Second kind, if this is supervised,

this is supervised the obvious other one

the obvious other one is called unsupervised.

is called unsupervised in that case I'm

In that case, I'm just going to give you a bunch of examples.

just going to give you a bunch of examples but I don't know the labels

But I don't know the labels associated with them.

associated with them I'm going to just

I'm going to just try and find what

try and find what are the natural ways

are the natural ways to group those examples

to group those examples together into

together into different models.

different models in some cases I may

And in some cases, I may know how many models are there.

know how many models are there in some

In some cases, I may want to just say what's

cases I may want to just say what's the

the best grouping I can find. OK.

best grouping I can find okay what I'm

What I'm going to do today is not a lot of code.

going to do today is not a lot of code I

I was expecting cheers for that, John, but I didn't get them.

was expecting cheers for that John but I

Not a lot of code.

didn't get them not a lot of code what

What I'm going to do is show you basically,

I'm going to do is show you basically

the intuitions behind doing this learning.

the intuitions behind doing this

And I"m going to start with my New England Patriots example.

learning I'm going to start with my New

So here are some data points about current Patriots players.

England Patriots example so here are some data points about current Patriots

And I've got two kinds of positions.

players and I've got two kinds sorry up

I've got receivers, and I have linemen.

positions I've got receivers and I have

And each one is just labeled by the name, the height in inches,

linemen and each one is just labeled by the name the height in inches and the

and the weight in pounds. OK? Five of each.

weight in pounds okay five of each if I

If I plot those on a two dimensional plot,

plot those on a two-dimensional plot

this is what I get. OK? No big deal.

this is what I get okay no big deal what

What am I trying to do?

am I trying to do I'm trying to learn

I'm trying to learn, are their characteristics

are there characteristics that

that distinguish the two classes from one another?

distinguish the two classes from one

And in the unlabeled case, all I have

another and in the unlabeled case all I

are just a set of examples.

have are just a set of examples so what

So what I want to do is decide what

I want to do is decide what makes two

makes two players similar with the goal of seeing,

players similar with the goal of seeing

can I separate this distribution into two or more natural groups.

can I separate this distribution into two or more natural groups similar as a

Similar is a distance measure.

distance measure says how do I take two

It says how do I take two examples with values

examples with values or features

or features associated, and we're

associated when decide how far apart are

going to decide how far apart are they? And in the unlabeled case, the simple way to do it is to say,

they and in the unlabeled case the way to do it is to say if I know that

if I know that there are at least k groups there--

there are at least K groups there in

in this case, I'm going to tell you there are

this case I'm going to tell you there

two different groups there--

are two different groups there how could

how could I decide how best to cluster things

I decide how best to cluster things

together so that all the examples in one group

together so that all the examples in one

are close to each other, all the examples in the other group

group are close to each other all the

are close to each other, and they're reasonably far apart. There are many ways to do it.

examples in the other group are close to each other and they're reasonably far apart there are many ways to do it I'm

I'm going to show you one.

going to show you one it's a very

It's a very standard way, and it works, basically, as follows.

standard way and it works basically as follows if all I know is that there are

If all I know is that there are two groups there,

two groups there I'm going to start by

I'm going to start by just picking two examples as my exemplars.

just picking two examples as my exemplar

Pick them at random.

z' pick them at random actually at

Actually at random is not great.

random is not great I don't want to pick

I don't want to pick too closely to each other.

too close to each other I'm going to try

I'm going to try and pick them far apart.

and pick them far apart but I picked two

But I pick two examples as my exemplars.

examples as my exemplars and for all the

And for all the other examples in the training data,

other examples in the training data I

I say which one is it closest to.

say which one is it closest to what I'm

What I'm going to try and do is create clusters

going to try and do is create clusters

with the property that the distances

with the property that the distances

between all of the examples of that cluster are small.

between all of the examples of that cluster are small the average distance

The average distance is small.

is small and see if I can find clusters

And see if I can find clusters that

it gets the average distance for both

gets the average distance for both clusters as small as possible.

cluster as small as possible this

This algorithm works by picking two examples,

algorithm works by picking two examples

clustering all the other examples by simply saying

clustering all the other examples by

put it in the group to which it's closest to that example.

simply saying put it in the group which into which it's closest to that example

Once I've got those clusters, I'm

once I've got those clusters I'm going

going to find the median element of that group.

to find the median element of that group

Not mean, but median, what's the one closest to the center?

not mean but meeting what's the one closest to the center and treat those as

And treat those as exemplars and repeat the process.

my exemplars and repeat the process and

And I'll just do it either some number of times

I'll just do it either some number of

or until I don't get any change in the process.

times or until I don't get any change in

So it's clustering based on distance.

the process so it's clustering based on

And we'll come back to distance in a second.

distance and we'll come back to distance in a second so here's what would happen

So here's what would have my football players.

with my football players if I just did

If I just did this based on weight,

this based on weight there's the natural

there's the natural dividing line.

dividing line and it kind of makes sense

And it kind of makes sense. All right?

right these three are obviously

These three are obviously clustered,

clustered and again it's just on this

and again, it's just on this axis. They're all down here.

axis they're all down here these seven

These seven are at a different place.

are at a different place there's a

There's a natural dividing line there.

natural dividing line there if I were to

If I were to do it based on height, not as clean.

do it based on height not as clean this

This is what my algorithm came up

is what my algorithm came up with as the

with as the best dividing line here,

best dividing line here meaning that

meaning that these four, again, just based on this axis

these four game just based on this axis

are close together.

are close together these six are close

These six are close together.

together but it's not nearly as clean

But it's not nearly as clean. And that's part of the issue we'll look at

and that's part of the issue we'll look

is how do I find the best clusters.

at us how do I find the best clusters if I use

If I use both height and weight, I

both height and weight I get that

get that, which was actually kind of nice, right?

which is actually kind of nice right

Those three cluster together. they're near each other,

those three cluster together they're

in terms of just distance in the plane.

near each other in terms of just

Those seven are near each other.

distance in the plane those seven are

There's a nice, natural dividing line through here.

near each other there's a nice natural dividing line through here and in fact

And in fact, that gives me a classifier.

that gives me a classifier this line is

This line is the equidistant line

the equidistant line between the Centers

between the centers of those two clusters.

of those two clusters meaning any point

Meaning, any point along this line

on along this line is the same distance

is the same distance to the center of that group

to the center of that group as it is to

as it is to that group.

that group and so any new example if

And so any new example, if it's above the line,

it's above the line I would say gets

I would say gets that label, if it's below the line,

that label if it's below the line gets

gets that label.

that label in a second we'll come back

In a second, we'll come back to look

to look at how do we measure the

at how do we measure the distances,

distances but the idea here is pretty

but the idea here is pretty simple. I want to find groupings near each other

simple I want to find groupings near

and far apart from the other group.

each other and far apart from the other group now suppose I actually knew the

Now suppose I actually knew the labels on these players.

labels on these are players I shouldn't

call them objects they're people these

These are the receivers.

players these are the receivers those

Those are the linemen.

are the linemen and for those of you or

And for those of you who are football fans,

football pan fans you can figure out

you can figure it out, right?

right those are the two tight ends

Those are the two tight ends.

they're much bigger

They are much bigger. I think that's Bennett and that's Gronk if you're really

I think that's Bennett and that's Gronk

a big Patriots fan.

if you're really a big Patriots fan but

But those are tight ends, those are wide receivers,

those are tight ends those are wide

and it's going to come back in a second,

receivers and it's going to come back in

but there are the labels.

a second but there are the labels now

Now what I want to do is say, if I could take advantage

what I want to do is say if I could take

of knowing the labels, how would I divide these groups up?

advantage of knowing the labels how would I divide these groups up and

And that's kind of easy to see.

that's kind of easy to see basic idea in

Basic idea, in this case, is if I've

this case is if I've got labelled groups

got labeled groups in that feature

in that feature space what I want to do

space, what I want to do is find a subsurface that naturally

is find a sub surface that naturally

divides that space.

divides that space now subsurface it's a

Now subsurface is a fancy word.

fancy word it says in the

It says, in the two-dimensional case,

two-dimensional case I don't want to

I want to know what's the best line,

know what's the best line if I can find

if I can find a single line, that separates all the examples

a single line that separates all the

with one label from all the examples of the second label.

examples with one label from all the examples of the second label we'll see

We'll see that, if the examples are well separated,

that if the examples are well separated

this is easy to do, and it's great.

this is easy to do and it's great but in

But in some cases, it's going to be

some cases it's going to be more

more complicated because some of the examples

complicated because some of the examples

may be very close to one another.

may be very close to one another and

And that's going to raise a problem

that's going to raise a problem that you

that you saw last lecture.

saw a last lecture I want to avoid

I want to avoid overfitting.

overfitting I don't want to create a

I don't want to create a really complicated surface

really complicated surface to separate

to separate things. And so we may have to tolerate a few incorrectly

things and so we may have to tolerate a

labeled things, if we can't pull it out.

few incorrectly labeled things if we can't pull it out

And as you already figured out, in this case,

and as you already figured out in this

with the labeled data, there's the best fitting line

case with the labeled data there's the

best fitting line right there anybody

Anybody over 280 pounds is going to be a great lineman.

over 280 pounds is going to be a great

Anybody under 280 pounds is more likely to be a receiver.

lineman anybody under 280 pounds is more likely to be a receiver okay

OK. So I've got two different ways of trying to think

so I've got two different ways of trying

about doing this labeling.

to think about doing this labeling I'm

I'm going to come back to both of them in a second. Now suppose I add in some new data. I want to label new instances.

going to come back to both of them in a second now suppose I add in some new data I want to label new instances now

Now these are actually players of a different position.

these are actually players of a

These are running backs.

different position these are running

But I say, all I know about is receivers and linemen.

backs but I say all I know about is receivers and linemen I get these two

I get these two new data points.

new data points I'd like to know are

I'd like to know, are they more likely to be

they more likely to be a receiver or

a receiver or a linemen?

alignment and there's the data for these

And there's the data for these two gentlemen. So if I go back to now plotting them,

two gentlemen so if I go back to now plotting them oh you notice one of the

oh you notice one of the issues.

issues so there are my linemen the red

So there are my linemen, the red ones are my receivers,

ones are my receivers the two black dots

the two black dots are the two running backs.

are the two running backs notice right

And notice right here. It's going to be really hard to separate those two

here it's going to be really hard to

examples from one another.

separate those two examples from one

They are so close to each other.

another they are so close to each other

And that's going to be one of the things

and that's going to be one of the things

we have to trade off.

we have to trade off but if I think

But if I think about using what I learned as a classifier with unlabeled data, there were my two clusters.

about using what I learned as a classifier with unlabeled data there were my two clusters now you see I've

Now you see, oh, I've got an interesting example.

got an interesting example this new

This new example I would say is clearly more

example I would say is clearly more like

like a receiver than a lineman.

a receiver than alignment but that one

But that one there, unclear. Almost exactly lies along that dividing line

there unclear almost exactly lies along

between those two clusters.

that dividing line between those two clusters and that would either say I

And I would either say, I want to rethink the clustering

want to rethink the clustering or I want

or I want to say, you know what?

to say you know what as I know maybe

As I know, maybe there aren't two clusters here.

there aren't two clusters here maybe

Maybe there are three.

there are three and I want to classify

And I want to classify them a little differently.

them a little differently so I'll come

So I'll come back to that. On the other hand, if I had used the labeled data,

back to that on the other hand if I'd

there was my dividing line.

use the labeled data there was my

This is really easy.

dividing line this is really easy both

Both of those new examples are clearly

of those new examples are clearly below

below the dividing line.

the dividing line they are clearly

They are clearly examples that I would

examples that I would categorize as

categorize as being more like receivers

being more like receivers than they are

than they are like linemen.

like linemen and I know it's a football

And I know it's a football example.

example if you don't like football pick

If you don't like football, pick another example.

another example but you get the sense of

But you get the sense of why I can use the data in a labeled case and the unlabeled case

why I can use the data in the labeled

to come up with different ways of building the clusters.

case in the unlabeled case to come up with different ways building the Buster's so what we're going to do over

So what we're going to do over the next 2 and 1/2 lectures is look at how can we

the next two and a half lectures is look

write code to learn that way of separating things out?

at how can we write code to learn that way of separating things out we're going

We're going to learn models based on unlabeled data.

to learn models based on unlabeled data

That's the case where I don't know what the labels are,

that's the case where I don't know what

by simply trying to find ways to cluster things together

the labels are by simply trying to find ways to cluster things together nearby

nearby, and then use the clusters to assign labels

and then use the clusters to assign

labels to new data and we're going to

And we're going to learn models by looking at labeled data

learn models by looking at labeled data

and seeing how do we best come up with a way of separating

and seeing how do we best come up with a way of separating with a line or a plane

with a line or a plane or a collection of lines, examples

or a collection of lines examples from

from one group, from examples of the other group.

one group from examples of the other

With the acknowledgment that we want to avoid overfitting,

group with the acknowledgment that we want to avoid overfitting we don't want

we don't want to create a really complicated system.

to create a really complicated system

And as a consequence, we're going

and as a consequence we're going to have

to have to make some trade-offs between what

to make some trade-offs between what we

we call false positives and false negatives.

call false positives and false negatives

But the resulting classifier can then label any new data

but the resulting classifier can then

by just deciding where you are with respect

label any new data by just deciding

to that separating line.

where you are with respect to that

separating line so here's what you're

So here's what you're going to see over the next 2

going to see over the next two and a

and 1/2 lectures.

half lectures every machine learning

Every machine learning method has five essential components.

method has five essential components we

We need to decide what's the training data,

need to decide what's the training data

and how are we going to evaluate the success of that system.

and how are we going to evaluate the

We've already seen some examples of that.

success of that system I've already seen some examples of that we need to decide

We need to decide how are we going

how are we going to represent each

to represent each instance that we're giving it.

instance that we're giving it I happen

I happened to choose height and weight for football players.

to choose height and weight for football

But I might have been better off to pick average speed

players but I might have been better off to pick average speed or I don't know

or, I don't know, arm length, something else.

arm length something else how do I

How do I figure out what are the right features.

figure out what are the right features

And associated with that, how do I measure distances

and associated with that how do I measure distances between those features

between those features? How do I decide what's close and what's not close?

how do I decide what's close and what's

Maybe it should be different, in terms of weight versus height,

not close maybe it should be different

in terms of weight versus height for

I need to make that decision.

example I need to make that decision and

And those are the two things we're

those are the two things we're going to

going to show you examples of today, how to go through that.

show you examples up to date how to go through that starting next week

Starting next week, Professor Guttag

professor Guttag is going to show you

is going to show you how you take those and actually start

how you take those and actually start

building more detailed versions of measuring clustering,

building more detailed versions of measuring clustering measuring

measuring similarities to find an objective function that you

similarities to find an objective

want to minimize to decide what is the best cluster to use.

function that you want to minimize to decide what is the best cluster to use

And then what is the best optimization method you want

and then what is the best optimization

to use to learn that model.

method you want to use to learn that model so let's start talking about

So let's start talking about features.

features I've got a set of examples

I've got a set of examples, labeled or not.

I need to decide what is it about those examples that's

I need to decide what is that about

useful to use when I want to decide what's

those examples that's useful to use when

close to another thing or not.

I want to decide what's close to another thing or not and one of the problems is

And one of the problems is, if it was really easy,

if it was really easy it would be really

it would be really easy. Features don't always capture what you want.

easy features don't always capture what

I'm going to belabor that football analogy,

you want I'm going to belabor that

but why did I pick height and weight.

football analogy but why did I pick

Because it was easy to find.

height and weight because it was easy to find but what is you know if you work

You know, if you work for the New England Patriots, what

for the New England Patriots what is the

is the thing that you really look for when you're asking,

thing that you really look for when

what's the right feature?

you're asking what's the right feature

It's probably some other combination of things.

it's probably some other combination

So you, as a designer, have to say what

things so you as a designer have to say

are the features I want to use.

what are the features I want to use I

That quote, by the way, is from one

quote by the way it's from one of the

of the great statisticians of the 20th century, which

great statisticians of the 20th century

I think captures it well.

which I think captures it well so

So feature engineering, as you, as a programmer,

feature engineering as you as a

comes down to deciding both what are the features

programmer comes down to deciding both

I want to measure in that vector that I'm going to put together,

what are the features I want to measure in that vector that I'm going to put

and how do I decide relative ways to weight it?

together and how do I decide relative ways to weight it so John and Anna and I

So John, and Ana, and I could have made our job

could have made our torch or this on Troy wrong terms on our job this term

this term really easy if we had sat down

really easy if we had sat down at the

at the beginning of the term and said, you know,

beginning of the term and said you know

we've taught this course many times.

we've taught this course many times

We've got data from, I don't know,

we've got data from I don't know John

John, thousands of students, probably over this time.

thousands of students probably over this

Let's just build a little learning algorithm

time let's just build a little learning

that takes a set of data and predicts your final grade.

algorithm it takes a set of data and predicts your final grade you don't have

You don't have to come to class, don't

to come class don't have to go through

have to go through all the problems,

all the problems as we'll just predict

because we'll just predict your final grade.

your final grade wouldn't that be nice

Wouldn't that be nice?

being make our job a little easier and

Make our job a little easier, and you may or may not like that idea.

you may or may not like that idea but I

But I could think about predicting that grade?

could think about predicting that great

Now why am I telling this example.

now why am I telling this example I was

I was trying to see if I could get a few smiles.

trying to see if I could get a few

I saw a couple of them there.

smiles I saw a couple of them there but

But think about the features.

think about the features what would I

What I measure? Actually, I'll put this on John because it's his idea.

measure I shall put this on John because

What would he measure?

it's his idea what would he measure well

Well, GPA is probably not a bad predictor of performance.

GPA is probably not a bad predictor

You do well in other classes, you're

performance you do well in other classes

likely to do well in this class.

you're likely to do well in this class

I'm going to use this one very carefully.

I'm going to use this one very carefully

Prior programming experience is at least a predictor,

prior programming experience is at least

but it is not a perfect predictor.

a predictor but it is not a perfect

Those of you who haven't programmed before,

predictor those of you haven't

in this class, you can still do really well in this class.

programmed before in this class you can

But it's an indication that you've seen other programming

still do really well in this class but it's an indication that you've seen

other programming languages on the other

On the other hand, I don't believe in astrology.

hand I don't believe in astrology so I

So I don't think the month in which you're born,

don't think the month in which you're

the astrological sign under which you were born

born the austra astrological sign under

has probably anything to do with how well you'd program.

which you were born it has probably anything to do with how well you'd program I doubt that I color

I doubt that eye color has anything to do

has anything to do with how well you

with how well you'd program.

program you get the idea some features

You get the idea. Some features matter, others don't.

matter others don't

Now I could just throw all the features in and hope that

now I could just throw all the features

the machine learning algorithm sorts out those it wants

in and hope that the machine learning algorithm sorts out those it wants to

to keep from those it doesn't.

keep from those it doesn't but I remind

But I remind you of that idea of overfitting.

you that idea of overfitting if I do

If I do that, there is the danger

that there is the danger that it will

that it will find some correlation between birth

find some correlation between birth

month, eye color, and GPA.

month I color and GPA and that's going

And that's going to lead to a conclusion

to lead to a conclusion that we really

that we really don't like.

don't like by the way in case you're

By the way, in case you're worried,

worried I can assure you that stew

I can assure you that Stu Schmill

shmall in the Dean of Admissions

in the dean of admissions department

department does not use machine learning

does not use machine learning to pick you.

to pick you he actually looks at a whole

He actually looks at a whole bunch of things

bunch of things because it's not easy to

because it's not easy to replace him with a machine-- yet.

replace him with a machine yep all right

All right. So what this says is we need to think about

so what this says is we need to think

how do we pick the features.

about how do we pick the features and

And mostly, what we're trying to do

mostly what we're trying to do is to

is to maximize something called the signal to noise ratio.

maximize something called the

Maximize those features that carry the most information,

signal-to-noise ratio maximize those features that carry the most information

and remove the ones that don't.

and remove the ones that don't so I want

So I want to show you an example of how

to show you an example of how you might

you might think about this. I want to label reptiles.

think about this I want to label reptiles I want to come

I want to come up with a way of labeling animals as,

up with a way of labeling animals as are

are they a reptile or not.

they a reptile or not and I give you a

And I give you a single example.

single example with a single example you

With a single example, you can't really do much.

can't really do much but from this

But from this example, I know that a cobra, it lays eggs,

example I know that a cobra is it lays

it has scales, it's poisonous, it's cold blooded,

eggs it has scales it's poisonous its

it has no legs, and it's a reptile.

cold-blooded it has no legs and it's a

So I could say my model of a reptile is well,

reptile so I could say my model of a

I'm not certain.

reptile is well I'm not certain I don't

I don't have enough data yet.

have enough data yet but if I give you a

But if I give you a second example,

second example and it also happens to be

and it also happens to be egg-laying,

egg-laying have scales poisonous

have scales, poisonous, cold blooded, no legs.

cold-blooded node lakes

There is my model, right?

there's my model right perfectly

Perfectly reasonable model, whether I design it

reasonable model whether I design it our

or a machine learning algorithm would

machine learning algorithm would do it

do it says, if all of these are true, label it as a reptile. OK?

says if all of these are true label it as a reptile okay and now I give you a

And now I give you a boa constrictor. Ah. It's a reptile.

boa constrictor ah it's a reptile but it

But it doesn't fit the model.

doesn't fit the model and in particular

And in particular, it's not egg-laying,

it's not egg-laying and it's not

and it's not poisonous.

poisonous so I've got to refine the

So I've got to refine the model.

model or the algorithms got to refine

Or the algorithm has got to refine the model.

the model and this I want to remind you

And this, I want to remind you, is looking at the features.

is looking at the feature so I started

So I started out with five features.

out with five features this doesn't fit

This doesn't fit. So probably what I should do is reduce it.

so probably what I should do is reduce

I'm going to look at scales.

it I'm going to look at scales I'm going

I'm going to look at cold blooded.

to look at cold-blooded I'm going to

I'm going to look at legs.

look at legs that captures all three

That captures all three examples.

examples again if you think about this

Again, if you think about this in terms of clustering,

in terms of clustering all three of them

all three of them would fit with that.

would fit with that okay

OK. Now I give you another example--

now I'll give you another example

chicken. I don't think it's a reptile.

chicken I don't think it's a reptile in

In fact, I'm pretty sure it's not a reptile.

fact I'm pretty sure it's not a reptile

And it nicely still fits this model, right? Because, while it has scales, which you may or not realize,

and it nicely still fits this model right because while it has scales which

it's not cold blooded, and it has legs.

you may not realize it's not cold-blooded and it has legs so it is a

So it is a negative example that reinforces the model.

negative example that reinforces the

model sounds good and now I'll give you

And now I'll give you an alligator.

an alligator it's a reptile

It's a reptile. And oh fudge, right?

you know fudge right it doesn't satisfy

It doesn't satisfy the model. Because while it does have scales and it is cold blooded,

the model because well it does have scales and it is cold-blooded it has

legs I'm almost done with the example

I'm almost done with the example.

but you see the point again I've got to

But you see the point. Again, I've got to think about how do I refine this.

think about how do I refine this and I

And I could by saying, all right.

could by saying all right let's make it

Let's make it a little more complicated-- has scales,

a little more complicated have scales

cold blooded, 0 or four legs--

cold-blooded zero or four legs I'm going

I'm going to say it's a reptile.

to say it's a reptile I'll give you the

I'll give you the dart frog.

dart frog not a reptiles and amphibian

Not a reptile, it's an amphibian.

and that's nice because it still

And that's nice because it still satisfies this.

satisfies it so it's an example outside

So it's an example outside of the cluster that says no scales, not cold blooded,

of the cluster that says no scales not cold-blooded but happens have four legs

but happens to have four legs. It's not a reptile. That's good.

it's not a reptile that's good and then

And then I give you-- I have to give you a python, right?

I give you a half to Python right I mean

I mean, there has to be a python in here.

there has to be a Python in here oh come

Oh come on. At least grown at me when I say that.

on at least groan at me when I say that

There has to be a python here.

there has to be a Python here and I give

And I give you that and a salmon.

you that in a salmon and now I am in

And now I am in trouble. Because look at scales, look at cold blooded, look at legs.

trouble because look at scales look at cold-blooded look at legs

I can't separate them.

I can't separate them on those features

On those features, there's no way

there's no way to come up with a way

to come up with a way that will correctly

that will correctly say that the Python

say that the python is a reptile and the salmon is not.

is a reptile and the salmon is not and

And so there's no easy way to add in that rule.

so there's no easy to way to add in that

And probably my best thing is to simply go back

rule and probably my best thing is to

to just two features, scales and cold blooded.

simply go back to just two features scales and cold blooded and basically

And basically say, if something has

say if something has scales and it's

scales and it's cold blooded, I'm going to call it a reptile.

cold-blooded I'm going to call it a

If it doesn't have both of those,

reptile if it doesn't have both of those I'm

I'm going to say it's not a reptile.

going to say it's not a reptile it won't

It won't be perfect.

be perfect it's going to incorrectly

It's going to incorrectly label the salmon.

label the salmon but I've made a design

But I've made a design choice here that's important.

choice here that's important and the

And the design choice is that I will have no false negatives.

design choice is that I will have no false negatives what that means is

What that means is there's not going

there's not going to be any instance of

to be any instance of something that's not a reptile that I'm

something that's not a reptile that I'm

going to call a reptile.

going to call a reptile I may have some

I may have some false positives.

false positives so I did that the wrong

So I did that the wrong way.

way if false negative says everything

A false negative says, everything

that's not a reptile I'm going to say

that's not a reptile I'm going to categorize that direction.

I'm going to categorize that direction I

I may have some false positives, in that,

may have some false positives in that I

I may have a few things that I will incorrectly

may have a few things that I will

label as a reptile.

incorrectly labeled as a reptile and in

And in particular, salmon is going

particular whoops sorry salmon is going

to be an instance of that.

to be an instance of that this trade-off

This trade off of false positives and false negatives

of false positives and false negatives

is something that we worry about, as we think about it.

is something that we worry about as we

Because there's no perfect way, in many cases,

think about it because there's no perfect way in many cases separate of

to separate out the data.

the data and if you think back to my

And if you think back to my example of the New England

example of the New England Patriots that

Patriots, that running back and that wide receiver were

running back on that wide receiver were

so close together in height and weight,

so close together in height weight there

there was no way I'm going to be able to separate them apart.

is no way I'm going to be able to

And I just have to be willing to decide

separate them apart and I just have to

how many false positives or false negatives

be willing to decide how many false positives or false negatives do I want

do I want to tolerate.

to tolerate once I figured out what

Once I've figured out what features to use, which is good,

features to use which is good then I

then I have to decide about distance.

have to decide about distance how do I

How do I compare two feature vectors?

compare to feature vectors they're going

I'm going to say vector because there could

to say vector because there be multiple

be multiple dimensions to it.

dimensions to it how do I decide how to

How do I decide how to compare them?

compare them because I want to use the

Because I want to use the distances to figure out either

distances to figure out either how to

how to group things together or how to find a dividing line

group things together or how to find a

that separates things apart.

dividing line that separates things

So one of the things I have to decide is which features.

apart so one of the things I have to decide is which features I also have to

I also have to decide the distance.

decide the distance and finally I may

And finally, I may want to decide how to weigh relative importance of different dimensions

want to decide how to weigh relative importance of different dimensions in

in the feature vector.

the feature vector some may be more

Some may be more valuable than others in making that decision.

valuable than others in making that

And I want to show you an example of that.

decision and I want to show you an example of that so let's go back to my

So let's go back to my animals.

animals I've got I started off with a

I started off with a feature vector that actually

feature vector that actually had five

had five dimensions to it. It was egg-laying, cold blooded, has scales,

dimensions to it it was when egg-laying cold-blooded has scales I forget what

I forget what the other one was, and number of legs.

the other one was a number of legs so

So one of the ways I could think about this

one of the ways I could think about this

is saying I've got four binary features and one integer

is saying I've got four binary features and one integer feature associated with

feature associated with each animal.

each animal and one way to learn to

And one way to learn to separate out reptiles from non reptiles

separate out reptiles from non reptiles

is to measure the distance between pairs of examples

is to measure the distance between pairs of

and use that distance to decide what's near each other

examples and use that distance to decide

and what's not.

what's near each other and what's not

And as we've said before, it will either

and as you said before it lead to be

be used to cluster things or to find a classifier surface that

used to cluster things or to find a

separates them.

classifier surface that separates them

So here's a simple way to do it.

so here's a simple way to do it for each

For each of these examples, I'm going to just let true

of these examples I'm going to just let

be 1, false be 0.

true be one false be zero so the first

So the first four are either 0s or 1s.

four are either zeros or ones and the

And the last one is the number of legs.

last one is the number of legs and now I

And now I could say, all right.

could say all right how do I measure

How do I measure distances between animals

distances between animals or anything

or anything else, but these kinds of feature vectors?

else about these kinds of feature vectors here we're going to use

Here, we're going to use something

something called the Minkowski metric or

called the Minkowski Metric or the Minkowski difference.

the Minkowski difference given two

Given two vectors and a power, p,

vectors and a power P we basically take

we basically take the absolute value

the absolute value of the difference

of the difference between each of the components

between each of the components of the

of the vector, raise it to the p-th power, take the sum, and take the p-th route of that.

vector raise it to the peeth power take the sum and take the piece root of that

So let's do the two obvious examples.

so let's do the two obvious examples if

If p is equal to 1, I just measure the absolute distance

P is equal to one I just measure the

between each component, add them up, and that's my distance.

absolute distance between each component add them up and that's my distance it's

It's called the Manhattan metric.

called the Manhattan metric the one

The one you've seen more, the one we saw last time,

you've seen more than we saw last time

if p is equal to 2, this is Euclidean distance, right?

if P is equal to two this is this is

It's the sum of the squares of the differences

Euclidean distance right it's the sum of the squares of the differences of the

of the components. Take the square root.

components take the square root take the

Take the square root because it makes

square root because it makes it have

it have certain properties of a distance.

certain properties of a distance that's

That's the Euclidean distance.

the sorry that's the Euclidean distance

So now if I want to measure difference between these two, here's the question.

so now if I want to measure difference between these two here's the question is

Is this circle closer to the star or closer to the cross?

this circle closer to the star or closer to the cross unfortunately I put the

Unfortunately, I put the answer up here.

answer up here but it differs depending

But it differs, depending on the metric I use.

on the metric I use right Euclidean

Right? Euclidean distance, well, that's square root of 2 times 2,

distance well that's square root of 2

so it's about 2.8.

times 2 so it's about 2.8 and that's 3

And that's three. So in terms of just standard distance in the plane,

so in terms of just standard distance in

we would say that these two are closer than those two are.

the plane we would say that these two are closer than those to our Manhattan

Manhattan distance, why is it called that?

distance why is it called that because

Because you can only walk along the avenues and the streets.

you can only walk along the avenues and

Manhattan distance would basically

the streets Manhattan distance would

say this is one, two, three, four units away.

basically say this is 1 2 3 4 units away

This is one, two, three units away.

this is 1 2 3 units away and under

And under Manhattan distance, this is closer,

Manhattan distance this is closer this

this pairing is closer than that pairing is.

pairing is closer than that pairing is

Now you're used to thinking Euclidean.

now you're used to thinking Euclidean

We're going to use that.

we're going to use that but this is

But this is going to be important

going to be important when we think

when we think about how are we comparing distances

about how are we comparing distances

between these different pieces.

between these different pieces so

So typically, we'll use Euclidean.

typically will use the Euclidean we're

We're going to see Manhattan actually has some value.

going to see Manhattan actually has some

So if I go back to my three examples-- boy, that's

value so if I go back to my three

a gross slide, isn't it?

examples boy that's a gross slide isn't

But there we go--

it but there we go

rattlesnake, boa constrictor, and dart frog.

rattlesnake boa constrictor and dart

There is the representation.

frog there's a representation I can ask

I can ask, what's the distance between them?

what's the distance between them in the

In the handout for today, we've given you a little piece

handout for today we've given you a

of code that would do that.

little piece of code that would do that

And if I actually run through it, I get,

and if I actually run through it I get

actually, a nice little result. Here

actually a nice little result here are

are the distances between those vectors using Euclidean metric.

the distances between those vectors using Euclidean metric whoops sorry I'm

I'm going to come back to them.

going to come back then but you can see

But you can see the two snakes, nicely, are

the two snakes nicely are reasonably

reasonably close to each other.

close to each other whereas the dart

Whereas, the dart frog is a fair distance away from that.

frog is a fair distance away from that

nice right that's a nice separation it

That's a nice separation that says there's

says there's a difference between these

a difference between these two. OK. Now I throw in the alligator.

two okay now I throw in the alligator

Sounds like a Dungeons & Dragons game.

sounds like a Dungeons and Dragons game

I throw in the alligator, and I want to do the same comparison.

I throw in the alligator and I want to do the same comparison and I don't get

And I don't get nearly as nice a result. Because now it says,

nearly as nice a result because now it

as before, the two snakes are close to each other.

says as before the two snakes are close to each other but it says that the dart

But it says that the dart frog and the alligator

frog and the alligator are much closer

are much closer, under this measurement,

under this measurement then either them

than either of them is to the other.

is to the other and to remind you right

And to remind you, right, the alligator and the two

the alligator and the two snakes I would

snakes I would like to be close to one another and a distance

like to be close to one another and a

away from the frog.

distance away from the Frog because I'm

Because I'm trying to classify reptiles versus not.

trying to classify reptiles versus not

So what happened here?

so what happened here well this is a

Well, this is a place where the feature engineering

place where the feature engineering is

is going to be important.

going to be important because in fact

Because in fact, the alligator differs from the frog

the alligator differs from the Frog in

in three features.

three features but and only in two

And only in two features from, say, the boa constrictor.

features from say the boa constrictor

But one of those features is the number of legs.

but one of those features is the number

And there, while on the binary axes,

of legs and their while on the binary

the difference is between a 0 and 1,

axes the difference is between a 0 & 1

here it can be between 0 and 4.

here it can between can be between 0 & 4

So that is weighing the distance a lot more than we would like.

so that is weighing the distance a lot

The legs dimension is too large, if you like.

more than we would like the legs dimension is too large if you like how

How would I fix this?

would I fix this this is actually I

This is actually, I would argue, a natural place

would argue a natural place to you

to use Manhattan distance.

is Manhattan distance why should I think

Why should I think that the difference

that the difference in the number of

in the number of legs or the number of legs difference

legs or the number of legs difference is

is more important than whether it has scales or not?

more important than whether it has scales or not why should I think that

Why should I think that measuring that distance

measuring that distance Euclidean wise

Euclidean-wise makes sense?

make sense they are really completely

They are really completely different measurements.

different measurements and in fact I'm

And in fact, I'm not going to do it,

not going to do it but if I ran

but if I ran Manhattan metric on this,

Manhattan metric on this it would get

it would get the alligator much closer to the snakes,

the alligator much closer to the snakes

exactly because it differs only in two features, not three.

exactly because it differs only in two features not three the other way I could

The other way I could fix it would

fix it would be to say I'm letting too

be to say I'm letting too much weight be associated

much weight be associated with the

with the difference in the number of legs.

difference in the number of legs so

So let's just make it a binary feature.

let's just make it a binary feature

Either it doesn't have legs or it does have legs.

either it doesn't have legs or it does have legs run the same classification

Run the same classification. And now you see the snakes and the alligator

and now you see the snakes and the alligator are all close to each other

are all close to each other. Whereas the dart frog, not as far away as it was before,

whereas the dart frog not as far away as

but there's a pretty natural separation, especially

well as before but there's a pretty natural separation especially using that

using that number between them.

number between what's my point

What's my point? Choice of features matters.

choice of features matters throwing too

Throwing too many features in may, in fact,

many features in may in fact give us

give us some overfitting.

some overfitting and in particular

And in particular, deciding the weights

deciding the weights that I want on

that I want on those features has a real impact.

those features has a real impact and you

And you, as a designer or a programmer,

as a designer a programmer have a lot of

have a lot of influence in how you think about using those.

influence and how you think about using those so feature engineering really

So feature engineering really matters.

matters how you pick the features what

How you pick the features, what you use is going to be important.

you use is going to be important okay

OK. The last piece of this then is we're

the last piece of this then is we're

going to look at some examples where we give you data, got

going to look at some examples where we give you data got features associated

features associated with them.

with them we're going to in some cases

We're going to, in some cases have them labeled,

have them labeled in other cases not and

in other cases not. And we know how now to think about how do we

we know how now to think about how do we

measure distances between them. John. JOHN GUTTAG: You probably didn't intend to say weights of features.

measure distances between them John

You intended to say how they're scaled.

sorry the scales that not though thank

ERIC GRIMSON: Sorry. The scales and not the-- thank you, John.

you John now I did take that back I did

No, I did. I take that back. I did not mean to say weights of features.

not mean to say waste of features I

I meant to say the scale of the dimension

meant to say the scale of the dimension

is going to be important here.

is going to be important to thank you

Thank you, for the amplification and correction.

for the for the amplification and

You're absolutely right. JOHN GUTTAG: Weights, we use in a different way,

correction you're absolutely right and

as we'll see next time.

we're going to see next time we're going

ERIC GRIMSON: And we're going to see next time why we're going to use weights in different ways.

to use weights a different way so

So rephrase it.

rephrase it block that out of your mind

Block that out of your mind. We're going to talk about scales and the scale on the axes

we're going to talk about scales and the

as being important here.

scale on the axes is being important

And we already said we're going to look

here and we already said we're going to

at two different kinds of learning,

look at two different kinds of learning

labeled and unlabeled, clustering and classifying.

labeled and unlabeled clustering and

And I want to just finish up by showing you two examples of that.

classifying and I want to just finish up by showing you two examples of that how

How we would think about them algorithmically,

we would think about them

and we'll look at them in more detail next time.

algorithmically and we'll look at them in more detail next time as we look at

As we look at it, I want to remind

it I want to remind you the things that

you the things that are going to be important to you.

are going to be important to you how do

How do I measure distance between examples?

I measure distance between examples

What's the right way to design that?

what's the right way to design that what

What is the right set of features to use in that vector? And then, what constraints do I want to put on the model?

is the right set of features to use in that vector and then what constraints do I want to put on the model in the case

In the case of unlabelled data, how

of unlabeled data how do I decide how

do I decide how many clusters I want to have?

many clusters I want to have because I

Because I can give you a really easy way to do clustering.

can give you a really easy way to do

If I give you 100 examples, I say build 100 clusters.

clustering I would give you a hundred examples I say build 100 clusters every

Every example is its own cluster.

example is this own cluster distance is

Distance is really good.

really good it's really close to itself

It's really close to itself, but it does a lousy job

but it does allow the job of labeling

of labeling things on it.

things on it so I have to think about

So I have to think about, how do I decide how many clusters, what's the complexity

how do I decide how many clusters what's

of that separating service?

the complexity of that separating

How do I basically avoid the overfitting problem,

service how do I basically avoid the

which I don't want to have? So just to remind you, we've already

overfitting problem which I don't want to have so just remind you we've already

seen a little version of this, the clustering method. This is a standard way to do it, simply repeating what

seen a little version of this the clustering method this is a standard way

we had on an earlier slide.

to do it simply repeating what we had on

If I want to cluster it into groups,

an earlier slide I want to cluster it

I start by saying how many clusters am I looking for?

into groups I start by saying how many clusters am I looking for pick an

Pick an example I take as my early representation.

example as take as my early

For every other example in the training data,

representation for every other example

put it to the closest cluster.

in the training data put it to the closest cluster once I've got those find

Once I've got those, find the median, repeat the process.

the median repeat the process and that

And that led to that separation.

led to that separation now once I've got

Now once I've got it, I like to validate it.

it I like to validate it and in fact I

And in fact, I should have said this better.

should have said this better those two

Those two clusters came without looking at the two black dots.

clusters came without looking at the the two black dots once I put the black dots

Once I put the black dots in, I'd

in I'd like to validate how well does

like to validate, how well does this really work?

this really work in that

And that example there is really not very encouraging.

example there is really not very

It's too close.

encouraging it's too close so that's a

So that's a natural place to say, OK, what if I did this

natural place to say okay what if I did

with three clusters?

this with three clusters that's what I

That's what I get. I like the that. All right? That has a really nice cluster up here.

get I like that right that has a really nice cluster up here fact that the

The fact that the algorithm didn't know the labeling

algorithm didn't know the labeling is

irrelevant there's a nice grouping of

There's a nice grouping of five. There's a nice grouping of four.

five there's a nice grouping of four and

And there's a nice grouping of three in between.

there's a nice grouping of three in

And in fact, if I looked at the average distance

between and in fact if I looked at the distance the average distance between

between examples in each of these clusters,

examples in each of these clusters it is

it is much tighter than in that example.

much tighter than in that example and so

And so that leads to, then, the question of should I

that what sorry that leads to then the

look for four clusters?

question of should I look for four

Question, please. AUDIENCE: Is that overlap between the two clusters

clusters question please yeah the

not an issue? ERIC GRIMSON: Yes. The question is, is the overlap between the two clusters

question is is the overlap between the

two clusters a problem no I just drew it

No. I just drew it here so I could let

here so I could let you see where those

you see where those pieces are.

pieces are but in fact if you like the

But in fact, if you like, the center is there.

boundary is that the center is there

Those three points are all closer to that center

those three points are all closer to

than they are to that center.

that Center than they are to that Center

So the fact that they overlap is a good question.

so the fact that they overlap so good

It's just the way I happened to draw them.

question is just the way I happen to

I should really draw these, not as

draw them I should really draw these not

circles, but as some little bit more convoluted surface.

as circles but as some little bit more convoluted surface okay having done

OK? Having done three, I could say should I look for four?

three I could say should I look for four

Well, those points down there, as I've already said,

well those points down there as I've

are an example where it's going to be

already said are an example where it's

hard to separate them out.

going to be hard to separate them out

And I don't want to overfit.

and I don't want to over fit because the

Because the only way to separate those out

only way to separate those out is the

is going to be to come up with a really convoluted cluster,

kind of is going to be to come up with a

which I don't like.

really convoluted cluster which I don't

like right let me finish with showing

Let me finish with showing you one other example

you one other example from the other

from the other direction. Which is, suppose I give you labeled examples.

direction which is suppose I give you labeled examples so again the goal is

So again, the goal is I've got features

I've got features associated with each

associated with each example.

example they're going to have multiple

They're going to have multiple dimensions on it.

dimensions on it but I also know the

But I also know the label associated with them.

label associated with them and I want to

And I want to learn what is the best

learn what is the best way to come up

way to come up with a rule that will let me take new examples

with a rule that will let me take new

and assign them to the right group. A number of ways to do this.

examples and assign them to the right group number of ways to do this you can

You can simply say I'm looking for the simplest surface that

simply say I'm looking for the simplest

will separate those examples.

surface that will separate those

In my football case that were in the plane, what's

examples in my football case that we're

the best line that separates them,

in the plane what's the best line that

which turns out to be easy.

separates them which turns out to be

I might look for a more complicated surface.

easy I might look for a more complicated

And we're going to see an example in a second

surface and we're going to see an

where maybe it's a sequence of line segments

example in a second where maybe it's a sequence of line segments that separates

that separates them out.

them out because there's not just one

Because there's not just one line that does the separation.

line that does the separation as before I

As before, I want to be careful.

want to be careful if I make it too

If I make it too complicated, I may

complicated I may get a really good

get a really good separator, but I overfit to the data.

separator but I over fit to the data and

And you're going to see next time.

you're going to see next time I'm going

I'm going to just highlight it here.

to highlight it here there's a third way

There's a third way, which will lead

which will lead to them almost the same

to almost the same kind of result

kind of result called K nearest

called k nearest neighbors.

neighbors and the idea here is I've got

And the idea here is I've got a set of labeled data.

a set of labeled data and what I'm going

And what I'm going to do is, for every new example,

to do is for every new example say find

say find the k, say the five closest labeled examples. And take a vote.

the K say the five closest labeled examples and take a vote if three out of

If 3 out of 5 or 4 out of 5 or 5 out of 5 of those labels

five or four out of five of five out of

are the same, I'm going to say it's part of that group.

five of those labels are the same I'm going to say it's part of that group and

And if I have less than that, I'm

if I have less than that I'm going to

going to leave it as unclassified.

leave it as unclassified and that's a

And that's a nice way of actually thinking

nice way of actually thinking about how

about how to learn them.

to learn them and let me just finish by

And let me just finish by showing you an example.

showing you an example and I won't use

Now I won't use football players on this one.

football players on this one I'll use a

I'll use a different example.

different example I'm going to give you

I'm going to give you some voting data.

some voting data I think this is

I think this is actually simulated data.

actually simulated data but these are a

But these are a set of voters in the United States

set of voters in the United States with

with their preference.

their preference they tend to vote

They tend to vote Republican.

Republican they tend to vote Democrat

They tend to vote Democrat. And the two categories are their age and how far away

and the two categories are their age and

they live from Boston.

how far away they live from Boston

Whether those are relevant or not, I don't know,

whether those are relevant or not I

but they are just two things I'm going to use to classify them.

don't know but they're just two things are going to use to classify them and

And I'd like to say, how would I fit a curve

I'd like to say how would I fit a curve

to separate those two classes?

to separate those two classes I'm going

I'm going to keep half the data to test.

to keep half the data to test I'm going

I'm going to use half the data to train.

to use half the data to train so if this

So if this is my training data, I

is my training data I could say what's

can say what's the best line that separates these?

the best line that separates these I

I don't know about best, but here are two examples. This solid line has the property that all the Democrats

don't know about best but here are two examples this solid line has the property that all the Democrats are on

are on one side.

one side everything on the other side is

Everything on the other side is a Republican,

a Republican but there are some

but there are some Republicans on this side of the line.

Republicans on this side of the line I

I can't find a line that completely separates these,

can't find a line that completely

as I did with the football players.

separates these as I did with the

But there is a decent line to separate them.

football players but there's a decent line to separate them here's another

Here's another candidate. That dash line has the property that on the right side

candidate that dash line has the property that on the right side you've

you've got-- boy, I don't think this is deliberate,

got boy I don't think this is deliberate

John, right-- but on the right side,

John right but on the right side you've

you've got almost all Republicans.

got almost all Republicans seems like

It seems perfectly appropriate.

perfectly appropriate one Democrat but

One Democrat, but there's a pretty good separation there.

there's a pretty good separation there

And on the left side, you've got a mix of things.

and on the left side you've got a mix of

But most of the Democrats are on the left side of that line.

things but most of the Democrats are on the left side of that line all right

the fact that left and right correlates

The fact that left and right correlates

with distance from

with distance from Boston is completely irrelevant here.

Boston's completely irrelevant here but

But it has a nice punch to it. JOHN GUTTAG: Relevant, but not accidental.

it has a nice punch to it but not

ERIC GRIMSON: But not accidental. Thank you.

accidental thank you all right so now

All right. So now the question is, how would I evaluate these?

the question is how would I evaluate

How do I decide which one is better?

these how do I decide which one's better

And I'm simply going to show you,

and I'm simply going to show you very

very quickly, some examples.

quickly some examples first one is to

First one is to look at what's called the confusion matrix.

look at it's called the confusion matrix

What does that mean? It says for this, one of these classifiers for example,

what does that mean it says for this one

the solid line.

of these classifiers for example the

Here are the predictions, based on the solid line

solid line here are the predictions

of whether they would be more likely to be

based on the solid line of whether they

Democrat or Republican.

would be more likely to be Democrat or

And here is the actual label.

Republican and here's the actual label

Same thing for the dashed line.

same thing for the dashed line and that

And that diagonal is important because those are

diagonal is important because those are

the correctly labeled results.

the correctly labeled results write it

Right? It correctly, in the solid line case,

correctly in the dashed line case solid

gets all of the correct labelings of the Democrats.

line case gets all of the correct labelings of the Democrats he gets half

It gets half of the Republicans right.

of the Republicans right but it has some

But it has some where it's actually Republican,

where it's actually a Republican but it

but it labels it as a Democrat.

labels it as a Democrat that we'd like

That, we'd like to be really large.

to be really large and in fact it leads

And in fact, it leads to a natural measure

to a natural measure called the accuracy

called the accuracy. Which is, just to go back to that,

which is just to go back to that we say

we say that these are true positives.

that these are true positives meaning I

Meaning, I labeled it as being an instance, and it really is.

labeled it as being an instance and it

These are true negatives.

really is these are true negatives I

I label it as not being an instance, and it really isn't.

label it as not being an instance and it

And then these are the false positives.

really isn't and then these are the false positives I label that as being in

I labeled it as being an instance and it's not,

instances or not and these are the false

and these are the false negatives.

negatives I label it as not being an

I labeled it as not being an instance, and it is.

instance and it is and an easy way to

And an easy way to measure it is to look at the correct labels

measure it is to look at the correct

over all of the labels.

labels over all of the labels the true

The true positives and the true negatives,

positives and the true negatives the

the ones I got right.

ones I got right and in that case both

And in that case, both models come up with a value of 0.7.

models come up with a value of 0.7 so

So which one is better?

which one's better well I should

Well, I should validate that.

validate that and I want to do that in a

And I'm going to do that in a second by looking at other data.

second by looking at other data we could

We could also ask, could we find something

also ask could we find something with

with less training error?

less training error this is only getting

This is only getting 70% right. Not great.

70% right not great well here's a more

Well, here is a more complicated model.

complicated model and this is where you

And this is where you start getting

start getting worried about overfitting

worried about overfitting.

now what I've done is I've come up with

Now what I've done, is I've come up with a sequence of lines that separate them.

a sequence of lines that separate them

So everything above this line, I'm going to say

so everything above this line I'm going

is a Republican.

to say is a Republican everything below

Everything below this line, I'm going to say is a Democrat.

this line I'm going to say is a Democrat

So I'm avoiding that one.

so I'm avoiding that one I'm avoiding

I'm avoiding that one.

that one I'm still capturing many of the

I'm still capturing many of the same things. And in this case, I get 12 true positives, 13 true negatives,

same things and in this case twelve true positives thirteen true

and only 5 false positives.

negatives and only five false positives

And that's kind of nice.

and that's kind of nice you can see the

You can see the 5. It's those five red ones down there.

five it's those five red ones down there

It's accuracy is 0.833.

its accuracy is 0.833 and now if I apply

And now, if I apply that to the test data, I get an OK result.

that to the test data I get an okay

It has an accuracy of about 0.6.

result as an accuracy of about point six I could use this idea to try and

I could use this idea to try and generalize to say could I

generalize I say could I come up with a

come up with a better model.

better model and you're going to see

And you're going to see that next time.

that next time there could be other ways

There could be other ways in which I measure this.

in which I measure this and I want to

And I want to use this as the last example.

use this as the last example another

Another good measure we use is called PPV, Positive Predictive

good measure we use is called PPV predictive positive it's right positive

Value which is how many true positives do I come up with out

predictive value which is how many true positives do I come up with out of all

of all the things I labeled positively.

the things I labeled positively and in

And in this solid model, in the dashed line,

the solid model in the dashed line I get

I can get values about 0.57.

values about point five seven the

The complex model on the training data is better.

complex model on the training data is

And then the testing data is even stronger.

better in the in the testing data is even stronger and finally two other

And finally, two other examples are called

examples are called sensitivity and

sensitivity and specificity. Sensitivity basically tells you what percentage

specificity sensitivity basically tells

did I correctly find.

you what percentage did I correctly find

And specificity said what percentage

and specificity said what percentage did

did I correctly reject.

I correctly reject and I show you this

And I show you this because this is

because this is where the trade-off

where the trade-off comes in.

comes in I could make sense that

If sensitivity is how many did I correctly

sensitivity is how many did I correctly

label out of those that I both correctly

label out of those that I both correctly

labeled and incorrectly labeled as being negative,

labeled incorrectly labeled as being

how many them did I correctly label

negative how many of them did I

as being the kind that I want?

correctly labeled as being the kind that

I can make sensitivity 1.

I want I can make sensitivity one label

Label everything is the thing I'm looking for.

everything is the thing I'm looking for

Great. Everything is correct.

great everything's correct but the

But the specificity will be 0.

specificity will be zero because I'll

Because I'll have a bunch of things incorrectly labeled. I could make the specificity 1, reject everything.

have a bunch of things incorrectly labeled I could make the specificity one reject everything saying nothing is an

Say nothing as an instance. True negatives goes to 1, and I'm in a great place there,

instance true negatives goes to zero but the sensitivity sorry true negatives goes to one and I'm

but my sensitivity goes to 0.

in a great place there but my sensitivity goes to zero I've got a

I've got a trade-off.

trade-off as I think about the machine

As I think about the machine learning algorithm I'm using

learning algorithm I using a my choice

and my choice of that classifier,

of that classifier I'm going to see a

I'm going to see a trade off where I can increase specificity at the cost of sensitivity or vice

trade-off where I can increase specificity at the cost of sensitivity

or vice versa and you'll see a nice

And you'll see a nice technique called ROC or Receiver Operator

technique called auro

Curve that gives you a sense of how you want to deal with that.

see a receiver operator curb that gives you a sense of how you want to deal with

And with that, we'll see you next time.

that and with that we'll see you next

We'll take your question off line

time we'll take your question offline if

if you don't mind, because I've run over time.

you don't mind cuz I run over time but

But we'll see you next time where Professor Guttag

we'll see you next time where Professor

will show you examples of this.

Guttag will show you examples of this

Similar videos:

Machine Learning 2 - Introduction to ML
Roshan
7.5K views 47 likes 1 dislikes 12 May 2014100% match

01 Machine Learning With R Introduction
RR ITEC
119 views 0 likes 0 dislikes 19 Feb 2018100% match

Introduction to Machine Learning
Introduction to Data Analytics
16.8K views 27 likes 2 dislikes 21 Jul 2015100% match

Lec 27: Introduction to Machine Learning
NPTEL IIT Guwahati
1.4K views 8 likes 0 dislikes 25 Sep 201975% match

Machine Learning on FPGAs: Introduction
Marco Winzker
1.7K views 64 likes 0 dislikes 18 Sep 202075% match

What is Machine Learning (ML)? A Basic Introduction
Eye on Tech
654 views 4 likes 1 dislikes 14 Nov 201975% match

ML Lecture 0-1: Introduction of Machine Learning
Hung-yi Lee
552.9K views 2.1K likes 36 dislikes 29 Sep 201775% match

A Gentle Introduction to Machine Learning
StatQuest with Josh Starmer
120.9K views 2K likes 17 dislikes 26 Nov 201875% match

Introduction to Ethics in Machine Learning
MIT OpenCourseWare
31.1K views 474 likes 32 dislikes 01 Oct 202075% match

Introduction to AI and Machine Learning - Overview
CloudxLab Official
868 views 7 likes 1 dislikes 25 Sep 201875% match

3.1: Introduction to Session 3 - What is Machine Learning?
The Coding Train
52.4K views 858 likes 5 dislikes 08 May 201775% match

What's Machine Learning?
LiveIntent
88 views 1 likes 0 dislikes 23 Mar 201667% match

What is Machine Learning?
Lawrence Livermore National Laboratory
4.7K views 86 likes 4 dislikes 05 Jul 201867% match

What Is Machine Learning?
SAS Software
4.5K views 25 likes 0 dislikes 07 Jul 201567% match

What is Machine Learning?
OxfordSparks
345.3K views 2.2K likes 55 dislikes 11 Jan 201767% match

What is Machine Learning?
Google Cloud Tech
787.2K views 12.8K likes 177 dislikes 24 Aug 201767% match

C ++ learning - #1 introduction
Nikola Djordjevic
9K views 143 likes 4 dislikes 04 Jul 201667% match

What is Machine Learning
Udacity
1K views 4 likes 0 dislikes 06 Jun 201667% match

What is Machine Learning?
Code.org
17.7K views 250 likes 11 dislikes 01 Dec 202067% match

What is machine learning?
National Science Foundation News
670 views 20 likes 2 dislikes 29 Mar 201867% match

What is machine learning?
National Science Foundation News
126 views 7 likes 0 dislikes 12 Apr 201867% match

What is Machine Learning?
Matthew Voke
57 views 6 likes 0 dislikes 01 Apr 202067% match

What is machine learning?
In 3 Minutes
2.9K views 12 likes 2 dislikes 18 Apr 201767% match

What is Machine Learning ?
Cubeware GmbH
6K views 4 likes 0 dislikes 25 Jan 202167% match

Introduction to big data and machine learning
Stéphane Déprès
2.8K views 15 likes 2 dislikes 21 Nov 201660% match

Introduction to Azure Machine Learning | AI7
Microsoft Visual Studio
2K views 9 likes 1 dislikes 17 Nov 201760% match

Week 1 - Lecture 1 - Introduction to Machine Learning
Machine Learning- Balaraman Ravindran
57.7K views 150 likes 9 dislikes 17 Jan 201660% match

Introduction to Spell (for Machine Learning in the Cloud)
The Coding Train
49.3K views 776 likes 12 dislikes 17 Sep 201860% match

Introduction - Georgia Tech - Machine Learning
Udacity
985 views 1 likes 1 dislikes 23 Feb 201560% match

Introduction - Georgia Tech - Machine Learning
Udacity
821 views 3 likes 1 dislikes 23 Feb 201560% match

Settings Privacy About Contact Donate