Spam Prevention - Web Development Udacity English English (auto-generated)

Spam Prevention - Web DevelopmentYouTube↗ IA↗(long shot)
Education 4.2K views 26 likes 2 dislikes 27 May 2012

Udacity 651K subscribers United States YouTube↗

Click to Play Video

So one other thing we spent almost no time on in the beginning

so one of the thing we spent almost no

and a lot of time on now is spam prevention.

time on the beginning and a lot of time

So you think on a site like reddit, where it's like the Wild West, there's no rules,

on now spam prevention do you think on a site like Reddit where it's like the

like anybody can submit anything, that it would just be quickly overrun with spam.

Wild West there's no rules like anybody can submit anything that'll just quickly

You'd be surprised. Now that reddit is big, yes. Reddit receives lots of spam.

be over on us you'd be surprised now that red it's big yes the reddit

But at the time you kind of--If you understand spammers' motivations, it's really easy to prevent.

receives lots of spam but at the time you kind of if you understand spammers motivations it's really easy to prevent

A spammer's motivation is basically they want links,

a spammers motivation is basically if

but an easy way to prevent their links from providing any value

they want links but an easy way to

is to add an attribute to your anchor tag that looks like this.

prevent their links from providing any value is to add an attribute to your anchor tag that looks like this so if

So if you have a link with this extra attribute, rel=nofollow, it basically tells Google

you have a link with this extra attribute rel equals nofollow basically

this link shouldn't be followed for search quality purposes.

tells google this link shouldn't be followed for search quality purposes

Because the way Google works, right, is it looks at a given page

because the way Google works right is it

and however many pages elsewhere on the Internet link to that page gives that page some authority.

looks at a given page and however many pages else run Internet link to that

And you can basically say I don't want to give this link any authority.

page gives that page some authority and you can basically say I don't wanna give

So every link on reddit has rel=nofollow until it has a certain number of points,

this link any authority and so every link on reddit has rel equals nofollow until it has a certain number of points

because we didn't want to put rel=nofollow on all links because we like the way the Internet works.

because we didn't want to put rel nofollow on all links because we like

And we figured if a link was good on reddit, that link should have some authority.

the way the internet works and we figured if a link was good on reddit that link should have some Authority

Understanding that spammers all they really want are links

understanding that spammers are all they

to increase the authority of their page to get more traffic there really helps.

really want our links that to increase the authority of their page to get more

So this is one thing we did. Another thing we did was...Spam is rarely cemented by an individual by hand.

traffic there really helps so this is one thing we did another thing we did was spam is really submitted by an

There's almost always automated tools.

individual by hand those almost almost always automated

Looking at the behaviors of these bots that were submitting contents to reddit.

tools looking at the behaviors of these bots that were submitting contents to

They always had these kind of gaps, behavior gaps, that humans would never do.

Reddit they always had these kind of gaps behavior gaps that humans would

For example, a bunch of users will all use the same password.

never do for example a bunch of users

Before we were hashing passwords, it was really easy for us to tell. They were all using the same passwords.

who all used the same password before we were hashing passwords it was really easy for us to tell they're all using

If there were 100 users all submitting crap but they had the same password, it was all spam,

the same passwords if there was a hundred users all submitting crap but they had the same password it was all

and we could just remove it.

spam and we could just remove it and

And that worked out really nice. Eventually we started, you know, hashing passwords, and we lost that ability.

that worked out really nice eventually we started you know hashing passwords

But we had learned some other tricks along the way.

and we lost that ability but we had

One is that the users didn't understand the site.

learned some other tricks along the way one is that the users didn't understand

They would always submit a link and then comment on their own link.

the site they would always submit a link

And users, legitimate users, rarely commented on their own link as the first comment.

and then comment on their own link and users legitimate users rarely kampf it on their own link as the first comment

And so we would just look for links that had 1 comment by the same author and mark it as spammers.

so we would just look for links that had one comment by the same author and

Spammers on a community site that behaves differently from other sites online,

market as spam BAM errs on a community set that behaves differently from other

noticing that these peopel use the site in weird ways was enough to catch most spam.

sites online noticing that these people use the site in weird ways was enough to

We didn't have to do any fancy, like, look for Viagra or look for suspicious-looking links.

catch the most spam we didn't have to do any fancy like look for Viagra or look

A lot of these behavior heuristics worked really, really well.

for suspicious-looking links a lot of these behavior sticks worked really

Submissionary. This is another heuristic. Legitimate users didn't submit that often.

really well submission rate this is another euro stick legitimate users didn't submit

Or if they did, the time between submissions was stagnated,

that often or if they did that the time between submissions was stagnated and

and spammers would often just submit a bunch of links at the same time.

spammers would often just submit a bunch

We look for those signs and ban the links.

of links at the same time we look for

The most important thing that reddit did, and still does, is don't let them know when they're caught.

those signs and ban the links most important thing that read it did and still does don't let them know when

When we'd ban a submission on reddit, that user would still see their submission on our hot page or on their user page.

they're caught when we banish mission on reddit that user would still see their submission on our hot page or on their

If their vote doesn't count, it would still look like it would count,

use Paige if their vote doesn't count it

and if you try to figure out on reddit if you vote counts or not,

would still look like a count and if you

it's really difficult to tell because on reddit there's a delay between when you click that button and when we update the score.

try to figure out on Reddit whether your vote counts or not it's really difficult to tell because unread if there's a delay between when you click that button

and when we update the score I'm not

I'm not going to get into the details, but suffice it to say, it's really, really hard.

going to get into the details but suffice to say it's really really hard

And we've done this, we've spent a lot of time, making it so that users don't know when they're caught.

and we've done this we spend a lot of time making it so that users don't know

And I've had to do this on ASCII Chan and on my blog as well

in the car and I've had to do this on ASCII Chan and on my blog as well

because a lot of you guys have submitted a bunch of crap to ASCII Chan and my blog that I didn't want to appear.

because a lot of you guys have submitted a bunch of crap to ASCII Chan and my

A lot of that stuff is just sitting in memcache and only you see it,

blog that I didn't want to appear a lot of that stuff is just sitting in

but if you were to change computers, you would find that that submission disappears.

memcache and only you see it but if you were to change computers you would find

I think being subtle about when you've detected that somebody is spamming is really, really important.

that that submission disappears I think being subtle about when you've detected that somebody's spamming is really

It's not the only solution, but security through obscurity works really, really well if you keep things obscure enough.

really important it's not the only solution but security through obscurity works really really well if you keep things obscure enough

Similar videos:

Web 简介
 Udacity
25.7K views 60 likes 0 dislikes 23 Feb 2015100% match

The Method Attribute - Web Development
Udacity
9.6K views 14 likes 8 dislikes 22 Apr 2012100% match

Scaling Databases - Web Development
Udacity
64.3K views 795 likes 7 dislikes 25 Apr 2012100% match

Most Common Method - Web Development
Udacity
9.7K views 5 likes 0 dislikes 23 Feb 2015100% match

Paragraph Tag - Web Development
Udacity
8.6K views 4 likes 0 dislikes 23 Feb 2015100% match

Shopping List Take 1 - Web Development
Udacity
841 views 1 likes 0 dislikes 23 Feb 2015100% match

Bold Tag - Web Development
Udacity
53K views 29 likes 0 dislikes 23 Feb 2015100% match

Correct URL - Web Development
Udacity
10.9K views 6 likes 1 dislikes 23 Feb 2015100% match

Types of Databases - Web Development
Udacity
8.5K views 43 likes 4 dislikes 25 Apr 2012100% match

World Wide Web - Web Development
Udacity
37.9K views 40 likes 1 dislikes 23 Feb 2015100% match

File Types
Udacity
34.4K views 11 likes 3 dislikes 23 Feb 2015100% match

User Agent Header - Web Development
Udacity
11.6K views 24 likes 7 dislikes 23 Feb 2015100% match

App Server Architecture - Web Development
Udacity
14K views 52 likes 5 dislikes 27 May 2012100% match

Missing End Tag
Udacity
43K views 15 likes 1 dislikes 23 Feb 2015100% match

Part 2 - Web Development
Udacity
643 views 2 likes 0 dislikes 07 May 2012100% match

Span and Div
Udacity
50.3K views 27 likes 1 dislikes 23 Feb 2015100% match

Rot13 - Web Development
Udacity
9.3K views 46 likes 8 dislikes 23 Apr 2012100% match

Differences Between Get and Post - Web Development
Udacity
78.3K views 516 likes 18 dislikes 22 Apr 2012100% match

File Types Solution - Web Development
Udacity
9.9K views 9 likes 0 dislikes 23 Feb 2015100% match

Cookie Headers - Web Development
Udacity
4.2K views 23 likes 1 dislikes 23 Feb 2015100% match

Datastore Types Solution - Web Development
Udacity
656 views 1 likes 0 dislikes 25 Apr 2012100% match

URL Encoding - Web Development
Udacity
13.7K views 17 likes 13 dislikes 22 Apr 2012100% match

Dynamic Content - Web Development
Udacity
6.1K views 2 likes 0 dislikes 23 Feb 201567% match

Best Browser - Web Development
Udacity
14.9K views 7 likes 0 dislikes 23 Feb 201567% match

Code Organization - Web Development
Udacity
7.2K views 25 likes 1 dislikes 27 May 201267% match

Whitespace Solution - Web Development
Udacity
15.7K views 10 likes 1 dislikes 23 Feb 201567% match

Course Outline - Web Development
Udacity
46.2K views 169 likes 0 dislikes 23 Feb 201567% match

Precomputed Caching - Web Development
Udacity
12.2K views 80 likes 0 dislikes 27 May 201267% match

Reddit Architecture - Web Development
Udacity
27.7K views 397 likes 3 dislikes 27 May 201267% match

Query Parameters - Web Development
Udacity
22.6K views 78 likes 3 dislikes 23 Feb 201567% match

Settings Privacy About Contact Donate