Problems with Memcachedb - Web Development Udacity English (auto-generated) English

Problems with Memcachedb - Web DevelopmentYouTube↗ IA↗(long shot)
Education 6K views 13 likes 0 dislikes 27 May 2012

Udacity 651K subscribers United States YouTube↗

Click to Play Video

so let's take it from from which of these pieces don't exist anymore all

Let's take it from which of these pieces don't exist anymore.

right so after you left but before I started memcachedb hit a scaling wall

All right. After you left but before I started, memcachedb hit a scaling wall,

and it just would not go any further the rights just were too fast for it

and it just would not go any further, the writes just were too fast for it.

was it was did you guys try adding more memcache DVD boxes I don't know you

Did you guys try adding more memcachedb boxes?

weren't there for that yeah that caste

I don't know actually. >>You went there for that. >>Yeah.

was not designed for the heavy load we

Memcachedb is not designed for the heavy load we were throwing at it.

were throwing out it's basically this is a memcache code bolted on top of Berkeley DB but just kind of neat but it

It's basically--it was the memcache code bolted on top of Berkeley db, which is kind of neat,

wasn't really kind of designed to work at that kind of our scale or our use

but it wasn't really kind of design to work at kind of our scale or use case,

case which was just basically sending as

which is just basically sending as many queries at it as you could.

many queries that isn't good so you guys got rid of this right and replace it

So you guys got rid of this? >>Right.

with Cassandra so it's a distributed no sequel database the way it works is you have rows and a row is charted by its key to somewhere in the ring of servers

And replace it with Cassandra. It's a distributed, NoSQL database. The way it works is you have rows, and a row is sharded by it's key to somewhere in the ring of servers

so you get automatic sharding across

so you get automatic sharding across this entire ring

this entire ring and when a row has columns inside of it and that's where the actual data is stored so this moves

when a row has columns inside of it and that's where the actual data is stored.

to Cassandra and was pretty similar

This moves to Cassandra and was pretty similar.

yeah so so I'll let you draw Cassandra in there and I'll explain a little bit

I'll let you draw Cassandra there and I'll explain a little bit more about--

more about so man we talked about with databases you can you can replicate them

Remember we talked about with database as you can replicate them where you send

where you send the same data to multiple machines which helps for load and durability this conflicts are here or

the same data to multiple machines, which helps for load and durability--this kind of picture here

you can shard them which is you send

or you can shard them which is you send some chunk of data to one machine

some chunk it out at the one machine and

and another chunk of data to another.

another chunk of data to another so let's say we're talking about in this case this is we started using this for the pre compute stuff first yeah so

Let's say we are talking about--in this case, we started using this for the precompute stuff first? >>Yeah.

you've got these listings every subreddit has like a hot page and that

You've got these listings--every subreddit has like a hot page and that will be precomputed

would be pre computed so you can store you know hot pages for some reddit's on

so you can store hot pages for some Reddits on this node

this node somehow pitches for edits on this node and all around so each node is not a an exact copy of every other node

and some hot pages for Reddits on this node and all around. So, each node is not an exact copy of every other node. >>Right. >>Now, is there some overlap?

right now there is some overlap right so it's configurable but in our case we're using replication factor of 3 which

Right. It's configurable, but in our case, we're using a replication factor of 3,

means that if a piece of data lives on this node it is also on this one and this one and

which means that if a piece of data lives on this node, it is also on this one and this one

that's that happens all around the ring

and that happens all around the ring.

and why do you do that

And why do you do that?

that's simultaneously a read can be serviced from any one of the three if if

Simultaneously a read can be serviced from any one of the 3 if we allow it to be,

we allow it to be which means that if one nodes going slow we don't go really

which means that if one node is going slow, we don't go really slow,

slow and also it means that if any one node just ceased to exist we wouldn't

and also it means that if anyone one node just cease to exist,

lose all our data for that segment of

we wouldn't lose all data for that segment of the data.

the data let's say you have content this guy this guy and this guy and we lose

Let's say you have content on this guy, this guy, and this guy, and you lose this node,

this note that's the content get redistributed no no no there they're

does the content get redistributed? >>No.

assigned a key space and you have to move tokens if you want to rebalance the

They're assigned a key space and you have to move tokens if you want to rebalance the ring.

ring okay and that's something that you as a developer have to do yeah that's it

Okay. That's something that you as a developer have to do?

that's one of the things as the team grows too is people's roles change you

Yeah, that's an operational thing >> Or systems? >>Yeah. One of the things that happens as the team grows too, as people's role has changed.

know when you're a small website you are the designer developer and sysadmin

When you're a small website, you are the designer, developer, sysadmin, all in one,

all-in-one and you guys have a whole team of operations guys yeah they're very good from well they're

and you guys have a whole team of operations guys. >>Two. Yeah. >>Okay. You have a team of two--they're very good, from what I've heard. They're extraordinarily good.

extremely good when I left it was me and

When I left, it was me and David who now works with me at Hipmunk

David who now works with me at Hipmunk basically developers and Ops guys and we

basically developers and OPs guys, and we're better developers

were better developers and we were ops guys so that explains some of us

than we were OPs guys, so that explains some of this.

Similar videos:

Memcached - Web Development
Udacity
13.5K views 115 likes 3 dislikes 23 Feb 2015100% match

Dynamic Content - Web Development
Udacity
6.1K views 2 likes 0 dislikes 23 Feb 2015100% match

Code Organization - Web Development
Udacity
7.2K views 25 likes 1 dislikes 27 May 2012100% match

More Handlers - Web Development
Udacity
9.1K views 9 likes 2 dislikes 22 Apr 2012100% match

Components of the Web Solution - Web Development
Udacity
9.6K views 19 likes 0 dislikes 23 Feb 2015100% match

Web Applications - Web Development
Udacity
19.5K views 59 likes 4 dislikes 02 Jun 2014100% match

Install Google App Engine
Udacity
7.1K views 40 likes 48 dislikes 17 Apr 2012100% match

Most Common Method - Web Development
Udacity
9.7K views 5 likes 0 dislikes 23 Feb 2015100% match

Paragraph Tag - Web Development
Udacity
8.6K views 4 likes 0 dislikes 23 Feb 2015100% match

Shopping List Take 1 - Web Development
Udacity
841 views 1 likes 0 dislikes 23 Feb 2015100% match

Span and Div Solution - Web Development
Udacity
17.4K views 7 likes 0 dislikes 23 Feb 2015100% match

Components of the Web
Udacity
28K views 54 likes 0 dislikes 23 Feb 2015100% match

World Wide Web - Web Development
Udacity
37.9K views 40 likes 1 dislikes 23 Feb 2015100% match

App Server Architecture - Web Development
Udacity
14K views 52 likes 5 dislikes 27 May 2012100% match

User Input - Web Development
Udacity
7.7K views 3 likes 0 dislikes 22 Apr 2012100% match

Part 2 - Web Development
Udacity
643 views 2 likes 0 dislikes 07 May 2012100% match

Valid Headers - Web Development
Udacity
7.9K views 1 likes 0 dislikes 23 Feb 2015100% match

Caching Techniques - Web Development
Udacity
34.4K views 216 likes 7 dislikes 23 Feb 2015100% match

Differences Between Get and Post - Web Development
Udacity
78.3K views 516 likes 18 dislikes 22 Apr 2012100% match

Frameworks - Web Development
Udacity
28.7K views 154 likes 25 dislikes 23 Feb 2015100% match

Security - Web Development
Udacity
5.7K views 64 likes 0 dislikes 23 May 2012100% match

Cookies - Web Development
Udacity
38.4K views 315 likes 11 dislikes 23 Feb 2015100% match

Datastore Types Solution - Web Development
Udacity
656 views 1 likes 0 dislikes 25 Apr 2012100% match

Best Browser - Web Development
Udacity
14.9K views 7 likes 0 dislikes 23 Feb 201567% match

Whitespace Solution - Web Development
Udacity
15.7K views 10 likes 1 dislikes 23 Feb 201567% match

Course Outline - Web Development
Udacity
46.2K views 169 likes 0 dislikes 23 Feb 201567% match

Precomputed Caching - Web Development
Udacity
12.2K views 80 likes 0 dislikes 27 May 201267% match

Reddit Architecture - Web Development
Udacity
27.7K views 397 likes 3 dislikes 27 May 201267% match

Bold Tag - Web Development
Udacity
53K views 29 likes 0 dislikes 23 Feb 201567% match

Query Parameters - Web Development
Udacity
22.6K views 78 likes 3 dislikes 23 Feb 201567% match

Settings Privacy About Contact Donate