Scaling - Web Development Udacity Japanese English

Scaling - Web DevelopmentYouTube↗ IA↗(long shot)
Education 12.5K views 81 likes 1 dislikes 27 May 2012

Udacity 651K subscribers United States YouTube↗

Click to Play Video

さて各キャッシュを動かすAPサーバと

So we had these app servers that were all running their own caches,

互いのレプリカである2つのデータベースを[br]手に入れました

and then we had a couple databases that were all replicas of each other.

この時片方のAPサーバのプログラムを実行する

At this point we added a load balancer, and this load balancer probably actually ran.

ロードバランサを追加しました

It was probably a program running on one of these app machines,

2台のAPサーバはキャッシュを同期し続けています

and these guys were still keeping their caches in sync using interacting with the databases directly.

Spreadによる同期は複雑で[br]APサーバの台数には限界があったので

And we had a limit ot how many app servers we can have because we had this complicated caching spread thing.

次はmemcachedレイヤを足しました

The next thing we added was the memcache layer.

APサーバにインメモリキャッシュを持たせる代わりに

So instead of these app services containing their end memory cache,

memcached経由で通信します

they would communicate via memcache.

キャッシュを同期する代わりに[br]APサーバ同士で共有したのです

So instead of having to keep their cache in sync, we just had 1 cache that was just shared among all of our app servers.

悲しいことに[br]ここに到達するまでに長くかかりすぎました

I'm sad it took us so long to figure this out because memcache existed when we started reddit,

memcachedを早く使えばよかったです

and we should have been using it from the beginning.

これですべてのキャッシュをmemcachedに移行し

This is what allowed us to get all of that state, all of that cache, out of the apps and into memcached

任意のAPサーバも増やせました

and allowed us to add apps arbitrarily.

APサーバは大きくなりキャッシュも同期し

Once we had that going, that allowed us to scale our apps

何も心配する必要がなくなりました

and they stayed in sync, and so we can add an app, lose an app. We didn't have to worry about it.

次はデータベースの負荷の問題です

The next thing we had to start dealing with was the database load.

耐久性とパフォーマンス向上のため[br]すでにレプリケートしていましたが

So we're already replicating for kind of durability and for performance reasons,

読み込んだデータを複数のマシンに[br]送れるようになりました

so we can spread our reads across multiple machines as we started segmenting on type.

またリンクとコメントを[br]別々に格納できるようになりました

So we'd have a database for just links; then we could separate comments out into its own database.

コメント用のデータベースもレプリケートします

And so these would still replicate to each other, but if you're only submitting a link,

リンクを投稿したらリンク専用のデータベースに

you only have to touch this database, and if you're only, like, reading a comment,

コメントはコメント専用に格納されるのです

for example, you only have to touch this database.

今でも基本的にこの方法で[br]データベースを拡張しています

And this is actually still basically the general setup reddit has today in terms of how the database is scaled.

最初にシャーディングを作ればよかったと思います

And we never wrote sharding in the beginning, and I really regret that.

Thing DBを書き直した時いつか必要になるから

When I rewrote the ThingDB, the second version of it, I had in the back of my head,

シャーディングを追加したほうがいいと思ったのですが

you know, I should add sharding, because we're going to need that someday.

thingの稼働に必死だったのでやめたのです

And then I just wanted to get the damn thing into production so I stopped.

巨大なシステムを作る時は困難なことを先にしないと

The big lession I've learned is when you're writing a big system like that,

2度とそのチャンスはやってこないことを学びました

if you don't do the hard parts up front, you may never get another opportunity to them,

データベースが大きすぎて[br]シャーディングの追加は大変です

because now the database is so big, that if we wanted to bolt on sharding, that's a huge project.

今やマシンやキャッシングを増やす方が簡単です

It's easier right now to just add bigger machines and more caching.

いつか誰かがやらねばなりません

It's not going to work forever, and somebody's going to have to bite the bullet and do that.

あの時やればもっと簡単でした

And it would have been a lot easier to do it at the time.

Thing DBに変えた時結合を使うのをやめたので

Since all of our queries, we stopped using joins when we switched to ThingDB,

結果的にシャーディングを作ってもよかったのです

sharding's actually fairly straightforward if you kind of do it right from the beginning.

APサーバのソフトは徐々に変えましたが[br]常にPythonを使っていました

Over time some of the software on these app servers changed so we've always been using Python.

最初のサーバは覚えていませんが

I don't remember what app server we used originally.

Redditでアーロンが作成したフレームワークの[br]web.pyに変えたのです

We switched from whatever we used initially to web.py, which is a framework that we wrote at reddit.

web.pyは今でもネットでシェアしています

Aaron was basically the main author of that, and it's still out there on the Internet somewhere.

ハンドラクラスとかGETやPOST関数の概念をもつ

And this is where the first time I recall seeing a framework that had kind of the notion

フレームワークを見たのは初めてでした

of a handler class and then functions for get and post,

以来その考え方に夢中になったのです

and I've become kind of addicted to thinking of web applications that way.

実はGAEやWebapp2もweb.pyの

Actually the Google app engine, the webapp2 framework,

デザインを継承しています

inherited a lot of that design from web.py, which is nice.

意志決定を模索していた時期だったので

Nice for me, at least, is that design decision has kind of stuck around for a little while

web.pyの導入は嬉しかったです

so I think that means it was a good one.

現在PythonはPylonsという[br]フレームワークを使っています

Now Python uses a web framer called Pylons, but it uses a very old version of Pylons.

Pylonsに変えた時web.pyの維持はやめました

And basically when we switched to Pylons, Aaron had stopped maintaining web.py.

第三者が作ったデータベースを使って

I didn't want to maintain it. We switched to something else maintained by somebody else.

それをバラバラにしてweb.pyのように機能させました

And then we basically shredded most of it and made it function just like web.py.

あとから考えれば自前のものにするべきでしたが

In hindsight we probably should have just written our own because that's effectively what we did,

結局Pylonsを使ったのです

but we did it on top of Pylons.

PylonsのReddit版はオープンソースになっています

So if you want to use the reddit version of Pylons, it's open source. It's online.

それはPylonsとは[br]まったく別物になっているのですが

But it doesn't resemble anything like the actual Pylons web framework at this point.

どうやらRedditではまだそれを使っているようです

And to my knowledge, that's still what they use today, this hacked up version of Pylons.

Similar videos:

Italics
Udacity
49.1K views 16 likes 2 dislikes 23 Feb 2015100% match

Best Browser - Web Development
Udacity
14.9K views 7 likes 0 dislikes 23 Feb 2015100% match

Code Organization - Web Development
Udacity
7.2K views 25 likes 1 dislikes 27 May 2012100% match

Content Type - Web Development
Udacity
7.5K views 5 likes 1 dislikes 22 Apr 2012100% match

Cache Architecture - Web Development
Udacity
7.2K views 16 likes 1 dislikes 27 May 2012100% match

Telnet Request - Web Development
Udacity
8.2K views 2 likes 0 dislikes 23 Feb 2015100% match

Whitespace
Udacity
47.8K views 26 likes 4 dislikes 23 Feb 2015100% match

Course Outline - Web Development
Udacity
46.2K views 169 likes 0 dislikes 23 Feb 2015100% match

Most Common Method - Web Development
Udacity
9.7K views 5 likes 0 dislikes 23 Feb 2015100% match

Paragraph Tag - Web Development
Udacity
8.6K views 4 likes 0 dislikes 23 Feb 2015100% match

Shopping List Take 1 - Web Development
Udacity
841 views 1 likes 0 dislikes 23 Feb 2015100% match

Bold Tag - Web Development
Udacity
53K views 29 likes 0 dislikes 23 Feb 2015100% match

Correct URL - Web Development
Udacity
10.9K views 6 likes 1 dislikes 23 Feb 2015100% match

Components of the Web
Udacity
28K views 54 likes 0 dislikes 23 Feb 2015100% match

Types of Databases - Web Development
Udacity
8.5K views 43 likes 4 dislikes 25 Apr 2012100% match

World Wide Web - Web Development
Udacity
37.9K views 40 likes 1 dislikes 23 Feb 2015100% match

App Server Architecture - Web Development
Udacity
14K views 52 likes 5 dislikes 27 May 2012100% match

Query Parameters - Web Development
Udacity
22.6K views 78 likes 3 dislikes 23 Feb 2015100% match

Part 2 - Web Development
Udacity
643 views 2 likes 0 dislikes 07 May 2012100% match

Valid Headers - Web Development
Udacity
7.9K views 1 likes 0 dislikes 23 Feb 2015100% match

Span and Div
Udacity
50.3K views 27 likes 1 dislikes 23 Feb 2015100% match

Google App Engine - Web Development
Udacity
10.1K views 13 likes 3 dislikes 17 Apr 2012100% match

Reddit Architecture 2 - Web Development
Udacity
11.1K views 88 likes 2 dislikes 27 May 2012100% match

World Wide Web Solution - Web Development
Udacity
12.6K views 14 likes 0 dislikes 23 Feb 2015100% match

The Action Attribute - Web Development
Udacity
7.5K views 7 likes 0 dislikes 22 Apr 2012100% match

Cookies - Web Development
Udacity
38.4K views 315 likes 11 dislikes 23 Feb 2015100% match

Datastore Types Solution - Web Development
Udacity
656 views 1 likes 0 dislikes 25 Apr 2012100% match

URL Encoding - Web Development
Udacity
13.7K views 17 likes 13 dislikes 22 Apr 2012100% match

Growing Reddit - Web Development
Udacity
82.7K views 484 likes 27 dislikes 27 May 2012100% match

Scaling Databases - Web Development
Udacity
64.3K views 795 likes 7 dislikes 25 Apr 201290% match

Settings Privacy About Contact Donate