概要信息：

Build High Performance
Weibo System
@TimYang

• µQìNJ
• 140†
• āÔ6ď6ŋçÉƛ
• ÚÏŷŴǂƇZWuġǂŮƹń…*
IJ6Ěų
• DŽü§_ƖݵQŋŕDžQCon 2010
• DŽµQþüŸ`Š5DžWDC 2010
Agenda




	



12
3 4
Part 1
ēƥ‡3
ĉ‚ƐúãŲ¯Ljơ?Ų¯\ï
ƵŬRçYëDJŋŕ(
ŧ»)Lj
„+ů;NJ
MySQL
• ƙdēƥ‡3Lj_âÌ?Ɩ
• Œşƾ
Ì?Ņĭ
• Ò hash Ì?
• ŗ¥Lj„6ŋ
• ŏō¬Lj„ user_timeline index
ƪƺ
• Pù»ťƮEǂƵŬÏŖÌ?
• ŶƪƝ¥Ƶċ
• īÄ
• 6ŋLjĉĺ 5 Ą
• ¦Ɩ
MySQL + cache ƪƺ
• ƴ—
• 
ŧ»
• cacheßÔʼny€LjƵŬ¡AN
• ƻ~¨Ƨ
• ĖIW0Ûƪƺ
NoSQL
“Databases are specializing – the “one 
size fits all” approach no longer applies.”
MongoDBŵűoˆ
NoSQL?
• PùºƆ
• Redis
• MongoDB
• ?›«ºƆ
• Cassandra
• HBase
īƒ
ƨęmİFØú#ìĕ4
ů‰İŒ(ǃ
ģ»ĊƉ
• MongoDB
•Redis
• HBase
• Cassandra
Redis - Ïä«
• ÏN‡3ă«Ĩů
• snapshot - 
Đ
• vm - ,ŠÜª
• diskstore - ,Šãäh
• aof - Ƥ§¿
Redis - ßÔʼny
• string: key value redisObject 16 bytes/item
• list: XhƦū 40 bytes / item
• hash: zipmap(<64)
• set/sorted set
Redis - Replication
• Ƥżrdbá#ƪƺ
•  ĵħņP
Redis - Žƥŭ@
Redis - Œ(
• ƵŬǀƝż:Ŷƪ
• ťŽ¶Ĵõ_īLjåÁğfailoveräĂ
• ólist/setßÔœüƵċ(optional)
ēƥ‡3
• MySQLLjŒşƾİēƥ‡3
• NoSQLLj{ŪMySQLcacheƫĿƲLj
'ìƵŬódƙİƽƼťG
Part 2
çűŇ
µQþü
(ŃĢ)
 ,


$
*#(
0%3
4#
5)
1,

	
"
52
86)&



7
+'4 -
!0,
/.
ôJ→Õ]→£ī
ƪƺ

ơ?WebŋŕĩƸwcacheßÔ
ŶƪLjīUŚìiƄ€NJ
_īİcacheƀęÜÅóĝěß
ÔLjŧßÔIƈæƃŘ¿
• ƎNJ
• =ŽNJ
“Web ŋŕī json û‡3W cache ƶž
Ēſǃ
úµQī json ßÔœüû‡Å
ó†Ĉ(MÎ,Š/¼)LjƵŬ2~5K†Ũ™
a, ī xml ƵŬ10k™a, ī protobuf ¡A
NgLjŎ^ó500†Ũǃ”
3ò6æ 00:47 ƇZ(495) | ŷŴ(134)
Tļ
RDBMS→
Key value (JSON)→
Protocol buffers(binary)
JSON Ĝ½
• DB
• Cache
• Message Queue
• API
PB $L
• Numeric: varint, from 1 byte
• &ƊřbLj&Ɗ†ĈfĻ
• ÚÏŻŰljJava, C++, Python...
• řůĵǀÞ
•  cache ñőœýDƫœýLjƓ
ć
ŨŎĿƫ
• “192.168.0.1” → “0xc0a80000” → varint
Benchmark
TextText
(
http://code.google.com/p/thrift-protobuf-compare/wiki/Benchmarking)
“We would like to provide public APIs that 
accept protocol buffers as well as XML, both 
because it is more efficient and because we're 
just going to convert that XML to protocol 
buffers on our end anyway.”
                                        -  Google     
ßÔœüLj’Rìŝ
©ć}ĨƵċ
• µQİZūì©ć}Ĩ
• ưijŸ`£ī|Lj£īIJ6ßÔÙ
,ƚėƇh©ćä«
• 7ƵŬ©ćİÙ,NJ
ƪƺ
ưijŋŕ|Lj“
Փ>ƪƺ”¾<
|LjƗľVr²
ơ?ìĬ¨Z
j©ć}Ĩśĕ¥Ĩůǃ
• &ŕ LAMP £īLjP
Ù,ćƿ|
W}Ĩçƫ|à*»ť±nǃ
• ©ćƬA}Ĩļ¡ 1ms İ}Ĩçƫ¦
Ʃ_ť%¬ƂƕƨY£WJÝƳǃ
• çıÖƬA}Ĩ»ťÑÿßÔǃ
©ćıÖ
• MQ stat
• MQ Processor stat
ƪƺ
èêīÄįŊ	LjßԄ+çÉ
ƛDÅóīÄ?
íķó“1úãŷŴ”Ljě¨SČ
óßÔNJ
ç»
ŧ»
• TimelineĔ¼ĐLjűßLj6ŋ\ïńJ
ƵŬWçÉƛŔÅóīÄ
• ůƯŢdLjHťĬeôJ‹Á
• ©ć}Ĩ
• ‘: cache ßÔ¢?¨
• RAMN
ŞİZūĐļ
1. ØZū
2. 4ƬA
3. ƬA}Ĩļ¡ż>
4. :cacheWßÔ¢master
5. ßÔ¢replicationDÅóŨě
ƪƺ
ƪƺ
• Ń5ćLjĶxç»W
ŧ»
ƪƺ
• Ń5ćLjĶxç»W
ŧ»
• ÛƓäĎ
ƪƺ
• Ń5ćLjĶxç»W
ŧ»
• ÛƓäĎ
• Ɠ
ć©ć}Ĩ
ƪƺ
• Ń5ćLjĶxç»W
ŧ»
• ÛƓäĎ
• Ɠ
ć©ć}Ĩ
• ƬA?ŏ

ć
• RAM is the new disk
• ÅóĝěßÔIƈD9‡
äĎ

“Percona Server now both SQL and NOSQL 
HandlerSocket, ÔĻ‰İđŹôJp_!ƅD
100rpsLjƣŜì12cores/24threads and 
380GB of RAM, mysql„ýƒĀī“ì

RedisLjÅóßÔƢID9‡Lj-īNoSQLä«
ûŶƪ”
2010-12-25 00:22 ƇZ(9) | ŷŴ(2) 
@jackbillow   “ 1. ØOPKİlookup 2.ð cache
•džňĨäƷLJ”
@kobe    “ ƑìƏŗÁ÷)LjÐinnodbİźVó
ݘ85ťī”
@TimYang “„ý
­ū_!5ơÜ49‡Lj 
InnoDB%ŦKC§

Adaptive Hash Indexesdžz
hashİ9‡ō¬LJ!ƋD9‡ßԢݻť”
äĎ
• ‘NoSQL®,ßÔİż¢
• Ɯƍƫ ļ¡ż[binlogÜ4Redis
• jbinlog https://github.com/tangfl/jbinlog 
Part 3
Ÿ`Õ]
ĘÀŭBšŒZĪ\NLjmİł
Ƈ\Ÿ`lappĪ¹vƫİł

Õ]ŵűŬŌ
• Š5
• Ʊĸ.Ë
• spam
• éī
• ĽŒ
Text
5t?›İ
Õ]n£ç
ƫW_ī»
ıÖ
7$N
• ÇƥŶƪÕ]
• GZIPUŚ
• ×ƘÕ]
Part 4
ôJčĨ
µQñž>ħ±n»ťVrìNJ
“@ƠĦZheng:“Ġ|Œ³ljưijŋŕİÏŖƏ
ũLjĠ1‘ƃåƮ|ƩǃÂ"İƏŗţž“ì
ÏŖÊĠ|Lj ”_ťƭ)Ġ1ǃ” //@Įǁ:
þüœİH¤“ìÖEĠǃ
@bianljîˆ"Ġ" http://t.cn/h0k4r VáƇ
Z(49)|VáŷŴ(15)”
2ò14æ 10:36 ƇZ(22) | ŷŴ(10) 
čĨ
ljƤü
• 2=ĎLjšì2IĎǃ
• ãJƟ´ƒ
VBǃ
čĨljŽƥ
• Žƥŭ@WıÖ
• Žƥöŭ@ƒLjöóÞıÖħóŋŕ
• ŋŕŀĞ\¿Ljť¸ƝŒ(
• JƌƝ|ƩŔŽƥŭ@ûÓÃ
čĨljŸâ
• Ɯƍæ·lu°˜8ZħcĹ©ž
• Õ]n£Ɲ¥
• Ĕ¼}ĨƝ¥
• cachekĥ
• ƌƝŒ(ƪƺťG
čĨqljƭŏ
• P
Jƪƺ¬ZƕƨYëLjƞÁà
*}Ĩ»ťƭǃ
• ƭŏLjĉ
J|IĤŁ¨6Hť

)$


!
	(
*, -&


MySQL
Redis
Memcached
Timeline
)
%


#"
 ,


' 

!
+

ć
• 2=ĎLj_!ƁïƔ
• ïƒİ‘ßÔRAMN
• ïƒİ?›«ä«
Q&A
@TimYang
ąƎI4ãĒµQÈøsƬ
ƩõÍŤ
杭州站 ·∙  2011年10月20日~22日 
www.qconhangzhou.com（6月启动） 
QCon北京站官方网站和资料下载 
www.qconbeijing.com