浅看rabbitmq的mnesia部署

转载：http://www.iteye.com/topic/643187

伸缩性：根据系统负载，可以在运行中过程中添加或者删除服务节点，改变系统处理规模。
mnesia是一个分布式数据库模块，由多个节点构成的数据库cluster，数据表的位置对应用是透明的。透过该特性，很容易构建出一个具有高伸缩性的系统。
rabbitmq是一个分布式的消息中间件，在mnesia-cluster的机制上可由多个节点共同构建。

rabbitmq在一个节点上初始化mnesia的过程概况如下：
1.启动mnesia，尝试连接到系统的其他节点上；
2.若无法连接到任何节点，表示该节点是系统中第一个启动的节点；
    2.1.在这种情况下可以检查数据库的老表，然后等待其他节点的连接；
3.若连接到一些节点，则这些节点会同步并合并schema表；
    3.1.在当前节点上创建schema表和其他数据表的副本后，该节点就和所连接的节点数据将保持同步和一致；

Java代码 
%%rabbit_mnesia.erl 
init_db(ClusterNodes) -> 
    case mnesia:change_config(extra_db_nodes, ClusterNodes -- [node()]) of 
    end 
%%rabbit_mnesia.erl
init_db(ClusterNodes) ->
    case mnesia:change_config(extra_db_nodes, ClusterNodes -- [node()]) of
    end
 启动mnesia后连接到其他节点。

Java代码 
%%rabbit_mnesia.erl 
    case mnesia:change_config(extra_db_nodes, ClusterNodes -- [node()]) of 
        {ok, []} ->  
%%rabbit_mnesia.erl
    case mnesia:change_config(extra_db_nodes, ClusterNodes -- [node()]) of
        {ok, []} ->
 无法连接到任何节点，则检查当前数据库目录下的老表是否正确。（数据库目录为空则建立新表）

Java代码 
%%rabbit_mnesia.erl 
    case mnesia:change_config(extra_db_nodes, ClusterNodes -- [node()]) of 
        {ok, [_|_]} -> 
            IsDiskNode = ClusterNodes == [] orelse %%ClusterNodes==[]主节点 
                lists:member(node(), ClusterNodes), 
            ok = wait_for_replicated_tables(), 
            ok = create_local_table_copy(schema, disc_copies), 
            ok = create_local_table_copies(case IsDiskNode of 
%%rabbit_mnesia.erl
    case mnesia:change_config(extra_db_nodes, ClusterNodes -- [node()]) of
        {ok, [_|_]} ->
            IsDiskNode = ClusterNodes == [] orelse %%ClusterNodes==[]主节点
                lists:member(node(), ClusterNodes),
            ok = wait_for_replicated_tables(),
            ok = create_local_table_copy(schema, disc_copies),
            ok = create_local_table_copies(case IsDiskNode of
成功连接到一些节点后，mnesia之间交换数据库元信息，并等待在当前节点上有磁盘副本(disc_copies)的表和cluster完成同步。
            ok = wait_for_replicated_tables(),
如果当前节点是数据存储节点，还要在该节点上建立一些表格的磁盘副本
            ok = create_local_table_copies(case IsDiskNode of

我做了些简单的实验来观察mnesia相互连接时的特性。

Java代码 
(a@localhost)1> mnesia:create_schema([node()]). 
ok 
(a@localhost)2> mnesia:start(). 
ok 
(a@localhost)3> mnesia:create_table(user, [{disc_copies, [node()]}]). 
{atomic,ok} 
(a@localhost)4> mnesia:info(). 
---> Processes holding locks <---  
---> Processes waiting for locks <---  
---> Participant transactions <---  
---> Coordinator transactions <--- 
---> Uncertain transactions <---  
---> Active tables <---  
user           : with 0        records occupying 304      words of mem 
schema         : with 2        records occupying 524      words of mem 
===> System info in version "4.4.10", debug level = none <=== 
opt_disc. Directory "/home/hwh/a" is used. 
use fallback at restart = false 
running db nodes   = [a@localhost] 
stopped db nodes   = []  
master node tables = [] 
remote             = [] 
ram_copies         = [] 
disc_copies        = [schema,user] 
disc_only_copies   = [] 
[{a@localhost,disc_copies}] = [schema,user] 
3 transactions committed, 0 aborted, 0 restarted, 1 logged to disc 
0 held locks, 0 in queue; 0 local transactions, 0 remote 
0 transactions waits for other nodes: [] 
ok 
(a@localhost)1> mnesia:create_schema([node()]).
ok
(a@localhost)2> mnesia:start().
ok
(a@localhost)3> mnesia:create_table(user, [{disc_copies, [node()]}]).
{atomic,ok}
(a@localhost)4> mnesia:info().
---> Processes holding locks <---
---> Processes waiting for locks <---
---> Participant transactions <---
---> Coordinator transactions <---
---> Uncertain transactions <---
---> Active tables <---
user           : with 0        records occupying 304      words of mem
schema         : with 2        records occupying 524      words of mem
===> System info in version "4.4.10", debug level = none <===
opt_disc. Directory "/home/hwh/a" is used.
use fallback at restart = false
running db nodes   = [a@localhost]
stopped db nodes   = []
master node tables = []
remote             = []
ram_copies         = []
disc_copies        = [schema,user]
disc_only_copies   = []
[{a@localhost,disc_copies}] = [schema,user]
3 transactions committed, 0 aborted, 0 restarted, 1 logged to disc
0 held locks, 0 in queue; 0 local transactions, 0 remote
0 transactions waits for other nodes: []
ok
 a节点上有两个磁盘表，分别是schema和user。

Java代码 
(b@localhost)1> mnesia:start(). 
ok 
(b@localhost)2> mnesia:change_config(extra_db_nodes, ['a@localhost', 'b@localhost', 'c@localhost']--[node()]). 
{ok,[a@localhost]} 
(b@localhost)1> mnesia:start().
ok
(b@localhost)2> mnesia:change_config(extra_db_nodes, ['a@localhost', 'b@localhost', 'c@localhost']--[node()]).
{ok,[a@localhost]}
 b节点尝试连接a,c节点，最终连接上了正在运行的a节点。连接之后b节点是什么状态呢？

Java代码 
(b@localhost)3> mnesia:info(). 
---> Processes holding locks <---  
---> Processes waiting for locks <---  
---> Participant transactions <---  
---> Coordinator transactions <--- 
---> Uncertain transactions <---  
---> Active tables <---  
schema         : with 2        records occupying 533      words of mem 
===> System info in version "4.4.10", debug level = none <=== 
opt_disc. Directory "/home/hwh/b" is NOT used. 
use fallback at restart = false 
running db nodes   = [a@localhost,b@localhost] 
stopped db nodes   = []  
master node tables = [] 
remote             = [user] 
ram_copies         = [schema] 
disc_copies        = [] 
disc_only_copies   = [] 
[{a@localhost,disc_copies}] = [user] 
[{a@localhost,disc_copies},{b@localhost,ram_copies}] = [schema] 
4 transactions committed, 0 aborted, 0 restarted, 0 logged to disc 
0 held locks, 0 in queue; 0 local transactions, 0 remote 
0 transactions waits for other nodes: [] 
ok 
(b@localhost)3> mnesia:info().
---> Processes holding locks <---
---> Processes waiting for locks <---
---> Participant transactions <---
---> Coordinator transactions <---
---> Uncertain transactions <---
---> Active tables <---
schema         : with 2        records occupying 533      words of mem
===> System info in version "4.4.10", debug level = none <===
opt_disc. Directory "/home/hwh/b" is NOT used.
use fallback at restart = false
running db nodes   = [a@localhost,b@localhost]
stopped db nodes   = []
master node tables = []
remote             = [user]
ram_copies         = [schema]
disc_copies        = []
disc_only_copies   = []
[{a@localhost,disc_copies}] = [user]
[{a@localhost,disc_copies},{b@localhost,ram_copies}] = [schema]
4 transactions committed, 0 aborted, 0 restarted, 0 logged to disc
0 held locks, 0 in queue; 0 local transactions, 0 remote
0 transactions waits for other nodes: []
ok
可以看到user表是remote的，在a节点上有磁盘副本，在b节点上没有任何类型的副本。schema表已经合并，分别存储在a,b节点上。
此时在b节点上就可操作user表，表的位置是透明的。

Java代码 
(b@localhost)4> mnesia:change_table_copy_type(schema, node(), disc_copies). 
{atomic,ok} 
(b@localhost)5> mnesia:add_table_copy(user, node(), disc_copies). 
{atomic,ok} 
(b@localhost)6> mnesia:info(). 
---> Processes holding locks <---  
---> Processes waiting for locks <---  
---> Participant transactions <---  
---> Coordinator transactions <--- 
---> Uncertain transactions <---  
---> Active tables <---  
user           : with 0        records occupying 304      words of mem 
schema         : with 2        records occupying 542      words of mem 
===> System info in version "4.4.10", debug level = none <=== 
opt_disc. Directory "/home/hwh/b" is used. 
use fallback at restart = false 
running db nodes   = [a@localhost,b@localhost] 
stopped db nodes   = []  
master node tables = [] 
remote             = [] 
ram_copies         = [] 
disc_copies        = [schema,user] 
disc_only_copies   = [] 
[{a@localhost,disc_copies},{b@localhost,disc_copies}] = [schema,user] 
6 transactions committed, 0 aborted, 0 restarted, 2 logged to disc 
0 held locks, 0 in queue; 0 local transactions, 0 remote 
0 transactions waits for other nodes: [] 
ok 
(b@localhost)4> mnesia:change_table_copy_type(schema, node(), disc_copies).
{atomic,ok}
(b@localhost)5> mnesia:add_table_copy(user, node(), disc_copies).
{atomic,ok}
(b@localhost)6> mnesia:info().
---> Processes holding locks <---
---> Processes waiting for locks <---
---> Participant transactions <---
---> Coordinator transactions <---
---> Uncertain transactions <---
---> Active tables <---
user           : with 0        records occupying 304      words of mem
schema         : with 2        records occupying 542      words of mem
===> System info in version "4.4.10", debug level = none <===
opt_disc. Directory "/home/hwh/b" is used.
use fallback at restart = false
running db nodes   = [a@localhost,b@localhost]
stopped db nodes   = []
master node tables = []
remote             = []
ram_copies         = []
disc_copies        = [schema,user]
disc_only_copies   = []
[{a@localhost,disc_copies},{b@localhost,disc_copies}] = [schema,user]
6 transactions committed, 0 aborted, 0 restarted, 2 logged to disc
0 held locks, 0 in queue; 0 local transactions, 0 remote
0 transactions waits for other nodes: []
ok
在b节点上建立schema表和user表的磁盘副本后，发现user表不再是remote属性了，可从本地直接读取。

先退出b节点，再退出a节点，然后只重启b节点。

Java代码 
(b@localhost)1> mnesia:start(). 
ok 
(b@localhost)2> mnesia:info(). 
---> Processes holding locks <---  
---> Processes waiting for locks <---  
---> Participant transactions <---  
---> Coordinator transactions <--- 
---> Uncertain transactions <---  
---> Active tables <---  
schema         : with 2        records occupying 542      words of mem 
===> System info in version "4.4.10", debug level = none <=== 
opt_disc. Directory "/home/hwh/b" is used. 
use fallback at restart = false 
running db nodes   = [b@localhost] 
stopped db nodes   = [a@localhost]  
master node tables = [] 
remote             = [] 
ram_copies         = [] 
disc_copies        = [schema,user] 
disc_only_copies   = [] 
[] = [user] 
[{b@localhost,disc_copies}] = [schema] 
2 transactions committed, 0 aborted, 0 restarted, 0 logged to disc 
0 held locks, 0 in queue; 0 local transactions, 0 remote 
0 transactions waits for other nodes: [] 
ok 
(b@localhost)3> mnesia:dirty_read(user, key). 
** exception exit: {aborted,{no_exists,[user,key]}} 
     in function  mnesia:abort/1 
(b@localhost)1> mnesia:start().
ok
(b@localhost)2> mnesia:info().
---> Processes holding locks <---
---> Processes waiting for locks <---
---> Participant transactions <---
---> Coordinator transactions <---
---> Uncertain transactions <---
---> Active tables <---
schema         : with 2        records occupying 542      words of mem
===> System info in version "4.4.10", debug level = none <===
opt_disc. Directory "/home/hwh/b" is used.
use fallback at restart = false
running db nodes   = [b@localhost]
stopped db nodes   = [a@localhost]
master node tables = []
remote             = []
ram_copies         = []
disc_copies        = [schema,user]
disc_only_copies   = []
[] = [user]
[{b@localhost,disc_copies}] = [schema]
2 transactions committed, 0 aborted, 0 restarted, 0 logged to disc
0 held locks, 0 in queue; 0 local transactions, 0 remote
0 transactions waits for other nodes: []
ok
(b@localhost)3> mnesia:dirty_read(user, key).
** exception exit: {aborted,{no_exists,[user,key]}}
     in function  mnesia:abort/1
 这时看到user表此时是不可用的，因为a,b节点共同维护user表的一致性，但是b节点先退出，所以user表的最终状态由a节点决定。
启动a节点之后，user表就变成可用状态了。

Java代码 
(a@localhost)1> mnesia:start(). 
(a@localhost)1> mnesia:start().
Java代码 
(b@localhost)4> mnesia:info().                
---> Processes holding locks <---  
---> Processes waiting for locks <---  
---> Participant transactions <---  
---> Coordinator transactions <--- 
---> Uncertain transactions <---  
---> Active tables <---  
user           : with 0        records occupying 304      words of mem 
schema         : with 2        records occupying 542      words of mem 
===> System info in version "4.4.10", debug level = none <=== 
opt_disc. Directory "/home/hwh/b" is used. 
use fallback at restart = false 
running db nodes   = [a@localhost,b@localhost] 
stopped db nodes   = []  
master node tables = [] 
remote             = [] 
ram_copies         = [] 
disc_copies        = [schema,user] 
disc_only_copies   = [] 
[{a@localhost,disc_copies},{b@localhost,disc_copies}] = [schema,user] 
3 transactions committed, 0 aborted, 0 restarted, 0 logged to disc 
0 held locks, 0 in queue; 0 local transactions, 0 remote 
0 transactions waits for other nodes: [] 
ok