Linux 软件阵列与低端硬件阵列卡性能对比

本文主要测试 linux 软件阵列卡与 LSI SAS 9211-8i 硬件阵列卡做 raid 0 的性能.

测试不太严谨,结果仅供参考.

测试结果:

从结果可以看出,低端的硬件阵列卡性能还不如软件阵列.

软件阵列顺序写 Block 的时候,大概比硬件阵列要快 4M/s 的样子. Rewrite 都要快 38M/s . 顺序读 Block 要快 168M/s .

但是软件阵列确实比硬件阵列消耗 CPU 些. 很多测试的结果软件阵列大概比硬件阵列多用了 5%~10% 的 CPU.

测试环境:

CPU: Pentium E5400
内存: 4G (可用 3782 M)
系统: Centos 7.2.1511
硬盘: 4 个同型号的 2TB 的硬盘,两个用来组硬件阵列,两个用来组软阵列.

LSI SAS 9211-8i 之所以称为低端阵列卡,因为阵列卡本身无缓存.

测试工具: bonnie++

相关信息:

列出所有硬盘:

[root@host220 ~]# fdisk -l
Disk /dev/sdb: 3998.0 GB, 3997997989888 bytes, 7808589824 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: dos
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1  4294967295  2147483647+  ee  GPT
Partition 1 does not start on physical sector boundary.

Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sdd: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

sdb 为由硬件阵列卡组的 raid0 ,两个 2TB,所以 sdb 大小将近有 4TB. sdc sdd 也是插在阵列卡上,但是直通的,所以直接显示 2TB 一个.

查看软阵列信息:

[root@host220 ~]# cat /proc/mdstat 
Personalities : [raid0] 
md0 : active raid0 sdd[1] sdc[0]
      3906766848 blocks super 1.2 512k chunks
      
unused devices: <none>

可以看到由 sdc sdd 两个硬盘组建成 md0 这个软 raid0 阵列.

查看挂载信息:

[root@host220 ~]# df -h | grep -E 'mnt|Mount'
Filesystem               Size  Used Avail Use% Mounted on
/dev/md0p1               3.6T   89M  3.4T   1% /mnt/raidtest/soft
/dev/sdb1                3.6T   89M  3.4T   1% /mnt/raidtest/hw

soft 文件夹挂载的是软阵列.hw 文件夹挂载的是硬阵列.

硬件阵列卡测试结果:

[root@host220 ~]# bonnie++ -u root -d /mnt/raidtest/hw/
Using uid:0, gid:0.
Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
host220.cs.lo 7560M   283  99 298715  53 102106  21  2699  98 190370  16 388.6   8
Latency             30693us     253ms     758ms   17693us     246ms     372ms
Version  1.96       ------Sequential Create------ --------Random Create--------
host220.cs.local    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 28154  91 +++++ +++ +++++ +++ 31745  90 +++++ +++ +++++ +++
Latency               169us     534us     554us      99us      29us      61us
1.96,1.96,host220.cs.local,1,1476076832,7560M,,283,99,298715,53,102106,21,2699,98,190370,16,388.6,8,16,,,,,28154,91,+++++,+++,+++++,+++,31745,90,+++++,+++,+++++,+++,30693us,253ms,758ms,17693us,246ms,372ms,169us,534us,554us,99us,29us,61us

防作弊截图 http://ww1.sinaimg.cn/large/c356df32gw1f8n0ei3chvj21hc0fzafh.jpg

软件阵列卡测试结果:

[root@host220 ~]# bonnie++ -u root -d /mnt/raidtest/soft/
Using uid:0, gid:0.
Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
host220.cs.lo 7560M   590  99 302015  50 140381  26  2738  96 359120  26 631.2  12
Latency             29796us     444ms     302ms   21224us     225ms     572ms
Version  1.96       ------Sequential Create------ --------Random Create--------
host220.cs.local    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 27982  89 +++++ +++ +++++ +++ 31718  88 +++++ +++ +++++ +++
Latency               164us     432us     468us      94us      30us      59us
1.96,1.96,host220.cs.local,1,1476076562,7560M,,590,99,302015,50,140381,26,2738,96,359120,26,631.2,12,16,,,,,27982,89,+++++,+++,+++++,+++,31718,88,+++++,+++,+++++,+++,29796us,444ms,302ms,21224us,225ms,572ms,164us,432us,468us,94us,30us,59us

防作弊截图 http://ww2.sinaimg.cn/large/c356df32gw1f8n0ehljthj21gy0hi79w.jpg

结果整理

bonnie++ 的命令输出的最后一行是 csv 的行,我们把它们都写到 /tmp/a.csv 文件里,再调用 cat a.csv | bon_csv2html .这样就把结果通过 html 表格来展示了.

bonnie++ 的结果解析请看参考此两篇文章.

Sequential Output 部分表示写文件的相关信息 Sequential Input 部分表示读文件的相关信息 Per Chr 表示以字符为单位读写文件 Block 表示以 block 为单位读写文件 Rewrite 表示修改并重写已经存在的文件的每一个 block K/sec 表示每秒读或写文件的速率，以 K 为单位 %CP 表示在某阶段执行操作时平均消耗的 CPU

Sequential Create 和 Radom Create 这两大类测试均是用创建，读取，删除大量的小文件来测试磁盘效率。

虽然有 nofollow ,但还是要贴一下自己博客的链接 -_- : https://blog.mangege.com/tech/2016/10/10/1.html