V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
wsgzao
V2EX  ›  程序员

使用 MegaCli64 快速查看 RAID 和定位故障硬盘实践

  •  
  •   wsgzao ·
    wsgzao · 2019-07-22 16:55:19 +08:00 · 2475 次点击
    这是一个创建于 1732 天前的主题,其中的信息可能已经有所发展或是发生改变。

    前言

    MegaCli 是 LSI 公司官方提供的 SCSI 卡管理工具,由于 LSI 被收购变成了现在的 Broadcom,所以想下载 MegaCli,需要去 Broadcom 官网查找。现在官方有 Storcli,整合了 LSI 和 3ware 所有产品,本文主要以 MegaCli64 为例,无论选择 MegaCli 还是 Storcli 只要在线上环境运行稳定可以获取正确数据就都是合适的方法。

    使用 MegaCli64 快速查看 RAID 和定位故障硬盘实践

    更新历史

    2019 年 07 月 21 日 - 初稿

    阅读原文 - https://wsgzao.github.io/post/megacli64/

    扩展阅读

    MegaCLI vs tw_cli vs Storcli

    Extracting the RAID-Controller Logs via MegaCLI


    MegaCli 简介

    MegaRaid 阵列卡管理工具。他可以查看当前 RAID 卡的所有信息,包括 RAID 卡型号、类型、磁盘状态、电池状态等等。学会了如何使用 MegaCli,我们可以在硬盘没有彻底嗝屁( Failed )之前,监测到是否已经出现预告警报错,不需要现场逐台巡检磁盘状态灯。

    1. 使用 Zabbix 监控硬盘或者使用 IPMI 接入 OOB(Out-of-Band)监控硬件状态
    2. 使用更加成熟的商业化工具管理如云霁科技 CloudBoot,拒绝被单一厂商绑架

    MegaCli64 官网地址: http://docs.avagotech.com/docs/12351587

    Linux 下默认路径: /opt/MegaRAID/MegaCli/MegaCli64

    MegaCli 安装

    1. Download the MegaCli utility from the Broadcom support site

    2. Unzip the downloaded files and install them to any directory of your installed operating system (e.g.: Linux – /tmp/megacli)

    3. From a terminal window in Linux, install the using the the following command:

      Note: root priviledges required to install MegaCLI:

      rpm -i MegaCli-8.07.14-1.noarch.rpm

    4. Then change directory into /opt/MegaRAID/MegaCLI/ and run any of the commands from the table below:

    |Command |Action | | --- | --- | |./MegaCli64 -FwTermLog -Dsply -aALL > /tmp/ttylog.txt |Creates the RAID controller log (ttylog) | |./MegaCli64 -PDList -aALL > /tmp/disks.txt |Creates a list with information about the RAID controllers, virtual disks and hard disks installed | |./MegaCli64 -LDInfo -LALL -aALL > /tmp/LDinfo.txt |Creates a list with information about existing RAID volumes and configurations | |./MegaCli64 -AdpAllInfo -aALL > /tmp/Adapterinfo.txt |Creates a list with information about RAID controller settings | |./MegaCli64 -AdpBbuCmd -aALL > /tmp/Battery.txt |Creates a detailed list of the battery status of the RAID controller (state of charge, learning cycle, etc.) | |./MegaCli64 -AdpEventLog -IncludeDeleted -f deleted.txt -aALL |Creates the RAID controller log (ttylog) with all information since very first controller initialization( Note: This file will always be saved in the MegaCLI root folder) |

    使用 MegaCli64 查看硬盘信息

    [root@localhost wangao]# MegaCli -ShowSummary -aALL
    
    System
    	Operating System:  Linux version 3.10.0-957.21.3.el7.x86_64
    	Driver Version: 07.705.02.00-rh1
    	CLI Version: 8.07.06
    
    Hardware
            Controller
                     ProductName       : PERC H730P Mini(Bus 0, Dev 0)
                     SAS Address       : 51866da0a1306300
                     FW Package Version: 25.5.0.0018
                     Status            : Optimal
            BBU
                     BBU Type          : BBU
                     Status            : Healthy
            Enclosure
                     Product Id        : BP13G+EXP
                     Type              : SES
                     Status            : OK
    
            PD
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 0
                    Vendor Id          : HGST
                    Product Id         : HUS726060AL5214
                    State              : Online
                    Disk Type          : SAS,Hard Disk Device
                    Capacity           : 5.457 TB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 1
                    Vendor Id          : HGST
                    Product Id         : HUS726060AL5214
                    State              : Online
                    Disk Type          : SAS,Hard Disk Device
                    Capacity           : 5.457 TB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 2
                    Vendor Id          : HGST
                    Product Id         : HUS726060AL5214
                    State              : Online
                    Disk Type          : SAS,Hard Disk Device
                    Capacity           : 5.457 TB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 3
                    Vendor Id          : HGST
                    Product Id         : HUS726060AL5214
                    State              : Online
                    Disk Type          : SAS,Hard Disk Device
                    Capacity           : 5.457 TB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 4
                    Vendor Id          : HGST
                    Product Id         : HUS726060AL5214
                    State              : Online
                    Disk Type          : SAS,Hard Disk Device
                    Capacity           : 5.457 TB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 5
                    Vendor Id          : HGST
                    Product Id         : HUS726060AL5214
                    State              : Online
                    Disk Type          : SAS,Hard Disk Device
                    Capacity           : 5.457 TB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 6
                    Vendor Id          : HGST
                    Product Id         : HUS726060AL5214
                    State              : Online
                    Disk Type          : SAS,Hard Disk Device
                    Capacity           : 5.457 TB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 7
                    Vendor Id          : HGST
                    Product Id         : HUS726060AL5214
                    State              : Online
                    Disk Type          : SAS,Hard Disk Device
                    Capacity           : 5.457 TB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 8
                    Vendor Id          : HGST
                    Product Id         : HUS726060AL5214
                    State              : Online
                    Disk Type          : SAS,Hard Disk Device
                    Capacity           : 5.457 TB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 9
                    Vendor Id          : HGST
                    Product Id         : HUS726060AL5214
                    State              : Online
                    Disk Type          : SAS,Hard Disk Device
                    Capacity           : 5.457 TB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 10
                    Vendor Id          : HGST
                    Product Id         : HUS726060AL5214
                    State              : Online
                    Disk Type          : SAS,Hard Disk Device
                    Capacity           : 5.457 TB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 11
                    Vendor Id          : HGST
                    Product Id         : HUS726060AL5214
                    State              : Online
                    Disk Type          : SAS,Hard Disk Device
                    Capacity           : 5.457 TB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 12
                    Vendor Id          : ATA
                    Product Id         : ST9250610NS
                    State              : Online
                    Disk Type          : SATA,Hard Disk Device
                    Capacity           : 232.375 GB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 13
                    Vendor Id          : ATA
                    Product Id         : ST9250610NS
                    State              : Online
                    Disk Type          : SATA,Hard Disk Device
                    Capacity           : 232.375 GB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 14
                    Vendor Id          : HGST
                    Product Id         : HUS726060AL5214
                    State              : Online
                    Disk Type          : SAS,Hard Disk Device
                    Capacity           : 5.457 TB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 15
                    Vendor Id          : HGST
                    Product Id         : HUS726060AL5214
                    State              : Online
                    Disk Type          : SAS,Hard Disk Device
                    Capacity           : 5.457 TB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 16
                    Vendor Id          : HGST
                    Product Id         : HUS726060AL5214
                    State              : Online
                    Disk Type          : SAS,Hard Disk Device
                    Capacity           : 5.457 TB
                    Power State        : Active
    
                    Connector          : 00<Internal><Encl Pos 1 >: Slot 17
                    Vendor Id          : HGST
                    Product Id         : HUS726060AL5214
                    State              : Online
                    Disk Type          : SAS,Hard Disk Device
                    Capacity           : 5.457 TB
                    Power State        : Active
    
    Storage
    
           Virtual Drives
                    Virtual drive      : Target Id 0 ,VD name
                    Size               : 232.375 GB
                    State              : Optimal
                    RAID Level         : 1
    
                    Virtual drive      : Target Id 1 ,VD name
                    Size               : 81.862 TB
                    State              : Optimal
                    RAID Level         : 5
    
    
    Exit Code: 0x00
    
    # check virutal disk / raid info
    if [[ -f /opt/MegaRAID/MegaCli/MegaCli64 ]]; then
        raidlevel=`sudo /opt/MegaRAID/MegaCli/MegaCli64 -LDInfo -Lall -aALL -NoLog | grep 'RAID Level' | awk -F ":" '{print $2}' | awk -F "," {'print $1}' | awk -F "-" '{print $2}'`
        disk_number=`sudo /opt/MegaRAID/MegaCli/MegaCli64 -PDList -aALL -NoLog | egrep 'Device Id' | awk 'BEGIN {RS=""; FS="\n"} {print NF}'`
        disk_type=`sudo /opt/MegaRAID/MegaCli/MegaCli64 -PDList -aALL -NoLog | egrep 'Media Type' | head -1 | awk -F ":" '{print $2}'`
    else
        raidlevel="none"
        disk_number="none"
        disk_type="none"
    fi
    
    # test
    [root@localhost wangao]# echo $raidlevel
    1 5
    [root@localhost wangao]# echo $disk_number
    18
    [root@localhost wangao]# echo $disk_type
    Hard Disk Device
    
    

    Ansible 安装 MegaCLi64

    ---
    - hosts: all
      become: yes
      gather_facts: no
    
      tasks:
        - name: upload MegaCli64
          copy:
            src: files/MegaCli-8.07.14-1.noarch.rpm
            dest: /tmp/MegaCli-8.07.14-1.noarch.rpm
            
        - name: install MegaCli64 rpm from a local file
          yum:
            name: /tmp/MegaCli-8.07.14-1.noarch.rpm
            state: present
            
        - name: create symlink for MegaCli64
          file:
            src: /opt/MegaRAID/MegaCli/MegaCli64
            dest: /usr/sbin/MegaCli
            state: link
    
    

    Ansible 使用 MegaCli64 检查故障硬盘

    ---
    - hosts: all
      become: yes
      gather_facts: no
    
      tasks:
        - name: Check Disk Status
          command: ./MegaCli64 -ShowSummary -aALL
          args:
            chdir: /opt/MegaRAID/MegaCli/
          register: result
        
        - name: Log check info
          lineinfile:
            dest: "{{inventory_dir}}/ansible.log"
            line: '[Ansiable Tasks: Disk Failure info] host={{inventory_hostname}}'
            insertafter: EOF 
            create: yes
          when: '"Failed" in result.stdout'
          delegate_to: localhost
          become: no
    
    
    # check log
    cat ansible.log
    [Ansiable Tasks: Disk Failure info] host=10.71.12.89
    
    
    2 条回复    2019-07-22 17:45:27 +08:00
    ryd994
        1
    ryd994  
       2019-07-22 17:25:23 +08:00 via Android
    你是当我们不会 rtfm 吗?

    @livid 硬广告:
    “使用更加成熟的商业化工具管理如 云霁科技 CloudBoot,拒绝被单一厂商绑架”
    wsgzao
        2
    wsgzao  
    OP
       2019-07-22 17:45:27 +08:00
    @ryd994 #1 我根据自己的经验做推荐( 4 年前 Cloudboot 刚出来用过开源版本,之后他们的产品被别人恶意抄袭还申请了国家专利),和厂商无利益相关。我们自己也会测试使用 Dell,HP,华为的商业化解决方案,但效果也并不理想。如果你有更好的商业化解决方案也可以直接推荐
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   我们的愿景   ·   实用小工具   ·   1534 人在线   最高记录 6543   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 27ms · UTC 17:05 · PVG 01:05 · LAX 10:05 · JFK 13:05
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.