Like
Like Love Haha Wow Sad Angry
5

一、介绍

  MegaRAID 是 LSI 的阵列管理工具,它可以帮助我们管理阵列,有图形化界面和命令行工具。我们这只关注 Linux 下的命令行工具,有两个工具,分别是 MegaCLI StorCLI 。不过 StorCLI 已经基本代替了MegaCLI,所以下面我们都基于 StorCLI 这个工具来。
  StorCLI 是专为 MegaRAID 产品线设计的命令行管理软件,说明文档:https://docs.broadcom.com/doc/12352476
  由于 LSI 被收购变成了现在的 Broadcom(博通),所以现在想下载 MegaCli 或者 StorCLI ,需要去 Broadcom 官网搜索下载即可,官方有详细的说明文档,官方网站:https://www.broadcom.com/
  StorCLI 下载地址:https://docs.broadcom.com/docs-and-downloads/raid-controllers/raid-controllers-common-files/007.1804.0000.0000_Unified_StorCLI.zip
  注意,以上工具只对 LSI 的存储控制卡起作用,如果是其它品牌的存储控制卡,建议去官方网站下载相应阵列管理工具。
  一般来说,PMC 厂商的存储控制卡(Adaptec)命令行工具为 Arcconf,LSI 厂商的存储控制卡命令行工具为 StorCLI。

二、安装

[root@controller ~]# rpm -ivh storcli-007.1804.0000.0000-1.noarch.rpm
[root@controller ~]# ln -s /opt/MegaRAID/storcli/storcli64 /usr/local/bin/storcli
[root@controller ~]# storcli64 -v

      StorCli SAS Customization Utility Ver 007.1804.0000.0000 Apr 09, 2021

    (c)Copyright 2021, Broadcom Inc. All Rights Reserved.

安装完毕之后,在 /opt 目录下会多出一个 MegaRAID/storcli 的目录。
工具路径:/opt/MegaRAID/storcli/storcli64

三、常用命令

以下是一些命令和含义:

命令 含义
add 向控制器添加/创建一个新元素,如 VD、Spare 等
delete 删除像 VD,Spare 这样的元素
show 显示有关元素的信息
set 为属性设置特定值
get 获取属性的特定值
compare 将特定值与属性进行比较
start 启动后台操作
stop 停止后台操作
pause 暂停后台操作
resume 恢复后台操作
download 将文件下载到指定设备
expand 扩展给定驱动器的大小
insert 插入丢失的新驱动器
transform 降级控制器
reset 重置控制器物理
split 拆分逻辑驱动器镜像
/cx 控制器特定命令。注意,x代表数字编号,all代表所有,下面的都一样
/ex 机箱特定命令
/sx 插槽/PD 特定命令
/vx 虚拟驱动器特定命令
/dx 磁盘组特定命令
/fall 外部配置特定命令
/px 物理特定命令
/[bbu|cv] 电池备份单元,缓存命令

1、基本查看命令

查看阵列卡总体信息:

[root@controller ~]# storcli show
CLI Version = 007.1804.0000.0000 Apr 09, 2021
Operating system = Linux 3.10.0-957.el7.x86_64
Status Code = 0
Status = Success
Description = None

Number of Controllers = 1
Host Name = controller
Operating System  = Linux 3.10.0-957.el7.x86_64

System Overview :
===============

---------------------------------------------------------------------------------
Ctl Model                 Ports PDs DGs DNOpt VDs VNOpt BBU sPR DS EHS ASOs Hlth 
---------------------------------------------------------------------------------
  0 LSIMegaRAIDSAS9240-8i     8   6   5     0   5     0 N/A On  -  Y      0 Opt  
---------------------------------------------------------------------------------

Ctl=Controller Index|DGs=Drive groups|VDs=Virtual drives|Fld=Failed
PDs=Physical drives|DNOpt=Array NotOptimal|VNOpt=VD NotOptimal|Opt=Optimal
Msng=Missing|Dgd=Degraded|NdAtn=Need Attention|Unkwn=Unknown
sPR=Scheduled Patrol Read|DS=DimmerSwitch|EHS=Emergency Spare Drive
Y=Yes|N=No|ASOs=Advanced Software Options|BBU=Battery backup unit/CV
Hlth=Health|Safe=Safe-mode boot|CertProv-Certificate Provision mode
Chrg=Charging | MsngCbl=Cable Failure

输出信息含义:
  Ctl:控制器编号
  Model:阵列卡型号
  PDS:物理驱动器
  DGS:驱动器组
  VDs:虚拟驱动器
  sPR:巡读
  EHS:紧急备用驱动器
  Hlth:健康状态

注:
  巡读:磁盘巡检读取。为了确保物理磁盘的运行状况和数据完整性,巡检读取功能会扫描潜在问题并解决这些问题。

查看所有的控制器、机箱、插槽id及相关信息:

[root@controller ~]# storcli /call /eall /sall show    
'CLI Version = 007.1804.0000.0000 Apr 09, 2021
Operating system = Linux 3.10.0-957.el7.x86_64
Controller = 0
Status = Success
Description = Show Drive Information Succeeded.

Drive Information :
=================

---------------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model               Sp Type 
---------------------------------------------------------------------------------
14:0     19 Onln   1   7.276 TB SATA HDD N   N  512B ST8000NM000A-2KE101 U  -    
14:1     25 Onln   0 558.406 GB SAS  HDD N   N  512B ST600MM0069         U  -    
14:2     26 Onln   0 558.406 GB SAS  HDD N   N  512B ST600MM0069         U  -    
14:3     24 Onln   2   7.276 TB SATA HDD N   N  512B ST8000NM000A-2KE101 U  -    
14:4     23 Onln   3   7.276 TB SATA HDD N   N  512B ST8000NM000A-2KE101 U  -    
14:5     22 Onln   4   7.276 TB SATA HDD N   N  512B ST8000NM000A-2KE101 U  -    
---------------------------------------------------------------------------------

EID=Enclosure Device ID|Slt=Slot No|DID=Device ID|DG=DriveGroup
DHS=Dedicated Hot Spare|UGood=Unconfigured Good|GHS=Global Hotspare
UBad=Unconfigured Bad|Sntze=Sanitize|Onln=Online|Offln=Offline|Intf=Interface
Med=Media Type|SED=Self Encryptive Drive|PI=Protection Info
SeSz=Sector Size|Sp=Spun|U=Up|D=Down|T=Transition|F=Foreign
UGUnsp=UGood Unsupported|UGShld=UGood shielded|HSPShld=Hotspare shielded
CFShld=Configured shielded|Cpybck=CopyBack|CBShld=Copyback Shielded
UBUnsp=UBad Unsupported|Rbld=Rebuild

输出信息含义:
  Controller = 0:表示只有一个控制器,id 为 0。
  EID:机箱编号。
  Slt:插槽编号。
  DID:设备编号
  State:设备状态。有onln、offln、jobd、ugood
  DG:磁盘组。
  size:磁盘容量
  intf:接口类型,可以看到有 SATA 和 SAS。
  Med:存储介质,SSD 或 HDD。
  SeSz:扇区大小
  Model:磁盘型号

注:
  onln:硬盘上线。即硬盘加入 RAID 组
  offln:硬盘离线。即硬盘脱离 RAID 组
  jobd:直通
  ugood:此状态下硬盘可用于创建 RAID 或者热备盘。

查看特定磁盘信息:

[root@controller ~]# storcli /c0/e14/s0 show        # 查看位于0号插槽的磁盘信息,show all可以看到更详细的信息
Drive Information :
=================

-------------------------------------------------------------------------------
EID:Slt DID State DG     Size Intf Med SED PI SeSz Model               Sp Type 
-------------------------------------------------------------------------------
14:0     19 Onln   1 7.276 TB SATA HDD N   N  512B ST8000NM000A-2KE101 U  -    
-------------------------------------------------------------------------------

[root@controller ~]# storcli /c0/e14/s0,1 show      # 查看位于0号和1号插槽的磁盘信息
---------------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model               Sp Type 
---------------------------------------------------------------------------------
14:0     19 Onln   1   7.276 TB SATA HDD N   N  512B ST8000NM000A-2KE101 U  -    
14:1     25 Onln   0 558.406 GB SAS  HDD N   N  512B ST600MM0069         U  -    
---------------------------------------------------------------------------------

[root@controller ~]# storcli /c0/e14/s0-2 show      # 查看位于0号到2号插槽的磁盘信息
---------------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model               Sp Type 
---------------------------------------------------------------------------------
14:0     19 Onln   1   7.276 TB SATA HDD N   N  512B ST8000NM000A-2KE101 U  -    
14:1     25 Onln   0 558.406 GB SAS  HDD N   N  512B ST600MM0069         U  -    
14:2     26 Onln   0 558.406 GB SAS  HDD N   N  512B ST600MM0069         U  -    
---------------------------------------------------------------------------------

查看虚拟驱动器信息:

[root@controller ~]# storcli /c0/vall show      # 查看所有虚拟驱动器信息
Virtual Drives :
==============

---------------------------------------------------------------
DG/VD TYPE  State Access Consist Cache Cac sCC       Size Name 
---------------------------------------------------------------
0/0   RAID1 Optl  RW     Yes     NRWTD -   ON  558.406 GB      
1/1   RAID0 Optl  RW     Yes     NRWTD -   ON    7.276 TB      
2/2   RAID0 Optl  RW     Yes     NRWTD -   ON    7.276 TB      
3/3   RAID0 Optl  RW     Yes     NRWTD -   ON    7.276 TB      
4/4   RAID0 Optl  RW     Yes     NRWTD -   ON    7.276 TB      
---------------------------------------------------------------

[root@controller ~]# storcli /c0/v0 show        # 查看0号虚拟驱动器信息   
---------------------------------------------------------------
DG/VD TYPE  State Access Consist Cache Cac sCC       Size Name 
---------------------------------------------------------------
0/0   RAID1 Optl  RW     Yes     NRWTD -   ON  558.406 GB      
---------------------------------------------------------------

查看磁盘组信息:

[root@controller ~]# storcli /c0/d0 show        # 查看0号磁盘组信息
TOPOLOGY :
========

-----------------------------------------------------------------------------
DG Arr Row EID:Slot DID Type  State BT       Size PDC  PI SED DS3  FSpace TR 
-----------------------------------------------------------------------------
 0 -   -   -        -   RAID1 Optl  N  558.406 GB dflt N  N   dflt N      N  
 0 0   -   -        -   RAID1 Optl  N  558.406 GB dflt N  N   dflt N      N  
 0 0   0   14:1     25  DRIVE Onln  N  558.406 GB dflt N  N   dflt -      N  
 0 0   1   14:2     26  DRIVE Onln  N  558.406 GB dflt N  N   dflt -      N  
-----------------------------------------------------------------------------

2、其它常用命令

不知道 set 要设置的属性名请help

1、RAID重建
storcli /cx set rebuildrate=x       # 整数,设置RAID重构速率
storcli /cx show rebuildrate        # 查看重构速率
storcli /cx/ex/sx show rebuild      # 查看某块盘的重构情况
storcli /cx/ex/sx pause rebuild     # 暂停重构

2、巡读
storcli /cx set patrolread=[on|off]    # 开启或关闭巡读
storcli /cx show patrolread            # 查看控制器巡读相关信息

3、全局热备盘
storcli /cx/ex/sx add hotsparedrive         # 设置某块盘为全局热备盘
storcli /cx/ex/sx delete hostsparedrive     # 删除某块全局热备盘
storcli /cx/ex/sx add hotsparedrive dgs=0   # 设置某块盘为某个磁盘组的局部热备盘

4、定位磁盘
storcli /cx/ex/sx start locate      # 打开磁盘指示灯
storcli /cx/ex/sx stop locate       # 关闭磁盘指示灯

5、虚拟驱动器(逻辑磁盘)
storcli /cx/vall show           # 显示逻辑磁盘信息
storcli /cx/vall show all       # 显示更详细的逻辑磁盘信息
storcli /cx/vx del              # 删除逻辑磁盘,此命令会等待cache中的数据写入磁盘
storcli /cx/vx del force        # 删除逻辑磁盘,此命令不会等待cache中的数据写入磁盘
storcli /cx/[vx|vall] set wrcache=wt/wb/awb   # 设置逻辑磁盘的写策略

6、物理磁盘初始化
storcli /cx/ex/sx start initialization      # 硬盘初始化
storcli /cx/ex/sx show initialization       # 查询初始化进度
storcli /cx/ex/sx stop initialization

7、Raid初始化
storcli /cx/vx start init          # 快速初始化
storcli /cx/vx start init full     # 完全初始化
storcli /cx/vx [show|pause|resume|stop] bgi     # 查看、暂停、继续、停止初始化

注:
 init:只把 RAID 的前 100MByte 空间进行全写0操作,随后此RAID的状态就变为“Optimal”。
 init full:把整个RAID都初始化为0,才会结束初始化过程,在此之前RAID状态为“initialization”。

8、其它
storcli /cx download file=filepath      # 升级raid卡固件,需要重启
storcli /cx flushcache                  # 清除raid卡、物理磁盘cache

再其它的就自行 help 吧,或者看官方说明文档吧,太多了。
推荐阅读:https://support.huawei.com/enterprise/zh/doc/EDOC1000004345/d02f498c

四、实例

1、创建RAID

[root@controller ~]# storcli /call/eall/sall show | sed '/---/,/^$/!d'
---------------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model               Sp Type 
---------------------------------------------------------------------------------
14:0     19 Onln   1   7.276 TB SATA HDD N   N  512B ST8000NM000A-2KE101 U  -    
14:1     25 Onln   0 558.406 GB SAS  HDD N   N  512B ST600MM0069         U  -    
14:2     26 Onln   0 558.406 GB SAS  HDD N   N  512B ST600MM0069         U  -    
14:3     24 Onln   2   7.276 TB SATA HDD N   N  512B ST8000NM000A-2KE101 U  -    
14:4     23 Onln   3   7.276 TB SATA HDD N   N  512B ST8000NM000A-2KE101 U  -    
14:5     22 Onln   4   7.276 TB SATA HDD N   N  512B ST8000NM000A-2KE101 U  -    
14:8     28 JBOD   - 558.911 GB SAS  HDD N   N  512B ST600MM0069         U  -    
---------------------------------------------------------------------------------

上面的编号 14:8 的 SAS 直通盘是我刚插进去的,现要将它做成Raid 0。
首先先改变磁盘状态,将直通状态改成 UGood 状态:

[root@controller ~]# storcli /c0/e14/s8 set good force
CLI Version = 007.1804.0000.0000 Apr 09, 2021
Operating system = Linux 3.10.0-957.el7.x86_64
Controller = 0
Status = Success
Description = Set Drive Good Succeeded.

[root@controller ~]# storcli /call/eall/sall show | sed '/---/,/^$/!d'
---------------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model               Sp Type 
---------------------------------------------------------------------------------   
14:8     28 UGood  - 558.406 GB SAS  HDD N   N  512B ST600MM0069         U  -    
---------------------------------------------------------------------------------

创建Raid 0:

[root@controller ~]# storcli /c0 add vd r0 size=all drives=14:8
CLI Version = 007.1804.0000.0000 Apr 09, 2021
Operating system = Linux 3.10.0-957.el7.x86_64
Controller = 0
Status = Success
Description = Add VD Succeeded.

[root@controller ~]# storcli /call/eall/sall show | sed '/---/,/^$/!d'
---------------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model               Sp Type 
---------------------------------------------------------------------------------
14:8     28 Onln   5 558.406 GB SAS  HDD N   N  512B ST600MM0069         U  -   
---------------------------------------------------------------------------------

[root@controller ~]# storcli /c0/v5 show | sed '/---/,/^$/!d'
---------------------------------------------------------------
DG/VD TYPE  State Access Consist Cache Cac sCC       Size Name 
---------------------------------------------------------------
5/5   RAID0 Optl  RW     Yes     NRWTD -   ON  558.406 GB      
---------------------------------------------------------------

删除刚创建的Raid 0:

[root@controller ~]# storcli /c0/v5 delete
CLI Version = 007.1804.0000.0000 Apr 09, 2021
Operating system = Linux 3.10.0-957.el7.x86_64
Controller = 0
Status = Success
Description = Delete VD succeeded

[root@controller ~]# storcli /c0/vall show | sed '/---/,/^$/!d'
---------------------------------------------------------------
DG/VD TYPE  State Access Consist Cache Cac sCC       Size Name 
---------------------------------------------------------------
0/0   RAID1 Optl  RW     Yes     NRWTD -   ON  558.406 GB      
1/1   RAID0 Optl  RW     Yes     NRWTD -   ON    7.276 TB      
2/2   RAID0 Optl  RW     Yes     NRWTD -   ON    7.276 TB      
3/3   RAID0 Optl  RW     Yes     NRWTD -   ON    7.276 TB      
4/4   RAID0 Optl  RW     Yes     NRWTD -   ON    7.276 TB      
---------------------------------------------------------------

创建 raid 命令格式用法:
storcli /cx add vd type=raid[0|1|5|6|10|50|60][Size=,,..|*all][name=,..] drives=e:s|e:s-x|e:s-x,y;e:s-x,y,z [PDperArray=x][SED][pdcache=on|off|*default][pi] [DimmerSwitch(ds)=default|automatic(auto)|*none|maximum(max)|MaximumWithoutCaching(maxnocache)][cachevd] [wt|*wb] [nora|*ra] [*direct|cached] [CachedBadBBU|*NoCachedBadBBU][Strip=<8|16|32|64|128|256|1024>] [AfterVd=X] [Spares = [e:]s|[e:]s-x|[e:]s-x,y][force]

2、硬盘点灯

(1)直通状态

此时 sdf 为直通状态。假如我要点亮sdf,先要根据盘符通过使用 smartctl 工具确认硬盘序列号。

[root@controller ~]# smartctl -a /dev/sdf | grep -i 'Serial number'
Serial number:        WFJ3ZNF2

然后通过硬盘序列号确认硬盘槽位号。

[root@controller ~]# storcli /call/eall/sall show  all | grep -B 2 'SN = WFJ3ZNF2'
Drive /c0/e14/s8 Device attributes :
==================================
SN = WFJ3ZNF2

使用 start locate 命令点亮硬盘定位指示灯。

[root@controller ~]# storcli /c0/e14/s8 start locate
CLI Version = 007.1804.0000.0000 Apr 09, 2021
Operating system = Linux 3.10.0-957.el7.x86_64
Controller = 0
Status = Success
Description = Start Drive Locate Succeeded.

# 关闭指示灯
[root@controller ~]# storcli /c0/e14/s8 stop locate

(2)Raid 状态

  此时 sda 为 Raid 1 状态。我要点亮sda,先要通过块设备信息确认硬盘盘符对应的 target ID。
  执行以下命令查询 sdf 对应的的 target ID。其中打印信息中 targetx:y:z 中的 z 即为 target ID。如下所示,“target ID” 为 0。

[root@controller ~]# lsscsi | grep sda 
[0:2:5:0]    disk    LSI      MR9240-8i        2.13  /dev/sdf

[root@controller ~]# ll /sys/block/ | grep sda
lrwxrwxrwx. 1 root root 0 Jul  7 15:24 sda -> ../devices/pci0000:80/0000:80:01.0/0000:81:00.0/host0/target0:2:0/0:2:0:0/block/sda

  通过 target ID 确认硬盘槽位号,执行以下命令查询所有硬盘信息,并将查询到的信息保存为文件。然后在文件中查找 “Target ID” 字段,在查找到的 “Target ID” 为 0 所在的硬盘信息中记录 “Device IDs” 对应的信息。

[root@controller ~]# storcli /call show alilog > sda.txt
[root@controller ~]# vim sda.txt
......
Detail Logical Drive Information
--------------------------------

Target ID                      : 0
......
Total size of the logical drive: 571808MB
Logical Drive owner's Device ID: 255
Number of Physical Devices  : 2
Device IDs:     25
Device IDs:     26
......

[root@controller ~]# storcli /call/eall/sall show | sed '/---/,/^$/!d'      # DID 就是 Device IDs
---------------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model               Sp Type 
---------------------------------------------------------------------------------  
14:1     25 Onln   0 558.406 GB SAS  HDD N   N  512B ST600MM0069         U  -    
14:2     26 Onln   0 558.406 GB SAS  HDD N   N  512B ST600MM0069         U  -    
---------------------------------------------------------------------------------
```bash
此时确认好了硬盘槽位号,依次点亮即可:
```bash
[root@controller ~]#  storcli /c0/e14/s1 start locate
[root@controller ~]#  storcli /c0/e14/s2 start locate

3、硬盘直通

注意,需要RAID阵列卡支持直通功能才行。如果硬盘里存在阵列信息,设置为直通状态是不会成功的要先初始化。

[root@controller ~]# storcli /c0 set jbod=on      # 开启RAID卡的硬盘直通功能
[root@controller ~]# storcli /c0 show jbod
Controller Properties :
=====================

----------------
Ctrl_Prop Value 
----------------
JBOD      ON    
----------------

[root@controller ~]# storcli /c0/e14/s8 set jbod

处于直通状态的硬盘是不能用于创建 Raid 和热备盘的,需要取消直通状态才行:

[root@controller ~]# storcli /call/eall/sall show | sed '/---/,/^$/!d' | grep -v '^14:[0-7]'
---------------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model               Sp Type 
---------------------------------------------------------------------------------
14:8     28 JBOD   - 558.911 GB SAS  HDD N   N  512B ST600MM0069         U  -    
---------------------------------------------------------------------------------

[root@controller ~]# storcli /c0/e14/s8 set good force      # 取消直通
CLI Version = 007.1804.0000.0000 Apr 09, 2021
Operating system = Linux 3.10.0-957.el7.x86_64
Controller = 0
Status = Success
Description = Set Drive Good Succeeded.

[root@controller ~]# storcli /call/eall/sall show | sed '/---/,/^$/!d' | grep -v '^14:[0-7]'
---------------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model               Sp Type 
---------------------------------------------------------------------------------
14:8     28 UGood  - 558.406 GB SAS  HDD N   N  512B ST600MM0069         U  -    
---------------------------------------------------------------------------------


参考文章:

https://docs.broadcom.com/doc/12352476
https://blog.51cto.com/mofesi/1309251 https://support.huawei.com/enterprise/zh/doc/EDOC1000004345/d02f498c
https://support.huawei.com/enterprise/zh/doc/EDOC1000041337/75f2d44b

Like
Like Love Haha Wow Sad Angry
5

发表评论