HP Proliant Raid
Материал из noname.com.ua
Raid на серверах HP
Мне достались 2 сервера с разными контроллерами
- H410 с кешем и P240 без кеша
|
|
|
|---|---|
lspci -s 09:00.0 -vv 09:00.0 RAID bus controller: Hewlett-Packard Company Smart Array G6 controllers (rev 01) Subsystem: Hewlett-Packard Company Smart Array P410 |
lspci -s 06:00.0 -vv 06:00.0 RAID bus controller: Hewlett-Packard Company Smart Array Gen9 Controllers (rev 01) Subsystem: Hewlett-Packard Company H240 |
09:00.0 RAID bus controller: Hewlett-Packard Company Smart Array G6 controllers (rev 01) Subsystem: Hewlett-Packard Company Smart Array P410 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin A routed to IRQ 37 Region 0: Memory at fbc00000 (64-bit, non-prefetchable) [size=2M] Region 2: Memory at fbbff000 (64-bit, non-prefetchable) [size=4K] Region 4: I/O ports at d800 [size=256] Expansion ROM at fbb00000 [disabled] [size=512K] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 Capabilities: [70] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <1us, L1 <8us ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W DevCtl: CorrErr- NonFatalErr+ FatalErr+ UnsupReq- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 256 bytes, MaxReadReq 4096 bytes DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend- LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM not supported ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp- LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s (ok), Width x8 (ok) TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP- LTR- 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- TPHComp- ExtTPHComp- AtomicOpsCap: 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, AtomicOpsCtl: ReqEn- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [ac] MSI-X: Enable+ Count=16 Masked- Vector table: BAR=0 offset=001c2000 PBA: BAR=0 offset=001c4000 Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol- UESvrt: DLP+ SDES+ TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 Kernel driver in use: hpsa Kernel modules: hpsa |
lspci -s 06:00.0 -vv 06:00.0 RAID bus controller: Hewlett-Packard Company Smart Array Gen9 Controllers (rev 01) Subsystem: Hewlett-Packard Company H240 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin A routed to IRQ 57 Region 0: Memory at fbd00000 (64-bit, non-prefetchable) [size=1M] Region 2: Memory at fbcffc00 (64-bit, non-prefetchable) [size=1K] Region 4: I/O ports at d800 [size=256] Expansion ROM at fbc00000 [disabled] [size=512K] Capabilities: [80] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2-,D3hot+,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [90] MSI: Enable- Count=1/32 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 Capabilities: [b0] MSI-X: Enable+ Count=64 Masked- Vector table: BAR=0 offset=00002000 PBA: BAR=0 offset=00003000 Capabilities: [c0] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <4us, L1 <1us ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W DevCtl: CorrErr- NonFatalErr+ FatalErr+ UnsupReq- RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ MaxPayload 256 bytes, MaxReadReq 4096 bytes DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend- LnkCap: Port #0, Speed 8GT/s, Width x8, ASPM not supported ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s (downgraded), Width x8 (ok) TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range B, TimeoutDis+ NROPrPrP- LTR- 10BitTagComp- 10BitTagReq- OBFF Via message, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- TPHComp+ ExtTPHComp- AtomicOpsCap: 32bit+ 64bit+ 128bitCAS+ DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, AtomicOpsCtl: ReqEn- LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS- LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [100 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol- UESvrt: DLP+ SDES+ TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 Capabilities: [300 v1] Secondary PCI Express LnkCtl3: LnkEquIntrruptEn- PerformEqu- LaneErrStat: 0 Kernel driver in use: hpsa Kernel modules: hpsa |
|
Модуль ядра: |
Модуль ядра: |
Утилиты
Для обоих контроллеров можно использовать новую утилиту ssacli (Smart Storage Admin CLI),
которая заменяет более старые hpacucli и hpssacli, однако на момент
начала работы я этого не знал, и для P410 использовал более старую утилиту
|
|
|
|---|---|
|
Файл:Hpacucli 9.40.1-1. amd64.deb wget https://downloads.linux.hpe.com/SDR/repo/mcp/ubuntu/pool/non-free/hpacucli_9.40.1-1._amd64.deb |
Файл:Ssacli-6.15-11.0 amd64.deb wget https://downloads.linux.hpe.com/SDR/repo/mcp/ubuntu/pool/non-free/ssacli-6.15-11.0_amd64.deb |
| Текст ячейки | Текст ячейки |
| Текст ячейки | Текст ячейки |
Сокращения, которые можно использовать в командах
chassisname=chcontroller=ctrllogicaldrive=ldphysicaldrive=pddrivewritecache=dwc
Просмотр конфигурации
P410 |
! H240
|
|---|---|
hpacucli ctrl all show config detail |
ssacli ctrl all show config detail |
hpacucli ctrl all show config
Smart Array P410 in Slot 1 (sn: PACCR9SYRIHU )
unassigned
physicaldrive 2I:1:3 (port 2I:box 1:bay 3, SAS, 300 GB, OK)
SEP (Vendor ID PMCSIERA, Model SRC 8x6G) 250 (WWID: 5001438006116C2F)
|
ssacli ctrl all show config Smart HBA H240 in Slot 255 (RAID Mode) (sn: PDNNK0BRHAE1PT) Port Name: 1I Port Name: 2I |
Smart Array P410 in Slot 1
Bus Interface: PCI
Slot: 1
Serial Number: PACCR9SYRIHU
Cache Serial Number: PAAVP9SYYM4C
RAID 6 (ADG) Status: Disabled
Controller Status: OK
Hardware Revision: C
Firmware Version: 6.64
Wait for Cache Room: Disabled
Surface Analysis Inconsistency Notification: Disabled
Post Prompt Timeout: 0 secs
Cache Board Present: True
Cache Status: Not Configured
Drive Write Cache: Disabled
Total Cache Size: 512 MB
Total Cache Memory Available: 400 MB
No-Battery Write Cache: Disabled
Battery/Capacitor Count: 0
SATA NCQ Supported: True
unassigned
physicaldrive 2I:1:3
Port: 2I
Box: 1
Bay: 3
Status: OK
Drive Type: Unassigned Drive
Interface Type: SAS
Size: 300 GB
Rotational Speed: 10000
Firmware Revision: HPD7
Serial Number: 6XP5CLK80000B335CPL7
Model: HP EG0300FBLSE
PHY Count: 2
PHY Transfer Rate: 6.0Gbps, Unknown
SEP (Vendor ID PMCSIERA, Model SRC 8x6G) 250
Device Number: 250
Firmware Version: RevC
WWID: 5001438006116C2F
Vendor ID: PMCSIERA
Model: SRC 8x6G
|
ssacli ctrl all show config detail Smart HBA H240 in Slot 255 (RAID Mode) Bus Interface: PCI Slot: 255 Serial Number: PDNNK0BRHAE1PT Cache Serial Number: PDNNK0BRHAE1PT Controller Status: OK Hardware Revision: B Firmware Version: 6.30 Firmware Supports Online Firmware Activation: False Wait for Cache Room: Disabled Surface Analysis Inconsistency Notification: Disabled Post Prompt Timeout: 0 secs Cache Board Present: False Drive Write Cache: Disabled Controller Memory Size: 0.2 SATA NCQ Supported: True Spare Activation Mode: Activate on physical drive failure (default) Controller Temperature (C): 33 Number of Ports: 2 Internal only Encryption: Not Set Express Local Encryption: False Driver Name: hpsa Driver Version: 3.4.20 WWN Port: 5001438042CA2050 Driver Supports SSD Smart Path: True PCI Address (Domain:Bus:Device.Function): 0000:06:00.0 Negotiated PCIe Data Rate: PCIe 2.0 x8 (4000 MB/s) Controller Mode: RAID Mode Pending Controller Mode: RAID Controller Mode Reboot: Not Required Port Max Phy Rate Limiting Supported: False Latency Scheduler Setting: Disabled Current Power Mode: MaxPerformance Survival Mode: Enabled Host Serial Number: 2M2812021S Sanitize Erase Supported: True Primary Boot Volume: Unknown (600508B1001C2F75860BC680F0767397) Secondary Boot Volume: Unknown (600508B1001C2F75860BC680F0767397) SPDM Supports Get Slot Certificate Chain: no SPDM Supports Get Controller Info : no SPDM Supports Get Slot Info : no SPDM Supports Set Import Certificate : no SPDM Supports Set Invalidate Slot : no Surface Scan Completion Supported: False Persistent Event Log Policy Change Supported: False
|
Статус контроллера
P410 |
H240
|
|---|---|
hpacucli ctrl all show status |
ssacli ctrl all show status |
Smart Array P410 in Slot 1 Controller Status: OK Cache Status: Not Configured |
Smart HBA H240 in Slot 255 (RAID Mode) Controller Status: OK |
Управление кэшированием
P410 |
H240
|
|---|---|
hpacucli ctrl slot=1 modify dwc=disable
Error: This operation is not supported with the current configuration. Use the
"show" command on devices to show additional details about the
configuration.
Reason: Controller not configured
|
ssacli controller slot=255 modify dwc=disable |
hpacucli ctrl slot=1 modify dwc=enable
Warning: Without the proper safety precautions, use of write cache on physical
drives could cause data loss in the event of power failure. To ensure
data is properly protected, use redundant power supplies and
Uninterruptible Power Supplies. Also, if you have multiple storage
enclosures, all data should be mirrored across them. Use of this
feature is not recommended unless these precautions are followed.
Continue? (y/n)
У меня не включился, думаю из-за Battery/Capacitor Count: 0 |
ssacli controller slot=255 modify dwc=enable
Warning: Without the proper safety precautions, use of write cache on physical
drives could cause data loss in the event of power failure. To ensure
data is properly protected, use redundant power supplies and
Uninterruptible Power Supplies. Also, if you have multiple storage
enclosures, all data should be mirrored across them. Use of this
feature is not recommended unless these precautions are followed.
Continue? (y/n) y
Без батареи но включилось: ssacli controller slot=255 show detail | grep 'Drive Write Cache' Drive Write Cache: Enabled |
Пересканировать устройства
P410 |
H240
|
|---|---|
hpacucli rescandetects newly added devices since the last rescan В моем случае ни к чему не привело - диск нашелся сам по себе |
ssacli rescan Мне не помогло, изначально диски не видились, не ясно был ли плохо вставлен кабель или помогло отключение и включение питания (изначально диски были вставлены на-горячую) |
Просмотр информации о дисках
P410 |
H240
|
|---|---|
|
Все диски: hpacucli controller slot=1 pd all show status physicaldrive 2I:1:3 (port 2I:box 1:bay 3, 300 GB): OK |
Все диски: ssacli controller slot=255 pd all show status physicaldrive 1I:0:7 (port 1I:box 0:bay 7, 300 GB): OK physicaldrive 1I:0:6 (port 1I:box 0:bay 6, 200 GB): OK physicaldrive 1I:0:8 (port 1I:box 0:bay 8, 300 GB): OK |
hpacucli ctrl slot=1 pd 2I:1:3 show detail |
ssacli controller slot=255 pd 1I:0:7 show detail |
Smart Array P410 in Slot 1
unassigned
physicaldrive 2I:1:3
Port: 2I
Box: 1
Bay: 3
Status: OK
Drive Type: Unassigned Drive
Interface Type: SAS
Size: 300 GB
Rotational Speed: 10000
Firmware Revision: HPD7
Serial Number: 6XP5CLK80000B335CPL7
Model: HP EG0300FBLSE
PHY Count: 2
PHY Transfer Rate: 6.0Gbps, Unknown
Во многих примерах приводят |
Smart HBA H240 in Slot 255
Array A
physicaldrive 1I:0:7
Port: 1I
Box: 0
Bay: 7
Status: OK
Drive Type: Data Drive
Interface Type: SAS
Size: 300 GB
Drive exposed to OS: False
Logical/Physical Block Size: 512/512
Rotational Speed: 10000
Firmware Revision: HPD7
Serial Number: 6XP5CLK80000B335CPL7
WWID: 5000C5005FBBAA91
Model: HP EG0300FBLSE
Current Temperature (C): 31
Maximum Temperature (C): 58
PHY Count: 2
PHY Transfer Rate: 6.0Gbps, Unknown
PHY Physical Link Rate: Unknown, Unknown
PHY Maximum Link Rate: Unknown, Unknown
Drive Authentication Status: Not Applicable
Sanitize Erase Supported: False
Shingled Magnetic Recording Support: None
|
|
physicaldrive 2I:1:3
Port: 2I
Box: 1
Bay: 3
|
Сам адрес диска, если он не известен можно посмотреть в physicaldrive 1I:0:7
Port: 1I
Box: 0
Bay: 7
|
Очистка
Перед очисткой, если диск находится в логической группе, его нужно вывести или удалить группу, например так:
ssacli ctrl slot=255 ld 1 delete
P410 |
H240
|
|---|---|
hpacucli ctrl slot=1 pd 2I:1:3 modify erase |
ssacli ctrl slot=255 pd 1I:0:7 modify erase |
Управление диодами на дисках
P410 |
H240
|
|---|---|
hpacucli ctrl slot=1 pd 2I:1:3 modify led=on |
ssacli ctrl slot=255 pd 1I:0:7 modify led=on |
hpacucli ctrl slot=1 pd 2I:1:3 modify led=off |
ssacli ctrl slot=255 pd 1I:0:7 modify led=off
|
Показать информацию (детально)
P410 |
H240
|
|---|---|
hpacucli ctrl slot=1 ld all show detail Error: The specified device does not have any logical drives. |
ssacli ctrl slot=255 ld all show detail Error: The specified device does not have any logical drives. |
Логических дисков пока нет
Создание дисков
Нельзя добавлять диски в процессе очистки, по-тому далее вместо 1I:0:7 использую 1I:0:8
P410 |
H240
|
|---|---|
|
логический диск (отдельный диск). Команда не возвращает вывод, результат смотреть отдельно. hpacucli ctrl slot=1 create type=ld drives=2I:1:3 raid=0 |
ssacli ctrl slot=255 create type=ld drives=1I:0:8 raid=0 |
|
Просмотреть что вышло? hpacucli ctrl slot=1 ld all show detail
Smart Array P410 in Slot 1
array A
Logical Drive: 1
Size: 279.4 GB
Fault Tolerance: 0
Heads: 255
Sectors Per Track: 32
Cylinders: 65535
Strip Size: 256 KB
Full Stripe Size: 256 KB
Status: OK
Caching: Enabled
Unique Identifier: 600508B1001C8823B6E46FDAC58243EC
Disk Name: /dev/sdb
Mount Points: None
Logical Drive Label: A7044617PACCR9SYRIHUC7C6
Drive Type: Data
|
Текст ячейки |
dmesg -T | tail -10 [Sun Sep 3 11:34:03 2023] loop7: detected capacity change from 0 to 8 [Sun Sep 3 16:36:18 2023] hpsa 0000:09:00.0: scsi 6:0:2:0: masked Direct-Access HP EG0300FBLSE PHYS DRV SSDSmartPathCap- En- Exp=0 [Mon Sep 4 08:48:25 2023] hpsa 0000:09:00.0: scsi 6:1:0:0: added Direct-Access HP LOGICAL VOLUME RAID-0 SSDSmartPathCap- En- Exp=1 [Mon Sep 4 08:48:25 2023] scsi 6:1:0:0: Direct-Access HP LOGICAL VOLUME 6.64 PQ: 0 ANSI: 5 [Mon Sep 4 08:48:25 2023] sd 6:1:0:0: [sdb] 585871964 512-byte logical blocks: (300 GB/279 GiB) [Mon Sep 4 08:48:25 2023] sd 6:1:0:0: [sdb] Write Protect is off [Mon Sep 4 08:48:25 2023] sd 6:1:0:0: [sdb] Mode Sense: 6b 00 00 08 [Mon Sep 4 08:48:25 2023] sd 6:1:0:0: Attached scsi generic sg2 type 0 [Mon Sep 4 08:48:25 2023] sd 6:1:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA [Mon Sep 4 08:48:25 2023] sd 6:1:0:0: [sdb] Attached SCSI disk
|
Текст ячейки |
|
Текст ячейки |
Текст ячейки
|
smartctl
P410 |
H240
|
|---|---|
smartctl -x /dev/sdb -d cciss,0 --all |
Текст ячейки 2
|
|
|
Текст ячейки 2
|
|
Текст ячейки |
Текст ячейки 2
|
|
Текст ячейки |
Текст ячейки 2
|
