Qlogic HBA, MPI Heartbeat stop
SN1100Q와 같은 Qlogic HBA의 fw가 "01.73.08"인 상황에서, 아래 패턴의 로그와 함께 운영 중 예기치 않은 포트 다운 또는 리부팅이 발생되는 경우 HBA fw의 오류일 수 있음
Linux
kernel: qla2xxx [0000:11:00.0]-ffff:0: MPI Heartbeat stop. Chip reset needed. MB0[8200h] MB1[100h] MB2[0h] MB3[0h]
kernel: qla2xxx [0000:11:00.0]-d013:0: MPI Heartbeat stop. FW dump needed
kernel: qla2xxx [0000:11:00.0]-d011:0: -> fwdt0 running...
kernel: qla2xxx [0000:11:00.1]-ffff:2: MPI Heartbeat stop. Chip reset needed. MB0[8200h] MB1[0h] MB2[0h] MB3[0h]
kernel: qla2xxx [0000:11:00.1]-00af:2: Performing ISP error recovery - ha=ffff92febd43c000.
kernel: qla2xxx [0000:11:00.0]-d011:0: -> fwdt1 running...
kernel: qla2xxx [0000:11:00.0]-d015:0: -> Firmware dump saved to buffer (0/ffffafa3b3ff1000) <7>
kernel: qla2xxx [0000:11:00.0]-00af:0: Performing ISP error recovery - ha=ffff92feb53e4000.
kernel: qla2xxx [0000:11:00.1]-0075:2: ZIO mode 6 enabled; timer delay (200 us).
kernel: qla2xxx [0000:11:00.1]-ffff:2: FC4 priority set to NVMe
kernel: qla2xxx [0000:11:00.0]-0075:0: ZIO mode 6 enabled; timer delay (200 us).
kernel: qla2xxx [0000:11:00.0]-ffff:0: FC4 priority set to NVMe
VMware
cpu0:2955620)WARNING: qlnativefc: vmhba1(12:0.0): MPI Heartbeat stop. Chip reset needed. MB0[8200h] MB1[100h] MB2[0h] MB3[0h]
cpu0:2955620)WARNING: qlnativefc: vmhba1(12:0.0): MPI Heartbeat stop. FW dump needed
cpu0:2955620)qlnativefc: vmhba1(12:0.0): Attempting to take firmware dump
cpu16:2955621)WARNING: qlnativefc: vmhba2(12:0.1): MPI Heartbeat stop. Chip reset needed. MB0[8200h] MB1[0h] MB2[0h] MB3[0h]
cpu78:66253)qlnativefc: vmhba2(12:0.1): Inside qlnativefcAbortIsp
cpu78:66253)qlnativefc: vmhba2(12:0.1): Performing ISP error recovery - ha= 0x430811132010.
cpu8:66252)qlnativefc: vmhba2(12:0.1): qlnativefcUnregSess sess 0x43081113c7d0 for deletion 50:06:0e:80:12:3c:fd:02
Action Plan
What: 1) HBA fw를 1.75.07로 업그레이드
2) 운영 중인 OS에 맞춰 HBA driver 업그레이드 (VMW 환경에서 필수)
- VMw 6.5 drv 2.1.101.0 or higher
- VMw 6.7 drv 3.1.46.0 or higher
- VMw 7.0 drv 4.1.22.0 or higher
- RHEL 7.8~ drv 10.02.01.00.a14-k1 or higher
- RHEL 8.2~ drv 10.02.01.00.a14-k1 or higher
- SLES12 sp4~ drv 10.02.01.00.a14-k1 or higher
- SLES15 sp1 drv 10.02.01.00.a14-k1 or higher
Workaround: Cold reboot 후 포트가 다시 활성화 되나, 패치 전까지 증상은 잠재된 상태일 수 있음
