H200

From k-space wiki
Jump to: navigation, search

DELL PERC H200 SAS controller is a cheap card that can be purchased from ebay for 25USD. It can be flashed with dummy HBA firmware (also known as IT mode) which disables all hardware RAID functionality (also known as IR mode) and exposes simple SAS controller which is exactly what you want when you're running software RAID, Btrfs or ZFS. DELL R710 is shipped with DELL PERC H700 card which can't be flashed with such firmware and vanilla LSI card does not work in the DELL integrated storage slot resulting in "Invalid PCIe card in storage slot" error during POST.

Now DELL supports DELL PERC H200 "integarated" card in R710, but flashing that card by the usual instructions would revert PCI vendor/product ID-s to LSI but read on below, this can be fixed.



Installing dummy HBA firmware

Many instructions can be found online.

The easiest way is to download LSI-9211-8i.zip which contains a) EFI shell which is necessary to use EFI binaries b) all the relevant firmware files

Following resets the hardware RAID config. Create yourself FreeDOS installation where you can run the exe files:

   megarec -sbrwrite 0 sbrempty...
   megarec -cleanflash 0

Now reboot to EFI shell, this can be accomplished by placing files in the .zip file to a FAT32 partition on a USB stick and booting it in a PC which supports booting from EFI.

Once EFI shell has booted up issue following to change working directory:

 fs0:

And then proceed to flash 6GBPSAS firwmare:

 sas2flash.efi -o -f 6gbpsas.fw

Reboot

Flash P7 version of the (IR mode?) firmware. This step is necessary because newer versions are not accepted by the previous firmware:

 sas2flash.efi -o -f 2118p7.bin


Reboot now

Flash the IT mode firmware:

 sas2flash.efi -o -f 2118it.bin

Program the SAS address, this is like MAC address of the card exposed on the SAS bus:

 sas2flash.efi -o -sasadd 5c81f660e20a1e00

Flash the boot ROM to enable booting from this card (and disks attached to it):

 sas2flash -o -b mptsas2.rom

This is where the story would end if you wouldn't try to use it in a box with PCI vendor/product ID whitelisting like DELL R710 and possibly others.

Using card in DELL R710

DELL R710 won't cough if any of the cards marked in the manual are advertised, this includes PERC H200 Integrated card.

Digging through the internet gives you listing of the ID-s:

       0072  SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]
               1000 3040  9210-8i
               1000 3080  9200-8e [LSI SAS 6Gb/s SAS/SATA PCIe x8 External HBA]
               1000 30b0  9200-8e [LSI SAS 6Gb/s SAS/SATA PCIe x8 External HBA]
               1028 1f1c  6Gbps SAS HBA Adapter
               1028 1f1d  PERC H200 Adapter
               1028 1f1e  PERC H200 Integrated
               1028 1f1f  PERC H200 Modular
               1028 1f20  PERC H200 Embedded
               1028 1f22  PERC H200 Internal Tape Adapter
               8086 350f  RMS2LL040 RAID Controller
               8086 3700  SSD 910 Series


Now the way to proceed here is to extract the SBR - that's basically OEM information blob of 256 bytes including card operation mode, PCI vendor/product ID-s and much more. Extract the blob from the card.

Boot to FreeDOS and run:

 megarec.exe -readsbr 0 lsi.sbr

Move the file to Linux box and run this Python script to patch it with new vendor/product ID:

   # https://marcan.st/2016/05/crossflashing-the-fujitsu-d2607/
   # http://blog.schmorp.de/2018-01-25-changing-the-pci-ids-on-lsi-20082108-based-megaraid-controllers.html
   VENDOR = 0x1028
   PRODUCT = 0x1F1E
   import struct
   with open("lsi.sbr", "rb") as fh:
       buf = fh.read()
       ov, = struct.unpack("H", buf[20:22])
       op, = struct.unpack("H", buf[22:24])
       assert buf[20:24] == buf[76+20:76+24]    
       assert sum([ord(j) for j in buf[0:76]]) % 0x100 == 0x5b
       print "Old checksum is:", repr(buf[75])
       print "Old vendor/product ID is %04x:%04x" % (ov, op)
       newbuf = buf[0:20] + struct.pack("HH", VENDOR, PRODUCT) + buf[24:75]
       with open("h200i-it.sbr", "wb") as fh:
           fh.write(newbuf)
           for i in range(0,255):
               if (i + sum([ord(j) for j in newbuf[0:75]])) % 0x100 == 0x5b:
                   csum = i
                   print "New checksum is: %x" % csum
                   break
           fh.write(struct.pack("B", csum))
           fh.write(buf[76:76+20])
           fh.write(struct.pack("HH", VENDOR, PRODUCT))
           fh.write(buf[76+24:76+75])
           fh.write(struct.pack("B", csum))
           fh.write(buf[76+76:])

Move generated h200i-it.sbr back to FreeDOS installation and boot the machine, flash with:

 megarec.exe -writesbr 0 h200i-it.sbr


Debricking bad flash

So when you reboot the box between sas2flsh, megarec, etc executions you can't really brick the card even if the whole Internet tells otherwise. Now a bad SBR could end with your card in some really messed up state where megarec.exe tells you "unable to open Diagnostic register" and sas2flash says it can't find any LSI SAS controllers.

Even in this case don't worry! megarec uses presumably PCI product/vendor ID to match the devices and that's why it won't see your badly flashed card. But if you know exact PCI address of the device you can still force flashing new SBR with lsirec.

This is a Linux program and if your Linux refuses to boot because it sees some really messed up PCI card you can tell Linux to disable loading driver for that card, eg add on boot prompt: modprobe.blacklist=mptsas modprobe.blacklist=megaraid_sas

Summary

With all these hacks DELL PERC H200 can be used in the R710's integrated storage slot, OS can be installed on it and OS can be booted as well. This leaves remaining four PCIe slots for Intel Optane, external SAS card, Intel X520-DA2 card and FC card for pretty epic FreeNAS installation.

Appendix: reformatting 520 byte sector disks

Some EMC storage boxes format disks in a such way (storage box metainformation 8 bytes per block?). With dummy SAS-es you would like to get back to "normal" sector sizes such as 512 or 4096 bytes per sector.

To list such disks in a multipath topology on FreeNAS:

 gmultipath list | grep "Sectorsize: 520" -3 | grep -a1 Consume  | grep Name |      

Now double check the list that it looks alright!

To reformat a disk:

 sg_format --format --six --size=512 /dev/da123