DELL PERC H200 SAS controller is a cheap card that can be purchased from ebay for 25USD. It can be flashed with dummy HBA firmware (also known as IT mode) which disables all hardware RAID functionality (also known as IR mode) and exposes simple SAS controller which is exactly what you want when you're running software RAID, Btrfs or ZFS.
DELL R710 is shipped with DELL PERC H700 card which can't be flashed with such firmware and vanilla LSI card does not work in the DELL integrated storage slot resulting in "Invalid PCIe card in storage slot" error during POST.
Now DELL supports DELL PERC H200 "integarated" card in R710, but flashing that card by the usual instructions would revert PCI vendor/product ID-s to LSI but read on below, this can be fixed.
Many instructions can be found online.
The easiest way is to download [https://nextcloud.k-space.ee/s/XGNw5ATkCBppkwc LSI-9211-8i.zip] ([https://d.arti.ee/LSI-9211-8i.zip mirror]) which contains a) EFI shell which is necessary to use EFI binaries b) all the relevant firmware files
Following resets the hardware RAID config. Create yourself FreeDOS installation where you can run the exe files:
megarec -sbrwrite 0 sbrempty...
megarec -cleanflash 0
Now reboot to EFI shell, this can be accomplished by placing files in the .zip file to a FAT32 partition on a USB stick and booting it in a PC which supports booting from EFI.
Once EFI shell has booted up issue following to change working directory:
fs0:
And then proceed to flash 6GBPSAS firwmare:
sas2flash.efi -o -f 6gbpsas.fw
Reboot
Flash P7 version of the (IR mode?) firmware. This step is necessary because newer versions are not accepted by the previous firmware:
sas2flash.efi -o -f 2118p7.bin
Reboot now
Flash the IT mode firmware:
sas2flash.efi -o -f 2118it.bin
Program the SAS address, this is like MAC address of the card exposed on the SAS bus:
sas2flash.efi -o -sasadd 5c81f660e20a1e00
Flash the '''BIOS boot ROM''' to enable booting from this card (and disks attached to it):
sas2flash -o -b [https://d.arti.ee/mptsas2.rom mptsas2.rom]
Flash '''UEFI boot ROM'''
sas2flash -o -b x64sas2.rom
This is where the story would end if you wouldn't try to use it in a box with PCI vendor/product ID whitelisting like DELL R710 and possibly others.
DELL R710 won't cough if any of the cards marked in the manual are advertised, this includes PERC H200 Integrated card.
Digging through the internet gives you listing of the ID-s:
0072 SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]
1000 3040 9210-8i
1000 3080 9200-8e [LSI SAS 6Gb/s SAS/SATA PCIe x8 External HBA]
1000 30b0 9200-8e [LSI SAS 6Gb/s SAS/SATA PCIe x8 External HBA]
1028 1f1c 6Gbps SAS HBA Adapter
1028 1f1d PERC H200 Adapter
1028 1f1e PERC H200 Integrated
1028 1f1f PERC H200 Modular
1028 1f20 PERC H200 Embedded
1028 1f22 PERC H200 Internal Tape Adapter
8086 350f RMS2LL040 RAID Controller
8086 3700 SSD 910 Series
Now the way to proceed here is to extract the SBR - that's basically OEM information blob of 256 bytes including card operation mode, PCI vendor/product ID-s and much more. Extract the blob from the card.
Boot to FreeDOS and run:
megarec.exe -readsbr 0 lsi.sbr
Move the file to Linux box and run this Python script to patch it with new vendor/product ID:
# https://marcan.st/2016/05/crossflashing-the-fujitsu-d2607/
# http://blog.schmorp.de/2018-01-25-changing-the-pci-ids-on-lsi-20082108-based-megaraid-controllers.html
VENDOR = 0x1028
PRODUCT = 0x1F1E
import struct
with open("lsi.sbr", "rb") as fh:
buf = fh.read()
ov, = struct.unpack("H", buf[20:22])
op, = struct.unpack("H", buf[22:24])
assert buf[20:24] == buf[76+20:76+24]
assert sum([ord(j) for j in buf[0:76]]) % 0x100 == 0x5b
print "Old checksum is:", repr(buf[75])
print "Old vendor/product ID is %04x:%04x" % (ov, op)
newbuf = buf[0:20] + struct.pack("HH", VENDOR, PRODUCT) + buf[24:75]
with open("h200i-it.sbr", "wb") as fh:
fh.write(newbuf)
for i in range(0,255):
if (i + sum([ord(j) for j in newbuf[0:75]])) % 0x100 == 0x5b:
csum = i
print "New checksum is: %x" % csum
break
fh.write(struct.pack("B", csum))
fh.write(buf[76:76+20])
fh.write(struct.pack("HH", VENDOR, PRODUCT))
fh.write(buf[76+24:76+75])
fh.write(struct.pack("B", csum))
fh.write(buf[76+76:])
Move generated h200i-it.sbr back to FreeDOS installation and boot the machine, flash with:
megarec.exe -writesbr 0 h200i-it.sbr
So when you reboot the box between sas2flsh, megarec, etc executions you can't really brick the card even if the whole Internet tells otherwise.
Now a bad SBR could end with your card in some really messed up state where megarec.exe tells you "unable to open Diagnostic register" and sas2flash says it can't find any LSI SAS controllers.
Even in this case don't worry! megarec uses presumably PCI product/vendor ID to match the devices and that's why it won't see your badly flashed card. But if you know exact PCI address of the device you can still force flashing new SBR with lsirec.
This is a Linux program and if your Linux refuses to boot because it sees some really messed up PCI card you can tell Linux to disable loading driver for that card, eg add on boot prompt: modprobe.blacklist=mptsas modprobe.blacklist=megaraid_sas
With all these hacks DELL PERC H200 can be used in the R710's integrated storage slot, OS can be installed on it and OS can be booted as well.
This leaves remaining four PCIe slots for Intel Optane, external SAS card, Intel X520-DA2 card and FC card for pretty epic FreeNAS installation.
Some EMC storage boxes format disks in a such way (storage box metainformation 8 bytes per block?). With dummy SAS-es you would like to get back to "normal" sector sizes such as 512 or 4096 bytes per sector.
To list such disks in a multipath topology on FreeNAS:
gmultipath list | grep "Sectorsize: 520" -3 | grep -a1 Consume | grep Name |
Now double check the list that it looks alright!
To reformat a disk:
sg_format --format --six --size=512 /dev/da123