Tag Archives: PCI-Express

HP MicroServer Gen8: two PCIe, too furious

1 Reply

The HP ProLiant MicroServer Gen8 was quite the impressive budget server when it was released in 2013.

Behold the power of some CPU cores, a maximum of 16GB of DDR3 RAM, and 4 to ?? disks

Intel was dominating desktop CPUs, and a typical desktop or workstation motherboard of this era had a single PCIe 3.0 x16 slot, and then several PCIe 2.0 slots (e.g. x4 and x1). The MicroServer Gen8, due to its small size (slightly larger than ITX), was limited to only one PCIe 2.0 x16 slot. You also did not have the option to bifurcate the PCIe x16 slot, as is common in recent years.

At least, that is what everyone thought. This adventure started when I found a very interesting blog post about bifurcating the PCIe x16 slot on Intel socket 1155:

As mentioned above, I want to set 16=8+4+4, so the values of CFG[5] and CFG[6] should be both 0. So we can ground CFG[5] and CFG[6] by connecting N35M35 and L37M37. (VSS is GND, and VCC is PWR)

This information is from the Desktop 3rd Generation Intel® Core™ Processor Family, Desktop Intel® Pentium® Processor Family, and Desktop Intel® Celeron® Processor Family datasheet:

Desktop 3rd Generation Intel® Core™ Processor Family, Desktop Intel® Pentium® Processor Family, and Desktop Intel® Celeron® Processor Family

The author found that by physically bridging the pads corresponding to CFG[5] and CFG[6] they could force the PCIe x16 slot to bifurcate into x8x4x4. Since there is no official support for PCIe bifurcation on the Ivy Bridge generation, this was an interesting finding and one which I feel went largely unnoticed.

The author used silver paste, which is more permanent than what I wanted. I decided to go a more reversible route: using kapton tape and very thin strips of aluminum foil (precision cut with Fiskars scissors from a roll of baking foil).

LGA1155 CPU modified for x8x4x4 bifurcation

It is important to note that by default CFG[5] and CFG[6] are pulled up to 1. Masking the pads with kapton tape is ineffective, they must be grounded to an adjacent GND pad.

WARNING: You can cause permanent damage to your CPU and motherboard by bridging incorrect pads. Always verify that the pads you have bridged are correct before installing the CPU into the socket.

So, I obtained HP MicroServer Gen8 and got to work testing this.

Bridging both CFG[5] and CFG[6] for x8x4x4 resulted in the system hanging indefinitely during the “Processor Initialization Complete” step in POST.

HP MicroServer Gen8 BIOS POST screen

Boot stuck at “Processor Initialization Complete”

Bridging only CFG[5] to set bifurcation to x8x8 did work, and the system passed POST.

LGA1155 CPU modified for x8x8 bifurcation

With this modification, you can install two PCIe devices in the PCIe x16 slot of the MicroServer Gen8. Unfortunately due to physical constraints in the chassis, it is not possible to install a half height card on top of a half height riser (e.g. x8 and dual NVMe) but you can install a dual or quad NVMe adapter and use two PCIe x4 lanes (e.g. for an M.2 SATA controller and an M.2 2.5/5G NIC). If you do not mind PCIe cards hanging out the side, you can use a dual x8 riser like the one pictured below.

HP MicroServer Gen8 with x8x8 riser installed

Here is the lspci output of two ConnectX-3 Pro 40G network cards installed in a MicroServer Gen8:

# lspci
00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)
00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)
00:06.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)
00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05)
00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5)
00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5 (rev b5)
00:1c.6 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 7 (rev b5)
00:1c.7 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 8 (rev b5)
00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a5)
00:1f.0 ISA bridge: Intel Corporation C204 Chipset LPC Controller (rev 05)
00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family 6 port Desktop SATA AHCI Controller (rev 05)
01:00.0 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Slave Instrumentation & System Support (rev 05)
01:00.1 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200EH
01:00.2 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Management Processor Support and Messaging (rev 05)
01:00.4 USB controller: Hewlett-Packard Company Integrated Lights-Out Standard Virtual USB Controller (rev 02)
03:00.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 Gigabit Ethernet PCIe
03:00.1 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 Gigabit Ethernet PCIe
04:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller (rev 03)
07:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
0a:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]

Of course the MicroServer Gen8 is very under powered and can only dream of saturating even a single 40G link.

# lspci -s 07:00.0 -vvv
07:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
        Subsystem: Hewlett-Packard Company InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+FLR-QSFP Adapter
        Physical Slot: 1
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 16
        Region 0: Memory at fbe00000 (64-bit, non-prefetchable) [size=1M]
        Region 2: Memory at f6000000 (64-bit, prefetchable) [size=32M]
        (...)
        Capabilities: [60] Express (v2) Endpoint, IntMsgNum 0
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <64ns, L1 unlimited
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 116W TEE-IO-
                DevCtl: CorrErr- NonFatalErr+ FatalErr+ UnsupReq-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- FLReset-
                        MaxPayload 256 bytes, MaxReadReq 4096 bytes
                DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend-
                LnkCap: Port #8, Speed 8GT/s, Width x8, ASPM L0s, Exit Latency L0s unlimited
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s (downgraded), Width x8
                        TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
                         10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
                         EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
                         FRS- TPHComp- ExtTPHComp-
                         AtomicOpsCap: 32bit- 64bit- 128bitCAS-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
                         AtomicOpsCtl: ReqEn-
                         IDOReq- IDOCompl- LTR- EmergencyPowerReductionReq-
                         10BitTagReq- OBFF Disabled, EETLPPrefixBlk-
                LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
                LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
                         EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
                         Retimer- 2Retimers- CrosslinkRes: unsupported
        (...)
        Kernel driver in use: mlx4_core
        Kernel modules: mlx4_core
# lspci -s 0a:00.0 -vvv
0a:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
        Subsystem: Hewlett-Packard Company InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+FLR-QSFP Adapter
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 16
        Region 0: Memory at fbf00000 (64-bit, non-prefetchable) [size=1M]
        Region 2: Memory at f8000000 (64-bit, prefetchable) [size=32M]
        (...)
        Capabilities: [60] Express (v2) Endpoint, IntMsgNum 0
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <64ns, L1 unlimited
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 116W TEE-IO-
                DevCtl: CorrErr- NonFatalErr+ FatalErr+ UnsupReq-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- FLReset-
                        MaxPayload 128 bytes, MaxReadReq 4096 bytes
                DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
                LnkCap: Port #8, Speed 8GT/s, Width x8, ASPM L0s, Exit Latency L0s unlimited
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s (downgraded), Width x8
                        TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
                         10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
                         EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
                         FRS- TPHComp- ExtTPHComp-
                         AtomicOpsCap: 32bit- 64bit- 128bitCAS-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
                         AtomicOpsCtl: ReqEn-
                         IDOReq- IDOCompl- LTR- EmergencyPowerReductionReq-
                         10BitTagReq- OBFF Disabled, EETLPPrefixBlk-
                LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
                LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
                         EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
                         Retimer- 2Retimers- CrosslinkRes: unsupported
        (...)
        Kernel driver in use: mlx4_core
        Kernel modules: mlx4_core

To anyone still running a MicroServer Gen8 and would like more expansion, I hope you find this information useful.

Gigabyte MJ11-EC1 PCIe Bifurcation

Leave a reply

The Gigabyte MJ11-EC1 motherboard is an ITX motherboard with an AMD EPYC 3151 (4C8T) onboard. These motherboards were being liquidated from the Gigabyte G431-MM0 GPU server in 2023, and could be purchased for around 60 Euros in the EU. The bare-bones G431-MM0 can still be purchased for around 170 Euros.

The MJ11-EC1 is very similar to the Gigabyte MJ11-EC0 with the main difference being the MJ11-EC0 has a PCIe x16 slot while the MJ11-EC1 has a SlimSAS (SFF-8654 8i) connector for use with the GPU riser in the G431-MM0.

You can purchase the SlimSAS cable (~18 Euros) and a PCIe riser (~15 Euros) from several AliExpress sellers. The added cost of the cable and PCIe riser does reduce the value proposition of the motherboard somewhat. Additionally, user testing showed that PCIe bifurcation was non-functional on the SlimSAS port, meaning only a single PCIe device could be recognized on the SlimSAS 8i port unless a PCIe switch was used.

However, I can demonstrate that bifurcation does work on the MJ11-EC1, and in fact it is possible to access all PCIe x16 lanes if you add the unpopulated U2_1 SFF-8654 connector. All the passive components for the SFF-8654 connector are already present on the motherboard, so only the physical connector needs to be added to unlock an additional 8 lanes of PCIe.

Two MJ11-EC1 motherboards, one with U2_1 unpopulated and one with a SlimSAS 8i connector soldered

However, PCIe Bifurcation does not work under every condition. The following scenarios do not work:

Cable Adapter Bifurcation working
SlimSAS 8i to dual 4i SlimSAS 4i to PCIe x4 No
SlimSAS 8i to 8i SlimSAS 8i to Dual NVMe No
Dual SlimSAS 8i to 8i PCIe x16 (JHHP1B) No

The first attempt was with a SlimSAS 8i to dual 4i cable. Unfortunately, bifurcation did not work with this cable, only one device was visible.

Only one device is recognized

The second attempt was with a SlimSAS 8i to dual NVMe adapter. Again, only the first NVMe device was visible, so I do not recommend purchasing this for use with the MJ11-EC1.

Only one device is recognized

I then tried this dual SlimSAS 8i to PCIe x16 adapter, which did not work at all. In my subsequent discussion with the vendor in the AliExpress dispute, it appears this adapter is only compatible with their PCIe x16 to SlimSAS riser. So despite using the SFF-8654 connector, it is not standards compliant with SlimSAS 8i and cannot be used with the MJ11-EC1. Do not purchase this.

This adapter does not work at all. Avoid purchasing the “2 Port SlimSAS 8i x2 to PCIe 4.0 x16 Slot Adapter Card SFF8654 Riser Card GEN4”

The following combinations are fully functional:

Cable Adapter BIOS configuration PCIe devices Bifurcation working
SlimSAS 8i to 8i PCIe x8 x8x8 2 Yes
SlimSAS 8i to 8i PCIe x8 with 4xNVMe riser, 2xNVMe x8x4x4 3 Yes
SlimSAS 8i to 8i PCIe x8 with 4xNVMe riser, 2xNVMe x4x4x4x4 4 Yes

The following SlimSAS 8i to PCIe x8 adapters were used during testing and worked as expected. The adapters were purchased with my own funds and I have no relationship to the brands or sellers.

CEACENT CNS41CX16W

The CEACENT CNS41CX16W places a decoupling capacitor (C21) in the path of the power connector. Cover this in glue/epoxy or it will get knocked off the board.

“SFF-8654 8i to PCIe 4.0 x16 External Graphics Card Adapter SFF-8654 8i Adapter Card” N-P548-A

The designer of this adapter does not seem to have considered the lack of clearance between the SFF-8654 and SATA power connector. I would call it “challenging” to plug in the SFF-8654 connector.

Most SFF-8654 to PCIe x8 adapters from China seem to have fundamentally flawed physical layouts, which is unfortunate given they are otherwise inexpensive and effective.

This NVMe adapter is great, my only wish is that they made a 1U compatible 2 NVMe version as inexpensive as the 4 NVMe model.

There are dual SlimSAS 8i to PCIe x16 adapters available, however they are cost prohibitive. Given the PCIe bifurcation options available in BIOS and the fact that there are 16 accessible PCIe lanes, I suspect a standards-compliant adapter (e.g. Ceacent CNS52CX16R) would work to expose all 16 lanes.

MJ11-EC1 with two PCIe x8 adapters; HP 544FLR-QSFP installed and BIOS configured for x8x8 bifurcation:

05:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
06:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
(...)
# lspci -s 05:00.0 -vvv
05:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
        Subsystem: Hewlett-Packard Company InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+FLR-QSFP Adapter
(...)
                LnkCap: Port #8, Speed 8GT/s, Width x8, ASPM L0s, Exit Latency L0s unlimited
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 8GT/s, Width x8
                        TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
# lspci -s 06:00.0 -vvv
06:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
        Subsystem: Hewlett-Packard Company InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+FLR-QSFP Adapter
(...)
                LnkCap: Port #8, Speed 8GT/s, Width x8, ASPM L0s, Exit Latency L0s unlimited
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 8GT/s, Width x8
                        TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

MJ11-EC1 with two PCIe x8 adapters, each with 4xNVMe adapters, NVMe 1 and 2 sockets populated; BIOS configured for x4x4x4x4 bifurcation:

05:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller 980 (DRAM-less)
06:00.0 Non-Volatile memory controller: KIOXIA Corporation NVMe SSD Controller BG4 (DRAM-less)
07:00.0 Non-Volatile memory controller: SK hynix 960GB TLC PCIe Gen3 x4 NVMe M.2 22110
08:00.0 Non-Volatile memory controller: Sandisk Corp WD PC SN810 / Black SN850 NVMe SSD (rev 01)
(...)
# lspci -s 05:00.0 -vvv
05:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller 980 (DRAM-less) (prog-if 02 [NVM Express])
        Subsystem: Samsung Electronics Co Ltd Device a801
(...)
                LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
                        ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 8GT/s, Width x4
                        TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
# lspci -s 06:00.0 -vvv
06:00.0 Non-Volatile memory controller: KIOXIA Corporation NVMe SSD Controller BG4 (DRAM-less) (prog-if 02 [NVM Express])
        Subsystem: KIOXIA Corporation NVMe SSD Controller BG4 (DRAM-less)
(...)
                LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <32us
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 8GT/s, Width x4
                        TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
# lspci -s 07:00.0 -vvv
07:00.0 Non-Volatile memory controller: SK hynix 960GB TLC PCIe Gen3 x4 NVMe M.2 22110 (prog-if 02 [NVM Express])
        Subsystem: SK hynix Device 0000
(...)
                LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM not supported
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 8GT/s, Width x4
                        TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
# lspci -s 08:00.0 -vvv
08:00.0 Non-Volatile memory controller: Sandisk Corp WD PC SN810 / Black SN850 NVMe SSD (rev 01) (prog-if 02 [NVM Express])
        Subsystem: Sandisk Corp WD PC SN810 / Black SN850 NVMe SSD
(...)
                LnkCap: Port #0, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <8us
                        ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 8GT/s (downgraded), Width x4
                        TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

This was a fun modification but the economic case is dubious at best. With the SFF-8654 8i cables being roughly 18 Euros each, and SlimSAS 8i to PCIe x8 adapters ranging in price from 18-22 Euros (+shipping) the additional cost to fully utilise 16 PCIe lanes easily exceeds the entire cost of the motherboard.

The SFF-8654 connector (Amphenol U10-B074-200T) proved very hard to source from Western Distributors as they are discontinued, and are not available at a reasonable price on AliExpress. I ended up purchasing them on Taobao via a Chinese based forwarding service. The cost for QTY 20 was 523¥ on Taobao, plus forwarding agent fees, shipping to Europe, and VAT.

Soldering the connector is also a nightmare. The ground plane on the MJ11-EC1 is very effective at dissipating heat. I used a pre-heater, hot air station (set to 200C with high flow to avoid melting the plastic), leaded solder, sticky flux, and still required some touch up work with a very fine tip on the soldering iron to fix bad connections.

It should be noted that bifurcation of the populated U2_2 SFF-8654 port works. Anyone owning the MJ11-EC1 wishing to do that just needs to flash the MJ11-EC0 BIOS via the BMC to expose PCIe bifurcation settings in BIOS, and they should be able to install two PCIe x4 devices (subject to the limitations mentioned above regarding cables/risers).