In Which I Savagely Impugn the Honor and Monoculture of IPMI & its Friends

IPMI+ Security Paper

Jan 19th, 2021 - small update

Spurred on by someone asking questions... it's been awhile since I've seriously looked at IPMI, but I poke at it from time to time... TLDR; not much has changed. Except....

- Redfish is a new OOB management standard and has come along as a potential successor to IPMI. Unfortunately, however, IPMI is still in every server... so instead of getting rid of it and replacing it with Redfish... you now have both Redfish and IPMI running in tandom. Adding additional attack surface and complexity to the guts of a server really isn't helping things out....

- Some vendors have a new, non-standard way (e.g. it isn't in the IPMI standard and I don't know of any who have published their algorithms/methods... what could go wrong?) around the RAKP problem of remotely leaking password hashes; Whether or not they've actually fixed the problem I don't know, but I suppose it's good that some of them at least tried to do something.

- Wikipedia had a pretty good writeup on some of the issues of IPMI security, but some years ago an anonymous contributer (sic) from a DC-area IP address took an ax to it and changed it to read as such:

Latest IPMI specification security improvements
However, this is only of historical value. The IPMI specification has been updated with RAKP+ and a stronger cipher that is computationally impractical to break. Vendors as a result have provided patches that remediate these vulnerabilities.[citation needed]

 

Citation needed indeed. There is no new magic RAKP+ that solves its problems (they added SHA256 support), there is no new spec (the last published in 2015) and there is no IPMI silver bullet; pretty much as bad as it was over a half-dozen years later. Indeed, some great research and discoveries continue to flow from the IPMI cesspool, including large numbers of security flaws (my personal fav is PCILeech, which can capture RAM using DMA over PCIe, just as I warned about back in the day.)

June 4th, 2014 - paper

A paper (11 pages) surveying IPMI and BMC security on the Internet; version 1.00 (June 4th, 2013):

Sold Down the River

A modestly lengthy paper (31 pages) on IPMI and BMC security; version 2.01 (August 12th, 2013):

IPMI: Freight Train to Hell, bloated director's cut.

- or -

IPMI: Express Train to Hell (one page, G-rated version; HTML/PDF;)

The one-page version is the express/single page/reader's digest one; it has various generalities I try to fully explain in the paper or supporting documents.

(Older material and first version of paper may be found here.)

Note #2. HD Moore put together a really fine set of methods to exploit various issues with IPMI. Required reading for some of the dangers. Dark times ahead (not because of his work ;))

Serious problem

Note #3. Zach Wikholm reported a nigh critical vulnerability (also reported last year, and I found about 30K then in a spot scan as well, but it's high time people started actually listening) in about a zillion and one (est :)) SuperMicro BMCs, as few as some interesting other problems. If you have a SM you really need to check this out. Spot checks reveal a LOT of vulnerable BMCs because of recovered passwords - for more see: Big Trouble in little BMC land

Kudos to Zach for finding these things, and Cari.net for supporting him.

Note #4. Facebook has put out OpenBMC, an interesting looking implementation that, in theory, may be placed on BMCs. Problematically most vendors (HP, Dell, IBM, etc.) won't let you install firmware that isn't signed by them... so you're out of luck. Plus, the low-level drivers and so on... who knows. I couldn't get it to build, myself, but let's remain hopeful. If anyone knows of (publically available) hardware that this will actually run on, drop me a line.

  1. Feedback
    1. Agree or no, feel free to drop me a line: zen @ either fish2.com or trouble.org.
    2. If you've done development/guts work with IPMI/BMCs and would care to chat, let me know!
  2. Q's, FAQs, WTFs
    1. IPMI Security Best Practices (Needs update with new version!)
    2. IPMI IFAQ (Infrequently (or never)) Asked Questions
  3. Tech Notes/Addendums
    1. A few methods to extract or capture an IPMI password
    2. Notes on breaking into/after IPMI stuff (work in progress)
  4. Misc paper additions
    1. Bibliography
    2. Test lab
  5. Very small bits of ipmi related software by me
    1. Tools and tidbits

Server Vendors

A very small assortment of server vendors, at least to provide some context; if you haven't heard of their version of IPMI, you've at least heard of them, assuming you're reading this on a computer. Most seem to simply give out the images, which contain the BMC's operating system and basic boot environment, but a few require a service contract or relationship with the vendor (which I didn't have.) I had access to the first 3 here, and some sketchy notes to the first 4; clicking the vendor icon to see more.

Vendor IPMI Flavor Latest Version BMC Flash Images
DelliDRAC (Integrated Dell Remote Access Card) iDRAC 7Anyone may download
Hewlett Packard iLO (Integrated Lights Out) iLO 4Anyone
Supermicro Supermicro Intelligent Management ? (IPMI 2.0) Anyone
LenovoIMM? IMM (legacy IBM?)Anyone
IBM IMM (Integrated Management Module) IMM2Requires service contract
Fujitsu iRMC (Integrated Remote Management Controller) iRMC S3Anyone
Oracle/Sun ILOM (Integrated Lights Out Manager) ILOM 3Requires service contract

Source for the S5520 Server Platforms - a dozen BMCs on various Intel boards - kudos to Intel! (Presumably they have others out there, I was sent this link.)

Firmware Vendors

Firmware Vendors - under the hood more vendors lurk; there are only a few places that make BMCs, or Baseboard Mgmt Controllers, the little computers that implement IPMI; it's often created by 3 or more different vendors - the chipmakers, the firmware software adder-onners, and a big vendor like IBM, Dell, HP, etc., which all have their own names for their flavor of IPMI. I've put up some notes on some of my findings when or if applicable. It's interesting to note the ubiquity of China in all of these.

Vendor Manufactured in...
Nuvoton Hong Kong and Shenzhen
Emulex ... at least some presence in Beijing and Shanghai
ATEN HQ in Taiwan, factories in Shenzhen.
Winbond Housed in Taiwan, also in Kunshan city, China.
Avocent Beijing and Guangzhou
ASPEED HQ'd in Taiwan, factories/subsidiary in China
Renesas Mainland China and Hong Kong