Winboot failed to load WinPE greater than 600MB on Dell R640

[vpnchoudhary]

Hi -

I am getting below error on dell R640. The machine has 170 GB RAM. So memory is not an issue. It appears that on this hardware Wimboot can only work with 1GB of address space.

The exact error is "There isn't enough memory available to create ramdisk device."
Status: 0xc000007

The wimboot info is captured before it hand over control to bootmgr.
https://1drv.ms/u/s!Ag6Nr700xzwc-HrQjwrg...2?e=CeWpWd

The top two screen shoots are from R640. Left hand side is trying to boot from a 650MB winpe whereas right hand side successfully booted from a 580 mb Winpe.
The bottom image is from a different sku of dell server with larger Winpe Image and it worked fine.

My guess based on looking at address space is that for larger winpe, the address space of bootmgr will overlap with the address space where BCD is stored. Hence the error.

R640->ipxe(wimboot)->win image 650MB | bootmgr start at 0x434a0 + wim (28e25c38) = 28E6 90D8 ->BCD starting point (25784074)

R640->ipxe(wimboot)->win image 580MB | bootmgr start at 0x434a0 + wim (243afee8) = 243F 3388 ->BCD starting point (2a23e074)

I can be completely off here but it does appear that Wimboot is not using the ram space effectively.

I am using the latest version of wimboot (2.6.0), I also tried with previous version but same result.

Any pointers on how to fix this issue ?

[simon]

A similar issue recently popped up on IRC, when trying to load an initramfs > 1G. It was due to the fact that ipxe can only access memory below 4GB in BIOS mode, and the BIOS will map random stuff there too, sometimes in a really fragmented manner that you don't get a nice large chunk of continuous memory, which is required. Maybe it's similar with wimboot?

I'm lazy and just copy & paste the relevant parts of the conversation:

Quote:<phibs> Anyone ever seen issues on certain nics PXE booting where the initrd is > 1G?
<phibs> I get: No space left on device (http://ipxe.org/34182006) on an Intel XXV710
...
<mcb30> phibs: with the BIOS builds (even the 64-bit BIOS builds) you are limited to loading images into the low 4G of memory
<phibs> yeah the image is only 1.2G though
<phibs> is this a BIOS / Vendor issue?
<phibs> seems to only be an issue on this new platform we are testing
<mcb30> Parts of the low 4G of address space will be reserved for PCI BARs, which means that it's not atypical for a BIOS to place only e.g. 2G of actual RAM within the low 4G of address space
<phibs> ahhhhhh
<mcb30> You can check the E820 memory map as reported by e.g. the Linux kernel log
<mcb30> (or by iPXE if you build with DEBUG=memmap)
<phibs> any idea on if it is fixable, or if I have to shrink my initrd?
<phibs> also thanks!
<mcb30> You'll need to shrink it or switch to UEFI
...
<phibs> mcb30: a weird thing is, when I enable above 4G decoding in the BIOS, it won't PXE at all!
<phibs> I get this lovely one
<phibs> https://img.ethernet.cards/screenshots/u...C7B964.png
...
<mcb30> phibs: that error makes some sense. If the PCI BAR is mapped above 4G then the PXE stack needs to be able to switch into 64-bit long mode to access it. This is why the 64-bit BIOS build of iPXE exists. It's very unlikely that Intel's PXE ROM can do that.
<phibs> mcb30: hmmmm some hosts won't boot without that enabled, this host won't boot with it enabled
<phibs> so the 64 bit build WILL work w/ that ?
<mcb30> phibs: the 64-bit BIOS build will be able to drive the NIC (assuming you're using ipxe.* or intelxl.* rather than undionly.kpxe), but (a) you'll still need a way to get iPXE loaded in the first place, (b) it will still be limited to whatever RAM the BIOS chooses to place below 4G
<mcb30> phibs: it's possible that if you place the PCI BARs above 4G then the BIOS would place more RAM below 4G, but that's not guaranteed - you'd have to test it
<phibs> undionly.kpxe
<phibs> I am very glad you are well versed into the terribleness that is x86 ;0
<mcb30> undionly.kpxe relies on the underlying PXE ROM's driver, so won't work. However, if you try booting from ipxe.usb (on a USB key), you should find that iPXE is able to detect and drive the NIC