iPXE Chooses Wrong Nework Interface

[necouchman]

So, I'm using iPXE with some Dell servers that have on-board Broadcom interfaces. Each of these systems has two on-board 5708S interfaces, one of which is connected to a DHCP-enabled network that should allow PXE booting. This works fine with the built-in management firmware.

I've generated the iPXE image for these network cards, and successfully used the uxdiag utility (replacement for b57udiag) to burn the iPXE ROM to the MBA space on the network card (apparently with uxdiag, mba replaces all of the pxe commands). However, I'm running into a problem with the boot. The iPXE ROM is loading from the correct Ethernet interface (I can see this by verifying the PCI ID that it uses), but it is configuring the wrong interface. On this particular server, the second Broadcom interface is the one that is DHCP-enabled, but it seems like iPXE is choosing either the first MAC address or the first PCI ID that it sees - not really sure which one. So, IPXE starts from the correct PCI card, but tries to boot from the other interface.

Anyone ever run into this before? Any way to configure the iPXE firmware that I've just burned to make sure it activates the correct interface?

-Nick

[robinsmidsrod]

Have you verified that indeed only net0 is being initialized and not net1 as well? ifstat net0 and ifstat net1 should tell you if they're both configured. You can replace the built-in script to do dhcp net0 followed by dhcp net1, but this shouldn't be required, as far as I know. Is a DHCP server running on both networks?

Which driver is the 5708s driven by? If it is the tg3, then that one is old (unless mcb30 has committed the new tg3 driver) and doesn't run very well (very slow). The other driver for broadcom cards (can't remember the name) I know nothing about.

Have you tried using the latest vendor PXE ROM and using chainloaded undionly.kpxe to drive it, or does that not work for your setup?

[necouchman]

(2011-10-12 10:01)robinsmidsrod Wrote:  Have you verified that indeed only net0 is being initialized and not net1 as well? ifstat net0 and ifstat net1 should tell you if they're both configured. You can replace the built-in script to do dhcp net0 followed by dhcp net1, but this shouldn't be required, as far as I know. Is a DHCP server running on both networks?

Which driver is the 5708s driven by? If it is the tg3, then that one is old (unless mcb30 has committed the new tg3 driver) and doesn't run very well (very slow). The other driver for broadcom cards (can't remember the name) I know nothing about.

Have you tried using the latest vendor PXE ROM and using chainloaded undionly.kpxe to drive it, or does that not work for your setup?

Yes, I have verified that net0 is the only card initialized - if I drop to the command line and attempt commands like "dhcp net1" I get errors that the card does not exist.

And, yes, if I chainload iPXE, it seems to work fine, so I can go that route, for now, but I may have a situation, soon, where I need to boot from a network that does not have a DHCP server, so I'd much rather have a working one burned into the cards.

After posting this message I went back and did even more digging, and I think I found out what at least one of the problems is. This particular chipset uses the bnx2 driver, which still uses the legacy etherboot API (there's even a warning at boot time about it). I seem to remember there being some issues with that API that caused some of these problems.

I know this is more a question for the developers list/forum, but I'll ask, anyway - what are the key differences in a driver between the two APIs - that is, if I were to look into switching the bnx2 driver away from the legacy API, what would need to change?

Thanks - Nick

[robinsmidsrod]

I don't know enough about the differences between the driver APIs, unfortunately. Hopefully someone else can help you out. If you don't hear anything in a couple of days I'd suggest you try the mailing list.