Broadcom BCM5709 cards not working

[skibumatbu]

I'm having issues with some HP DL360 g7 servers with BCM5709 NICs in them.

I pulled the latest ROM from the iPXE rom-o-matic and when running on new hardware, it works great. But on the Broadcom NICs the iso doesn't see NICs at all. I found a few links in the forums but no real resolution:

http://forum.ipxe.org/showthread.php?tid=23 (Plans to support card Broadcom Net Extreme II - BCM5709 (14e4:1639)) but that one has no resolution.

There is someone who seems to have done work with this: http://ipxe.org/gsoc/bnx2 . Following their git repo I find this https://git.ipxe.org/people/dewyatt/ipxe...heads/bnx2 However, when building an iso from that tree the system hangs when I run ifopen on the NIC. By hang I mean that I run ifopen and then a few seconds later the shell seems to lock being unable to type anything in and nothing prints out.

So... Has anyone gotten these NIC's to work with iPXE? Should I use a different driver (if so how?)?

Thanks!

[SebastianRoth]

When pulling from rom-o-matic.eu add "bnx2,pci,netdevice" (without the quotes) in the field "Enable Debug". Or if you compile dewyatt's code then add "DEBUG=bnx2,pci,netdevice" to the make call (again without the quotes).

Then boot with this new binary and hopefully you will see some more useful debug output. Please take a picture and post here.

[skibumatbu]

I can't upload an attachment here, so I posted the photos to Imgur: http://imgur.com/a/MYUdg

This was using the dewyatt's tree. The ifopen doesn't generate much in the debug output, sadly. Is there a way to generate more debug for y'all to help see?

Thanks!

[SebastianRoth]

Can you please take a picture of the full output you get on screen. The blue and purple messages I see don't seam to be a huge problem.

What does ifstat show after the 'ifopen net0' or does if hang after ifopen? Are you sure net0 is where you have your cable connected? Tried others? Maybe add 'dhcp' to the debug and try to get an IP via the 'dhcp' command on the shell. Any errors you see?

Unfortunately there aren't many debug statements - DBGC(...) - in the bnx2 driver code. Maybe try adding some yourself and see what you get. I'd start in function bnx2_open on line 1173...

[skibumatbu]

What does ifstat show after the 'ifopen net0' or does if hang after ifopen?

Ifstat won't show anything... I tried to show it in the output (the lone "i" at the bottom of the console is where the system hung completely.

Are you sure net0 is where you have your cable connected? Tried others?

Good question... From a hardware perspective, the cable is in where net0 should be. But the interesting bit is that I can open and close all the other interfaces without hanging the device. But once I do net0 everything hangs. To me that means I am in the right port and that the issue is somewhere in the fact that we have a link. But I like your idea in adding debug lines. I'll give that a shot.

---

FYI... I added the debug statements all over bnx2_open and have more information. It appears that I'm failing AFTER the code leaves that function. It returns 0 and seems to hang in the calling function.

I'm going to look at DHCP next. I don't have DHCP on that subnet (which is why I'm using ipxe) but it will be interesting to see what happens.

---

Here is the latest screengrab: http://imgur.com/C2uhIky

I couldn't capture more up front because it scrolls off the screen too quickly.

One thing I notice is that looking at the C code, the ifopen only calls the netdev_open function... Given the output, the system hangs outside of the netdev and bnx2 open functions. Is there something going on in the background (checking links, etc). that I may not be catching?

[SebastianRoth]

(2016-03-22 14:43)skibumatbu Wrote:  I'm going to look at DHCP next. I don't have DHCP on that subnet (which is why I'm using ipxe) but it will be interesting to see what happens.

From what we see with your debugging statements added I'd say don't bother about DHCP right now. Well you might see a DHCP request packet leaving but I guess it will hang pretty soon after that anyway because 'dhcp' does just 'ifopen' as well!

I have to admit that I a still very new to the iPXE code and I don't know enough about that happens in the background. I'll try to find out but we might need some hint from some of the iPXE devs on this as well.

By the way... have you tried undionly/snponly yet? Legacy BIOS or UEFI??

[skibumatbu]

Thanks. lets hope a developer reads this and can take a look. Should I repost this in the developer forum/mailing list?

I am running in legacy bios mode. When I try undionly, I get a "Try to free Memory without Signature" error. I don't see alot of information on how to resolve that in google.

[SebastianRoth]

(2016-03-22 18:34)skibumatbu Wrote:  Thanks. lets hope a developer reads this and can take a look. Should I repost this in the developer forum/mailing list?

The developers are more active on the IRC channel I think. Give that a try if you like. But I am sure they will read this here as well.

(2016-03-22 18:34)skibumatbu Wrote:  I am running in legacy bios mode. When I try undionly, I get a "Try to free Memory without Signature" error. I don't see alot of information on how to resolve that in google.

Searching the web for that error I found this very intersting discussion: http://forum.ipxe.org/showthread.php?tid=7719 (as well read through the linked mailing list topic: http://lists.ipxe.org/pipermail/ipxe-dev...01703.html ) Unfortunately the gist is that UNDI won't work for the BCM5709 (14e4:1639) NIC...


Trying to investigate a little more you might want to add some debugging to see which drivers are being notified:

Code:

diff --git a/src/net/netdevice.c b/src/net/netdevice.c
index b834c3c..cf870f8 100644
--- a/src/net/netdevice.c
+++ b/src/net/netdevice.c
@@ -121,8 +121,10 @@ static void netdev_notify ( struct net_device *netdev ) {
        struct net_driver *driver;

        for_each_table_entry ( driver, NET_DRIVERS ) {
-               if ( driver->notify )
+               if ( driver->notify ) {
+                       DBGC( netdev , "Notifying '%s'\n" , driver->name );
                        driver->notify ( netdev );
+               }
        }
}

Here is what it looks like when I run this in a virtual machine (QEMU): http://imgur.com/pEdFP9a

My net0 comes up (link up) even before I run 'ifopen net0'! As well you might want to keep an eye on the NIC LED to see if link is up or down or coming up when you do 'ifopen'!

Another idea I had. What if the hang occurs due to a received packet instead of some background thing going on. Connect you host to a dump mini switch which is not further connected to your LAN. Then see what happens when you 'ifopen net0'. Keep an eye on the LED...

[jb303]

Hi,
Add me to this list.
I have a lot of broadcom network cards in my server and I think the 10gig network adapter is making it fail.
(I have Broadcom QLogic 57800, 57810 and NetXtreme BCM5719 in my server.)
Is there any way to select a specific nic to use (Ignoring all others)?

[surcouf]

Hi,

I'm having also the same issue on HP Proliant DL360 G7 and BCM5709.
I've tryied to add RHEL7 initrd.img to isolinux by editing ISO image and iPXE can execute a small script with shell as uniq command.

But no network device is available...

BCM5709 need a firmware to work : how can we had these firmware to iPXE image ?

Cheers,

--
Raphaël 'SurcouF' Bordet