iMAC with Broadcom 14e4-1686 BCM57766 : link status down

[mastacontrola]

(2017-03-15 15:29)speleo14 Wrote:  

(2017-03-15 14:43)mastacontrola Wrote:  I don't have an account, though creating one shouldn't be too problematic.

Yes it is, because it's not for free (Apple.... ).

(2017-03-15 14:43)mastacontrola Wrote:  That said, as this does appear to be a firmware issue, if we can't hold our breaths that a fix will come to place, might it be easier to find out a "change log" of what was changed between firmware B05 vs. B15? Even if it means something was added/adjusted that might be able to be corrected for in the iPXE BCM driver handling? Of course this would lead down a road that would essentially make the Mac's only usable on an 'ipxe' created driver system (Leaving out SNP/SNPONLY I suppse?)

Just thoughts in my head. I don't expect we'll see Apple pushing out a fix for this because of iPXE not working, but maybe we can get what was changed so the Opensource community might be able to provide a work around.

I agree that Apple will probably not fix this (or at least not in the near future), but I have no idea how and where to get any information on what was changed in the firmware...

I checked the Sierra installer and indeed there are 2 install packages there that refer to firmware (EmbeddedOsFirmware.pkg and FirmwareUpdate.pkg); unfortunately, both are .pkg files of the newer type (single binaries, not directories) so there is no possibility to look into them.
Additionally, I checked if the new firmware version for the Late 2015 iMac is available as a download, but it isn't (and neither is the older on that is working).

I have absolutely no experience with Boot ROMs and firmware, so I'm out of ideas what we could try. Anyone else? I have enough (different) Macs here that I can use for testing if anybody has any suggestions on WHAT to test.

According to here:
https://support.apple.com/en-us/HT201518

The latest firmware is now B16? Any chance of updating and seeing if something was addressed there?

[speleo14]

(2017-03-15 15:48)mastacontrola Wrote:  

(2017-03-15 15:29)speleo14 Wrote:  

(2017-03-15 14:43)mastacontrola Wrote:  I don't have an account, though creating one shouldn't be too problematic.

Yes it is, because it's not for free (Apple.... ).

(2017-03-15 14:43)mastacontrola Wrote:  That said, as this does appear to be a firmware issue, if we can't hold our breaths that a fix will come to place, might it be easier to find out a "change log" of what was changed between firmware B05 vs. B15? Even if it means something was added/adjusted that might be able to be corrected for in the iPXE BCM driver handling? Of course this would lead down a road that would essentially make the Mac's only usable on an 'ipxe' created driver system (Leaving out SNP/SNPONLY I suppse?)

Just thoughts in my head. I don't expect we'll see Apple pushing out a fix for this because of iPXE not working, but maybe we can get what was changed so the Opensource community might be able to provide a work around.

I agree that Apple will probably not fix this (or at least not in the near future), but I have no idea how and where to get any information on what was changed in the firmware...

I checked the Sierra installer and indeed there are 2 install packages there that refer to firmware (EmbeddedOsFirmware.pkg and FirmwareUpdate.pkg); unfortunately, both are .pkg files of the newer type (single binaries, not directories) so there is no possibility to look into them.
Additionally, I checked if the new firmware version for the Late 2015 iMac is available as a download, but it isn't (and neither is the older on that is working).

I have absolutely no experience with Boot ROMs and firmware, so I'm out of ideas what we could try. Anyone else? I have enough (different) Macs here that I can use for testing if anybody has any suggestions on WHAT to test.

According to here:
https://support.apple.com/en-us/HT201518

The latest firmware is now B16? Any chance of updating and seeing if something was addressed there?

Hm, no, that's for the MacBook Pro Mid 2014, not for the iMac (and it's from 2015).
The newest firmware that's available for download for iMacs is for the iMac Mid 2014, and dates from 2015.

[mastacontrola]

Just trying not to lose this thread in the mix of others as this is still a problem. I don't know if it's a problem we in the iPXE community can fix or if we actually do need to talk with Apple to try to get a suitable fix for this.

More information can be seen here:
https://forums.fogproject.org/topic/1026...broke-ipxe

While I realize the link is specific to the FOG Project, I think we can use a few different areas to try to get more information. It may be possible to fix, but with Apple's love to keep things down I don't know.

[NiKiZe]

(2017-06-13 14:44)mastacontrola Wrote:  I don't know if it's a problem we in the iPXE community can fix or if we actually do need to talk with Apple to try to get a suitable fix for this.

As has already been confirmed to be an issue with newer firmware, but not with older firmware, so contacting Apple should definitely be done.
With that said this is also an issue on non Apple machines if the NIC ends up in the same state. What is needed is to find out which register in the NIC is affected, so 2 machines with the same nic (one working firmware and one with issues) and then dumping the nics registers might give information about what needs to be done. and maybe comparing this to some other driver (linux?) but even better if we could get datasheets.

[speleo14]

(2017-06-13 15:24)NiKiZe Wrote:  

(2017-06-13 14:44)mastacontrola Wrote:  I don't know if it's a problem we in the iPXE community can fix or if we actually do need to talk with Apple to try to get a suitable fix for this.

As has already been confirmed to be an issue with newer firmware, but not with older firmware, so contacting Apple should definitely be done.
With that said this is also an issue on non Apple machines if the NIC ends up in the same state. What is needed is to find out which register in the NIC is affected, so 2 machines with the same nic (one working firmware and one with issues) and then dumping the nics registers might give information about what needs to be done. and maybe comparing this to some other driver (linux?) but even better if we could get datasheets.

I have enough machines with different firmware and NIC, so if I can help here, please let me know. However, I'm afraid I'd need some advice what to do exactly and how to do it.
I may be able to open a case with Apple engineering, but this will still need some time.

[SebastianRoth]

(2017-06-13 15:24)NiKiZe Wrote:  What is needed is to find out which register in the NIC is affected, so 2 machines with the same nic (one working firmware and one with issues) and then dumping the nics registers might give information about what needs to be done. and maybe comparing this to some other driver (linux?) but even better if we could get datasheets.

Bumping this topic as I hope we can get this fixed. Unfortunately I don't know much about about NIC registers yet but I am willing to dive in and learn if you could give me some advice on where to start reading. Code, books, URLs or just keywords that I can look up would be great. In case you already have some code that we could run on two different Macs to dump the NIC registers that would be awesome.

(2017-06-23 12:04)speleo14 Wrote:  I may be able to open a case with Apple engineering, but this will still need some time.

Did you get in contact with Apple yet. Maybe they have a general idea on what has changed that could break PXE booting (link state up).

Just found this by accident. Might be related.

Thanks in advance!

[curtis.larsen]

I can confirm that the same hardware version of iMac with different firmware causes this problem. Sorry, I'm sure this isn't big news to anyone.

iMac16,2 with ROM IM162.0206.B00 is able to send DHCP requests.
iMac16,2 with ROM IM162.0207.B20 is unable to send DHCP requests.

The second one had Sierra installed on it, which changed the ROM.

If there are any iPXE developers who want to work on this, I'm available to do as much debugging as you need. I'm a C/C++ programmer, used to command line utilities. I supported Michael Brown a few years ago when he put the first Intel iMac support into iPXE.

Thanks to all of the developers.

[curtis.larsen]

(2017-03-09 18:01)NiKiZe Wrote:  In that case I only think that boot ROM might be relevant.

Does any of these work with snponly.efi?

On my system with the newer firmware:
$ cat testscript.ipxe
#!ipxe
dhcp ||
ifstat
shell
$ make bin-x86_64-efi/snponly.efi EMBED=testscript.ipxe

I get the output shown in the linked image.



The MAC address *:4d:dd is the one that shows up in DHCP when the system boots into OSX.

[curtis.larsen]

After a little more debugging, it appears that the first reported error is in netdevice.c:net_rx(), where an unknown network protocol of 'aaaa' is received.

aaaa is 0101010101010101. That's too coincidental for me to be comfortable. Is that the pattern that uninitialized memory has?

I'm seeing this with
make bin-x86_64-efi/snponly.efi DEBUG=netdevice,nii EMBED=testscript.ipxe

[NiKiZe]

(2017-08-18 02:23)curtis.larsen Wrote:  After a little more debugging, it appears that the first reported error is in netdevice.c:net_rx(), where an unknown network protocol of 'aaaa' is received.

aaaa is 0101010101010101. That's too coincidental for me to be comfortable. Is that the pattern that uninitialized memory has?

I'm seeing this with
make bin-x86_64-efi/snponly.efi DEBUG=netdevice,nii EMBED=testscript.ipxe

Are you printing out the reg, and can you run this on a working machine as well? (and do we know if snponly.efi works with working firmware?)

If you can join us on IRC maybe you and Michael could figure this one out?

[rathore4u]

Bumping up.

Is there any solution to this issue?
I am having the same on MacMini7,1 and MacMini6,2 with latest firmwares.
I have working and non-working MacMini's for debugging and can help with that.

Thanks,

[NiKiZe]

I'm looking in to getting one machine with this problem, Could anyone provide exact model numbers on machines having this issue?

What I'm asking here is actual part numbers here, not model identifiers
but I found: https://support.apple.com/en-gb/HT201894

So if anyone could confirm a part number that has this issue that would be great.

[rathore4u]

(2018-05-28 17:18)NiKiZe Wrote:  I'm looking in to getting one machine with this problem, Could anyone provide exact model numbers on machines having this issue?

I have seen it on Macmini6,2 and Macmini7,1 after latest updates.

[letouzey]

Hi all

I'm having a similar issue here : "link is down" message while the network is ok on MacOS and linux. I'm using a MacMini 7,1 with network card NetXtreme BCM57766 (14e4-1686).

More detail on the screenshot (sorry for the poor quality of the image):


Oddly, when I test ipxe.efi from under a EFI shell, the first attempt gives me the above message, but then if I exit ipxe.efi and start it again, this time I'm welcomed by a "link is up" message (see image below). But trying "dhcp" then does not set up the network, on the contrary I'm back to "link is down" afterwards.


Anything I can try to help solving this issue ?

Regards
Pierre letouzey

[letouzey]

Update : I've also tried on many other Mac Mini 7,1 in our lab room, ipxe works great on them !
So the only problematic machine is the macmini 7,1 I used for initial testing. AFAICT this problematic machine has the exact same hardware as the other ones, but a slightly different system on it. In particular, the firmware is more recent (2017 instead of 2015). But I'm not sure yet that this recent firmware is to blame, it could be other settings like Apple SIP mode enabled or not. I'll continue some experiments to see if I can pinpoint which difference is causing the ipxe issue.

For now, I can just say that:
- many Macmini 7,1 with firmware MM71.88Z.0220.B03.1506051117 (dated 6/5/2015) do work fine
- one Macmini 7,1 with firmware MM71.88Z.0220.B29.1706181928 (dated 6/18/2017) is having the "link down" issue with ipxe.

Best,
Pierre Letouzey