Loading PNG for console on older hardware broken

[pxe_curious]

I had built a undionly.kpxe back in April 2017 and that was working fine.

I recently rebuilt it with latest commits as of July 17, 2018.

The following command worked before:

Code:

console --picture ${main-url}/ipxe/background.png --left 48 --right 48 --top 40 --bottom 105

With the latest build of the undionly.kpxe file, that command fails with the following:
http://ipxe.org/err/2d8c20

Per the error page, this is apparently traced back to arch/x86/interface/pcbios/vesafb.c (line 344). But this file has not been changed since the last time I built that binary. I locked at a few other files that might have been related and that were changed, but the changes there do not look like they could be causing this.

The hardware that is now failing is a Supermicro X7SBE, just for reference - it is old . Other, newer, hardware works fine.

When I restored the previous version of the undionly.kpxe file, the menu and background picture continued to work like expected.

The full error message I am seeing at the console, after the background image is successfully downloaded, is:

Code:

Could not configure console: No such file or directory (http://ipxe.org/2d8c203b)
Could not boot image: No such file or directory (http://ipxe.org/2d8c203b)
No more network devices

Press Ctrl-B for the iPXE command line...

And the script dies at that point and does not show a menu.

Does anyone have any insight as to what changes could have caused this?

[NiKiZe]

(2018-08-08 21:06)pxe_curious Wrote:  The following command worked before:

Code:

console --picture ${main-url}/ipxe/background.png --left 48 --right 48 --top 40 --bottom 105

With the latest build of the undionly.kpxe file, that command fails with the following:
http://ipxe.org/err/2d8c20

So something is not detected correctly, I think there is other posts about this, but my suggestion is to add || at the end of that command, that way if the console mode change fails than it's just that that fails, and not the whole boot process.
Here we are http://forum.ipxe.org/showthread.php?tid=8004

Also from the error page you linked:

ipxe.org/2d8c203b Wrote:This error most likely means the console command was unable to find a usable resolution in the VESA BIOS mode list.
You can compile your binary with DEBUG=vesafb to get a list of which resolutions was detected when running the console command.

(2018-08-08 21:06)pxe_curious Wrote:  When I restored the previous version of the undionly.kpxe file, the menu and background picture continued to work like expected.

The full error message I am seeing at the console, after the background image is successfully downloaded, is:

Code:

Could not configure console: No such file or directory (http://ipxe.org/2d8c203b)
Could not boot image: No such file or directory (http://ipxe.org/2d8c203b)
No more network devices

Press Ctrl-B for the iPXE command line...

And the script dies at that point and does not show a menu.

Does anyone have any insight as to what changes could have caused this?

I must also ask, you said that old build works, new does not. And this is on the exact same terminal? so that it isn't that one is over Networked KVM while the other is not or similar?

[pxe_curious]

(2018-08-08 21:33)NiKiZe Wrote:  

(2018-08-08 21:06)pxe_curious Wrote:  The following command worked before:

Code:

console --picture ${main-url}/ipxe/background.png --left 48 --right 48 --top 40 --bottom 105

With the latest build of the undionly.kpxe file, that command fails with the following:
http://ipxe.org/err/2d8c20

So something is not detected correctly, I think there is other posts about this, but my suggestion is to add || at the end of that command, that way if the console mode change fails than it's just that that fails, and not the whole boot process.
Here we are http://forum.ipxe.org/showthread.php?tid=8004

Also from the error page you linked:

ipxe.org/2d8c203b Wrote:This error most likely means the console command was unable to find a usable resolution in the VESA BIOS mode list.
You can compile your binary with DEBUG=vesafb to get a list of which resolutions was detected when running the console command.

(2018-08-08 21:06)pxe_curious Wrote:  When I restored the previous version of the undionly.kpxe file, the menu and background picture continued to work like expected.

The full error message I am seeing at the console, after the background image is successfully downloaded, is:

Code:

Could not configure console: No such file or directory (http://ipxe.org/2d8c203b)
Could not boot image: No such file or directory (http://ipxe.org/2d8c203b)
No more network devices

Press Ctrl-B for the iPXE command line...

And the script dies at that point and does not show a menu.

Does anyone have any insight as to what changes could have caused this?

I must also ask, you said that old build works, new does not. And this is on the exact same terminal? so that it isn't that one is over Networked KVM while the other is not or similar?

Thank you NiKiZe for the quick reply. I will be adding the double pipes to the end of the line, I was already heading that way ;-)

Re your question: Yes, same server connected to regular monitor (not KVM). Doesn't boot with new binary, but does on old one.

If the double pipes resolve it, I will leave it alone. Just couldn't figure out what the difference if between the binaries and the fact that it states that it didn't find the file, when the download was definitely successful.

[NiKiZe]

(2018-08-09 00:12)pxe_curious Wrote:  If the double pipes resolve it, I will leave it alone. Just couldn't figure out what the difference if between the binaries and the fact that it states that it didn't find the file, when the download was definitely successful.

Please run it with the debug build.
If possible it is good to get the data so that we can figure out what have gone bad. And also help others that might see this in the future.

If we don't get anything from the debug output then running a bisect to find out exactly when this broke would be extremely helpful
http://ipxe.org/howto/bisect

[pxe_curious]

(2018-08-09 22:47)NiKiZe Wrote:  Please run it with the debug build.
If possible it is good to get the data so that we can figure out what have gone bad. And also help others that might see this in the future.

If we don't get anything from the debug output then running a bisect to find out exactly when this broke would be extremely helpful
http://ipxe.org/howto/bisect

NiKiZe,

Tried a debug build but got no additional error messages than what I was getting before.

Using double pipes did address my issue and allows the menu to fall back to using text mode which is fine.

I also gave the git bisect a try but was not able to build a working version of undionly.kpxe, even with the version I thought I had used. I am not sure if this problem might be related to the fact that I upgraded my build box from Ubuntu 17 to 18. I might try again with a fresh build environment in the future.

One additional thing that I did at the iPXE command line is this:

console -x 800 -y 600

Gives me the "could not configure console: No such file or directory" error. But:

console -x 640 -y 480

Worked!

When it errors out, it returns to a 720x400 resolution. But trying the console command with that resolution will also result in an error.

I tried several more times with 800x600 and it jumped to 1024x768. This is really all over the place.

[NiKiZe]

This all suggests to me that it is the VESA implementation of that specific box that is broken or maybe does some magic...
Does that motherboard have IPMI support?
Could it be that some kind of SOL console or similar is enabled, and that simply tries to force the box to not leave text mode?

[pxe_curious]

(2018-08-16 07:32)NiKiZe Wrote:  This all suggests to me that it is the VESA implementation of that specific box that is broken or maybe does some magic...
Does that motherboard have IPMI support?
Could it be that some kind of SOL console or similar is enabled, and that simply tries to force the box to not leave text mode?

I would agree with you except that the undionly.kpxe I built more than a year ago based on commit fd6d1 does work perfectly, background image and all. Switching between video resolutions also works fine.
I built the same binary again today with the same commit and it has the bug.
I think it has something to do with my build environment and I am going to rebuild it today and see what happens.

[pxe_curious]

I nuked and rebuilt my build environment, going back to Ubuntu 16.
Built the older commit and it worked.
Built the latest commit and it worked as well.

Not sure what it was in my build environment that borked it, but the problem is not with the iPXE code. Issue resolved.

[NiKiZe]

I see, awesome!
Did you try the failing builds on any other hardware to see if this issue could be reproduced?
If it can it might be worth submitting to the mailing list as well.

[pxe_curious]

(2018-08-16 18:43)NiKiZe Wrote:  Did you try the failing builds on any other hardware to see if this issue could be reproduced?

I tried the "broken" binaries on all kinds of hardware and had no issues. But I never checked the really old stuff. I tested on Dell PowerEdge 11 through 14th generation and Supermicro X8/9/10/11, which worked fine, but didn't go all the way back to Supermicro X7.

But the new builds now work on everything.

[NiKiZe]

(2018-08-16 19:24)pxe_curious Wrote:  I tried the "broken" binaries on all kinds of hardware and had no issues. But I never checked the really old stuff. I tested on Dell PowerEdge 11 through 14th generation and Supermicro X8/9/10/11, which worked fine, but didn't go all the way back to Supermicro X7.

But the new builds now work on everything.

Still just trying to figure out what the factor here is. What CPU is on the board that it fails on? Maybe newer compilers create broken binaries for older CPUs

[pxe_curious]

The motherboard works with Core 2 Duo and Quad procs.

The particular machine I was testing with has a Q9400.

[pxe_curious]

Just a follow-up:
Rebuilt my build environment with a fresh Ubuntu 18 install and the binaries built in on that platform do show the video mode problem on older hardware. Binaries built on Ubuntu 16 are ok.
Binaries built on Ubuntu 18 are also a little bit smaller compared to those built on Ubuntu 16 (~300 bytes for PCBIOS, ~800 bytes for EFI), so maybe something missing?
Build machine itself is the same, the OS version is the only thing that is different.

[pxe_curious]

Also built binaries on CentOS 6.10/7.5. In both cases the binaries that were built exhibited the flawed behavior and failed to load the background image on older hardware.