Threaded Mode | Linear Mode

jwillis84 · 2014-08-13, 07:36

All bugs I am aware of have been fixed.

If you clone the github project

# git clone -b usb-net-drivers https://github.com/johntwillis/ipxe

Change to ipxe and type 'make' it will proceed to build the default ipxe images.

I used the bin/ipxe.iso image to successfully boot from a USB ASIX Adapter, obtain a dhcp address and download itest.ipxe script file and used that to download a tomrtbt kernel and initrd and then booted from that.

Any further work will be to support more hardware; USB host controller types and USB devices.

tohtoris · 2014-08-19, 20:09

Hi jwillis84,

I would be interest to try out pxe booting a Lenovo Yoga 2 pro with asix based Lenovo usb dongle.

I tried building you ipxe branch on a 64bit Ubuntu 14.04 but ran in to a problem.

Code:

git clone -b usb-net-drivers https://github.com/johntwillis/ipxe

cd ipxe/src/

make bin/ipxe.iso ####    same result with make bin/asix.iso

....

.....

[BUILD] bin/tg3_phy.o

[BUILD] bin/asix.o

drivers/net/usb/asix.c: In function ‘asix_get_phy_addr’:

drivers/net/usb/asix.c:151:2: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]

  DBG("asix_get_phy_addr() returning 0x%04x\n", *((uint16_t *)buf));

  ^

drivers/net/usb/asix.c: In function ‘asix_poll’:

drivers/net/usb/asix.c:281:20: error: variable ‘packet’ set but not used [-Werror=unused-but-set-variable]

  uint8_t *buffer, *packet;

                    ^

drivers/net/usb/asix.c: In function ‘asix_88178_probe’:

drivers/net/usb/asix.c:410:6: error: variable ‘gpio0’ set but not used [-Werror=unused-but-set-variable]

  int gpio0 = 0;

      ^

cc1: all warnings being treated as errors

make: *** [bin/asix.o] Virhe 1

antti@vonwright:~/devel/ipxe/src$

Would you happen to have any idea what could be wrong?

***robinsmidsrod*** · 2014-08-25, 11:16

tohtoris: Those are warnings you get from your compiler. The code should be fixed to not emit these compiler warnings. The iPXE Makefile is designed to halt on warnings, as these can cause problems. You might be able to get it to compile with NO_WERROR=1 flag to make, but those warnings should really be fixed by jwillis84 to avoid the problem alltogether.

jwillis84 · 2014-08-26, 05:22

(2014-08-19 20:09)tohtoris Wrote: Hi jwillis84,

I would be interest to try out pxe booting a Lenovo Yoga 2 pro with asix based Lenovo usb dongle.

I tried building you ipxe branch on a 64bit Ubuntu 14.04 but ran in to a problem.

Code:

git clone -b usb-net-drivers https://github.com/johntwillis/ipxe cd ipxe/src/ make bin/ipxe.iso #### same result with make bin/asix.iso .... ..... [BUILD] bin/tg3_phy.o [BUILD] bin/asix.o drivers/net/usb/asix.c: In function ‘asix_get_phy_addr’: drivers/net/usb/asix.c:151:2: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing] DBG("asix_get_phy_addr() returning 0x%04x\n", *((uint16_t *)buf)); ^ drivers/net/usb/asix.c: In function ‘asix_poll’: drivers/net/usb/asix.c:281:20: error: variable ‘packet’ set but not used [-Werror=unused-but-set-variable] uint8_t *buffer, *packet; ^ drivers/net/usb/asix.c: In function ‘asix_88178_probe’: drivers/net/usb/asix.c:410:6: error: variable ‘gpio0’ set but not used [-Werror=unused-but-set-variable] int gpio0 = 0; ^ cc1: all warnings being treated as errors make: *** [bin/asix.o] Virhe 1 antti@vonwright:~/devel/ipxe/src$

Would you happen to have any idea what could be wrong?

I used CentOS 5.9 and the gcc compiler available for that version of Linux, and did not receive the errors your seeing. I believe its a matter of differing gcc defaults for warning messages between CentOS and Ubuntu.

The default compile options for the ipxe project treat warnings as errors, and that is halting the build.

I am away from my home on business this week, but I will look into fixing these for Ubuntu when I return.

Thanks for reporting the errors and thanks for the detail on the version and edition of Linux in which you encountered the errors.

jwillis84 · 2014-09-02, 08:28

(2014-08-19 20:09)tohtoris Wrote: Hi jwillis84,

I would be interest to try out pxe booting a Lenovo Yoga 2 pro with asix based Lenovo usb dongle.

I tried building you ipxe branch on a 64bit Ubuntu 14.04 but ran in to a problem.

Would you happen to have any idea what could be wrong?

I just pushed changes to the Git repository, it will now finish compiling on 64bit Ubuntu 14.04.1 LTS.

hefty · 2014-09-08, 03:14

I added my own ASIX 88178 to the hardware list in asix.c, the device could be found on 1/4 of the systems I use but still didn't function with a dhcp timeout. And, unfortunately, I won't be able to use it just yet as 3 of the 4 computers I'm using use incompatible usb host controllers.

This will make ipxe/pxe much more usable for many people. I'll be paying close attention to this thread. Do you happen to have a blog jwillis84?

ps: I found a long list of ASIX 88xxx devices here, which includes both vendor and product ids.
http://cateee.net/lkddb/web-lkddb/USB_NET_AX8817X.html

tohtoris · (This post was last modified: 2014-09-08 13:54 by tohtoris.)

(2014-09-02 08:28)jwillis84 Wrote: I just pushed changes to the Git repository, it will now finish compiling on 64bit Ubuntu 14.04.1 LTS.

Sorry for my late reply. I was able to build the ipxe after your changes! I made just a quick test with a usb thumb drive, and was able to boot to the ipxe. Still the network boot didn't work and I quess that perhaps my network card is not supported. It just says "No more network devices" and continues to normal boot.

Ethernet usb-dongle is Lenovo U2L 100-Y1

Code:

Bus 001 Device 003: ID 17ef:7203 Lenovo

[273388.380067] usb 1-7: new high-speed USB device number 3 using ehci-pci

[273388.525211] usb 1-7: New USB device found, idVendor=17ef, idProduct=7203

[273388.525219] usb 1-7: New USB device strings: Mfr=1, Product=2, SerialNumber=3

[273388.525224] usb 1-7: Product: U2-Ethernet

[273388.525227] usb 1-7: Manufacturer: DAEC            

[273388.525231] usb 1-7: SerialNumber: 000000

[273388.837262] asix 1-7:1.0 eth1: register 'asix' at usb-0000:00:1d.7-7, ASIX AX88772 USB 2.0 Ethernet, 00:b5:6d:04:81:73

I'll try it again later... User error is always possible.

jwillis84 · (This post was last modified: 2014-09-08 17:49 by jwillis84.)

Code:

Bus 001 Device 003: ID 17ef:7203 Lenovo

[273388.525211] usb 1-7: New USB device found, idVendor=17ef, idProduct=7203

Currently the driver will not recognize your device and load.

The line above says your device id should be:

USB_ROM(0x17ef, 0x7203, "asix", "Lenovo U2L 100-Y1", 0),

In the ipxe/src/drivers/usb/net/usb/asix.c source code file.

The USB_ROM strings are used to "recognize" a device that is connected to the USB host controller.

Basically ipxe "starts" the host controller, then probes for what is attached, it gets id strings back from the connected devices and checks its inventory of drivers, of which the asix.c driver is "one". When it finds a match, it loads that driver and uses it to "start" the device and produces an ethernet interface in ipxe kernel space, which the normal network drivers then see and try to perform dhcp to initialize.. and the rest "as they say" is history.. things just work.

I'll add the string to the source code this evening and push it to git hub.

Your feedback was valuable, and extends the reach of this driver to provide service for other users that have a device like yours.

Its also interesting in that your device is a 1000 Mbps device [ AX88772 ] or 1 Gig Ethernet adapter, where I have only tested with an [ AX 88178 ] which is a 100 Mbps device.

Currently the USB stack for ipxe was designed with 100 Mbps in mind. Since it is based on the Linux USB stack it should only run at USB 2.0 speeds. In theory this should not be a problem, but in practice "anything" could happen. So your results will be very interesting.

iPXE generally is a bootstrapper.. meaning it downloads only a few hundered MBytes before starting another kernel and disk image with more sophisticated USB device drivers.. so speed isn't a major concern.

I am looking into a USB3.0 host controller driver and was going to look into verifying AX88772 support in the near future, but ther are so many branching directions in which I could focus my efforts, pursuing all of them at the same time.. just isn't practical.

USB2.0 host controllers were based on the EHCI device with companion controllers, UHCI and OHCI and pretty much worked transparently as UHCI or OHCI if there wasn't an EHCI driver available.

USB3.0 host controllers got a complete redesign when that spec came out, and Linux rewrote the USB stack it provided to support the higher speeds. The XHCI device was an attempt to cleanup and simplify the device driver requirements, but it appears the vendors successfully (to them) made each implementation of XHCI unique to the vendors that more than one XHCI device driver will be necessary.

Exasperating...

I tend to agree that more USB2.0 device drivers are far more important at this instance in time, but reality is more XHCI host controllers will be appearing on laptop and desktop motherboards in the near future.

tohtoris · 2014-09-09, 06:36

Hi, just a quick reply

(2014-09-08 17:31)jwillis84 Wrote: Currently the driver will not recognize your device and load.

The line above says your device id should be:

USB_ROM(0x17ef, 0x7203, "asix", "Lenovo U2L 100-Y1", 0),

In the ipxe/src/drivers/usb/net/usb/asix.c source code file.

The USB_ROM strings are used to "recognize" a device that is connected to the USB host controller.

I tried making the following change by myself to src/drivers/net/usb/asix.c

Code:

static struct usb_device_id asix_88772_ids[] = {

        USB_ROM(0x2001, 0x3c05, "asix", "DLink DUB-E100", 0),

        USB_ROM(0x0b95, 0x772a, "asix", "ASIX AX88772A", 0),

        USB_ROM(0x17ef, 0x7203, "asix", "Lenovo U2L 100-Y1", 0),

};

Maybe I added the string to a wrong place or something else also needs changes, but with this modification I still got same results. I+ ll take a look at this again later.

By the way here's another asix adapter for you from my colleques desk Wink

Code:

[333676.600058] usb 1-7: new high-speed USB device number 7 using ehci-pci

[333676.738255] usb 1-7: New USB device found, idVendor=0b95, idProduct=1790

[333676.738263] usb 1-7: New USB device strings: Mfr=1, Product=2, SerialNumber=3

[333676.738267] usb 1-7: Product: AX88179

[333676.738271] usb 1-7: Manufacturer: ASIX Elec. Corp.

[333676.738274] usb 1-7: SerialNumber: 00000000000241

[333677.170272] ax88179_178a 1-7:1.0 eth1: register 'ax88179_178a' at usb-0000:00:1d.7-7, ASIX AX88179 USB 3.0 Gigabit Ethernet, 1a:2b:3c:4d:62:f3

[333677.172493] usbcore: registered new interface driver ax88179_178a

jwillis84 · (This post was last modified: 2014-09-09 07:29 by jwillis84.)

(2014-09-09 06:36)tohtoris Wrote: Hi, just a quick reply

(2014-09-08 17:31)jwillis84 Wrote: Currently the driver will not recognize your device and load.

The line above says your device id should be:

USB_ROM(0x17ef, 0x7203, "asix", "Lenovo U2L 100-Y1", 0),

In the ipxe/src/drivers/usb/net/usb/asix.c source code file.

The USB_ROM strings are used to "recognize" a device that is connected to the USB host controller.

I tried making the following change by myself to src/drivers/net/usb/asix.c

Code:

static struct usb_device_id asix_88772_ids[] = { USB_ROM(0x2001, 0x3c05, "asix", "DLink DUB-E100", 0), USB_ROM(0x0b95, 0x772a, "asix", "ASIX AX88772A", 0), USB_ROM(0x17ef, 0x7203, "asix", "Lenovo U2L 100-Y1", 0), };

Maybe I added the string to a wrong place or something else also needs changes, but with this modification I still got same results. I+ ll take a look at this again later.

By the way here's another asix adapter for you from my colleques desk

Code:

[333676.600058] usb 1-7: new high-speed USB device number 7 using ehci-pci [333676.738255] usb 1-7: New USB device found, idVendor=0b95, idProduct=1790 [333676.738263] usb 1-7: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [333676.738267] usb 1-7: Product: AX88179 [333676.738271] usb 1-7: Manufacturer: ASIX Elec. Corp. [333676.738274] usb 1-7: SerialNumber: 00000000000241 [333677.170272] ax88179_178a 1-7:1.0 eth1: register 'ax88179_178a' at usb-0000:00:1d.7-7, ASIX AX88179 USB 3.0 Gigabit Ethernet, 1a:2b:3c:4d:62:f3 [333677.172493] usbcore: registered new interface driver ax88179_178a

I just added the same and did a git push, but if you've already tried it then perhaps something else isn't working.

Lenovo has a long forum thread on PXE booting with this Adapter here: http://forums.lenovo.com/t5/X-Series-Thi...977/page/2

One of the most striking things is they mention [be sure its plugged into the USB 2.0 port]

Its kind of standard practice these days to make laptops/desktops with lots of USB3.0 ports and with exactly one USB2.0 port.

USB goes by different symbols and names to indicate USB speed type

USB 1.1 Full Speed (FS)
USB 2.0 High Speed (HS)
USB 3.0 Super Speed (SS)

And sometimes they mark the ports with tiny lettering you almost need a bright light and magnifying glass to see.

Most commonly they mark the USB3.0 ports with tiny "ss" letters and leave one port totally [unmarked] that would be the USB2.0 port and its intended only for provisioning the laptop.

Sometimes All of the USB3.0 ports merely have a "dot" and the one without a dot is the USB2.0 port.

What your looking for is a [unique] port that stands out as "different" from all of the others.. that will be the one USB2.0 port for PXE Booting.

In my experience, right handed people tend to plug a wireless mouse or external keyboard into the [upper right hand side] of the laptop, and that "tends" to be the USB2.0 port.. on Desktops it gets even "weirder" as they play with unusual colors and and stickers.. there is no standard.

Also you usually have to "activate" Legacy BIOS and PXE booting in the BIOS due to a strange United States Presidential order which made it a violation to sell desktops to the US Government with PXE "enabled" by default in the BIOS.. I think it was under George Bush and they were worried about malware infecting machines when they are first unwrapped and booted up.. since that time I've seen a lot of systems defaulted to "PXE disabled". And around the same time F12 became the default for selecting Boot options manually at boot up. Now its sort of a pseudo standard for any company that hopes to sells PCs to the US Government.

Also UEFI really doesn't play nice with PXE BIOS routines.. I know there is a lot of current work to get that working.. and I've seen some nice menus come of it.. but its been flaky as can be. Just enable BIOS Legacy support if you can.

The AX88179 is a USB3.0 adapter and I'm pretty sure it just won't work in a USB2.0 port, but you can try. And as I've said there isn't a xHCI host controller driver currently available for iPXE.

There is a slim chance it will work simply because it shares heritage with the AX88772 and AX88178 but I would be surprised if it did. We're talking about jumping about six years in hardware technology.. even motherboards change quite a bit in that time.

[Just a quick clarification]

The current iPXE USB support has USB host drivers for UHCI and OHCI and seem to work with EHCI that covers USB1.1 and USB2.0 ports because those ports will be attached inside the laptop to host controller hardware that iPXE can load a driver for and look for things attached to those ports.

The current iPXE USB support does not have a USB host driver for xHCI, so it will not load a driver to use the USB3.0 ports [at all] anything attached to a USB3.0 port will not even be detected. (unless) the USB3.0 host controller can fall back to emulating a USB2.0 host controller.. which I strongly doubt. I could be wrong, I just can't prove this yet.

But if you can get the USB Ethernet adapter plugged into a USB2.0 port it should work.

My current focus is on deriving an xHCI host controller driver for iPXE.. but it may take some time.

Here is a handy guide to the Chipsets and their USB version/speed:

http://www.asix.com.tw/download.php?sub=...ItemID=131

kfortner · 2014-09-11, 22:14

John,

I am not using an ISO to PXE boot but undionly. Does any of this work help undionly work for USB?

jwillis84 · (This post was last modified: 2014-09-12 04:13 by jwillis84.)

(2014-09-11 22:14)kfortner Wrote: John,

I am not using an ISO to PXE boot but undionly. Does any of this work help undionly work for USB?

Catching up.. iPXE (with this USB support) will now compile to any image type you like. It was a problem compiling to the USB image type because I was using VMware player to test the image and did not realize the image it created was of the proper type for a VMware boot.

Essentially if the resulting USB image doesn't end up with a specific number of bytes or virtual "tracks" VMware Player will ignore it and fail to boot.

I've been defaulting to the .iso image format simply to avoid the extra step of "padding" the resulting image so that the USB image type will boot properly in VMware Player. The same could be true for some hardware types trying to boot from the USB image.. which might explain why some USB devices can be booted from, but other USB devices cannot be booted from when using the image. (I've always had variable results with USB boot devices and chocked that up to the USB thumb drive quality versus the USB emulation support for that USB device in the BIOS of the particular laptop or desktop I was using.. this revelation might explain a lot..)

Setting that issue to the side for a moment.

If the USB image boots for you, iPXE will search for any PCI devices that correspond to a UHCI or OHCI host controller for USB and start them up.

iPXE will then start the built-in USB Stack (this code added to iPXE) and probe any USB devices connected to the USB2.0 ports.

If it finds a USB Ethernet Adapter that it "recognizes" it will then load an iPXE USB device driver for that Adapter and next create a normal Ethernet device type in the iPXE kernel space.

At this point iPXE resume control and treats the device like any other Ethernet device, it does not distinguish between device types once the Ethernet device "exists" in kernel space.

I have not tried UNDI support within iPXE.

About 14 years ago I was very interested in UNDI support for providing a device driver to support installing Windows 2000/2003.

Essentially UNDI leaves a "stub" of a driver running in memory that augments the BIOS routines and presents a "Universal" Ethernet device that the corresponding UNDI driver in windows operating system can recognize and attach to, combining "forces" the two merge into a custom driver that lets the "Installer" or even the full "Operating System" boot from the network.

They were popular for "boot from net" devices around the SunRay time I believe and LTSP.

In theory iPXE could have UNDI support.. I think it does.. but I have never tested it.

In my mind its rather like booting iPXE to load the driver, scooping out the brains to leave a Zombie body and then pouring in a new operating system.. sounds creepy when you look it.. but also cool that you can do it.

At the time everything bootable was 16 bit, and even Windows had to jump back and forth between 32 bit mode and 16 bit mode to use the BIOS routines for UNDI.. they were generally only written in 16 bit code.

Today, 14 years later.. I think they would have overcome many of those limitations.. and I would guess the UNDI would leave the USB components running in memory to continue providing network service.. to iPXE this code just looks like weird but familar PCI driver code.

I'm sorry I can't answer your question directly.

I just don't know.

Thinking simply.. I think it should support UNDI if the UNDI feature is working with other PCI device drivers.

I don't have a specific test scenario in mind so I can't promise to test it for you.

I would "guess" your looking to support either an embedded OS, or an installer OS like WinPE.

If the iPXE boot loader is all you have to keep upgrading, to add support for strange and new USB Ethernet devices with the latest greatest USB chipsets, then the embedded OS, or installer OS can remain stable and strictly configured to use the iPXE UNDI support.. and never have to see a hardware driven upgrade ever again.

That's a big win.

iPXE is a small code base to test and maintain, an embedded OS or installer OS often is huge, lumbering.. and delicate.. they barely survive the install process before crashing or otherwise going off the rails.

If you could bring my imagination down to earth and give me a little more information what your targeting or expecting.. I might be able to dream up a way of testing the UNDI support your seeking.

kfortner · 2014-09-12, 15:55

(2014-09-12 04:08)jwillis84 Wrote:
(2014-09-11 22:14)kfortner Wrote: John,

I am not using an ISO to PXE boot but undionly. Does any of this work help undionly work for USB?

Catching up.. iPXE (with this USB support) will now compile to any image type you like. It was a problem compiling to the USB image type because I was using VMware player to test the image and did not realize the image it created was of the proper type for a VMware boot.

Essentially if the resulting USB image doesn't end up with a specific number of bytes or virtual "tracks" VMware Player will ignore it and fail to boot.

I've been defaulting to the .iso image format simply to avoid the extra step of "padding" the resulting image so that the USB image type will boot properly in VMware Player. The same could be true for some hardware types trying to boot from the USB image.. which might explain why some USB devices can be booted from, but other USB devices cannot be booted from when using the image. (I've always had variable results with USB boot devices and chocked that up to the USB thumb drive quality versus the USB emulation support for that USB device in the BIOS of the particular laptop or desktop I was using.. this revelation might explain a lot..)

Setting that issue to the side for a moment.

If the USB image boots for you, iPXE will search for any PCI devices that correspond to a UHCI or OHCI host controller for USB and start them up.

iPXE will then start the built-in USB Stack (this code added to iPXE) and probe any USB devices connected to the USB2.0 ports.

If it finds a USB Ethernet Adapter that it "recognizes" it will then load an iPXE USB device driver for that Adapter and next create a normal Ethernet device type in the iPXE kernel space.

At this point iPXE resume control and treats the device like any other Ethernet device, it does not distinguish between device types once the Ethernet device "exists" in kernel space.

I have not tried UNDI support within iPXE.

About 14 years ago I was very interested in UNDI support for providing a device driver to support installing Windows 2000/2003.

Essentially UNDI leaves a "stub" of a driver running in memory that augments the BIOS routines and presents a "Universal" Ethernet device that the corresponding UNDI driver in windows operating system can recognize and attach to, combining "forces" the two merge into a custom driver that lets the "Installer" or even the full "Operating System" boot from the network.

They were popular for "boot from net" devices around the SunRay time I believe and LTSP.

In theory iPXE could have UNDI support.. I think it does.. but I have never tested it.

In my mind its rather like booting iPXE to load the driver, scooping out the brains to leave a Zombie body and then pouring in a new operating system.. sounds creepy when you look it.. but also cool that you can do it.

At the time everything bootable was 16 bit, and even Windows had to jump back and forth between 32 bit mode and 16 bit mode to use the BIOS routines for UNDI.. they were generally only written in 16 bit code.

Today, 14 years later.. I think they would have overcome many of those limitations.. and I would guess the UNDI would leave the USB components running in memory to continue providing network service.. to iPXE this code just looks like weird but familar PCI driver code.

I'm sorry I can't answer your question directly.

I just don't know.

Thinking simply.. I think it should support UNDI if the UNDI feature is working with other PCI device drivers.

I don't have a specific test scenario in mind so I can't promise to test it for you.

I would "guess" your looking to support either an embedded OS, or an installer OS like WinPE.

If the iPXE boot loader is all you have to keep upgrading, to add support for strange and new USB Ethernet devices with the latest greatest USB chipsets, then the embedded OS, or installer OS can remain stable and strictly configured to use the iPXE UNDI support.. and never have to see a hardware driven upgrade ever again.

That's a big win.

iPXE is a small code base to test and maintain, an embedded OS or installer OS often is huge, lumbering.. and delicate.. they barely survive the install process before crashing or otherwise going off the rails.

If you could bring my imagination down to earth and give me a little more information what your targeting or expecting.. I might be able to dream up a way of testing the UNDI support your seeking.

What I am trying to accomplish is to PXE boot into a WinPE environment to then run a set of code that deploys the operating system to the target system from the WinPE environment. Today this works great on PCI based NICs, but for USB NICs the undionly.kpxe does not seem to work. I am dynamically building the iPXE boot script using a PHP script and the iPXE web call to determine what WinPE deployment environment to use. Is there a way to use the ISO build of iPXE to make the same call to my PHP and then chain boot to WinPE from the ISO of iPXE?

jwillis84 · (This post was last modified: 2014-09-12 22:01 by jwillis84.)

Quote:What I am trying to accomplish is to PXE boot into a WinPE environment to then run a set of code that deploys the operating system to the target system from the WinPE environment. Today this works great on PCI based NICs, but for USB NICs the undionly.kpxe does not seem to work. I am dynamically building the iPXE boot script using a PHP script and the iPXE web call to determine what WinPE deployment environment to use. Is there a way to use the ISO build of iPXE to make the same call to my PHP and then chain boot to WinPE from the ISO of iPXE?

That's a scenario I can test.

It would be helpful to know which WinPE version, or the target OS.

We are up to WinPE version 5.1 which spans Thirteen "released" versions.

As a Guess I'll look at testing WinPE 3.0 or WinPE 3.1 since Microsoft seems to be "burning" anything Win Eight related and taking chisel and hammer to all the monuments. If your using something post 3.1 please let me know.. and you have my condolences. Undecided

I'm guessing some people already have access to WinPE 6.0 but I do not.. should be available with two weeks however. I can't wait to dig into it.

[As for boot from ISO over http and then chain load?] Yes I do think that will work.. but you would still need something local, in ROM, NVRAM, SD card, USB or a USB DVDROM to bootstrap the whole process.. and at that point why not just boot PXE and load WinPE into a ramdisk and continue from there.

Like I said.. I got the USB image booting in VMware.. it just took an extra step.. and was trivial to do.. i didn't want to bog down the focus on USB support in the Git repo to address the USB image issue.. I felt it was some one elses problem

the BIOS looks at the USB MMD devices like NVRAM, USB Flash or SD Cards and either supports them as bootable or not.. they load iPXE which loads a USB host controller driver and then probes for attached USB Ethernet devices.. if it finds one.. that loads an iPXE USB Ethernet device driver..

.. and then I presume if properly configured.. iPXE loads the next stage boot loader from the network (WinPE ntloader?) into RAM.. then releases its memory and jumps to the ntloader.. which boots from its ramdisk, looks for the iPXE undi device driver signature in memory, and loads the WinPE undi device driver.. they mate.. and become the UNDI driver for WinPE that just happens to be attached to a USB Ethernet device already started.. and continue to pass traffic

I don't really know if the current undionly.kpxe support is completely in 32 bit space, or if somehow it expects the replacement kernel to "adopt" the iPXE driver code and provide runtime cycles for it.. the old 16 bit version kind of left it running in 16 bit space as a Terminate Stay Resident stub that got calls and woke up to process network traffic and went to sleep while the 32 bit OS was running.. it was more a high load "polling" mechanism flipping back and forth to catch and resend missed packets. I feel almost certain they do it a different way now.

You would almost need a kind of virtual machine environment that shared the environment back and forth.. the new OS in one vm, the old UNDI driver code running in a second.. and they mtually communicate through the UND Interface.. I just don't know .. but its intensely interesting (of course that Apes or Mimicks the original TSR model and used the DOS and Windows environment as virtual machines.

I will look into it.

kfortner · 2014-09-15, 15:16

(2014-09-12 21:42)jwillis84 Wrote:
Quote:What I am trying to accomplish is to PXE boot into a WinPE environment to then run a set of code that deploys the operating system to the target system from the WinPE environment. Today this works great on PCI based NICs, but for USB NICs the undionly.kpxe does not seem to work. I am dynamically building the iPXE boot script using a PHP script and the iPXE web call to determine what WinPE deployment environment to use. Is there a way to use the ISO build of iPXE to make the same call to my PHP and then chain boot to WinPE from the ISO of iPXE?

That's a scenario I can test.

It would be helpful to know which WinPE version, or the target OS.

We are up to WinPE version 5.1 which spans Thirteen "released" versions.

As a Guess I'll look at testing WinPE 3.0 or WinPE 3.1 since Microsoft seems to be "burning" anything Win Eight related and taking chisel and hammer to all the monuments. If your using something post 3.1 please let me know.. and you have my condolences.

I'm guessing some people already have access to WinPE 6.0 but I do not.. should be available with two weeks however. I can't wait to dig into it.

[As for boot from ISO over http and then chain load?] Yes I do think that will work.. but you would still need something local, in ROM, NVRAM, SD card, USB or a USB DVDROM to bootstrap the whole process.. and at that point why not just boot PXE and load WinPE into a ramdisk and continue from there.

Like I said.. I got the USB image booting in VMware.. it just took an extra step.. and was trivial to do.. i didn't want to bog down the focus on USB support in the Git repo to address the USB image issue.. I felt it was some one elses problem

the BIOS looks at the USB MMD devices like NVRAM, USB Flash or SD Cards and either supports them as bootable or not.. they load iPXE which loads a USB host controller driver and then probes for attached USB Ethernet devices.. if it finds one.. that loads an iPXE USB Ethernet device driver..

.. and then I presume if properly configured.. iPXE loads the next stage boot loader from the network (WinPE ntloader?) into RAM.. then releases its memory and jumps to the ntloader.. which boots from its ramdisk, looks for the iPXE undi device driver signature in memory, and loads the WinPE undi device driver.. they mate.. and become the UNDI driver for WinPE that just happens to be attached to a USB Ethernet device already started.. and continue to pass traffic

I don't really know if the current undionly.kpxe support is completely in 32 bit space, or if somehow it expects the replacement kernel to "adopt" the iPXE driver code and provide runtime cycles for it.. the old 16 bit version kind of left it running in 16 bit space as a Terminate Stay Resident stub that got calls and woke up to process network traffic and went to sleep while the 32 bit OS was running.. it was more a high load "polling" mechanism flipping back and forth to catch and resend missed packets. I feel almost certain they do it a different way now.

You would almost need a kind of virtual machine environment that shared the environment back and forth.. the new OS in one vm, the old UNDI driver code running in a second.. and they mtually communicate through the UND Interface.. I just don't know .. but its intensely interesting (of course that Apes or Mimicks the original TSR model and used the DOS and Windows environment as virtual machines.

I will look into it.

I am currently using WinPE 5.1, and from my experience on the ASIX USB, undionly.kpxe will load, and pull an IP, but either the http file transfer or wimboot process will fail out.

jwillis84 · (This post was last modified: 2014-09-16 03:04 by jwillis84.)

[/quote]

I am currently using WinPE 5.1, and from my experience on the ASIX USB, undionly.kpxe will load, and pull an IP, but either the http file transfer or wimboot process will fail out.
[/quote]

I haven't finished looking into this yet, but came across mention of a "second type" of undionly target called the undionly.kkpxe

You might try compiling to that target type

# make bin/undionly.kkpxe

And see if that works

I'm a little alarmed at what I have read about the undionly.kpxe support -- I may misunderstand it.. but from what I have read, people say its meant to remove the undionly.kpxe driver from memory once it has copied the boot image over the network using the combined (iPXE-undionly)/(BIOS undi) driver .. so that the boot loader its about to chain to can fully access the UNDI support in the BIOS.. since we know the BIOS does not have "general" USB Ethernet adapter driver support that will not work. It might work if (a) the BIOS has an UNDI driver interface built into it (b) the USB Ethernet adapter being used "just happens" to be supported by the BIOS for PXE booting (which if it did why bother with iPXE, other than to provide nicer menus)

The .kkpxe target type seems to avoid releasing the memory allocated to the iPXE code.. or does it in a different way.. but I don't think iPXE provides a generic UNDI interface that a secondary bootloader could bond with.. rather it seems to be all about using the UNDI driver interface in the BIOS (if there is support for UNDI).

UNDI in a way depends on two virtual machines, be that BIOS Interrupts or a service running as a virtual machine to expose an UNDI interface that an UNDI driver can access to get network services. The UNDI interface was specified a long long time ago.. we've moved way beyond Terminate Stay Resident and mere BIOS Network interrupts to provide a generic UNDI interface.. I would have thought.. but maybe I am wrong.

If the UNDI support in iPXE is really the top-half of the driver in the iPXE bootloader or OS trying to run on the hardware.. then its only going to provide support if there is already an UNDI interface in the BIOS for iPXE.

Chaining "network" drivers is sort of what UNDI does, like Chaining "bootloaders".. I'm not 100 percent sure.. but I don't think iPXE is in the business of loading its drivers to provide a generic UNDI interface.. it would then have to stay resident and accept calls from the top-layer UNDI driver and return control back and forth.. not impossible, but probably not very stable.

It's as if what you'd like were an UNDI "bus" driver, that finds UNDI support, then re-exposes that to the subsequent bootloader to come "after" iPXE, but if the BIOS already had UNDI support.. you wouldn't need iPXE.

More likely you'd like iPXE to "stay resident" and provide it's USB bus and USB device driver support, then expose a generic "UNDI" interface that a subsequent bootloader could bond with and continue. In that way iPXE would become a BIOS "add-on" ROM providing a UNDI interface for a BIOS that natively "has no" UNDI support.

If it is that way, then what your seeking in an UNDI "bus driver" kind of like the USB "bus driver" to provide a place later "UNDI device" drivers could "plug-in" for service.

What really makes me think this is not possible without a new project to add a "UNDI" bus driver to supply the "interface" is that I read several people saying ".kpxe" [get's out of the way] and [ only one driver is compiled into the .kpxe image ]

***robinsmidsrod*** · 2014-09-21, 12:13

jwillis84: iPXE has an UNDI driver, and is able to take advantage of an existing PXE firmware when loaded via it (that is what undionly.kpxe does). iPXE also provide its native drivers as an UNDI interface when booting via USB/CD/ROM. So you can have iPXE on both sides of the equation. What undionly.kpxe does it uses the generic networking interface that the firmware provides and use it as a "virtual" network device.

Sometimes this doesn't work quite right, because the underlying firmware has bugs. The UNDI driver in iPXE does a lot of crazy stuff to try to work around various firmware bugs, but sometimes it's just not possible to make it work. In that case you'll either need to use a native iPXE driver or get a firmware update for your PXE-enabled NIC.

When a USB NIC comes with PXE support in it, iPXE should be able to run via undionly.kpxe. But that all depends on the quality of the PXE firmware on the USB NIC. That is why I'm looking closely at your project, as it might help us to finally get basic support for USB NICs, just as we have basic support for WiFi NICs (although we only support Atheros and Realtek).

Keep up the good work! I hope someone will have time to review your code soon. Maybe you should ask Mareo again. I recall him saying he would have time in a couple of weeks, but that's a while ago now, so maybe he's got some spare time on his hands.

mithro · 2014-09-23, 04:54

Hello,

I gave your repository at https://github.com/johntwillis/ipxe a try to see if I could PXE boot my Lenovo X1 Carbon (2013 model) using an Apple USB->Ethernet adapter.

The Apple USB Ethernet adapter is a AX88772 asix adapter with IDs 05ac:1402. See the dmesg logs below;

Quote:[87658.231359] usb 3-2.4: new high-speed USB device number 30 using xhci_hcd
[87658.272752] usb 3-2.4: New USB device found, idVendor=05ac, idProduct=1402
[87658.272758] usb 3-2.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[87658.272763] usb 3-2.4: Product: Apple USB Ethernet Adapter
[87658.272766] usb 3-2.4: Manufacturer: Apple Inc.
[87658.272769] usb 3-2.4: SerialNumber: 2D8CB1
[87658.633762] asix 3-2.4:1.0 eth1: register 'asix' at usb-0000:08:00.0-2.4, ASIX AX88772 USB 2.0 Ethernet, 487:05:e9:40:52
[87663.795160] asix 3-2.4:1.0 eth1: link up, 100Mbps, full-duplex, lpa 0xCDE1

See https://github.com/johntwillis/ipxe/pull/1

I wasn't able to make this work though as the Carbon doesn't have a UHCI (nor OHCI) controller inside the machine. It does have both a USB3.0 xHCI controller and USB2.0 EHCI controller. (The xHCI controller disappears if you disable USB3.0 support in the BIOS.)

Thanks for your work so far. Hopefully iPXE will grow xHCI support sometime in the future.

jwillis84 · (This post was last modified: 2014-09-23 06:59 by jwillis84.)

(2014-09-21 12:13)robinsmidsrod Wrote: jwillis84: iPXE has an UNDI driver, and is able to take advantage of an existing PXE firmware when loaded via it (that is what undionly.kpxe does). iPXE also provide its native drivers as an UNDI interface when booting via USB/CD/ROM. So you can have iPXE on both sides of the equation. What undionly.kpxe does it uses the generic networking interface that the firmware provides and use it as a "virtual" network device.

Sometimes this doesn't work quite right, because the underlying firmware has bugs. The UNDI driver in iPXE does a lot of crazy stuff to try to work around various firmware bugs, but sometimes it's just not possible to make it work. In that case you'll either need to use a native iPXE driver or get a firmware update for your PXE-enabled NIC.

When a USB NIC comes with PXE support in it, iPXE should be able to run via undionly.kpxe. But that all depends on the quality of the PXE firmware on the USB NIC. That is why I'm looking closely at your project, as it might help us to finally get basic support for USB NICs, just as we have basic support for WiFi NICs (although we only support Atheros and Realtek).

Keep up the good work! I hope someone will have time to review your code soon. Maybe you should ask Mareo again. I recall him saying he would have time in a couple of weeks, but that's a while ago now, so maybe he's got some spare time on his hands.

Yes, I think we are on similar lines of thought.

I have not actually looked at what iPXE undionly.kpxe actually does, I've merely read up on discussions about it in the forums. So my information is second hand.

The existing USB support that has been tested as working for the ASIX driver is available, but its a question of what undionly.kpxe and undionly.kkpxe does with it.

UNDI originally exposed either the original underlying BIOS firmware support for an UNDI virtual interface, or alternatively stayed in memory to provide an UNDI interface based on the booting PXE bootloader. Peter Anvin would probably have a better grasp upon it than I since he wrote PXELinux and I believe it did provide some kind of UNDI support.

A different way of behaving though is to put a priority on exposing the BIOS firmware support for the UNDI virtual interface by turning control over to the next kernel to boot in the chain, or by cleaning up the system and then turning control over to the next kernel in the boot chain. (One is cleaner than the other and might lead to less problems) That is what I think .kpxe and .kkpxe does based on the forum conversations I read.. but those posters could have misunderstood UNDI and/or may have not read the code for themselves.

Ultimately I just need to look at the code.

(2014-09-23 06:16)jwillis84 Wrote: Ultimately I just need to look at the code.

Ok I've found the start of the UNDI support in ipxe/src/arch/i386/drivers/net/undi.c and undionly.c

Michael Brown c2007

undionly.c code has a note about two types of build

make bin/undionly.kpxe

make bin/undi.dsk

The first one is pure UNDI without regard to bus type, I would guess that's for when there is UNDI support in the BIOS firmware.

The second one is for UNDI+plus with "other" drivers, I would guess that's for when there is no UNDI support in the BIOS firmware for the devices that iPXE supports... like USB Ethernet devices

Checking the bin/undi.dsk.tmp.map the usb bus appears to be getting compiled in, but I don't see an entry for the asix.o driver, could be it needs to be told to add in that driver or drivers, so that the iPXE resident UNDI can load an iPXE driver to support the new USB bus being compiled into the undi.dsk image.

I'll have to spend more time figuring out how the build system decides what to compile into the undi.dsk image type, I would guess its a special Makefile list somewhere.

As ignorant as I am, stumbling around in the codebase.. I almost feel like leaving bread crumbs..

(2014-09-15 15:16)kfortner Wrote:
(2014-09-12 21:42)jwillis84 Wrote:
Quote:What I am trying to accomplish is to PXE boot into a WinPE environment to then run a set of code that deploys the operating system to the target system from the WinPE environment. Today this works great on PCI based NICs, but for USB NICs the undionly.kpxe does not seem to work. I am dynamically building the iPXE boot script using a PHP script and the iPXE web call to determine what WinPE deployment environment to use. Is there a way to use the ISO build of iPXE to make the same call to my PHP and then chain boot to WinPE from the ISO of iPXE?

That's a scenario I can test.

It would be helpful to know which WinPE version, or the target OS.

We are up to WinPE version 5.1 which spans Thirteen "released" versions.

As a Guess I'll look at testing WinPE 3.0 or WinPE 3.1 since Microsoft seems to be "burning" anything Win Eight related and taking chisel and hammer to all the monuments. If your using something post 3.1 please let me know.. and you have my condolences.

I'm guessing some people already have access to WinPE 6.0 but I do not.. should be available with two weeks however. I can't wait to dig into it.

[As for boot from ISO over http and then chain load?] Yes I do think that will work.. but you would still need something local, in ROM, NVRAM, SD card, USB or a USB DVDROM to bootstrap the whole process.. and at that point why not just boot PXE and load WinPE into a ramdisk and continue from there.

Like I said.. I got the USB image booting in VMware.. it just took an extra step.. and was trivial to do.. i didn't want to bog down the focus on USB support in the Git repo to address the USB image issue.. I felt it was some one elses problem

the BIOS looks at the USB MMD devices like NVRAM, USB Flash or SD Cards and either supports them as bootable or not.. they load iPXE which loads a USB host controller driver and then probes for attached USB Ethernet devices.. if it finds one.. that loads an iPXE USB Ethernet device driver..

.. and then I presume if properly configured.. iPXE loads the next stage boot loader from the network (WinPE ntloader?) into RAM.. then releases its memory and jumps to the ntloader.. which boots from its ramdisk, looks for the iPXE undi device driver signature in memory, and loads the WinPE undi device driver.. they mate.. and become the UNDI driver for WinPE that just happens to be attached to a USB Ethernet device already started.. and continue to pass traffic

I don't really know if the current undionly.kpxe support is completely in 32 bit space, or if somehow it expects the replacement kernel to "adopt" the iPXE driver code and provide runtime cycles for it.. the old 16 bit version kind of left it running in 16 bit space as a Terminate Stay Resident stub that got calls and woke up to process network traffic and went to sleep while the 32 bit OS was running.. it was more a high load "polling" mechanism flipping back and forth to catch and resend missed packets. I feel almost certain they do it a different way now.

You would almost need a kind of virtual machine environment that shared the environment back and forth.. the new OS in one vm, the old UNDI driver code running in a second.. and they mtually communicate through the UND Interface.. I just don't know .. but its intensely interesting (of course that Apes or Mimicks the original TSR model and used the DOS and Windows environment as virtual machines.

I will look into it.

I am currently using WinPE 5.1, and from my experience on the ASIX USB, undionly.kpxe will load, and pull an IP, but either the http file transfer or wimboot process will fail out.

That is very interesting that it pulled an IP.. I seem to recall having a second nic be detected in addition to the one I thought I was booting from and after it failed on the asix it tried to dhcp from that.. unfortunately it was vmware virtual nic and wasn't of practical use.

But that is promising, maybe your right that it is some other subsystem failing. Were I in your shoes I would be using Wireshark to observe the traffic and make sure of what I was seeing, and maybe even witnessing what is failing in the protocol or timing. I guess that assumes you have one end in a virtual machine.. or a passive hub, or monitor port on a switch where you can observe the network traffic.

jwillis84 · (This post was last modified: 2014-09-26 04:08 by jwillis84.)

Learning GitHub versus Git.

My fork still retains all the work, but its all been merged back into the Master branch, and then changes in the parent ipxe/ipxe project changes since the original Forking were pulled in to bring the code base up to date.

I did things in a really inept, bizarre way.. it felt like flying one of those model airplanes through an acrobatic backflip.. but I learned a lot about GitHub.. really I wish the Tutorials and User Guides for GitHub were less formal, they read like first year Calculus books.. slowly explaining things.. and building on topics. When all I wanted to do was 'Fly like the Eagles'..

Studying how everyone else seems to be doing things "seems" possible to me now.

And Hurray! No conflicts with the mainstream ipxe/ipxe code, the Merge went automatic with no conflicts.

If ever someone does get a chance to audit the new code from gPXE brought in and tweaked to work in iPXE for whatever criteria is being impressed upon silly coders like me.. perhaps it will be an easy "Pull" request.

I'm looking in several directions now, WinPE 5.1 testing with UNDI.dsk, xHCI controller support (this could take some time), and more USB Ethernet device chipsets.. really the new chipsets I'd like to focus on are USB3.0 capable, but some more USB2.0 should be possible.

Opinion: The old USB code, PCI drivers for the USB controller, and the USB Ethernet drivers appeared to be more a proof of concept and there weren't a lot of actual pci:vendor:device Identifiers, so instead of actually "finding" a recognized device on the bus.. it just assumed that device was on the bus and proceeded to use it. Not a bad assumption, since iPXE is often target compiled to support only one device driver at a time.. but the megalithic ipxe.dsk target would pull in everything and then it appears only one driver would load and work at runtime. Good luck guessing which one.

I know iPXE isn't trying to be a complete operating system, but the intent of the original authors did appear to be to eventually smoothly support all USB controller and USB Ethernet devices regardless of the ipxe.dsk image and regardless of which combination was in play, truly plug and play. I'd really like to honor that.

So I am collecting pci:vendor:device id codes, and usb:vendor:device id codes and beginning to insert them into the codebase. That could enlarge the codebase and might be against design preferences to keep ipxe as light and small as possible. I just don't know what the outcome will be. Which ever way it goes, I will try to be flexible.

Part time I'm also learning GitHub and the culture surrounding iPXE. A lot of that is on this board. Now some of it is on my personal blog. I think GitHub has a Wiki per forked project.. and I'll see about moving my verbose comments over there. That way if anyone goes to to the https://github.com/johntwillis/ipxe/wiki they will have [on topic] reference information.

Honestly I'm a lot more talk than code. So its going to be a little project "just a hobby, won't be big and professional like gnu" Smile