iPXE problems with iSCSI Boot and Windows

[brownmit]

Well.

I'm making loads of progress with iPXE, and I love it. But, i'm having tons of problems with windows and iSCSI boot. So, let me back up and give you some troubleshooting information.

I am using both undionly.kpxe and ipxe.kpxe, built from a git pull from a few weeks back. version is 1.0.0 + (d0bfd)

my DHCP server is configured to give out undionly.kpxe, and I have a script that loads from which I can chain over to the full ipxe.kpxe when needed.

I have successfully built a linux diskless client that boots from iSCSI. Its screaming fast. My iSCSI server is an HP DL380 G7 running ubuntu server and the iscsitarget software stack. Seems to work very well, no problems with either using iSCSI to boot ISO images and also my linux diskless image.

However, when I try using it with Windows, it is crazy, terribly slow. I have a custom PE image that I load using iPXE and winboot, works great.

So, here's what I tried to get windows installed:

Strategy #1

grab the shell and do a sanhook iscsi:192.168.15.25::::iqn.2016-01.com.mjbrowns:mbstudio
(works, maps to 0x80)

proceed to winboot my PE image. Image takes longer than usual to boot, but when it finishes, I can see the ISCSI device, but any I/Os to it take forever. diskpart clean (to clean the disk headers) took over 10 minutes!

I've tried this sequence inserting a variety of:
set keep-san 1
set netX/gateway 0.0.0.0

to no avail.

Strategy #2

I also put a disk in, booted, mapped the iSCSI LUN and cloned the boot disk (using Paragon Partition Manger) to the ISCSI disk, then shut down, removed the drive and tried to do a sanboot - it basically just seems to hang on the logo screen and goes nowhere.

I've tried Strategy #1 using both physical diskless client and virtual machine in VMware Workstation running on my laptop -same thing happens.

Any suggestions?

[brownmit]

(2016-02-17 00:12)brownmit Wrote:  Well.

<snip>

I may be making a habit of getting frustrated, posting a message to the forums, then figuring it out myself Once again, here's what I learned that may help other people.

First of all, and most importantly, Microsoft has broken installation of Windows 10 via ISCSI. You can get it to deploy the image but you will end up with an inaccessible boot device error when it goes on to the second phase of installation.

The short answer is that what you need to do is to follow this process:
1...Install Windows 10 on a local hard drive
2...iPXE boot with iSCSI sanhook device to a PE environment loaded up with HD copy tools
3...Copy your fully installed Windows 10 HD to the iSCSI device
4...Shut Down, remove local HD, boot to iSCSI device.

There are however, some things you really need to know, that I learned the hard way from various experimentation.

1...With Windows, this will NOT work with undionly.kpxe. You must use the full iPXE stack or when windows switches out of preboot it will lose the connection to your iSCSI drive and you end up with inaccessible boot device.
2...With Windows 8 and later, when you first boot off the iSCSI HD, you will get an error about unsigned drivers. You can deal with this in two ways. One, hit F8 and select the option to enable unsigned drivers, or Two, make sure the PE environment you used in step 2 above has EasyBCD or something similar so you can tweak this flag in the BCD before you reboot.
3...Its incredibly important to have set netX/gateway 0.0.0.0 before your sanboot command otherwise it will take a VERY long time to boot (10 min +)
4...It is helpful to set initiator-iqn properly in your ipxe script. This can avoid issues with secured iSCSI devices.
5...I'm not certain if it's important or not to use root-path; it seemed to be important when doing my Windows 7 iSCSI install testing, but once installed it doesn't seem to be needed.

Here's my boot script that works, again, using the full ipxe.pxe/ipxe.lkrn rather than undionly.kpxe

#!ipxe
set iqn-base iqn.2016-01.com.mydomain
set client mysystem
set initiator-iqn ${iqn-base}:${client}
set root-path iscsi:${next-server}::::${iqn-base}:${client}_d0
set netX/gateway 0.0.0.0
sanboot -k ${root-path} || goto doError

Here's a copy of my script that I used to boot into a PE environment with the iSCSI LUN attached:

#!ipxe

set iqn-base iqn.2016-01.com.mydomain
set client mysystem
set initiator-iqn ${iqn-base}:${client}
set root-path iscsi:${next-server}::::${iqn-base}:${client}_d0
set netX/gateway 0.0.0.0
set keep-san 1
sanhook -d 0x80 ${root-path} || goto doError

set hbase http://<path to pxe/ipxe root of server>
set wpath ${hbase}/rescue64 # path to the PE image
kernel ${hbase}/wimboot || goto doError
initrd ${wpath}/bcd BCD || goto doError
initrd ${wpath}/boot.sdi boot.sdi || goto doError
initrd ${wpath}/bootmgr.exe bootmgr.exe || goto doError
initrd ${wpath}/boot.wim boot.wim || goto doError
boot

:doError
shell

[biatche]

Following your instructions, I installed Win10 onto a local ssd, started iscsi initiator & press yes. Booted to w10pese, used macrium free to clone the disk to an iscsi disk (scst). Booting into this w10 iscsi disk gives me "Preparing automatic repair"

what have i done wrong?

[pazhanii]

Windows 10 bcd look for disk signature to match the one it installed. Make sure you create a valid boot entry in bcd pointing to to the new clone iscsi disk. otherwise the said error appears. The tricky part lies in bcd entry. You can even try it with easybcd software. But while doing, bootmgr version should be same of that you used to install.

[biatche]

has this to do with "driver signature"? which ive already disabled via f8 prior to copying to iscsi disk.

so how did you fix it?

[pazhanii]

Disk signature means here the valid bcd id entry which points out the winboot. Even if you clone the disk the actual bcd boot entry will vary. That's why windows throw the error. In windows 7 it might work, But from win 8 and windows 10, bootmgr version got complicated. So unless you create exact bcd entry which windows expect, nothing will happen. If you are clear about bcd and bootmgr version and its working model then you can make it work.

[biatche]

Im playing around with bcd now, i hope you are right because im not entirely convinced since you can already see the windows logo and the loading animation. if its bootmgr, wouldnt i see just text?

[biatche]

went ahead to erase current bcd and rebuilt it via recovery commany prompt (booted via iscsi btw).... same thing is happening..... something else aint right.

[biatche]

to test concept, i had just cloned from one local disk to another local disk and it booted just fine without doing any bcd nonsense. so the problem is iscsi related.

[MultimediaMan]

Bringing the answer from ipxedevel.list: Please use the latest GIT Version (cc9f31 or later); MCB noted a regression and fixed in this commit:

<quote>librm] Do not unconditionally preserve flags across virt_call()

Commit 196f0f2 ("[librm] Convert prot_call() to a real-mode near
call") introduced a regression in which any deliberate modification to
the low 16 bits of the CPU flags (in struct i386_all_regs) would be
overwritten with the original flags value at the time of entry to
prot_call().

The regression arose because the alignment requirements of the
protected-mode stack necessitated the insertion of two bytes of
padding immediately below the prot_call() return address. The
solution chosen was to extend the existing "pushfl / popfl" pair to
"pushfw;pushfl / popfl;popfw". The extra "pushfw / popfw" appears at
first glance to be a no-op, but fails to take into account the fact
that the flags restored by popfl may have been deliberately modified
by the protected-mode function.

Fix by replacing "pushfw / popfw" with "pushw %ss / popw %ss". While
%ss does appear within struct i386_all_regs, any modification to the
stored value has always been ignored by prot_call() anyway.

The most visible symptom of this regression was that SAN booting would
fail since every INT 13 call would be chained to the original INT 13
vector.

Reported-by: Vishvananda Ishaya <vishvananda@gmail.com>
Reported-by: Jamie Thompson <forum.ipxe@jamie-thompson.co.uk></quote>

Best,

M^3

[biatche]

ill try latest ipxe. problem is, for some reason.. my latest ipxe build has been auto-restarting on this test box. so i had used my older compiles to do the testing.

[biatche]

on the latest ipxe right now.. still getting the same problem.

funny thing is i can enter command prompt and list the drives content, which does mean that theres iscsi connectivity.

[MultimediaMan]

Sounds like there is another problem, then. Possibly with the installed OS.

[biatche]

I don't know, but cloning local disk to local disk works fine. So if something isn't right, it should be something related to iscsi/network... i doubt its ipxe/iscsi related problem but more on windows, but anyway thought brownmin would have the answer to it since he has already done it.

[MultimediaMan]

(2016-03-13 09:33)biatche Wrote:  I don't know, but cloning local disk to local disk works fine. So if something isn't right, it should be something related to iscsi/network... i doubt its ipxe/iscsi related problem but more on windows, but anyway thought brownmin would have the answer to it since he has already done it.

There is a LOT to iSCSI booting, the iBFT and the running OS settings ~must~ match, otherwise the OS won't know where the target is.

What OS is this?

[biatche]

win10, as topic.

it was fine in win7 though, im quite tempted to test win7 again, although i dont wish to use win7 anymore.

[BofA]

Hello!
I have almost the same problem with Win10. I've done all as described here, clone Win10 from local drive to iSCSI drive, and then remove local drive. Win10 boots from iSCSI first time with no problems, averything work, but after first restart/shutdown i get "INACCESSIBLE BOOT DEVICE". There is no such problem with Win8. What's wrong with Win10?

[scan80269]

I just achieved a breakthrough in installing Win10 directly to iSCSI LUNs. I have posted some info on how this is done on the ComputerAudiophile.com site:

http://www.computeraudiophile.com/f10-mu...ndex3.html

I've been able to get clients to boot Win10 diskless since early this year, but only recently did I figure out how to do Win10 installations over-the-network into iSCSI LUNs. My previous attempts were plagued by either the dreaded "INACCESSIBLE BOOT DEVICE" or the error "iSCSI deployment is disabled since no NICs referenced in the iBFT can be resolved to actual NT-visible devices" that blocks OS installation into the iSCSI LUN, but now I can install Win10 into iSCSI LUNs without going through physical hard disks or removable storage like USB flash drives.

[scan80269]

(2016-08-13 02:55)scan80269 Wrote:  I just achieved a breakthrough in installing Win10 directly to iSCSI LUNs. I have posted some info on how this is done on the ComputerAudiophile.com site:

http://www.computeraudiophile.com/f10-mu...ndex3.html

I've been able to get clients to boot Win10 diskless since early this year, but only recently did I figure out how to do Win10 installations over-the-network into iSCSI LUNs. My previous attempts were plagued by either the dreaded "INACCESSIBLE BOOT DEVICE" or the error "iSCSI deployment is disabled since no NICs referenced in the iBFT can be resolved to actual NT-visible devices" that blocks OS installation into the iSCSI LUN, but now I can install Win10 into iSCSI LUNs without going through physical hard disks or removable storage like USB flash drives.

I'd like to clarify that my success in installing Win10 directly into iSCSI virtual drive is limited to Windows 10 Anniversary Edition (Version 1607 "Redstone 1"). The same procedure continues to fail with Windows 10 Version 1511 ("Threshold 2") and Windows 10 RTM (build 10240).

It may also depend on whether the NIC in the system has inbox driver support by the OS. Apparently Win10 Anniversary Edition has improved inbox NIC support to enable over-the-network OS installation into iSCSI virtual drive.

[Snowbird]

(2016-08-22 19:36)scan80269 Wrote:  

(2016-08-13 02:55)scan80269 Wrote:  I just achieved a breakthrough in installing Win10 directly to iSCSI LUNs. I have posted some info on how this is done on the ComputerAudiophile.com site:

http://www.computeraudiophile.com/f10-mu...ndex3.html

I've been able to get clients to boot Win10 diskless since early this year, but only recently did I figure out how to do Win10 installations over-the-network into iSCSI LUNs. My previous attempts were plagued by either the dreaded "INACCESSIBLE BOOT DEVICE" or the error "iSCSI deployment is disabled since no NICs referenced in the iBFT can be resolved to actual NT-visible devices" that blocks OS installation into the iSCSI LUN, but now I can install Win10 into iSCSI LUNs without going through physical hard disks or removable storage like USB flash drives.

I'd like to clarify that my success in installing Win10 directly into iSCSI virtual drive is limited to Windows 10 Anniversary Edition (Version 1607 "Redstone 1"). The same procedure continues to fail with Windows 10 Version 1511 ("Threshold 2") and Windows 10 RTM (build 10240).

It may also depend on whether the NIC in the system has inbox driver support by the OS. Apparently Win10 Anniversary Edition has improved inbox NIC support to enable over-the-network OS installation into iSCSI virtual drive.

Yes, I just wanted to confirm the same thing as you, Win10 Anniversary Edition has a better NIC support and I was able to install it, I had no success with all the previous version of Win10...