Windows 8/Server 2012

[hech]

Hi,

I'm going to setting up a diskless and headless computing cluster. (no monitor, keyboard, mouse)

Right now I'm in testing phase, here's my setup:
1 Win Server 2012 with MSDHCP, IIS 8, iSCSI Target and a TFTP server for undionly.
1 Diskless Node with display so I can monitor progress
1 Diskless Headless Node
Gigabit LAN
WinPE4

Windows Server Unattended Installation finishes then boots to OS at first, then I shut it down.

This is where my problems start: I turn it back on, it loads for a long time then fails, it restarts automatically then boots to Windows fine.

This is my real problem: When the boot fails and it restarts, its MAC address are all zeros. When it has its original MAC address it does NOT boot, when it restarts with the MAC address 00 00 00 00 00 that is when it boots to windows.

PS: I will be using differencing VHD's with this, one child VHD per node.

I know nothing about PHP, dynamic scripts, and databases so I used this simple script I made:

Code:

!ipxe

:retry
dhcp net0 || goto retry
set initiator-iqn computenode
set arch amd64
set keep-san 1
sanboot --drive 0x80 iscsi:${root-path}::::${mac} ||
module amd64/media/bootmgr                     bootmgr        ||
module amd64/media/Boot/BCD                    BCD        ||
module amd64/media/Boot/Fonts/segmono_boot.ttf segmono_boot.ttf    ||
module amd64/media/Boot/Fonts/segoe_slboot.ttf segoe_slboot.ttf    ||
module amd64/media/Boot/Fonts/wgl4_boot.ttf    wgl4_boot.ttf    ||
module amd64/media/Boot/boot.sdi               boot.sdi        ||
module amd64/media/sources/boot.wim            boot.wim        ||
chain wimboot || reboot

Any help would be greatly appreciated.

[robinsmidsrod]

Your initiator-iqn and iscsi: target path doesn't look very proper. You initiator-iqn should look something like what you can see at http://ipxe.org/cfg/initiator-iqn and your iscsi target should look like what you can see at http://ipxe.org/cmd/sanboot.

Also I notice you specify the arch variable but never use it. Your module lines should look like this: module ${arch}/media/bootmgr bootmgr. There is also no need to specify net0 to when you're doing dhcp. The dhcp command tries all network devices in order until it finds one that has a response.

You might be suffering from the iBFT default gateway issue. Try set netX/gateway 0.0.0.0 before the sanboot command and see if it helps.

If you get an all-zeros MAC address something is very wrong. That should not happen. What kind of hardware are those machines using? It would be beneficial if you can capture the output of the ifstat command after having executed dhcp using the iPXE shell. That should give us more clues.

I'm going to assume you're using the latest version of wimboot and ipxe? A precompiled version of iPXE can be downloaded from http://rom-o-matic.eu/ and wimboot is at http://ipxe.org/wimboot.

[hech]

Thank you very much for replying.

My initiator iqn and iscsi target name works. But I will try using the iqn standard. Maybe something like:

Code:

set initiator-iqn iqn.2010-04.org.ipxe:${mac:hexhyp}
sanboot iscsi:10.0.4.1:::1:iqn.2010-04.org.ipxe:${mac:hexhyp}

I used "module amd64/media/bootmgr" because all nodes will be x64.

Will try using "set netX/gateway 0.0.0.0".

We will be using commodity hardware.
I have three of this motherboard for testing:
http://www.asrock.com/mb/Intel/H61M-VG4/
Ivy Bridge for processors.

I'm thinking it's a hardware issue. Maybe the NIC - Realtek RTL8111E? Will try a different mobo.

I will do what I can to get more data.

Installs smoothly, restarts, loads for a long time then goes bluescreen and restarts again with the MAC addresses all zero then boots to desktop.(I have to set a fixed script for the iscsi target so it would boot because it is based on MAC address)

It does not boot to desktop when it has the right MAC address, it boots only when it is all zero. Unfortunately, I can't be stuck with that because I can't have nodes with the same MAC addresses. We will be having 200+ compute nodes.

https://docs.google.com/file/d/0B93pgI0J...sp=sharing
https://docs.google.com/file/d/0B93pgI0J...sp=sharing

Yes I am using latest wimboot and ipxe. Latest WinPE.

Will retry everything on hyper-v. I have been on this for 16hours X 7days now. Happy I'm doing progress little by little, but I'm stuck on this.

Thank you again.

[hech]

I tried this script on a desktop and a laptop, still not working:

Code:

#!ipxe

:retry
dhcp || goto retry
set initiator-iqn iqn.2010-04.org.ipxe:computenode
set keep-san 1
set netX/gateway 0.0.0.0
sanboot iscsi:192.168.1.2::::iqn.1991-05.com.microsoft:server11-bc-5f-f4-b0-3f-f4 ||
module amd64/media/bootmgr                     bootmgr        ||
module amd64/media/Boot/BCD                    BCD        ||
module amd64/media/Boot/Fonts/segmono_boot.ttf segmono_boot.ttf    ||
module amd64/media/Boot/Fonts/segoe_slboot.ttf segoe_slboot.ttf    ||
module amd64/media/Boot/Fonts/wgl4_boot.ttf    wgl4_boot.ttf    ||
module amd64/media/Boot/boot.sdi               boot.sdi        ||
module amd64/media/sources/boot.wim            boot.wim        ||
chain wimboot || reboot

Just installs, restarts after installation, loads a long time then ends in a blue screen, and restarts again.

with the desktop it restarts with mac 00 00 00 00 00 00 but can't get past DHCP loading anymore. I turn it off for a while then the original MAC returns but it just repeats.

with the laptop it restarts, it mounts iscsi target and loads a long long time until blue screen then restarts again.

here's the ifstat on the desktop:
net0: bc:5f:f4:b0:3f:f4 using undionly on UNDI-PCI02:00.0 (open)
[Link:up, TX:3 TXE:0 RX:4 RXE:0]

I only have one NIC per node.

Does it matter if I disable the built-in SATA controller, or if it's IDE or AHCI?

What is a mystery to me is that it boots to desktop(and is fast) during the restart to all zero MAC address. will get ifstat when it happens again.

All hardware(mobo, procie, ram, switch, cables, peripherals) were just bought a week ago specifically for this project.

Thanks again.

[hech]

I also tried windows 7 just now, both 32 and 64-bit.

It says error on reboot after installation.

On 32bit, it says:
Failed to load because a required file is missing, or corrupt.
File is Rt630x86.sys

On 64bit:
Rt630x64.sys
Says wrong digital signature. I already tried disabling driver signature enforcement.

I already tried including the basic drivers on the boot.wim for winPE, boot.wim, and install.wim.

[MultimediaMan]

You're having problems with your Windows PE files. Realtek NICs aren't really all that bad... it's that there are a lot of counterfeit/clone "Not-So-Realteks" out there. And Realtek isn't very particular about how OEMs use their chips.

My suggestion is to try a different NIC.

[hech]

Thanks for the advice man. All three of my test systems and the laptop I used has realtek. Will try a different one soon. I was already content with realtek for a long time and then this.

[MultimediaMan]

(2013-07-21 13:56)hech Wrote:  Thanks for the advice man. All three of my test systems and the laptop I used has realtek. Will try a different one soon.

I feel your pain... I had some high hopes for some motherboards only to have them dashed by buggy hardware.

[hech]

Okay! This is now resolved. I bought an Intel Mobo with an Intel NIC and everything went smooth.

Thanks for all the help guys.

[robinsmidsrod]

hech: About the realtek motherboards. I noticed you were using undionly.kpxe. You might've had more luck with ipxe.pxe, as that would've used a native iPXE Realtek driver. Your issue with a broken MAC might be related to buggy behavior in the onboard PXE firmware. If you specifically cold-boot the machine and boot it using ipxe.usb (from a USB stick), does it still give you all zeros MAC? If that works, but ipxe.pxe loaded using chainloading does not, then it is pretty obvious the built-in firmware has a problem. If only undionly.kpxe ends up giving you an all-zeros MAC, then ipxe.pxe might be possible to use (if you control all the nodes in the network and it turned out to work). Just make sure you cold-boot your machines during testing. Warm reboot has been known to give false test results. Unplugging the power cable is usually the safest choice (because of how Wake-On-Lan works) to ensure you get a proper cold-boot.

Also, I noticed your root-path in your DHCP setup is just an IP address. AFAIK, that won't work with iPXE unless you're using a specialized iPXE script. If you meant to boot the iscsi target, you need to put the full IQN address to the target there.

[hech]

how do I choose which driver to use?

problem is back. I installed server 2012, it reboots, goes to desktop. I shut it down, when I turn it back on, it gives me the long loading time then bluescreen then reboots again. This is with the Intel NIC.

Quote:Also, I noticed your root-path in your DHCP setup is just an IP address. AFAIK, that won't work with iPXE unless you're using a specialized iPXE script. If you meant to boot the iscsi target, you need to put the full IQN address to the target there.

I matched that with a specialized iPXE script. I have no problem with attaching.

I replaced my two asrock mobos that has realtek nic with two intel mobos. one with intel NIC and one with realtek nic.

How do I choose which driver to use?

[hech]

i found a driver specific for my intel NIC: 82579lm

what's the difference with pxe and kpxe and kkpxe? which should I use?

i'm using ipxe.pxe now, it made loading boot.wim much slower, like 100Mbps or 10Mbps. do I have to change the default "115200" baud rate/com speed?

still have the problem, it installs fine, reboots to desktop. when i shut it down, it doesn't boot to desktop anymore.

[MultimediaMan]

(2013-07-25 04:42)hech Wrote:  i found a driver specific for my intel NIC: 82579lm

what's the difference with pxe and kpxe and kkpxe? which should I use?

i'm using ipxe.pxe now, it made loading boot.wim much slower, like 100Mbps or 10Mbps. do I have to change the default "115200" baud rate/com speed?

still have the problem, it installs fine, reboots to desktop. when i shut it down, it doesn't boot to desktop anymore.

Hech; RE: Wim loading slowly. I have reason to believe there's a bug in there somewhere. I'm going to post the problem on the developer's lists.

[hech]

the download was slow. i even saw the boot.sdi downloading.

i tried undionly.kpxe, and both the ipxe.pxe and ipxe.kpxe with specific driver for my intel 82579lm.

installs and reboots to windows smoothly but when I shut it down it won't boot to desktop anymore.

[robinsmidsrod]

It could be _some_ issue with the iPXE 82579LM driver. I've using the same, just the 82579V model on my motherboard, and I do recall us fixing a bug on that particular card. Could you use ipxe.pxe and run through the driver tests at http://ipxe.org/dev/driver and add a report to the table if it is successful? If you get errors then they should be reported to the mailing-list.

[robinsmidsrod]

(2013-07-26 05:30)hech Wrote:  my chip is intel 82579lm, I downloaded a 82579lm ipxe.pxe. on cold-boot and restarts, it boots to windows with chip name 82579lm but when I do a shutdown and turn it on again without unplugging, the chip name becomes 82579v, that is when it does not boot. how does it become 82579v? what do I do about it?

That is exactly the same that happened on my desktop computer, but the other way around. I had to get the core developers to debug this issue and fix it. It is because of some oddball EEPROM issue.

I'd suggest building with DEBUG=intel:3 and serial, and then send off a bug report to the mailing-list with the details. Coming online and doing it on IRC is even better. I would also suggest you go through the driver test checklist on the wiki and report which tests fail and which pass.

[hech]

Sorry. I got it all wrong. my chip is 82579v and it changes bus-id to 82579lm on most restarts and shutdowns.

I updated BIOS version from 44 to 48, from 48 to 99, and from 99 to 117.

I also found this update for NVM Update Utility for Intel 82579V Gigabit Ethernet PHY Network Connection.
https://downloadcenter.intel.com/Detail_...pkw=82579v

This utility resolves an issue where during system resume, the Intel® 82579V Gigabit Ethernet PHY Network Connection erroneously reports the device id as an Intel® 82579LM Gigabit Ethernet Controller Network Connection, resulting in a Windows* Code 10 error and loss of network connection.

Not all systems with the Intel 82579V Network connection will see this problem. Systems using Microsoft Windows* 8 are more susceptible to the issue.

If you experience this error, Intel recommends that you update the non-volatile memory for your network connection by downloading and running the NVM Update Utility.


I also included NDIS 6.3 drivers for x64 to the winPE boot.wim, win8 boot.wim, and win8 install.wim.

Finally! It is working as it should.

[robinsmidsrod]

hech: Great that you finally resolved your issues.

[spooch]

(2013-07-28 15:55)hech Wrote:  ...
I also included NDIS 6.3 drivers for x64 to the winPE boot.wim, win8 boot.wim, and win8 install.wim.

Finally! It is working as it should.

Were you able to sysprep your image directly on iscsi disk? I tried this with win8 (64bit), but after sysprep, the reboot hanged

[hech]

I was. Try not using unattend files. And you need same motherboard and same NIC I think. I tried different board with same NIC, it hangs also. Sorry for late reply. I didn't try it on Win7.

(2013-08-16 03:08)spooch Wrote:  

(2013-07-28 15:55)hech Wrote:  ...
I also included NDIS 6.3 drivers for x64 to the winPE boot.wim, win8 boot.wim, and win8 install.wim.

Finally! It is working as it should.

Were you able to sysprep your image directly on iscsi disk? I tried this with win8 (64bit), but after sysprep, the reboot hanged