Using iPXE to install OneFS (FreeBSD 7) via Razor-server

[isi_spike]

New install of razor-server using iPXE 1.0.0+ (86285)
Razor Server version: 0.15.0
Razor Client version: 0.15.1

I am a bit of a PXE noob, but I have banged my head against it enough to know the ropes.

I am working on doing a razor-server install of OneFS (http://www.emc.com/storage/isilon/isilon...system.htm) onto a OneFS hardware node. The currently accepted method for installing OneFS onto a node is to use dd to copy a .img file onto a USB stick

Code:

dd if=/tmp/OneFS_v7.1.1.84_reimg.img  of=/dev/disk2 bs=64k

and then boot from that USB stick. When the node boots from the USB it automatically begins the install process and you are prompted with "Are you sure you wish to destroy the boot drive partitions (yes/no)?" over the serial console for the node, if you enter yes a couple of times it installs just fine. If I can get razor-server to do the same thing, mission complete, I have separate scripts that can take over from the point where you are prompted for input on the serial console. I am not having any issues with razor-server and it appears to be doing everything that it should do. I can install CentOS onto normal bare metal compute nodes without issues using the same razor-server setup.

My problem is that I am unsure of how to effectively replicate the process of booting from this .img file via iPXE as if I were doing it via the local USB port. I have tried dozens of different options in my boot_install.erb (the file containing the iPXE instructions in razor_server), here are a few of my failures, from my memory (so they might not be 100% what I used at the time, but I tried just about everything), to give you an idea.

Note: I am using tftp to transfer the larger files as http (the default) is unreliable for me and tftp is reliable, altho slow.

#!ipxe
sleep 3
imgfree
imgexec tftp://192.168.0.200/OneFS_v7.1.1.84_reimg.img

#!ipxe
sleep 3
kernel <%= repo_url("memdisk") %>
initrd tftp://192.168.0.200/OneFS_v7.1.1.84_reimg.img
boot

#!ipxe
sleep 3
kernel <%= repo_url("memdisk") %> iso
initrd tftp://192.168.0.200/OneFS_v7_1_1_1.iso
boot

I am not even sure that this .iso would work, but it is converted from the .img file and may work. I have mostly abandoned using a .iso as it is a level of abstraction that seems unnecessary and likely to cause problems.



Any help, examples or suggestions on what to try would be greatly appreciated! Thank you in advance.

[isi_spike]

I have found a similar workflow that is known as working and I am unable to reproduce it in my iPXE configuration. I think that if I can get this workflow functioning that I should be able to use the same method to get my privies one working. Here is the configuration file that is being used in PXE for this new method.

SERIAL 0 115200 0
DEFAULT BR_RIPT_BSD10
PROMPT 0
MENU TITLE Force BR_RIPT_BSD10 boot

LABEL BR_RIPT_BSD10
menu label netboot:BR_RIPT_BSD10
kernel memdisk
append initrd=freebsd/BR_RIPT_BSD10-mfsroot.gz raw

MENU end


How do I correctly reproduce this in iPXE? I tried to copy this over in a more 1:1 way, but append does not work in iPXE.


EDIT: I was able to get the .gz file to boot. Unfortuantely it did not do what I needed either so I am still exploring the .img route. To note, this is the config that worked for it:

sleep 3
kernel <%= repo_url("memdisk") %>
imgargs memdisk raw
initrd <%= repo_url("B_RIPT_BSD10_307-mfsroot.gz") %>
boot

[isi_spike]

Something which a colleague noted about our USB install method is that it may be hard coded (I know.. don't get me started on this) to expect the device root to be mounted on /dev/da0s1a.

Is there a way to have the device show up at that mount point via an iPXE configuration value?


EDIT: Upon further investigation it looks like it is ending up with 1s1a by default and I need it to be 0s1a. Can anyone provide any ideas or even random guesses on this one?

[robinsmidsrod]

It seems like this OneFS is based on some form of BSD operating system. You probably need to pull apart the image and figure out how to load the kernel and ramdisk separately (with kernel and initrd ipxe commands). My guess is that the installer uses some form of squashfs/temp-filesystem that must be downloaded from the install image (via CD/USB drivers). You most likely have to specify that this should be loaded over the network instead using a kernel command-line. I would try to search for how this BSD it is based off does PXE booting. You might find luck there. If not you might need to unpack the image and inspect its boot process.

[isi_spike]

Thank you for replying robinsmidsrod.

OneFS is indeed based on FreeBSD, most versions of it are based on FreeBSD 7 which is rather old and doesn't have mush support for PXE. I was able to build a special build of OneFS that is based on FreeBSD 10 and then mash that into a format that can be loaded into a memdisk. This gets me to the point of running OneFS out of memory and I am then able to use a convoluted shell script to reimage the node using any build that I want. I am still having issues getting the machine to boot after this script runs, but I have confirmed that the OS is there.

Once I have a solid process for doing this I will see about getting it published.

[robinsmidsrod]

Feel free to publish a howto here, or I'd also welcome a paragraph or two on networkboot.org under the tutorials section. I'm sure several people would be interested in how you did the reimage thingy as well. OneFS is some form of storage-related OS, right?

[isi_spike]

(2014-12-06 21:03)robinsmidsrod Wrote:  OneFS is some form of storage-related OS, right?

Yeah, its a product which consists of a OS built on top of FreeBSD and a hardware platform.