Installation and Configuration of CentOS7 Diskless OS for x86_64, armv7, and aarch64
Table of Contents
1. Installation and Configuration Intro
The following guide describes the installation and configuration of CentOS7 for use with diskless clients running x86_64
(e.g. VME Controllers), armv7
(VTPs), or aarch64
(MOLLERADC).
It's my hope that this will be useful to anyone left to do this on their own.
This guide assumes that you have a DHCP that can be configured with these parameters
filename "pxelinux.0"; next-server 129.57.xxx.xxx;
Where next-server
should reflect the IPv4 address of your (tftp) host machine.
It also assumes that your host machine and VME/VTP have DNS entries.
dnsmasq is a program that can be used to handle TFTP, DHCP, and DNS. Here's a good write-up I found describing it's use with pxelinux: PXELinux using Proxy DHCP
1.1. The Boot Process
1.1.1. VME
BIOS configured for PXE (check your settings from these pages)
- DHCP provides info to connect to bootloader
- ip address
- tftp server address (your machine's hostname)
- tftp filename to load (
pxelinux.0
)
- pxelinux
- boot environment configured
pxelinux.cfg/<address files>
- tftp to get kernel and initial ramdisk
- boot environment configured
- kernel
- loads up operating system filesystem using NFS
1.1.2. VTP
uboot loaded on power up
- uboot
- environment loaded from
env.txt
on microSD - tftp to get kernel and initial ramdisk
- environment loaded from
- kernel
- loads up operating system filesystem using NFS
1.1.3. MOLLERADC
uboot configured for PXE
- DHCP provides info to connect to uboot
- ip address
- tftp server address (your machine's hostname)
- tftp filename to load (
pxelinux.0
)
- pxelinux
- boot environment configured
pxelinux.cfg/<address files>
- tftp to get kernel and initial ramdisk
- boot environment configured
- kernel
- loads up operating system filesystem using NFS
2. Host packages to install
nfs-utils, tftp-server, syslinux-tftpboot, syslinux
- I installed these on my host system using yum
yum install nfs-utils tftp-server syslinux-tftpboot syslinux
3. Download
3.1. centos7_dist
- Clone the
centos7_dist
repo from github
git clone --depth 1 https://code.jlab.org/fedaq/coda/utils/centos7_dist.git
or on site
git clone --depth 1 /site/coda/contrib/devel/centos7_dist
3.2. tgz’s
Download and install the root and tftp pair for your module(s).
- VTP (armv7):
- VME (x86_64):
- MOLLERADC (aarch64):
I downloaded these and moved them into the
tgz
directorycentos7_dist/tgz/root_x86_64-rc5.tgz centos7_dist/tgz/tftp_x86_64-rc2.tgz
4. Configure the distribution
4.1. VME OS
I put together the following configuration for my system:
# The local paths for NFS exports to diskless paths: /, /home, and snapshot files # using base path # /usr/local/diskless # # Uncomment and set, if running the diskless_* scripts on a machine # that's not the indented NFS server. Otherwise the default is NFS_SERVER_HOSTNAME=$(hostname -s) #NFS_SERVER_HOSTNAME="daqfs.jlab.org" NFS_PATH=/usr/local/diskless/CentOS7/x86_64 NFS_ROOT_PATH=${NFS_PATH}/root NFS_SNAPSHOT_PATH=${NFS_PATH}/snapshot NFS_HOME_PATH=${NFS_PATH}/home NFS_EXPORTS=centos7-x86_64.exports # # The hostname of our vme controller # CLIENT_HOSTNAME[0]="davme6" # # root filesystem and kernel boot files # TGZ_ROOT=tgz/root_x86_64-rc5.tgz TGZ_TFTP=tgz/tftp_x86_64-rc2.tgz # # kernel boot files specific to TGZ_TFTP version # VMLINUZ=vmlinuz-3.10.0-1062.9.1.el7.x86_64 INITRD=initramfs-3.10.0-1062.9.1.el7.x86_64.img # # tftp files # TFTP_PATH=/var/lib/tftpboot TFTP_PXELINUX_CFG_PATH=${TFTP_PATH}/pxelinux.cfg TFTP_PXELINUX_CFG_DEFAULT=centos7-x86_64.default TFTP_KERNEL_PATH=${TFTP_PATH}/CentOS7-x86_64-Diskless # # Template files # TEMPLATE_PATH=templates YP_CONF_TEMPLATE=${TEMPLATE_PATH}/yp.conf READONLY_ROOT_TEMPLATE=${TEMPLATE_PATH}/readonly-root PXECONFIG_ADDR_TEMPLATE=${TEMPLATE_PATH}/pxelinux.cfg_ADDR AUTOFS_HOME_TEMPLATE=${TEMPLATE_PATH}/auto.home UBOOT_ENV_TEMPLATE=${TEMPLATE_PATH}/env.txt
NFS_PATH
This is where the centos7 x86_64 filesystem will be unpacked.
- I updated this path to this location on my host system
NFS_PATH=/usr/local/diskless/CentOS7/x86_64
CLIENT_HOSTNAME[n]
- I only have one VME controller at the moment, it's hostname will be
davme6.jlab.org
. But's short hostname is what is needed here:
CLIENT_HOSTNAME[0]=davme6
- I only have one VME controller at the moment, it's hostname will be
TGZ_ROOT, TGZ_TFTP, VMLINUZ, INITRD
- These ought to be defined in an sample configuration file provided with the tgz's. They contain the root filesystem, and boot files needed for the diskless client.
TFTP_PATH
Sometimes this path will be different for different types of tftp servers.
- I'm using tftp-server on RHEL 7. So this path is the default:
TFTP_PATH=/var/lib/tftpboot
The scripts will generate the necessary configuration files and commands to install them (as root or with sudo)
4.1.1. diskless_ROOTOS.sh
I ran this command using my config file as the only argument
./diskless_ROOTOS.sh walkthrough_example.cfg
# # Generating yp.conf # # ... Generated: output/yp.conf # # Generating home.autofs # # ... Generated: output/auto.home # # Generating readonly-root # # ... Generated: output/readonly-root # # # Extract TGZ_ROOT=tgz/root_x86_64-rc5.tgz to # # NFS_ROOT_PATH=/usr/local/diskless/CentOS7/x86_64/root install -b -p -m 755 -d /usr/local/diskless/CentOS7/x86_64/root tar xzf tgz/root_x86_64-rc5.tgz -C /usr/local/diskless/CentOS7/x86_64/root # # # Make the snapshot and home directories install -b -p -m 755 -d /usr/local/diskless/CentOS7/x86_64/snapshot install -b -p -m 755 -d /usr/local/diskless/CentOS7/x86_64/home # # # To install yp.conf install -b -p -m 644 output/yp.conf /usr/local/diskless/CentOS7/x86_64/root/etc/yp.conf # # # To Install auto.home install -b -p -m 644 templates/home.autofs /usr/local/diskless/CentOS7/x86_64/root/etc/auto.master.d/home.autofs install -b -p -m 644 output/auto.home /usr/local/diskless/CentOS7/x86_64/root/etc/auto.master.d/auto.home # # # To Install readonly-root # install -b -p -m 644 output/readonly-root /usr/local/diskless/CentOS7/x86_64/root/etc/sysconfig/readonly-root
- This one generates a number of config files for the client OS and provides the installation commands to be run in sequence using sudo or as the root user.
4.1.2. diskless_NFSexports.sh
This script generates the exports files needed for NFS
./diskless_NFSexports.sh walkthrough_example.cfg
# # Generating NFS exports... # ... Generated: output/centos7-x86_64.exports # # To install NFS exports: install -b -p -m 644 output/centos7-x86_64.exports /etc/exports.d/centos7-x86_64.exports
4.1.3. diskless_PXE-TFTP.sh
This script generates the configuration and installation commands for PXE and TFTP.
./diskless_PXE-TFTP.sh walkthrough_example.cfg
# # Generating pxelinux config files # ... Linked: output/81391D92 to centos7-x86_64.default # # To Install pxelinux config files install -b -p -m 755 -d /var/lib/tftpboot/pxelinux.cfg/ install -b -p -m 644 output/centos7-x86_64.default /var/lib/tftpboot/pxelinux.cfg/ # # Link: /var/lib/tftpboot/pxelinux.cfg/81391D92 to centos7-x86_64.default ln -sf centos7-x86_64.default /var/lib/tftpboot/pxelinux.cfg/81391D92 # # Extract TGZ_TFTP=tgz/tftp_x86_64-rc2.tgz to # TFTP_KERNEL_PATH=/var/lib/tftpboot/CentOS7-x86_64-Diskless install -b -p -m 755 -d /var/lib/tftpboot/CentOS7-x86_64-Diskless/ tar xzvf tgz/tftp_x86_64-rc2.tgz -C /var/lib/tftpboot/CentOS7-x86_64-Diskless
4.2. VTP OS
If you've got a VTP, the configuration will be very similar, mostly swapping x86_64 with armv7.
CLIENT_HOSTNAME[n]
I only have one VTP, it's hostname will be
davtp1.jlab.org
. It's short hostname is all that's needed here:CLIENT_HOSTNAME[0]="davtp1"
VTP_MAC_ADDR
This parameter is used in the generation of the VTPs environment file. The MAC address is unique for each VTP, and can be obtained from Ben Raydo.
VTP_MAC_ADDR="ce:ba:f0:03:cc:ff"
Execute the scripts that were called for the VME OS using the armv7 config file.
4.2.1. diskless_UBOOT.sh
This script generates the uboot env.bin
that needs to be loaded onto the microSD used by the VTP
./diskless_UBOOT.sh walkthrough_vtp_example.cfg
# # Generating U-Boot Environment # output/env.bin now ready for VTP microSD
- Copy the resulting file
output/env.bin
to the VTPs microSD FAT partition.
4.3. MOLLERADC OS
The MOLLERADC (aarch64) configuration is similar to the VME (x64_64) configuration. u-boot for the MOLLERADC uses PXE. The MAC address of the POE++ connection should be labeled on the front panel.
the relevant scripts to use with the example
./diskless_ROOTOS.sh molleradc_example.cfg ./diskless_NFSexports.sh molleradc_example.cfg ./diskless_PXE-TFTP.sh molleradc_example.cfg
- skip the home directory and auto.home install
- a
coda
user + home directory is already provided within theroot_aarch64-rc2.tgz
distribution
- a
5. Enable and start the NFS-server service
As root, execute
systemctl enable nfs-server.service systemctl start nfs-server.service
6. Enable and start the TFTP server
As root, execute
systemctl enable tftp systemctl start tftp
7. First Login
7.1. Setup daq account
- I logged in as root and set up the home directory for the daq account and set its password
ssh root@davme6 /usr/sbin/mkhomedir_helper daq passwd daq
- Change the root password while you're at it.
- Now you can log in as the daq user
ssh daq@davme6
8. Fixes
8.1. VTP Booting sometimes halts (fixed in rc6)
in serial debug connection, booting stops/halts at
[ OK ] Reached target Local File Systems (Pre). Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Found device /dev/ttyPS0.
8.1.1. Update rhel-readonly.service
- Change the
/etc/systemd/system/rhel-readonly.service [Unit]
section to read:
[Unit] Description=Configure read-only root support DefaultDependencies=no Conflicts=shutdown.target systemd-tmpfiles-setup.service Before=shutdown.target emergency.service emergency.target local-fs.target systemd-random-seed.service After=systemd-remount-fs.service