Installation and Configuration of CentOS7 Diskless OS for x86_64, armv7, and aarch64

Table of Contents

1. Installation and Configuration Intro

The following guide describes the installation and configuration of CentOS7 for use with diskless clients running x86_64 (e.g. VME Controllers), armv7 (VTPs), or aarch64 (MOLLERADC). It's my hope that this will be useful to anyone left to do this on their own.

This guide assumes that you have a DHCP that can be configured with these parameters

filename "pxelinux.0";
next-server 129.57.xxx.xxx;

Where next-server should reflect the IPv4 address of your (tftp) host machine.

It also assumes that your host machine and VME/VTP have DNS entries.

dnsmasq is a program that can be used to handle TFTP, DHCP, and DNS. Here's a good write-up I found describing it's use with pxelinux: PXELinux using Proxy DHCP

1.1. The Boot Process

1.1.1. VME

BIOS configured for PXE (check your settings from these pages)

  1. DHCP provides info to connect to bootloader
    • ip address
    • tftp server address (your machine's hostname)
    • tftp filename to load (pxelinux.0)
  2. pxelinux
    • boot environment configured pxelinux.cfg/<address files>
    • tftp to get kernel and initial ramdisk
  3. kernel
    • loads up operating system filesystem using NFS

1.1.2. VTP

uboot loaded on power up

  1. uboot
    • environment loaded from env.txt on microSD
    • tftp to get kernel and initial ramdisk
  2. kernel
    • loads up operating system filesystem using NFS

1.1.3. MOLLERADC

uboot configured for PXE

  1. DHCP provides info to connect to uboot
    • ip address
    • tftp server address (your machine's hostname)
    • tftp filename to load (pxelinux.0)
  2. pxelinux
    • boot environment configured pxelinux.cfg/<address files>
    • tftp to get kernel and initial ramdisk
  3. kernel
    • loads up operating system filesystem using NFS

2. Host packages to install

nfs-utils, tftp-server, syslinux-tftpboot, syslinux
  • I installed these on my host system using yum
yum install nfs-utils tftp-server syslinux-tftpboot syslinux

3. Download

3.1. centos7_dist

  • Clone the centos7_dist repo from github
git clone --depth 1 https://code.jlab.org/fedaq/coda/utils/centos7_dist.git

or on site

git clone --depth 1 /site/coda/contrib/devel/centos7_dist

3.2. tgz’s

Download and install the root and tftp pair for your module(s).

  • VTP (armv7):
  • VME (x86_64):
  • MOLLERADC (aarch64):
  • I downloaded these and moved them into the tgz directory

    centos7_dist/tgz/root_x86_64-rc5.tgz
    centos7_dist/tgz/tftp_x86_64-rc2.tgz
    

4. Configure the distribution

4.1. VME OS

I put together the following configuration for my system:

# The local paths for NFS exports to diskless paths: /, /home, and snapshot files
#   using base path
#       /usr/local/diskless
#
# Uncomment and set, if running the diskless_* scripts on a machine
#  that's not the indented NFS server.  Otherwise the default is NFS_SERVER_HOSTNAME=$(hostname -s)
#NFS_SERVER_HOSTNAME="daqfs.jlab.org"

NFS_PATH=/usr/local/diskless/CentOS7/x86_64

NFS_ROOT_PATH=${NFS_PATH}/root
NFS_SNAPSHOT_PATH=${NFS_PATH}/snapshot
NFS_HOME_PATH=${NFS_PATH}/home

NFS_EXPORTS=centos7-x86_64.exports

#
# The hostname of our vme controller
#
CLIENT_HOSTNAME[0]="davme6"

#
# root filesystem and kernel boot files
#
TGZ_ROOT=tgz/root_x86_64-rc5.tgz
TGZ_TFTP=tgz/tftp_x86_64-rc2.tgz

#
# kernel boot files specific to TGZ_TFTP version
#
VMLINUZ=vmlinuz-3.10.0-1062.9.1.el7.x86_64
INITRD=initramfs-3.10.0-1062.9.1.el7.x86_64.img

#
# tftp files
#
TFTP_PATH=/var/lib/tftpboot
TFTP_PXELINUX_CFG_PATH=${TFTP_PATH}/pxelinux.cfg
TFTP_PXELINUX_CFG_DEFAULT=centos7-x86_64.default
TFTP_KERNEL_PATH=${TFTP_PATH}/CentOS7-x86_64-Diskless

#
# Template files
#
TEMPLATE_PATH=templates
YP_CONF_TEMPLATE=${TEMPLATE_PATH}/yp.conf
READONLY_ROOT_TEMPLATE=${TEMPLATE_PATH}/readonly-root
PXECONFIG_ADDR_TEMPLATE=${TEMPLATE_PATH}/pxelinux.cfg_ADDR
AUTOFS_HOME_TEMPLATE=${TEMPLATE_PATH}/auto.home
UBOOT_ENV_TEMPLATE=${TEMPLATE_PATH}/env.txt
  • NFS_PATH

    This is where the centos7 x86_64 filesystem will be unpacked.

    • I updated this path to this location on my host system
    NFS_PATH=/usr/local/diskless/CentOS7/x86_64
    
  • CLIENT_HOSTNAME[n]

    • I only have one VME controller at the moment, it's hostname will be davme6.jlab.org. But's short hostname is what is needed here:
    CLIENT_HOSTNAME[0]=davme6
    
  • TGZ_ROOT, TGZ_TFTP, VMLINUZ, INITRD
    • These ought to be defined in an sample configuration file provided with the tgz's. They contain the root filesystem, and boot files needed for the diskless client.
  • TFTP_PATH

    Sometimes this path will be different for different types of tftp servers.

    • I'm using tftp-server on RHEL 7. So this path is the default:
    TFTP_PATH=/var/lib/tftpboot
    

The scripts will generate the necessary configuration files and commands to install them (as root or with sudo)

4.1.1. diskless_ROOTOS.sh

  • I ran this command using my config file as the only argument

    ./diskless_ROOTOS.sh walkthrough_example.cfg
    
    # 
    # Generating yp.conf
    # 
    # ... Generated: output/yp.conf
    # 
    # Generating home.autofs
    # 
    # ... Generated: output/auto.home
    # 
    # Generating readonly-root
    # 
    # ... Generated: output/readonly-root
    # 
    # #   Extract TGZ_ROOT=tgz/root_x86_64-rc5.tgz to 
    # #   NFS_ROOT_PATH=/usr/local/diskless/CentOS7/x86_64/root
    install -b -p -m 755 -d /usr/local/diskless/CentOS7/x86_64/root
    tar xzf tgz/root_x86_64-rc5.tgz -C /usr/local/diskless/CentOS7/x86_64/root
    # 
    # #  Make the snapshot and home directories
    install -b -p -m 755 -d /usr/local/diskless/CentOS7/x86_64/snapshot
    install -b -p -m 755 -d /usr/local/diskless/CentOS7/x86_64/home
    # 
    # #   To install yp.conf
    install -b -p -m 644 output/yp.conf /usr/local/diskless/CentOS7/x86_64/root/etc/yp.conf
    # 
    # #   To Install auto.home
    install -b -p -m 644 templates/home.autofs /usr/local/diskless/CentOS7/x86_64/root/etc/auto.master.d/home.autofs
    install -b -p -m 644 output/auto.home /usr/local/diskless/CentOS7/x86_64/root/etc/auto.master.d/auto.home
    # 
    # #   To Install readonly-root
    # 
    install -b -p -m 644 output/readonly-root /usr/local/diskless/CentOS7/x86_64/root/etc/sysconfig/readonly-root
    
  • This one generates a number of config files for the client OS and provides the installation commands to be run in sequence using sudo or as the root user.

4.1.2. diskless_NFSexports.sh

  • This script generates the exports files needed for NFS

    ./diskless_NFSexports.sh walkthrough_example.cfg
    
    # 
    # Generating NFS exports...
    #   ... Generated: output/centos7-x86_64.exports
    # 
    # To install NFS exports:
    install -b -p -m 644 output/centos7-x86_64.exports /etc/exports.d/centos7-x86_64.exports
    

4.1.3. diskless_PXE-TFTP.sh

  • This script generates the configuration and installation commands for PXE and TFTP.

    ./diskless_PXE-TFTP.sh walkthrough_example.cfg
    
    # 
    # Generating pxelinux config files
    #   ... Linked: output/81391D92 to centos7-x86_64.default
    # 
    # To Install pxelinux config files
    install -b -p -m 755 -d /var/lib/tftpboot/pxelinux.cfg/
    install -b -p -m 644 output/centos7-x86_64.default /var/lib/tftpboot/pxelinux.cfg/
    # 
    # Link: /var/lib/tftpboot/pxelinux.cfg/81391D92 to centos7-x86_64.default
    ln -sf centos7-x86_64.default /var/lib/tftpboot/pxelinux.cfg/81391D92
    # 
    # Extract TGZ_TFTP=tgz/tftp_x86_64-rc2.tgz to 
    # TFTP_KERNEL_PATH=/var/lib/tftpboot/CentOS7-x86_64-Diskless
    install -b -p -m 755 -d /var/lib/tftpboot/CentOS7-x86_64-Diskless/
    tar xzvf tgz/tftp_x86_64-rc2.tgz -C /var/lib/tftpboot/CentOS7-x86_64-Diskless
    

4.2. VTP OS

If you've got a VTP, the configuration will be very similar, mostly swapping x86_64 with armv7.

  • CLIENT_HOSTNAME[n]

    I only have one VTP, it's hostname will be davtp1.jlab.org. It's short hostname is all that's needed here:

    CLIENT_HOSTNAME[0]="davtp1"
    
  • VTP_MAC_ADDR

    This parameter is used in the generation of the VTPs environment file. The MAC address is unique for each VTP, and can be obtained from Ben Raydo.

    VTP_MAC_ADDR="ce:ba:f0:03:cc:ff" 
    

Execute the scripts that were called for the VME OS using the armv7 config file.

4.2.1. diskless_UBOOT.sh

This script generates the uboot env.bin that needs to be loaded onto the microSD used by the VTP

./diskless_UBOOT.sh walkthrough_vtp_example.cfg
# 
# Generating U-Boot Environment
#  output/env.bin now ready for VTP microSD
  • Copy the resulting file output/env.bin to the VTPs microSD FAT partition.

4.3. MOLLERADC OS

The MOLLERADC (aarch64) configuration is similar to the VME (x64_64) configuration. u-boot for the MOLLERADC uses PXE. The MAC address of the POE++ connection should be labeled on the front panel.

  • the relevant scripts to use with the example

    ./diskless_ROOTOS.sh molleradc_example.cfg
    ./diskless_NFSexports.sh molleradc_example.cfg
    ./diskless_PXE-TFTP.sh molleradc_example.cfg
    
  • skip the home directory and auto.home install
    • a coda user + home directory is already provided within the root_aarch64-rc2.tgz distribution

5. Enable and start the NFS-server service

  • As root, execute

    systemctl enable nfs-server.service 
    systemctl start nfs-server.service 
    

6. Enable and start the TFTP server

  • As root, execute

    systemctl enable tftp
    systemctl start tftp
    

7. First Login

7.1. Setup daq account

  • I logged in as root and set up the home directory for the daq account and set its password
ssh root@davme6
/usr/sbin/mkhomedir_helper daq
passwd daq
  • Change the root password while you're at it.
  • Now you can log in as the daq user
ssh daq@davme6

8. Fixes

8.1. VTP Booting sometimes halts (fixed in rc6)

  • in serial debug connection, booting stops/halts at

    [  OK  ] Reached target Local File Systems (Pre).
             Starting udev Kernel Device Manager...
    [  OK  ] Started udev Kernel Device Manager.
    [  OK  ] Started Dispatch Password Requests to Console Directory Watch.
    [  OK  ] Reached target Paths.
    [  OK  ] Found device /dev/ttyPS0.
    

8.1.1. Update rhel-readonly.service

  • Change the /etc/systemd/system/rhel-readonly.service [Unit] section to read:
[Unit]
Description=Configure read-only root support
DefaultDependencies=no
Conflicts=shutdown.target systemd-tmpfiles-setup.service
Before=shutdown.target emergency.service emergency.target local-fs.target systemd-random-seed.service
After=systemd-remount-fs.service

Author: Bryan Moffit <moffit@jlab.org>

Created: 2025-08-23 Sat 07:32

Validate