This is the first post of a series in which I’m going to show you how to install your own OpenStack Infrastructure from scratch.
The main problem: How can I manage the OS installation on tons of servers?
The first problem that you are going to encounter installing OpenStack, is how to manage the installation of your bare metal servers. If your OpenStack setup is small enough, you can always install the OS one by one, but that’s not going to work if you want to scale it up in the future, so I’m going to show you how to install your bare metal servers using DHCP + PXE Boot + NFS. There are other options to accomplish this like Foreman or MAAS, but I’ve decided to talk about this one because the simplicity and the versatility of this method.
The Solution: DHCP + PXE
For these series, I’m going to use CentOS 7, but the steps should be pretty similar for any other OS.
First of all you’ll need to install dnsmasq. dnsmasq provides DNS and DHCP services. Also, it provides a PXE server which will use on this tutorial.
Install the required packages. You can do it issuing this command:
yum install dnsmasq
dnsmasq config file is located at /etc/dnsmasq.conf. The default file is full of options. I recommend you to start with a blank file and use these options:
interface=eth0 bind-interfaces domain=your domain #DHCP range-leases dhcp-range=eth0,172.16.64.0,static,255.255.252.0,1h dhcp-host=18:de:d7:a1:be:64,172.16.64.213,1h # PXE dhcp-boot=pxelinux.0,pxeserver,172.16.64.11 # Gateway dhcp-option=3,172.16.64.11 # DNS dhcp-option=6,172.16.20.1 # Broadcast Address dhcp-option=28,172.16.67.255 # NTP Server dhcp-option=42,172.16.64.11 pxe-prompt="Press F8 for menu.", 10 pxe-service=x86PC, "Install CentOS 7 from network server 172.16.64.11", pxelinux enable-tftp tftp-root=/var/lib/tftpboot
You’ll need to change the following options:
- Interface: network interface on which you want to listen to DHCP requests from your servers.
- bind-interfaces: if you want to listen only to DHCP requests on the interface defined before.
- domain: your own domain if you have one.
- dhcp-range: the first parameter references your network interface, the second one the network range for your DHCP server, the third one defines that only hosts defined on this file will get an IP address from dnsmasq, the fourth is the subnet mask and the last one the duration of the DHCP lease.
- dhcp-host: use this variable to define which hosts should get an IP address from this server. Format for this variable is host-mac-address,IP address,duration of the lease. You can define as many hosts as you want.
- dhcp-boot=pxelinux.0,pxeserver,172.16.64.11: replace the IP statement with your own IP.
- dhcp-option=3,172.16.64.11: gateway for your hosts.
- dhcp-option=6,172.16.20.1: DNS server for your hosts.
- dhcp-option=28,172.16.67.255: broadcast address for your hosts.
- dhcp-option=42,172.16.64.11: NTP server for your hosts.
- pxe-prompt: a small text to display after PXE Boot.
- pxe-service: use x86PC for 32/64 bit architectures, refer to the documentation if you want to use something different here.
- enable-tftp: enables the built-in tftp server.
- tftp-root: the location for all netbooting files.
SYSLINUX is a linux bootloader designed to run from an MS-DOS/Windows FAT file system. It is limited to Intel/AMD hardware. Over time, the Syslinux project expanded to include support for booting natively from CD-ROMS (ISOLINUX), linux file systems (EXTLINUX) and over PXE (PXELINUX), and that’s why are we using it.
To install it, just issue the following command:
yum install syslinux
Now, let’s move ahead and install the tftp-server and copy the syslinux bootloaders to the tftp folder, by default /var/lib/tftpboot
yum install tftp-server cp -r /usr/share/syslinux/* /var/lib/tftpboot
PXE Server configuration file
Your PXE server reads the required configuration from /var/lib/tftpboot/pxelinux.cfg. You can define specific configurations based on GUID or MAC addresses. If no specific configuration is found, PXE will use default to boot your bare-metal servers. Create the required files / folders to accomplish this:
mkdir /var/lib/tftpboot/pxelinux.cfg touch /var/lib/tftpboot/pxelinux.cfg/default
If you want to define a configuration for a host based on his MAC address, create a file using his MAC address as a name. For example:
You can use this simple menu to install your server:
default menu.c32 prompt 0 timeout 300 ONTIMEOUT local menu title Boot Menu label 1 menu label Install CentOS 7 x64 kernel centos7/vmlinuz append initrd=centos7/initrd.img method=nfs://172.16.64.11/opt/centos/ devfs=nomount label 4 menu label Boot from local drive
Pay attention to method line. This line indicates that we’ll use NFS to retrieve all necessary files in order to install our host. You can use other methods, like an HTTP server or an FTP.
Getting the files and installing NFS Server
We need the kernel and init from CentOS 7. In order to get these files, download CentOS 7 image and get the required files from the ISO file. Mount the ISO file to get the neccesarry files:
mount -o loop /path/to/centos-dvd.iso /mnt mkdir /var/lib/tftpboot/centos7 cp /mnt/images/pxeboot/vmlinuz /var/lib/tftpboot/centos7 cp /mnt/images/pxeboot/initrd.img /var/lib/tftpboot/centos7
Last but not least, we need to install an NFS server to share the required files at PXE boot. First of all, you’ll need to create a directory to store CentOS 7 installer files, which will be exported later through NFS. In my example this directory is opt/centos/ but you can choose whatever you want.
mkdir /opt/centos chmod -R 777 /opt/centos
Install NFS server and start all the required services.
yum install nfs-utils systemctl enable rpcbind systemctl enable nfs-server systemctl enable nfs-lock systemctl enable nfs-idmap systemctl start rpcbind systemctl start nfs-server systemctl start nfs-lock systemctl start nfs-idmap
After that, edit /etc/exports to add our directory to the list of exported directories by NFS.
WARNING: As you can see, we’re exporting to everywhere, so be careful because you can be exporting this directory to the world if your server is not secured properly. Feel free to change this line according to your setup.
And to finish, copy the files from /mnt to /opt/centos and restart nfs service:
cp -r /mnt/* /opt/centos systemctl restart nfs-server
After that, boot your uninstalled hosts using PXE and enjoy!
Bonus points: Kickstart files
You can fully automate the installation of your servers using kickstart files
Just replace this:
You can find an example of a kickstart file here.
And that’s all. Any constructive criticism is always welcome!
On my last job at Institute For Biocomputation and Physics of Complex Systems (BIFI) I grew up with OpenStack, Docker and a lot of different technologies related to the cloud enviroment. Also, I had to deal with a lot of mad scientists!
Lover of automatization using Ansible, I keep the OpenStack up and running at Datio, but I’ve also have to deal with a lot of mad people! 🙂