Automated VM Creation

This section covers how to automatically create and configure VMs using a predefined template and scripts.

Directory Structure

To ensure everything is organized, here’s the recommended directory structure for the automation process:

├── /root/
│   └── scripts/
│       └── elitical/
│           ├── elitical_dev_generate.py    # Automation script
│           ├── startup-script.sh           # First boot configuration script
│           └── source/
│               └── elitical-dev-vms.csv    # CSV file with VM details
├── /data1/
│   ├── guest-images/                       # VM disk images
│   ├── guest-xml/                          # VM XML configurations
│   └── guest-keys/                         # SSH keys for VMs
└── /data2/
    └── adm-images/                         # Template disk images

Prerequisites

Before running the automation script, ensure you have the following:

1. Input CSV File

This file contains the necessary data for each VM. It should have the following columns:

  • name: Name of the VM
  • emp_id: Employee ID
  • work: Work description
  • branch: Branch details
  • host_ip: Host IP address
  • devbox_ip: IP address for the devbox
  • mac_address: MAC address for the VM
  • node_name: Name of the node
  • jenkins_user: Jenkins username
  • jenkins_password: Jenkins password
  • private_key: Private SSH key
  • email: Email address

Ensure this file is in .csv format.

2. startup-script.sh File

This script is responsible for setting up essential services and configurations for the VM once it is created.

#!/bin/bash

# Stop and restart SSH services
systemctl stop sshd ssh || echo "sshd fail"
cp /etc/ssh/sshd_config /opt/sshd_backupfile.bkp || echo "copy fail"
apt purge openssh-server -y || echo "purge failed"
apt install openssh-server -y || echo "install failed"
cp /opt/sshd_backupfile.bkp /etc/ssh/sshd_config || echo "restore failed"

# Replace machine IDs
cp /etc/machine-id /etc/machine-id.bkp || echo "backup failed"
echo "mac-id" > /etc/machine-id || echo "replace failed"

# Update hosts file
cp /etc/hosts /etc/hosts.bkp || echo "backup failed"
sed -i 's/172.21.4.38/iptobereplaced/g' /etc/hosts || echo "update failed"
echo "iptobereplaced elitical-in-dev1a-node01" >> /etc/hosts || echo "append failed"

# Set up SSH keys
mkdir -p /home/devbox/.ssh /home/devopsadmin/.ssh || echo "directory creation failed"
echo "pubkeytobereplaced" > /home/devbox/.ssh/authorized_keys || echo "key setup failed"
echo "<jenkins-public-key>" > /home/devopsadmin/.ssh/authorized_keys || echo "admin key setup failed"

# Firewall configuration
ufw allow log from "developerip" to any port "39195" proto tcp || echo "ufw rule failed"

Ensure this script is prepared and accessible before running the automation script.

Automation Script Overview

The automation script (elitical_dev_generate.py) does the following:

  1. Reads input data from the .csv file.
  2. Prepares a VM template by copying an existing disk image.
  3. Configures the VM based on the startup-script.sh file.
  4. Defines the new VM with a unique configuration.

The following script automatically generates VMs from the template using the provided input data.

import subprocess
import csv

def run_command(command):
    print(f"Running command: {command}")
    subprocess.run(command, shell=True, check=True)

source_details = "../source/elitical-dev-vms.csv"
source_disk_img = '/data2/adm-images/elitical-dev-tmpl.qcow2'
source_xml = '/data1/guest-xml/dev-template.xml'
base_firstboot_script = '/root/scripts/elitical/vms/startup-script.sh'
base_uuid = '00000000-0000-0000-0000-'

with open(source_details) as details:
    details_reader = csv.DictReader(details, delimiter=',')
    for row in details_reader:
        name = row['name']
        emp_id = row['emp_id']
        work = row['work']
        branch = row['branch']
        host_ip = row['host_ip']
        devbox_ip = row['devbox_ip']
        mac_address = row['mac_address']
        node_name = row['node_name']
        jenkins_user = row['jenkins_user']
        jenkins_password = row['jenkins_password']
        private_key = row['private_key']
        email = row['email']

        target_name = node_name
        target_keys_dir = f'/data1/guest-keys/{target_name}'
        target_public_key_file = f'{target_keys_dir}/{target_name}.pub'
        target_disk_img = f'/data1/guest-images/{target_name}.qcow2'
        target_xml = f'/data1/guest-xml/{target_name}.xml'
        target_firstboot_script = f'/root/scripts/elitical/{target_name}.sh'

        target_bridge_mac = mac_address
        target_host_mac = target_bridge_mac.replace('20', '10')
        target_uuid = base_uuid + target_bridge_mac.replace(':', '')

        run_command(f"mkdir -p {target_keys_dir}")
        run_command(f'ssh-keygen -t ed25519 -N "" -C "{target_name}" -f {target_keys_dir}/{target_name}')

        with open(source_xml, 'r') as file:
            content = file.read()
            content = content.replace('dev-ein0034', target_name)
            content = content.replace('00000000-0000-0000-0000-0a010d20de35', target_uuid)
            content = content.replace('0a:01:0d:20:de:35', target_bridge_mac)
            content = content.replace('0a:01:0d:10:de:35', target_host_mac)
            content = content.replace('/data1/guest-images/dev-template.qcow2', target_disk_img)

        with open(target_xml, 'w') as file:
            file.write(content)

        with open(target_public_key_file, 'r') as pub_key_file:
            public_key = pub_key_file.read().strip()

        with open(base_firstboot_script, 'r') as script:
            script_content = script.read()
            script_content = script_content.replace("mac-id", target_uuid)
            script_content = script_content.replace("iptobereplaced", devbox_ip)
            script_content = script_content.replace("pubkeytobereplaced", public_key)
            script_content = script_content.replace("developerip", host_ip)

        with open(target_firstboot_script, "w") as start_script:
            start_script.write(script_content)

        run_command(f"cp {source_disk_img} {target_disk_img}")
        run_command(f"virt-sysprep -a {target_disk_img}")
        run_command(f"virt-sysprep --hostname {target_name} -a {target_disk_img} --firstboot {target_firstboot_script}")
        run_command(f"virsh define {target_xml}")

Commands

The script will execute commands like:

  • Create disk image: Copy a base image and modify it for each VM.
  • Generate SSH keys: Set up SSH keys for the VM.
  • Configure first boot script: Customize the startup script for each VM.
  • Sysprep and define VM: Run virt-sysprep and configure the VM with virsh.

Steps to Run the Automation

  1. Prepare Input Files:

    • Ensure the .csv file (elitical-dev-vms.csv) is ready with the correct data.
    • Ensure the startup-script.sh is up to date and configured for your environment.
  2. Run the Script: Execute the following command in your terminal to start the VM creation process:

    python3 elitical_dev_generate.py
    
  3. Verify VM Creation: After the script completes, verify that the VMs have been created and configured properly:

    • Use virsh list to check the status of the VMs.
    • Connect to the VMs using SSH to ensure they are correctly configured and running.