Automating Linux Installation using Vagrant and VirtualBox

In this tutorial I will explain in detail how we can setup the environment to deploy one or more Linux guest operating system virtually.

Prerequisites

VirtualBox should be installed
Vagrant should be installed

My Environment

Host Operating System - Windows 10 Enterprise

VirtualBox Version - Version 6.1.18

Vagrant Version - 2.2.14

Git Bash Shell - 2.28.0.windows.1

What is VirtualBox?

Oracle VM VirtualBox is a free and open-source hypervisor developed by Oracle Corporation. It allows us to run virtual machines on our physical host machine without affecting the host OS.

We call the operating system where VirtualBox is installed as "Host OS" and the OS running under VirtualBox VM's are called as "Guest OS".

While creating VM's we can specify the specs (Disk size, RAM, CPU cores etc.) we want for our VM to have.

VirtualBox runs on all major distributions such as Windows, Mac OSX and Linux.

What is Vagrant?

As per its official documentation "Vagrant is a tool for building and managing virtual machine environments in a single workflow."

Vagrant enables us to create, configure, and manage virtual machines using its extremely easy command line interface. It acts as a layer of software installed between a virtualization tool (such as VirtualBox, Docker, Hyper-V) and a VM.

Downloading and Installing VirtualBox and Vagrant

VirtualBox can be downloaded from here - virtualbox.org/wiki/Downloads

Vagrant can be downloaded from here - vagrantup.com/downloads

Once downloaded please follow the installation prompts and install both the tools as any other software we install on Windows or Mac OSX.

Tweaks on Windows -

If you wish to use VirtualBox on Windows, you must ensure that Hyper-V is not enabled on Windows. You can turn off the feature by running this Powershell command: Disable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V-All

Tweaks on Mac OSX -

Mac OSX users may see a warning saying the installation failed. Nothing to worry here. What you need to do is eject the VirtualBox image from: Finder > Devices. Then go to System Preferences > Security & Privacy and select ‘Allow’ to make the change: As we have allowed the exception, download the .dmg file again and you should be fine with your installation.

For more details you can follow the individual manuals here -

virtualbox.org/manual

vagrantup.com/docs/installation

Now since you have VirtualBox and Vagrant installed and ready to use let us start using them and create our first virtual machine.

Installing Ubuntu VM on VirtualBox using Vagrant CLI

In order to install Ubuntu or any other OS VM for that reason we need an installation image from somewhere.

Here is the Vagrant boxes repository hosted on Vagrant cloud - app.vagrantup.com/boxes/search

For my demo purpose I will be using ubuntu/bionic64 image which is an official Ubuntu 18.04 LTS release build.

Step 1 -> Create a local directory and Add the Vagrant box to it

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~
$ mkdir LCO_linux_install_demo

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~
$ cd LCO_linux_install_demo/

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$ mkdir shared_resources

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$ ls
shared_resources/

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$

Step 2 -> Create Vagrantfile and add the Vagrant box

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$ touch Vagrantfile

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$ vagrant box add ubuntu/bionic64
==> box: Loading metadata for box 'ubuntu/bionic64'
    box: URL: https://vagrantcloud.com/ubuntu/bionic64
==> box: Adding box 'ubuntu/bionic64' (v20210315.1.0) for provider: virtualbox
    box: Downloading: https://vagrantcloud.com/ubuntu/boxes/bionic64/versions/20210315.1.0/providers/virtualbox.box
Download redirected to host: cloud-images.ubuntu.com
    box:
==> box: Successfully added box 'ubuntu/bionic64' (v20210315.1.0) for 'virtualbox'!

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$

You can view the list of all vagrant boxes using following command ->

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$ vagrant box list | grep bionic64
ubuntu/bionic64     (virtualbox, 20210315.1.0)

Initialize the Vagrant environment -> The following command will initialize the Vagrant environment in current working directory.

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$ vagrant.exe init ubuntu/bionic64 --force
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.

This initializes the current directory to be a Vagrant environment by creating an initial Vagrantfile if one does not already exist.

If a first argument is given, it will prepopulate the config.vm.box setting in the created Vagrantfile.

If a second argument is given, it will prepopulate the config.vm.box_url setting in the created Vagrantfile.

--force - If specified, this command will overwrite any existing Vagrantfile.

Now have a look at the Vagrantfile content.

$ grep -v '^ *#' Vagrantfile

Vagrant.configure("2") do |config|

  config.vm.box = "ubuntu/bionic64"

end

This is the minimal configuration a Vagrantfile can have. There are multiple options which we should be using to specify our environment requirements. We will learn about them in next section.

Step 3 -> Bring up the virtual machine by using vagrant up

This command creates and configures guest machines according to your Vagrantfile.

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$ vagrant.exe up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'ubuntu/bionic64'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'ubuntu/bionic64' version '20210315.1.0' is up to date...
==> default: Setting the name of the VM: LCO_linux_install_demo_default_1615985580781_62799
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: Warning: Connection aborted. Retrying...
    default:
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default:
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!

Congratulation! if you see these messages after running vagrant up command.

Sometimes you might get Guest Additions related warnings/errors. Here is the fix -> First need to shutdown the VM using vagrant halt command.

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$ vagrant.exe halt
==> default: Attempting graceful shutdown of VM...

And then install the vagrant-vbguest plugin using following command.

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$ vagrant plugin install vagrant-vbguest
Installing the 'vagrant-vbguest' plugin. This can take a few minutes...
Installed the plugin 'vagrant-vbguest (0.29.0)'!

Now to avoid further more errors run vagrant reload command and all the errors will be gone.

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$ vagrant.exe reload
==> default: Checking if box 'ubuntu/bionic64' version '20210315.1.0' is up to date...
==> default: Clearing any previously set forwarded ports...
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: Warning: Connection aborted. Retrying...
    default: Warning: Connection reset. Retrying...
==> default: Machine booted and ready!
[default] GuestAdditions 6.1.18 running --- OK.
==> default: Checking for guest additions in VM...
==> default: Mounting shared folders...
    default: /vagrant => C:/Users/Jinendra/LCO_linux_install_demo
==> default: Machine already provisioned. Run `vagrant provision` or use the `--provision`
==> default: flag to force provisioning. Provisioners marked to run always will still run.

Step 4 -> Log in to the guest Ubuntu VM

In order to login to Ubuntu Guest VM run the vagrant ssh command.

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$ vagrant.exe ssh
Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-137-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Wed Mar 17 13:04:19 UTC 2021

  System load:  0.04              Processes:             100
  Usage of /:   3.2% of 38.71GB   Users logged in:       0
  Memory usage: 13%               IP address for enp0s3: 10.0.2.15
  Swap usage:   0%


0 packages can be updated.
0 of these updates are security updates.

New release '20.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.


vagrant@ubuntu-bionic:~$

That's all. You now have witnessed Ubuntu Server installation using Vagrant and running through VirtualBox on your computer.

To exit from the Ubuntu console type exit to exit out of the server, and then you can run vagrant halt to shut it down.

Let us see how does it look at VirtualBox UI. The highlighted one is our VM.

Customizing Vagrantfile

In this section I will put my own LCO Linux cluster's (1 master node, 2 worker nodes) environment's Vagrant file which you can leverage to create your own cluster.

$ cat Vagrantfile
# -*- mode: ruby -*-
# vi: set ft=ruby :

ENV['VAGRANT_NO_PARALLEL'] = 'yes'

Vagrant.configure(2) do |config|


  # Linux Master Server
  config.vm.synced_folder "shared_resources/", "/home/vagrant/shared_resources"
  config.vm.define "lco-linux-test-master" do |lcolinuxmaster|
    lcolinuxmaster.vm.box = "bento/ubuntu-18.04"
    lcolinuxmaster.vm.hostname = "lco-linux-test-master.example.com"
    lcolinuxmaster.vm.network "private_network", ip: "172.82.82.100"
    lcolinuxmaster.vm.provider "virtualbox" do |v|
      v.name = "lco-linux-test-master"
      v.memory = 4096
      v.cpus = 2
        end
  end

  NodeCount = 2

  # Linux Worker Nodes
  (1..NodeCount).each do |i|
    config.vm.synced_folder "shared_resources/", "/home/vagrant/shared_resources"
    config.vm.provision :shell, path: "bootstrap.sh"
    config.vm.define "lco-linux-test-worker#{i}" do |workernode|
      workernode.vm.box = "bento/ubuntu-18.04"
      workernode.vm.hostname = "lco-linux-test-worker#{i}.example.com"
      workernode.vm.network "private_network", ip: "172.82.82.10#{i}"
      workernode.vm.provider "virtualbox" do |v|
        v.name = "lco-linux-test-worker#{i}"
        v.memory = 2048
        v.cpus = 2
          end

    end
  end

end

Highlights from the above file -

config.vm.network - Configures networks on the machine. Follow here for VirtualBox provider specific networking options.
config.vm.provider - Configures provider-specific configuration, which is used to modify settings which are specific to a certain provider.
config.vm.synced_folder - Configures synced folders on the machine, so that folders on your host machine can be synced to and from the guest machine.
config.vm.box - It is a shorthand to a box in HashiCorp's Vagrant Cloud then this value does not need to be specified.
config.vm.hostname - The hostname the machine should have. Defaults to nil. If nil, Vagrant will not manage the hostname. If set to a string, the hostname will be set on boot. If set, Vagrant will update /etc/hosts on the guest with the configured hostname.

Here is the bootstrap.sh file which will be executed once the systems are booted and become ready to use.

$ cat bootstrap.sh
#!/bin/bash
echo ""
echo ""
echo "Checking your Linux release:"
echo ""
cat /etc/os-release
echo ""
echo ""
echo "BOOTSTRAP:"
echo "----------------------------------------"
echo "update ssh config"
sudo su -
sed -i 's/#\?\(PermitRootLogin\s*\).*$/\1 yes/' /etc/ssh/sshd_config
systemctl restart sshd
apt-get update -y
apt-get install vim lshw nmap telnet gcc perl kernel-headers kernel-devel -y
/sbin/rcvboxadd setup

Here I am changing some ssh configurations and installing basic packages.

You can modify both the files Vagrantfile and bootstrap.sh as per your requirements. They both should be present in the same folder where we are initiating vagrant.
Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$ ls
bootstrap.sh*  shared_resources/  Vagrantfile

It's time to provision the guest virtual machines.

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$ vagrant.exe up
Bringing machine 'lco-linux-test-master' up with 'virtualbox' provider...
Bringing machine 'lco-linux-test-worker1' up with 'virtualbox' provider...
Bringing machine 'lco-linux-test-worker2' up with 'virtualbox' provider...

...
...

This may take a few minutes to complete but it should finish without any errors.

All our machines are up and running. Verify their status.

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$ vagrant.exe status

Verify them on VirtualBox UI -

To destroy the test environment you created run the following command -

Jinendra@DESKTOP-IEH4IQ8 MINGW64 ~/LCO_linux_install_demo
$ vagrant.exe destroy
    lco-linux-test-worker2: Are you sure you want to destroy the 'lco-linux-test-worker2' VM? [y/N] y
==> lco-linux-test-worker2: Destroying VM and associated drives...
    lco-linux-test-worker1: Are you sure you want to destroy the 'lco-linux-test-worker1' VM? [y/N] y
==> lco-linux-test-worker1: Destroying VM and associated drives...
    lco-linux-test-master: Are you sure you want to destroy the 'lco-linux-test-master' VM? [y/N] y
==> lco-linux-test-master: Destroying VM and associated drives...

That's all for this tutorial. Now its just a matter of modifying the Vagrantfile and within a few minutes your environment will be ready.

Hope you like the article. Stay Tuned for more.

Thank you. Happy learning!