In this post we will show a way to pass an ec2 ami to vmdk format to be able to use it in Virtualbox and Vagrant, in this tutorial we used an ami created by the community with Debian Jessie as an example. However, when it comes to exporting ami's there are limitations you can find them in the following link. likewise, it is not guaranteed that this method works with other OS tools or versions different from those mentioned here, however, in the same way, maybe if it works as it is or implementing some adaptations.
- VirtualBox guest additions
- Recommended same Debian version installation disk (net install) as the exported OS image
In first step we need to create an S3 bucket or we can use an existing one, you can create an Amazon S3 bucket for storing the exported instances and grant VM Import/Export permission to access the bucket. If you already have a bucket and want to use it, you can use it instead of creating a bucket.
(Optional) create an S3 bucket
- Open the Amazon S3 console at https://console.aws.amazon.com/s3/.
2. Choose Create Bucket. 3. In the **Create a Bucket** dialog box, do the following:
- For Bucket Name, type a name for your bucket. This name must be unique across all existing bucket names in Amazon S3. Don’t use capital letter, spaces, dashes or underscores.
- For Region, select a region.
- Choose Create. When the bucket is created, the details pane opens, then in the following options do:
- name and region --> put a name and region
- configure options --> leave untouched
- set permissions --> leave selected all options
- On the buckets list choose the bucket just created
- in the popuped window select click on permissions
- in the submenu options select Access Control List
- then in Access for other AWS accounts select +Add account
- enter user
email@example.com minimum options marked, List objects, write objects and read bucket permissions
- click on save
Prerequisites to export a VM (instance)
To export a VM from Amazon EC2, install the AWS CLI on the instance to achieve this follow the instructions in this link. after install it you need to configure AWS with your credentials for example:
$ aws configure AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY Default region name [None]: us-west-2 Default output format [None]: json
Export the isntance
To export your instance, use the create-instance-export-task command. You must provide the information required to properly export the instance to the selected virtualization format. The exported file is saved in the S3 bucket that you specify. As you can see in the next example:
$ aws ec2 create-instance-export-task --instance-id id --target-environment target_enviroment --export-to-s3-task DiskImageFormat=disk_image_format,ContainerFormat=ova,S3Bucket=bucket,S3Prefix=prefix
- id = The ID of the instance.
- target_environment = The target environment (vmware | citrix | microsoft)
- disk_image_format = The disk image format (VMDK | VHD).
- bucket = The name of the S3 bucket.
- prefix = The image is written to an object in the S3 bucket using the following S3 key: prefix/export-i-xxxxxxxx.format (for example, my-exports/export-i-12345678.ova).
Monitor Instance Export
To monitor the exporting process of your instance, use the following describe-export-tasks command:
aws ec2 describe-export-tasks --export-task-ids export-i-12345678
When the export has been finished you can open your s3 bucket and if you don't see nothing don't forget to refresh the page and you will see a folder with containing the ova file from the recent exporting EC2 AMI image, then you can select and download it to your local enviroment.
Prepare virtual disk image
Next we need to extract from ova file the virtual disk file (.vmdk) and mount the image in Virtualbox as a secondary HD with an OS guest with the same OS AMI version recommended, we can use the net install version since we only need the repair option to install the GRUB (if necessary) and make little changes needed to boot the system correctly with Vagrant.
First we need to create a new virtual machine in Virtualbox with follow characteristics:
- name: whatever you want
- type: Linux
- version: Debian
In memory size 2048 is recommended for this purpose
In the hard disk option, we need to choose “use an existing virtual hard disk file” option and open and navigate to path where we previously extracted virtual disk (.vmdk) from the ova file and use them, then push the create button.
Now we can see the new virtual machine in in the left pane, the next step is configure it, to do that select the VM and press the settings button, we need to modify the next options as is described.
- In the system tab we need to deselect in the boot order the Floppy option and be sure the boot order be like the next image.
- In storage tab we need to add the installation disk of the Debian OS system, click on the empty controller IDE and then click in the disc icon to open, search and select the iso image
- In the audio tab options, we need to disable it.
- In network tab we need to confirm that the first adapter network is enabled and attached to NAT
That it’s all for the VM config now is time to run it so we start the machine and in the Options menu from the Debian installation we choose the Advanced options > Graphical rescue mode.
Next we choose the basic config options like Language, location, Keyboard, hostname, domain and time zone. This is only important to this process since this is only required to get to the next step
Next we can select the rescue mode type and we chose the device to use as root file then select
In the next step choose the rescue operations what would be Execute a shell in
In the shell we need to modify or add the password for the users root and admin that come by default in EC2 AMI with the passwd command for each user, for the user root vagrant recommends set the password “vagrant”, for user admin you can put whatever you want
Next press go back button and in the File options for the VM select close option and power off the machine, after that we need to disable the optical boot from the config options for the machine or unmount the attached iso installation image for the optical drive
Then we need to start the VM again but now will boot from the virtual hard drive, in these steps the first boot will take a long time (3 or 4 minutes) to boot, that's because we need to disable some options that the EC2 AMI of amazon has by default
Now the VM is booted we need to access it with the user root and the pass “vagrant” to do next modifications and changes to adapt it and can be functional with Vagrant.
- (optional) change hostname
Type the following command to edit
/etc/hostnameusing nano or vi text editor:
$ sudo nano /etc/hostname
Delete the old name and setup new name.
Next Edit the
$ sudo nano /etc/hosts
Replace any occurrence of the existing computer name with your new one, example:
The changes take effect after reboot the system.
- uninstall cloud-init or disable it by changing the name of
cloud-init-cfgwith different name with this we resolve the long boot time because these programs trying to connect intranet Amazon addresses, toverify the above first run the command
$ systemd-analyze blameand you will see something like next image.
After that you can continue with the method you prefer to disable cloud-init, if you choose uninstall it then execute the follow commands:
To just uninstall cloud-init
sudo apt-get remove cloud-init.
To Uninstall it and remove dependencies
sudo apt-get remove --auto-remove cloud-init.
To purge your config/data
sudo apt-get purge --auto-remove cloud-init.
Add user and home path for Vagrant
By default, Vagrant expects a "vagrant" user to SSH into the machine as. This user should be setup with the insecure keypair that Vagrant uses as a default to attempt to SSH. Also, even though Vagrant uses key-based authentication by default, it is a general convention to set the password for the "vagrant" user to "vagrant". This lets people login as that user manually if they need to.
With the next command:
$ sudo useradd -m -s $(which bash) -G sudo vagrant will create a new user with a home dir, bash login, shell and the ability to sudo
Then to create the password for the vagrant user do
passwd vagrant and enter vagrant as a new password.
(optional) Root Password: "vagrant"
Vagrant does not actually use or expect any root password. However, having a generally well-known root password makes it easier for the general public to modify the machine if needed.
Publicly available base boxes usually use a root password of "vagrant" to keep things easy.
to do that type
$ passwd root and enter vagrant as a new password.
This is important! Many aspects of Vagrant expect the default SSH user to have passwordless sudo configured. This lets Vagrant configure networks, mount synced folders, install software, and more.
To do that you can configure it (usually using visudo) to allow passwordless sudo for the "vagrant" user. This can be done with the following line at the end of the configuration file:
vagrant ALL=(ALL) NOPASSWD:ALL
Or add file in path
/etc/sudoers.d/vagrant with the same line before added.
Vagrant's documentation suggests that you modify the SSH configuration to avoid reverse DNS lookups. To do this, open
/etc/ssh/sshd_config, find the line which reads
#UseDNS yes and change it to
UseDNS no, if there are no line create at the end of file.
Then restart the SSH service with
sudo service ssh restart after that be sure the service is running with the command
sudo service ssh status.
Enable access via non secure key
To configure SSH access with the insecure keypair, place the public key into the
~/.ssh/authorized_keys file for the "vagrant" user.
to do that open a login shell for the vagrant user (
su - vagrant), download its public key and set the right permissions with next suggested steps:
$ mkdir -p /home/vagrant/.ssh $ chmod 0700 /home/vagrant/.ssh $ wget https://raw.githubusercontent.com/mitchellh/vagrant/master/keys/vagrant.pub -O /home/vagrant/.ssh/authorized_keys $ chmod 0600 /home/vagrant/.ssh/authorized_keys $ chown -R vagrant:vagrant /home/vagrant/.ssh
leave the vagrant’s user shell
(optional) Permitrootlogin only with key authentication
Allows root login only with public key authentication. This is often used with shell scripts and automated tasks.
/etc/ssh/sshd_config, uncomment and change the line
next restart ssh service: service ssh restart
(optional) Update and upgrade
It is recommended to update and upgrading before packaging the box
$ sudo apt update --yes $ sudo apt upgrade --yes
You'll want to reboot the machine now as probably installs a new kernel and we want to compile VirtualBox' guest additions with the newest kernel available.
VirtualBox Guest Additions must be installed so that things such as shared folders can function. Installing guest additions also usually improves performance since the guest OS can make some optimizations by knowing it is running within VirtualBox.
Before installing the guest additions, you will need the linux kernel headers and the basic developer tools. On Ubuntu, you can easily install these like so:
$ sudo apt-get install linux-headers-$(uname -r) build-essential dkms gcc make perl
To install via the command line, you can find the appropriate guest additions version to match your VirtualBox version by selecting the appropriate version here. The examples below use 5.2.0, which was the latest stable VirtualBox version at the time of writing.
$ wget http://download.virtualbox.org/virtualbox/5.2.0_RC1/VBoxGuestAdditions_5.2.0_RC1.iso $ sudo mkdir /media/VBoxGuestAdditions $ sudo mount -o loop,ro VBoxGuestAdditions_5.2.0_RC1.iso /media/VBoxGuestAdditions $ sudo sh /media/VBoxGuestAdditions/VBoxLinuxAdditions.run $ rm VBoxGuestAdditions_5.2.0_RC1.iso $ sudo umount /media/VBoxGuestAdditions $ sudo rmdir /media/VBoxGuestAdditions
Package the box
we have almost created the Vagrant box. Now we just have to create a
package.box file. We can do it with the help of vagrant package command. In your host computer create a folder where will be save the
$ mkdir ~/foldername $ cd ~/foldername $ vagrant package --base <name-of-VM-in-GUI>
After that you wil have a
package.box file in your actual folder.
Use this box
You can now import this base box into Vagrant with the command:
$ vagrant box add name package.box