Bootstrapping Instances via Metadata and Public Cloud Metadata Support!

OpenStack and several other cloud management platforms like (Amazon, CloudStack, etc) provide a metadata service that allows one to pass in additional information to an instance at boot time. This can be helpful in order to automate the configuration within an instance.

One use case people usually use this service for is to have one’s ssh-key automatically pulled into your instance when it boots so you can ssh into an instance without having a hard coded password in the image. There are several other neat things one can do with it, such as specifying a script for your instance to run in order to bootstrap setup configuration.

In this blog post we’ll use the metadata service to boot up several instances at once and have them automatically install devstack for us all from a single command. Then, try to see if we can get this working on a few of the big OpenStack public cloud providers.

To start, I’m assuming you already have a working OpenStack environment (with metadata) and a ubuntu-12.04 image that has cloud-init already installed. Cloud-init is a set of scripts that needs to be present in your guest instance in order to leverage the metadata service easily. You can grab the image I’m using for this from here if you need one: https://cloud-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img and upload it to glance OR you can use the HP cloud (hpcloud.com) and follow along as this also works there too!

First we’ll create an ssh key if you don’t already one:

$ ssh-keygen 
Generating public/private rsa key pair.
Enter file in which to save the key (/home/ubuntu/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/ubuntu/.ssh/id_rsa.
Your public key has been saved in /home/ubuntu/.ssh/id_rsa.pub.
The key fingerprint is:
a7:96:9b:dd:e6:a6:1e:33:a9:80:b2:13:47:ab:ea:3a ubuntu@ubuntu
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|                 |
|                 |
|    .            |
|   . .  S .      |
|  . o.   + .     |
|  .+. . + =      |
|E oo   o = =o    |
|=+..    +.+=o    |
+-----------------+

Upload it to nova:

$ nova keypair-add --pub_key ~/.ssh/id_rsa.pub mykey

Lists the uploaded key:

$ nova keypair-list
+-------+-------------------------------------------------+
| Name  | Fingerprint                                     |
+-------+-------------------------------------------------+
| mykey | a7:96:9b:dd:e6:a6:1e:33:a9:80:b2:13:47:ab:ea:3a |
+-------+-------------------------------------------------+

Boot an instance and specify the key, when the instance comes up  the cloud-init scripts will automatically grab the key and put it in the right place.

$ nova boot --image ubuntu-server-12.04 --flavor 4 --key-name mykey my_vm

Once the instance boots you’ll be able to ssh into using your ssh key:

$ ssh ubuntu@<instance_ip>

Next, we’re going to automate the installation of OpenStack via devstack. Below is a simple script that runs the commands required to setup devstack. We’re going to put this script in a file called setup_devstack.sh which we’ll pass via nova boot.

#!/bin/bash
sudo apt-get update
sudo apt-get install -y git
git clone https://github.com/openstack-dev/devstack  /home/ubuntu/devstack
cat > /home/ubuntu/devstack/localrc << "EOF" ENABLED_SERVICES=g-api,g-reg,key,mysql,n-api,n-cond,n-cpu,n-crt,n-obj,n-sch,q-agt,q-dhcp,q-l3,q-lbaas,q-meta,q-svc,q-vpn,quantum,rabbit,horizon,n-novnc,n-xvnc DATABASE_PASSWORD=password RABBIT_PASSWORD=password SERVICE_TOKEN=password SERVICE_PASSWORD=password ADMIN_PASSWORD=password EOF # HOST_IP is needed for devstack script. Determine IP from metadata service for the the heck of it :) HOST_IP=`curl 169.254.169.254/latest/meta-data/local-ipv4 | awk '{split($0,array,",")} END{print array[1]}'` echo HOST_IP=$HOST_IP >> /home/ubuntu/devstack/localrc
chown -R ubuntu:ubuntu /home/ubuntu/devstack/
su ubuntu -c /home/ubuntu/devstack/stack.sh &

To pass this script to the instance you need to specify an additional parameter –user-data with the script path to nova boot as shown below:

$ nova boot --image  ubuntu-server-12.04 --flavor 5 --key-name mykey --user-data ~/setup_devstack.sh devstack_vm

When the instance boots up, cloud-init will fetch your ssh key and then start running your script. In order to figure out how far your script has run, you can use nova console-log to find out where it is (as the console output is written to the serial port):

$ nova console-log devstack_vm
Unpacking sqlite3 (from .../sqlite3_3.7.9-2ubuntu1.1_amd64.deb) ...
Selecting previously unselected package unzip.
Unpacking unzip (from .../unzip_6.0-4ubuntu2_amd64.deb) ...
Selecting previously unselected package vbetool.
Unpacking vbetool (from .../vbetool_1.1-2ubuntu1_amd64.deb) ...
Selecting previously unselected package x11-utils.
Unpacking x11-utils (from .../x11-utils_7.6+4ubuntu0.1_amd64.deb) ...
Selecting previously unselected package xbitmaps.
Unpacking xbitmaps (from .../xbitmaps_1.1.1-1_all.deb) 

Downloading packages…..

$ nova console-log devstack_vm
+ mysql -uroot -ppassword -h127.0.0.1 -e 'DROP DATABASE IF EXISTS neutron_ml2;'
+ mysql -uroot -ppassword -h127.0.0.1 -e 'CREATE DATABASE neutron_ml2 CHARACTER SET utf8;'
+ /usr/local/bin/neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head
INFO  [alembic.migration] Context impl MySQLImpl.
INFO  [alembic.migration] Will assume non-transactional DDL.
INFO  [alembic.migration] Running upgrade None -> folsom, folsom initial database
INFO  [alembic.migration] Running upgrade folsom -> 2c4af419145b, l3_support
INFO  [alembic.migration] Running upgrade 2c4af419145b -> 5a875d0e5c, ryu
INFO  [alembic.migration] Running upgrade 5a875d0e5c -> 48b6f43f7471, DB support for service types
INFO  [alembic.migration] Running upgrade 48b6f43f7471 -> 3cb5d900c5de, security_groups

Setting up the neutron database…..

$ nova console-log devstack_vm
Horizon is now available at http://10.0.0.9/
Keystone is serving at http://10.0.0.9:5000/v2.0/
Examples on using novaclient command line is in exercise.sh
The default users are: admin and demo
The password: password
This is your host ip: 10.0.0.9
2014-06-27 22:21:00.883 | stack.sh completed in 1089 seconds.

Done!

At this point I wanted to see how easy it would be to test this out on some of the public OpenStack clouds and see if they were providing this metadata service.  I started with HP cloud and found a ubuntu image

$ glance --os-region-name  region-b.geo-1 image-show 75d47d10-fef8-473b-9dd1-fe2f7649cb41
No handlers could be found for logger "keystoneclient.httpclient"
+----------------------------------------------+----------------------------------------------------------------------------------+
| Property                                     | Value                                                                            |
+----------------------------------------------+----------------------------------------------------------------------------------+
| Property 'architecture'                      | x86_64                                                                           |
| Property 'com.hp__1__bootable_volume'        | True                                                                             |
| Property 'com.hp__1__image_lifecycle'        | active                                                                           |
| Property 'com.hp__1__image_type'             | disk                                                                             |
| Property 'com.hp__1__os_distro'              | com.ubuntu                                                                       |
| Property 'com.hp__1__vendor'                 | Canonical                                                                        |
| Property 'com.ubuntu.cloud__1__milestone'    | release                                                                          |
| Property 'com.ubuntu.cloud__1__official'     | True                                                                             |
| Property 'com.ubuntu.cloud__1__published_at' | 2014-06-11T19:07:04                                                              |
| Property 'com.ubuntu.cloud__1__serial'       | 20140606                                                                         |
| Property 'com.ubuntu.cloud__1__stream'       | server                                                                           |
| Property 'com.ubuntu.cloud__1__suite'        | precise                                                                          |
| Property 'com.ubuntu.cloud_images.official'  | True                                                                             |
| Property 'description'                       | Ubuntu Server 12.04 LTS (amd64 20140606) for HP Public Cloud. Ubuntu Server is   |
|                                              | the world's most popular Linux for cloud environments. Updates and patches for   |
|                                              | Ubuntu 12.04 LTS will be available until 2017-04-26. Ubuntu Server is the        |
|                                              | perfect platform for all workloads from web applications to NoSQL databases and  |
|                                              | Hadoop. More information regarding Ubuntu Cloud is available from                |
|                                              | http://www.ubuntu.com/cloud and instructions for using Juju to deploy workloads  |
|                                              | are available from http://juju.ubuntu.com EULA: http://www.ubuntu.com/about      |
|                                              | /about-ubuntu/licensing Privacy Policy: http://www.ubuntu.com/privacy-policy     |
| Property 'os_type'                           | linux-ext4                                                                       |
| Property 'os_version'                        | 12.04                                                                            |
| checksum                                     | a224dbd167ef6648c5b892b2d4b54780                                                 |
| container_format                             | bare                                                                             |
| created_at                                   | 2014-06-11T18:42:12                                                              |
| deleted                                      | False                                                                            |
| disk_format                                  | qcow2                                                                            |
| id                                           | 75d47d10-fef8-473b-9dd1-fe2f7649cb41                                             |
| is_public                                    | True                                                                             |
| min_disk                                     | 8                                                                                |
| min_ram                                      | 0                                                                                |
| name                                         | Ubuntu Server 12.04 LTS (amd64 20140606) - Partner Image                         |
| owner                                        | 10014302369510                                                                   |
| protected                                    | False                                                                            |
| size                                         | 260768256                                                                        |
| status                                       | active                                                                           |
| updated_at                                   | 2014-06-11T19:07:05                                                              |
+----------------------------------------------+----------------------------------------------------------------------------------+


Then, I tried the same command that I had run locally:

$ nova --os-region-name region-b.geo-1 boot --image 75d47d10-fef8-473b-9dd1-fe2f7649cb41 --flavor 102 --key-name mykey --user-data ~/setup_devstack.sh devstack_instance_hp

Everything worked as expected even nova console log :)

Next, I decided to test out the Rackspace cloud; Found a ubuntu-12.04 image and booted it the same way I had done locally and on the HP cloud. On the rackspace cloud I needed to pass –insecure to nova :/ . In addition, it was hard to figure out the credentials, though I opened a help chat and someone was able to provide that info quickly. I recommended that they should add a “Download OpenStack RC File” button to the UI to make it easier, hopefully they do that.

Anyways, Booted an instance via:

$ nova --insecure --os-region-name ORD boot --flavor 4 --image ffa476b1-9b14-46bd-99a8-862d1d94eb7a --key-name mykey --user-data ~/setup_devstack.sh devstack_rax

Sadly, nova console-log didn’t work here  :( ,  though I do understand that this an extension to the nova-api and probably has some added support cost enabling it.

$ nova --insecure --os-region-name ORD console-log devstack_rax
ERROR (BadRequest): There is no such action: os-getConsoleOutput (HTTP 400) (Request-ID: req-80a4115d-e61a-4db8-afb1-c17b4e779f49)

Once the VM came up I was able to ssh into it so the metadata bits did work for getting my ssh key though I wasn’t able to find my userdata script within the instance (/var/lib/cloud/instance/user-data.txt) and it hadn’t run. I didn’t do too much digging around why this wasn’t working but I guess RAX doesn’t support this yet. Hopefully in the future they will, unless they do and I’m doing something wrong.  Would be curious to know.

Hopefully you found this useful/interesting! I plan to do a follow up post on the inner workings of the metadata service within OpenStack.

This entry was posted in openstack, Uncategorized. Bookmark the permalink.

One Response to Bootstrapping Instances via Metadata and Public Cloud Metadata Support!

  1. Pingback: OpenStack Community Weekly Newsletter (June 27 – July 4) » The OpenStack Blog

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>