Building a Multi-Tier Application With OpenStack

In this blog post I’m going to give you a walk through on how to build a simple multi-tier application on OpenStack. Before I do this, I want to give a little background about the features that went into OpenStack during the Grizzly release cycle that enabled this. First I’ll talk about security groups and then about Load-balancer-as-a-Service.

During the Grizzly cycle of OpenStack there was a lot of work that went into security groups, in particular security groups involving Quantum. If you aren’t familiar with Quantum, it is a relatively new OpenStack project that became core in Folsom and its goal is to handle the networking part of OpenStack, which makes sense that security groups should be implemented there. In Folsom, if one wanted to use security groups, one would need to use Nova’s security group implementation, which had a few limitations that we wanted to fix. The first limitation was that Nova security groups implementation did not work if one wanted to use overlapping ip addresses. In addition, Nova’s security groups did not support egress filtering unless a tenant enforced this themselves within the instance. Egress filtering allows tenants to enforce which end hosts and protocols their instances are able to initiate communication with, which is useful if one wants to lock down who their instances can communicate with.

The last part of the security group work was to implement the ability for Nova to proxy its security group calls to Quantum. This is important because it allows one to create an instance via Nova and specific security groups in Quantum, which helps reduce orchestration requirements (i.e: first creating a port in quantum with specific security groups and then tell nova to use that port). In addition, this proxy layer allows existing scripts and tools to continue to work even if using quantum security groups, allows Nova’s EC2 security group implementation to work with quantum, and lastly allows the Nova security group api to work in conjunction with overlapping ips as the calls are proxied to Quantum, which handles this.

In addition, to security groups, Load-balancer-as-a-Service (LBaaS) was another feature added to Quantum during grizzly. LBaaS allows the ability to provision on demand loadbalancers pragmatically, which in my opinion is pretty freaking cool! This allows one to create several instances all running the same application and then distribute the load across them in order to scale out an application and provide high availability.

Multi-Tier Application Walk Through

Consider the scenario where a tenant wants a multi-tier application that consists of web servers and database servers. The tenant only wants to allow HTTP port 80 to be accessible to the internet from the web servers and only allows the web servers to be able to communicate with the database servers over port 3306 to access the database. The tenant also wants to have a jump box accessibly from the internet to ssh to and then ssh from that to any of the web servers or database servers if needed. This increases the security by not having your web servers and database severs directly¬†ssh-able¬†from the internet. Lastly, the tenant wants the requests to be loadbalanced between the two web servers. To demonstrate this using OpenStack, we’ll use a project called devstack to quickly setup OpenStack.

Get the devstack code:

$ git clone https://github.com/openstack-dev/devstack
$ cd devstack

Next, create a file called localrc which is used to tell devstack which components we want it to setup and install and put the following content below in it. For this demo we’ll be using the Open vSwitch Quantum plugin.

ENABLED_SERVICES=g-api,g-reg,key,n-api,n-crt,n-obj,n-cpu,n-sch,n-cauth,horizon,mysql,rabbit,sysstat,cinder,c-api,c-vol,c-sch,n-cond,quantum,q-svc,q-agt,q-dhcp,q-l3,q-meta,q-lbaas,n-novnc,n-xvnc,q-lbaas
DATABASE_PASSWORD=password
RABBIT_PASSWORD=password
SERVICE_TOKEN=password
SERVICE_PASSWORD=password
ADMIN_PASSWORD=password

Next run ./stack.sh and grab a cup of coffee as this will take a few minutes to complete as it downloads all the required packages and code.

In order to start using OpenStack, you’ll need to authenticate as a tenant. To do this, run the following command in order to put the demo user’s credentials in your environment.

$ source openrc demo demo

The devstack script automatically creates two networks for you ‘private’ and ‘public’. The ‘public’ network we’ll use to allocate floating ips out of later. Running quantum net-list will show this:

$ quantum net-list
+--------------------------------------+---------+--------------------------------------------------+
| id                                   | name    | subnets                                          |
+--------------------------------------+---------+--------------------------------------------------+
| 02e0a203-8349-47fc-8d61-8987b4197f1c | public  | c1787d38-fa26-48c9-9f36-d2f563b38c70             |
| 812fb7bd-3ad9-4583-b9f0-55c9ab5a7d55 | private | a00e7146-d77c-4835-93d0-5ab743f3aee6 10.0.0.0/24 |
+--------------------------------------+---------+--------------------------------------------------+

First we’ll create the three security groups we’ll need to contain the members: web, database and ssh.

$ quantum security-group-create web
$ quantum security-group-create database
$ quantum security-group-create ssh

Now we’ll add rules into these security groups for their desired functionality.

Allow all HTTP Port 80 traffic to web security group:

$ quantum security-group-rule-create --direction ingress --protocol TCP --port-range-min 80 --port-range-max 80 web

Allow database severs to be accessed from web servers:

$ quantum security-group-rule-create --direction ingress --protocol TCP --port-range-min 3306 --port-range-max 3306 --remote-group-id web database

Allow Jump host to ssh into database servers and webservers

$ quantum security-group-rule-create --direction ingress --protocol TCP --port-range-min 22 --port-range-max 22 --remote-group-id ssh database
$ quantum security-group-rule-create --direction ingress --protocol TCP --port-range-min 22 --port-range-max 22 --remote-group-id ssh web

Allow outside world to be able to access the jumpbox over port 22 for ssh:

$ quantum security-group-rule-create --direction ingress --protocol tcp  --port-range-min 22 --port-range-max 22 ssh

Let’s now boot some vms using these security groups. First, run quantum net-list to obtain the private network uuid that we are going to be using:

$ quantum net-list
+--------------------------------------+---------+--------------------------------------------------+
| id                                   | name    | subnets                                          |
+--------------------------------------+---------+--------------------------------------------------+
| 02e0a203-8349-47fc-8d61-8987b4197f1c | public  | c1787d38-fa26-48c9-9f36-d2f563b38c70             |
| 812fb7bd-3ad9-4583-b9f0-55c9ab5a7d55 | private | a00e7146-d77c-4835-93d0-5ab743f3aee6 10.0.0.0/24 |
+--------------------------------------+---------+--------------------------------------------------+

Then, we’ll run nova image-list to determine the images available to boot our instances with. Since we’re using devstack, the script automatically uploaded an image to glance for us to use.

$ nova image-list
+--------------------------------------+---------------------------------+--------+--------+
| ID                                   | Name                            | Status | Server |
+--------------------------------------+---------------------------------+--------+--------+
| fe2a01fe-202a-4b4d-b98c-2cbcc5c72d18 | cirros-0.3.1-x86_64-uec         | ACTIVE |        |
| 79b967f1-4f4e-4876-a2c0-edf647337e38 | cirros-0.3.1-x86_64-uec-kernel  | ACTIVE |        |
| 4d14dc5d-ef4a-451e-9ef8-bba3f8aaf66d | cirros-0.3.1-x86_64-uec-ramdisk | ACTIVE |        |
+--------------------------------------+---------------------------------+--------+--------+

Boot four instances: two web severs, one database server, and our ssh jump box.

Note: When running nova boot without –nic net-id= an instance will be allocated a port on every network the tenant owns. In this case the tenant only owns one network so we are leaving off the –nic net-id=812fb7bd-3ad9-4583-b9f0-55c9ab5a7d55 for simplicity so the commands can be copied and pasted.

Boots two instances named web_server1 and web_server2 on the private network using the cirros image and part of the web security group:

$ nova boot --image cirros-0.3.1-x86_64-uec --security_groups web --flavor 1 web_server1
$ nova boot --image cirros-0.3.1-x86_64-uec --security_groups web --flavor 1 web_server2

Boot database server:

$ nova boot --image cirros-0.3.1-x86_64-uec --security_groups database --flavor 1 database_server1

Boot ssh jump host:

$ nova boot --image cirros-0.3.1-x86_64-uec --security_groups ssh --flavor 1 jumpbox

Boot client instance that we’ll use to access the web servers from (Since we did not specify a security group this instance will be part of a ‘default’ security group which allows the instance to make outgoing connections to anyone but only accept incoming connections from members of this same security group):

$ nova boot --image cirros-0.3.1-x86_64-uec --flavor 1 client

Running nova list will display the status of the instances. After a few seconds all of the instances should go to an ACTIVE status.

$ nova list
+--------------------------------------+------------------+--------+------------------+
| ID                                   | Name             | Status | Networks         |
+--------------------------------------+------------------+--------+------------------+
| d4305afe-4985-45c1-9337-f6fd4f1c83f1 | client           | ACTIVE | private=10.0.0.7 |
| 5c4128a7-3133-43ee-8a98-ba8ff8fda65f | database_server1 | ACTIVE | private=10.0.0.5 |
| 50cbfad8-004a-4385-aa94-23d0a335fffa | jumpbox          | ACTIVE | private=10.0.0.6 |
| ec0ada74-c978-468e-9f5f-251c5dc0c710 | webserver1       | ACTIVE | private=10.0.0.3 |
| c66d0545-44af-476c-8d27-6d45c91e7e71 | webserver2       | ACTIVE | private=10.0.0.4 |
+--------------------------------------+------------------+--------+------------------+

To make the jumpbox publicly accessible on the internet we’ll need to assign a floating IP to it. To do this first create a floating IP via:

$ quantum floatingip-create public
Created a new floatingip:
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| fixed_ip_address    |                                      |
| floating_ip_address | 172.24.4.227                         |
| floating_network_id | 02e0a203-8349-47fc-8d61-8987b4197f1c |
| id                  | d67ec5ae-04e3-4e87-a6a5-7cc04a3d5608 |
| port_id             |                                      |
| router_id           |                                      |
| tenant_id           | d8293c068214472d8008000a79d369da     |
+---------------------+--------------------------------------+

Next, we need to determine the port id of the jumpbox:

$ quantum port-list
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                                                       |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
| 4cfc707f-f78a-4bd9-9c27-bb6f8b01dfcc |      | fa:16:3e:03:f7:c4 | {"subnet_id": "82294d13-5e55-4ae2-bc95-9d4e09ab4ebf", "ip_address": "10.0.0.2"} |
| 5620c0d6-046e-49de-b4ab-5d8948a78cbf |      | fa:16:3e:12:3b:75 | {"subnet_id": "82294d13-5e55-4ae2-bc95-9d4e09ab4ebf", "ip_address": "10.0.0.5"} |
| 5912c994-be8e-4b9e-8fe9-6bcb1f1c89dd |      | fa:16:3e:ba:04:b7 | {"subnet_id": "82294d13-5e55-4ae2-bc95-9d4e09ab4ebf", "ip_address": "10.0.0.1"} |
| 5cfcf996-4a3a-4a68-84e5-5c68d06832b4 |      | fa:16:3e:47:2e:72 | {"subnet_id": "82294d13-5e55-4ae2-bc95-9d4e09ab4ebf", "ip_address": "10.0.0.6"} |
| a219f389-c792-4087-a3a6-d8b9c681c170 |      | fa:16:3e:71:ce:0a | {"subnet_id": "82294d13-5e55-4ae2-bc95-9d4e09ab4ebf", "ip_address": "10.0.0.4"} |
| edeb0b7b-89f1-4018-9eae-fbf70bcee7ac |      | fa:16:3e:59:c8:ea | {"subnet_id": "82294d13-5e55-4ae2-bc95-9d4e09ab4ebf", "ip_address": "10.0.0.3"} |
| faa8718a-2efd-443f-a1b0-29c06c281e72 |      | fa:16:3e:5a:9d:a9 | {"subnet_id": "82294d13-5e55-4ae2-bc95-9d4e09ab4ebf", "ip_address": "10.0.0.7"} |
+--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+

and find the id that matches the IP address of the jumphost (10.0.0.6) and associate it via:

$ quantum floatingip-associate d67ec5ae-04e3-4e87-a6a5-7cc04a3d5608 5cfcf996-4a3a-4a68-84e5-5c68d06832b4
Associated floatingip d67ec5ae-04e3-4e87-a6a5-7cc04a3d5608

Now you should be able to ssh to the jumbox via with password cubswin:) :

$ ssh cirros@172.24.4.227

(Optional) To access your instances from horizon, point your web browser at the IP address of this box and it should take you to the horizon landing page. Log in via demo/password and then set the current project to ‘demo’ (if not already set) and click on the instances tab. This should display all of your running instances and should allow you to access them via VNC.

After logging into the jumpbox you’ll be able to ssh into your webserver1, webserver2, and database server via:

$ ssh 10.0.0.3
$ ssh 10.0.0.4
$ ssh 10.0.0.5

but none of those instances will be able to ssh to each other. The point of this instance is so that you do not need to have all of your other instances publicly addressable and directly accessible via the internet.

Now let’s log in to web_server1 and web_server2 (via ssh or via horizon) and setup a simple web server to handle requests and reply with who they are:

# On web_server 1 
$ while true; do echo -e 'HTTP/1.0 200 OK\r\n\r\nweb_server1' | sudo nc -l -p 80 ; done 

# on web_server 2
$ while true; do echo -e 'HTTP/1.0 200 OK\r\n\r\nweb_server2' | sudo nc -l -p 80 ; done

Now, log in to your client vm. From there if you run:

$ wget -O - http://10.0.0.3/
Connecting to 10.0.0.3 (10.0.0.3:80)
web_server1
               100% |************************************| 12 0:00:00 ETA

$ wget -O - http://10.0.0.4/
Connecting to 10.0.0.4 (10.0.0.4:80)
web_server2
               100% |************************************| 12 0:00:00 ETA

This demonstrates that our simple web server is working on our two web server instances.

We can demonstrate that the web security group is working correctly by killing our simple web server and changing the port number. (Note: to kill the web server you may need to hold control + c for a second in order for it to break out of the while loop before another instance of nc is created.)

# On web_server 1 
$ while true; do echo -e 'HTTP/1.0 200 OK\r\n\r\nweb_server1' | sudo nc -l -p 81 ; done

Now on the client run:

$ wget -O - http://10.0.0.3:81
Connecting to 10.0.0.3 (10.0.0.3:81)
wget: can't connect to remote host (10.0.0.3): Connection timed out

As you can see the request is never answered as expected because our web security group does not allow port 81 ingress. Now let’s set the web server to run on port 80 again.

At this point were going to provision loadbalancer via quantum order to load balance requests between our two web server instances.

First we determine the subnet uuid of the private network:

$ quantum subnet-list
+--------------------------------------+------+-------------+--------------------------------------------+
| id                                   | name | cidr        | allocation_pools                           |
+--------------------------------------+------+-------------+--------------------------------------------+
| a00e7146-d77c-4835-93d0-5ab743f3aee6 |      | 10.0.0.0/24 | {"start": "10.0.0.2", "end": "10.0.0.254"} |
+--------------------------------------+------+-------------+--------------------------------------------+

Create a loadbalancer pool:

$ quantum lb-pool-create --lb-method ROUND_ROBIN --name mypool --protocol HTTP --subnet-id a00e7146-d77c-4835-93d0-5ab743f3aee6

and then associate our two web servers with this pool.

$ quantum lb-member-create --address 10.0.0.3 --protocol-port 80 mypool
$ quantum lb-member-create --address 10.0.0.4 --protocol-port 80 mypool

Now, let’s create a health monitor, which checks to make sure our instances are still running and associate that with the pool:

$ quantum lb-healthmonitor-create --delay 3 --type HTTP --max-retries 3 --timeout 3
Created a new health_monitor:
+----------------+--------------------------------------+
| Field          | Value                                |
+----------------+--------------------------------------+
| admin_state_up | True                                 |
| delay          | 3                                    |
| expected_codes | 200                                  |
| http_method    | GET                                  |
| id             | ba9ed221-848a-4001-be0e-3a0314981396 |
| max_retries    | 3                                    |
| status         | PENDING_CREATE                       |
| tenant_id      | 53b0a7fe38b747b78e84207a34e46857     |
| timeout        | 3                                    |
| type           | HTTP                                 |
| url_path       | /                                    |
+----------------+--------------------------------------+

$ quantum lb-healthmonitor-associate ba9ed221-848a-4001-be0e-3a0314981396 mypool
Associated health monitor ba9ed221-848a-4001-be0e-3a0314981396

Let’s now create a VIP (Virtual IP Address) that when accessed the loadblancer will direct the request to either instance to loadbalance the request.

$ quantum lb-vip-create --name myvip --protocol-port 80 --protocol HTTP --subnet-id  a00e7146-d77c-4835-93d0-5ab743f3aee6 mypool
Created a new vip:
+------------------+--------------------------------------+
| Field            | Value                                |
+------------------+--------------------------------------+
| address          | 10.0.0.8                             |
| admin_state_up   | True                                 |
| connection_limit | -1                                   |
| description      |                                      |
| id               | e91236dc-b920-4302-bdfa-a6ec029ea128 |
| name             | myvip                                |
| pool_id          | 6aea6769-2cdf-4258-bbe7-60fed0965be9 |
| port_id          | bd9609f3-4c26-4b85-90f0-3e3139fc5c0e |
| protocol         | HTTP                                 |
| protocol_port    | 80                                   |
| status           | PENDING_CREATE                       |
| subnet_id        | a00e7146-d77c-4835-93d0-5ab743f3aee6 |
| tenant_id        | 53b0a7fe38b747b78e84207a34e46857     |
+------------------+--------------------------------------+

Finally, let’s test out the loadbalancer. From the client instance we should be able to run wget at 10.0.0.8 and see that it loadbalancers our requests.

$ wget -O - http://10.0.0.8/
Connecting to 10.0.0.8 (10.0.0.8:80)
web_server1
               100% |************************************| 12 0:00:00 ETA

$ wget -O - http://10.0.0.8/
Connecting to 10.0.0.8 (10.0.0.8:80)
web_server2
               100% |************************************| 12 0:00:00 ETA

$ wget -O - http://10.0.0.8/
Connecting to 10.0.0.8 (10.0.0.8:80)
web_server1
               100% |************************************| 12 0:00:00 ETA

$ wget -O - http://10.0.0.8/
Connecting to 10.0.0.8 (10.0.0.8:80)
web_server2
               100% |************************************| 12 0:00:00 ETA

From the output above you can see that the the requests are being handled by web_server1 then web_server2 in a round robin fashion.

Now to make our VIP publicly accessible via the internet we need to create another floating IP:

$ quantum floatingip-create public
Created a new floatingip:
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| fixed_ip_address    |                                      |
| floating_ip_address | 172.24.4.228                         |
| floating_network_id | 02e0a203-8349-47fc-8d61-8987b4197f1c |
| id                  | c1dad50e-fd98-4863-9d42-8d383ad441e4 |
| port_id             |                                      |
| router_id           |                                      |
| tenant_id           | d8293c068214472d8008000a79d369da     |
+---------------------+--------------------------------------+

Determine the port_id for the VIP:

$ quantum port-list
+--------------------------------------+------------------------------------------+-------------------+---------------------------------------------------------------------------------+
| id                                   | name                                     | mac_address       | fixed_ips                                                                       |
+--------------------------------------+------------------------------------------+-------------------+---------------------------------------------------------------------------------+
| 1334c522-14ad-4420-bc1a-53031ed68df9 | vip-37a3eeff-c24b-425f-8f29-c8288e1081c6 | fa:16:3e:e6:87:de | {"subnet_id": "82294d13-5e55-4ae2-bc95-9d4e09ab4ebf", "ip_address": "10.0.0.8"} |
| 4cfc707f-f78a-4bd9-9c27-bb6f8b01dfcc |                                          | fa:16:3e:03:f7:c4 | {"subnet_id": "82294d13-5e55-4ae2-bc95-9d4e09ab4ebf", "ip_address": "10.0.0.2"} |
| 5620c0d6-046e-49de-b4ab-5d8948a78cbf |                                          | fa:16:3e:12:3b:75 | {"subnet_id": "82294d13-5e55-4ae2-bc95-9d4e09ab4ebf", "ip_address": "10.0.0.5"} |
| 5912c994-be8e-4b9e-8fe9-6bcb1f1c89dd |                                          | fa:16:3e:ba:04:b7 | {"subnet_id": "82294d13-5e55-4ae2-bc95-9d4e09ab4ebf", "ip_address": "10.0.0.1"} |
| 5cfcf996-4a3a-4a68-84e5-5c68d06832b4 |                                          | fa:16:3e:47:2e:72 | {"subnet_id": "82294d13-5e55-4ae2-bc95-9d4e09ab4ebf", "ip_address": "10.0.0.6"} |
| a219f389-c792-4087-a3a6-d8b9c681c170 |                                          | fa:16:3e:71:ce:0a | {"subnet_id": "82294d13-5e55-4ae2-bc95-9d4e09ab4ebf", "ip_address": "10.0.0.4"} |
| edeb0b7b-89f1-4018-9eae-fbf70bcee7ac |                                          | fa:16:3e:59:c8:ea | {"subnet_id": "82294d13-5e55-4ae2-bc95-9d4e09ab4ebf", "ip_address": "10.0.0.3"} |
| faa8718a-2efd-443f-a1b0-29c06c281e72 |                                          | fa:16:3e:5a:9d:a9 | {"subnet_id": "82294d13-5e55-4ae2-bc95-9d4e09ab4ebf", "ip_address": "10.0.0.7"} |
+--------------------------------------+------------------------------------------+-------------------+---------------------------------------------------------------------------------+

Associate VIP port with floating IP:

$ quantum floatingip-associate c1dad50e-fd98-4863-9d42-8d383ad441e4  1334c522-14ad-4420-bc1a-53031ed68df9
Associated floatingip c1dad50e-fd98-4863-9d42-8d383ad441e4

At this point the VIP port is a member of the ‘default’ security group which does not allow ingress traffic unless you are also part of this security group so we need to update the VIP port to be a member of the web security group so that requests from the internet are allowed to pass (not just from our client instance).
Get the web security group uuid:

$ quantum security-group-list
+--------------------------------------+----------+-------------+
| id                                   | name     | description |
+--------------------------------------+----------+-------------+
| 4a65c786-b7a2-4158-a8fd-8ceff9d40337 | ssh      |             |
| 51eaf14f-d873-4e6b-ac12-648ba0bf4ce0 | web      |             |
| e11e7617-2683-48f2-a474-8634721248d2 | default  | default     |
| e637afb3-831e-4298-8e54-ceddedcd33ee | database |             |
+--------------------------------------+----------+-------------+

Update VIP port to be a member of the web security group:

$ quantum port-update 1334c522-14ad-4420-bc1a-53031ed68df9 --security_groups list=true 51eaf14f-d873-4e6b-ac12-648ba0bf4ce0
Updated port: 1334c522-14ad-4420-bc1a-53031ed68df9

At this point your VIP is publicly addressable:

$ wget -O - http://172.24.4.228
Connecting to 172.24.4.228 (172.24.4.228:80)
web_server1
               100% |************************************| 12 0:00:00 ETA

$ wget -O - http://172.24.4.228
Connecting to 172.24.4.228 (172.24.4.228:80)
web_server2
               100% |************************************| 12 0:00:00 ETA

To demonstration high availability, we’ll go and delete our web_server1 instance to simulate a failure.

$ nova delete web_server1

After our health monitor detects this. it will stop sending request to web_server1 and we can see this happening here as web_server2 handles all the requests:

$ wget -O - http://172.24.4.228
Connecting to 172.24.4.228 (172.24.4.228:80)
web_server2
               100% |************************************| 12 0:00:00 ETA

$ wget -O - http://172.24.4.228
Connecting to 172.24.4.228 (172.24.4.228:80)
web_server2
               100% |************************************| 12 0:00:00 ETA

I’d like to give a shout out to the whole OpenStack community on the amazing amount of work that was done in Grizzly and I’m excited for what’s in store for Havana!

This entry was posted in openstack, Uncategorized. Bookmark the permalink.

10 Responses to Building a Multi-Tier Application With OpenStack

  1. dino says:

    Thanks for the great guide. .This is way better than using stackops in learning openstack.

  2. Gramps Biz says:

    ditto! :)

  3. Vinay Bannai says:

    This is pretty cool.

  4. Tom Ellis says:

    Great guide. Thanks!

    The only thing I had to do ontop of this was add port 22 to the default group to allow me to ssh into the client vm, although I could have attached it to another security group with the rule.

  5. livemoon says:

    Good work. An intro about using new future in G version.

  6. Pingback: Installing OpenStack Grizzly with DevStack | NetworkStatic | Brent Salisbury's Blog

  7. Malini Bhandaru says:

    Excellent article demonstrating LBaaS and Security groups in Quantum. Thank you! – Malini

  8. curtis says:

    Thanks a lot for this guide. A great way to start into quantum. Much appreciated.

  9. edwardo says:

    Thanks a million, this made my day.

  10. Roman Rader says:

    Great explanation! Finally understood several things about OS.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>