Explanation of cloudinit.d services¶
The primary building block of cloudinit.d is the service. For readers familiar with init.d, the cloudinit.d service is akin to the program in init.d. On this page we will describe the cloudinit.d service construct and how cloudinit.d starts and configures its services.
Often times a service is composed of many processes running in a single VM. This is not strictly true. In fact, a service can be created entirely without a VM, and further many cloudinit.d service’s can all run on the same machine. However, in the classic case a VM is started specifically to accommodate the needs of a single server. That VM is launched by cloudinit.d, a set of user defined configuration scripts are then copied into that VM and run to contextualize it. The health of that service is then monitored by another service program.
The presentation at the beginning of this page illustrates this.
There are three programs that are associated with a service:
- bootpgm This program runs once to setup the service. Often times it will download and install software (apt-get/yum) and then configure that software for use considering the values and locations of other services in the boot.
- readypgm This program is run to check the status and health of the service. It can be, and typically is, run many times. As an example, if the service’s goal was to serve http, the readypgm would connect to localhost:80, download a known web page and check its content. If all is well the readypgm returns 0 and the service is reported as working. If not, the services is marked as down and the cloud application is in need of a repair.
- terminatepgm The terminate program is run when a service is shutdown. It is there to nicely cleanup resources associated with the service.
All of these programs are user defined and written as part of the process of creating a cloudinit.d service. None of the programs are mandatory. If you can create a service that does not need one of them, by all means ignore the unneeded ones. However we have found that they tend to be needed.
As an example let us say that we want to create a service that runs an apache2 web server. We find a base ubuntu VM image on our cloud and we associate it with our service. Now we must create the pgms needed to turn this VM into a cloudinit.d service. Some possible programs are shown below:
These programs can be written in any scripting language available on the VM on which they will be run.
#!/bin/bash sudo apt-get install apache2 exit $?
#!/bin/bash wget http://localhost exit $?
#!/bin/bash /etc/init.d/apache2 stop exit $?
All of the above programs are scp‘ed to the hosting machine and then run via ssh. Thus the user must have ssh credentials that will work on the associated cloud and VM, and port 22 must be open to that VM. This is a common scenario for infrastructure clouds.
A bootlevel is a set of services that can all be started at the same time. In other words, a set of services that have no dependencies on each other. These services are launched at the same time. The boot level is considered complete when all of the services’ bootpgms successfully complete.
All of the information about where (what cloud) a particular service is to be run on is set at the service level. Thus a cloudinit.d application can run across many clouds. It is entirely possible (and encouraged) to have an application launched with cloudinit.d such that some services are in a private cloud and others are in a public cloud. Right now cloudinit.d works on EC2, Nimbus, and Eucalyptus using the Boto library, and on many other clouds using the libcloud library.
Selecting a cloud is done using the iaas and iaas_url variables. To use EC2 (with the Boto library), select the region with the iaas variable. To use another cloud than EC2 with the Boto library, set iaas_url to the cloud API endpoint (in this case the value of iaas does not matter). Finally, to use the Libcloud library instead of Boto, set iaas to libcloud-<libcloud driver> and iaas_url to the cloud API endpoint.
To use the EU region of Amazon EC2:
To use the Nimbus cloud in FutureGrid Hotel with the Boto library:
To use the Nimbus cloud in FutureGrid Hotel with the Libcloud library:
iaas: libcloud-nimbus iaas_url: https://svc.uc.futuregrid.org:8444
The goal of cloudinit.d is to orchestrate many services to work in concert together to form a single application. In order for this to be possible the services need a way to discover information about each other. For example, if we are making a web server backed by a database we would make the database one service and the webserver another (as is the case in our wordpress example). Because the web server is dependent upon the database, we would put the database at bootlevel 1 and the web server at boot level 2.
However, just having the web server wait for the database to be ready is not enough. The web server must know the IP address and the port number of the database in order to connect to it. Further they likely need some sort of shared secret for making a secure connection. Cloudinit.d handles the exchange of this, and similar types of dependency information. Any service is allowed to lookup another service (that is at a lower boot level) and request an attribute from it. There is a small set of statically defined attributes that a service has (ex: hostname, IaaS instance id, etc) and the service can further defined its own setup attributes.
This secure exchange of service defined attributes is what makes cloudinit.d a powerful tool.