Multi-Cloud Clusters

Guide for launching auto-configuring clusters across multiple clouds.


Prerequisites

  • Experimental support for this feature is present in cloud-client-012 and later.
  • You must have access to multiple Nimbus clouds running TP2.2 or later. EC2 is not supported for multi-cloud clusters at this time.
  • You should be familiar with usage of the cloud-client for single cluster contextualization described in the Client Quickstart and One-click Clusters sections.

Introduction

Multi-cloud cluster deployment happens through the meta-cloud-client (bin/meta-cloud-client.sh). However, you'll need to use the regular cloud-client to handle uploading your images to each cloud.

Deployment documents

In addition to the usual cluster definition document, you must also provide a document that describes how the cluster members should be spread across the available clouds. The deployment document largely matches the structure of the cluster document. You must have a workspace entry in the deployment for each workspace in the cluster definition.

Below is a sample cluster definition followed by a corresponding deployment document: 

<cluster xmlns="https://www.globus.org/2008/06/workspace/metadata/logistics">

  <workspace>
    <name>head-node</name>
    <image>my-head-node</image>
    <quantity>1</quantity>
    <nic wantlogin="true">public</nic>
  </workspace>

  <workspace>
    <name>compute-nodes</name>
    <image>my-compute-node</image>
    <quantity>2</quantity>
    <nic>public</nic>
  </workspace>

</cluster>

<deployment xmlns="https://www.globus.org/2008/06/workspace/metadata/logistics">

  <workspace>
    <name>head-node</name>

    <deploy>
      <cloud>nimbus</cloud>
      <quantity>1</quantity>
    </deploy>

  </workspace>

  <workspace>
    <name>compute-nodes</name>

    <deploy>
      <cloud>nimbus</cloud>
      <quantity>1</quantity>
    </deploy>

    <deploy>
      <cloud>stratus</cloud>
      <quantity>1</quantity>
      <image>stratus-compute-node</image>
    </deploy>

  </workspace>

</deployment>

In this example, the single head-node workspace is deployed to the cloud named nimbus. Of the two compute-nodes workspaces, one is deployed to nimbus and one to stratus. The stratus deployment uses a special image name which will override the one provided in the cluster document.

Configuration

Cloud configuration files live in conf/clouds/. Each cloud has its own properties file. The name of the file is important; it is the unique cloud identifier in the deployment documents. Cloud properties files can usually be obtained from the site administrator for your cloud. Below is an example properties file for the Nimbus cloud: 

###########################################################################
# Host+port of Virtal Workspace Service (not the URL)
###########################################################################

vws.factory=tp-vm1.ci.uchicago.edu:8445


###########################################################################
# Host+port of image repository (not the URL)
###########################################################################

vws.repository=tp-vm1.ci.uchicago.edu:2811


###########################################################################
# Virtal Workspace Service identity, to verify we are talking to the
# right machine.
###########################################################################

vws.factory.identity=/O=Grid/OU=GlobusTest/OU=simple-workspace-ca/CN=host/tp-vm1.ci.uchicago.edu


###########################################################################
# Image repository identity, to verify we are talking to the right machine.
###########################################################################

vws.repository.identity=/O=Grid/OU=GlobusTest/OU=simple-workspace-ca/CN=host/tp-vm1.ci.uchicago.edu

Usage

The meta-cloud-client is launched much like the cloud-client:

$ ./bin/meta-cloud-client.sh --run --cluster cluster.xml --deploy deploy.xml --hours 1

If all goes well, your cluster will be deployed to all clouds involved and contextualized using the Context Broker referenced in the conf/cloud.properties configuration file.