Introduction

Topics covered in this chapter:

Amazon Elastic Compute Cloud (EC2) is part of Amazon Web Services (AWS http://aws.amazon.com). Using either a web interface or a set of API calls the service allows to run virtual machines on the Amazon infrastructures. A virtual machine instance is started from an Amazon Machine Image (AMI): this is the image of the machine to run (it includes both the operating system and a set of pre-installed applications).

The AEC2 Mosel package is a subset of the AWS API for Mosel. It enables a model to:

• manage basic AWS functionality like creating keys, defining security credentials, building AMIs, ...
• start/stop and query the status of running virtual machines in the Cloud
• start and use Mosel instances on virtual machines just like they were running on the local network (using the standard mmjobs functionality)

This package is also available as an extension to XPRD (Java version) with which a Java program using XPRD can manage instances created in EC2. This extension uses the same configuration files as the Mosel package.

Overview

The AEC2 package consists in a Mosel package (aec2.bim) and the program aec2setup. In addition to these components, system commands to connect to remote hosts using the SSH protocol are also required. On Unix-like platforms, the programs ssh and scp are usually available; on Windows systems, specific programs pcmdgen.exe, mpscp.exe and mplink.exe are included in the package distribution.

Using Amazon EC2 requires the availability of machine images (AMI) in order to start virtual machines. For the particular case of Mosel, the AMI must have Xpress installed and accept connections for running remote Mosel instances either via SSH or using the xprmsrv server: the provided setup program (aec2setup) can be used to create appropriate images. This program, that is to be run once only, also results in a set of configuration files required to initialise the package when it is used by a model.

With the help of the mmjobs module a Mosel model can already start and manage remote instances for creating distributed applications. The AEC2 package adds a set of routines to this framework for managing EC2 virtual machines and generating the appropriate connection strings for the connect function of mmjobs AEC2 is designed such that enabling an existing distributed model to use resources in the Cloud merely requires the user to add a few statements to the original Mosel model source code. For instance, consider the following model that compiles and then executes the model "mymodel.mos" on some remote host "myserver":

model dist
uses 'mmjobs'

declarations
 minst: Mosel
 rmod: Model
end-declarations

if connect(minst, "myserver")<>0 then
 exit(1)
end-if

if compile(minst, "", "rmt:mymodel.mos", "tmp:m.bim")<>0 then
 exit(2)
end-if

load(minst, rmod, "tmp:m.bim")
run(rmod)
wait
dropnextevent
unload(rmod)
disconnect(minst)
end-model

The following example performs the same operations but using a virtual machine in EC2 instead of a local server:

model distec2
uses 'mmjobs','aec2'

declarations
 ainst: EC2instance
 minst: Mosel
 rmod: Model
end-declarations

if not loadAEC2Config("aec2.acf") then  ! load AEC2 configuration
 exit(-1)
end-if

ainst:=runInstance                      ! start a virtual machine in the Cloud
if not waitInstanceReady(ainst, 300, true) then
 exit(-2)
end-if

if connect(minst,getConnString(ainst))<>0 then
 exit(1)
end-if

if compile(minst, "", "rmt:mymodel.mos", "tmp:m.bim")<>0 then
 exit(2)
end-if

load(minst, rmod, "tmp:m.bim")
run(rmod)
wait
dropnextevent
unload(rmod)
disconnect(minst)

terminateInstance(ainst)                 ! terminate the virtual machine
end-model

Note the use of function getConnString as input parameter for the mmjobs function connect: it generates a valid connection string for the specified Amazon instance (either for an SSH or xprmsrv link depending on the default protocol).

Setup and creation of an Amazon Machine Image (AMI) for Mosel

The AEC2 settings are stored in a set of configuration files. The 'aec2setup' program will configure your AWS account as needed and create these configuration files. One of the tasks performed by this program is to create a dedicate AMI running the Linux operating system. To this end, an Xpress Linux package (either 32bit or 64bit depending on the needs) together with its associated license file are required to run the program.

To run the setup program, copy into the same directory the Xpress Linux archive (file name in the form "xp?.?.?_linux*_setup.tar"), the "xpauth.xpr" license and the "aec2setup" program itself then, from a command prompt type:

aec2setup

During the process, the program will ask various questions, in most cases a default answer is provided (noted in square brackets): simply hit enter if you do not know what to do. As part of the procedure a virtual machine is started in the Cloud and Xpress is installed on it in order to create the AMI to be used later on. Note that:

• starting a virtual machine and creating the AMI may be quite long operations (up to several minutes): do not interrupt the program during these operations
• in case of failure, you can restart the program: it will recover information already setup.

Once the setup has completed, the following files have been created:

• "aec2.aid" the secret key-pair to access your AWS account. This file must be kept secret as it gives full access to your AWS account
• "aec2.ask" the private key to log into virtual machines. The information stored in this file cannot be regenerated. If it is lost or corrupted, a new security credential will have to be created (i.e. delete the file and re-run aec2setup)
• "aec2.acf" the main configuration file (that references the 2 others)

These 3 files are required by the AEC2 package when running a model using it.