XPRD: Remote model execution without local installation

In a distributed Mosel application, the local Mosel (master) model that we have seen in the previous section can be replaced by an XPRD (C or Java) program. The Mosel remote invocation library (XPRD) makes it possible to build applications requiring the Xpress technology that run from environments where Xpress is not installed—including architectures for which Xpress is not available. XPRD is a self-contained library (i.e. with no dependency on the usual Xpress libraries) that provides the necessary routines to start Mosel instances either on the local machine or on remote hosts and control them in a similar way as if they were invoked through the Mosel libraries. Besides the standard instance and model handling operations (connect/disconnect, compile-load-run, stream redirection), the XPRD library supports the file handling mechanisms of mmjobs (transparent file access between instances) as well as its event signaling system (events can be exchanged between the application and running models).

Mosel				XPRD

Figure 5: Remote model execution with and without local installation

The following examples show how to replace the Mosel master model in the examples from Section Working with remote Mosel instances by a Java program. The Xpress distribution equally includes C versions of these examples in the subdirectory examples/mosel/WhitePapers/MoselPar/XPRD. The Mosel (sub)models remain unchanged from the versions shown in the previous section.

Executing a model remotely

The structure of a Java program for compiling and launching the Mosel model rtparams.mos on a remote Mosel instance remains exactly the same as what we have seen for the Mosel master model in Section Executing a submodel remotely, including the use of the rmt driver indicating that the Mosel source is located on a remote machine. The generated BIM file is saved into Mosel's temporary working directory (using the tmp driver).

import com.dashoptimization.*;
import java.lang.*;
import java.io.*;

public class runrtdistr
{
 public static void main(String[] args) throws Exception
 {
  XPRD xprd=new XPRD();
  XPRDMosel mosInst=null;
  XPRDModel modPar:=null;

       // Use the name or IP address of a machine in
       // your local network, or "" for current node
  String NODENAME = "";
                               // Open connection to a remote node
  mosInst=xprd.connect(NODENAME);
                               // Compile the model file
  mosInst.compile("", "rmt:rtparams.mos", "tmp:rp.bim");
                               // Load bim file into the remote instance
  modPar=mosInst.loadModel("tmp:rp.bim");
                               // Run-time parameters
  modPar.execParams = "PARAM1=" + 2 + ",PARAM2=" + 3.4 +
                     ",PARAM3='a string'" + ",PARAM4=true";
  modPar.run();                // Run the model
  xprd.waitForEvent();         // Wait for model termination
  xprd.dropNextEvent();        // Ignore termination event message

  System.out.println("`rtparams' returned: " + modPar.getResult());

  mosInst.disconnect();        // Disconnect remote instance
 }
}

For compiling and running this Java program only the XPRD library is required, the corresponding commands typically look as follows (Windows version, assuming that xprd.jar is located in the current working directory):

javac -cp xprd.jar:. runrtdistr.java
java -cp xprd.jar:. runrtdistr 

Note: In this example, we assume that we know of a specific Mosel instance that we wish to connect to. The program example in Section Finding available Mosel servers shows how to search for available xprmsrv servers on the local network.

Parallel models in distributed architecture

The following Java program runrtpardistr.java extends the simple example from the previous section to running A=10 model instances on B=5 Mosel instances, assigning 2 models to each instance. All submodels are started concurrently and the Java programm waits for their termination.

import com.dashoptimization.*;
import java.lang.*;
import java.io.*;

public class runrtpardistr
{
 static final int A=10;
 static final int B=5;

 public static void main(String[] args) throws Exception
 {
  XPRD xprd=new XPRD();
  XPRDMosel[] mosInst=new XPRDMosel[B];
  XPRDModel[] modPar=new XPRDModel[A];
  String[] NODENAMES=new String[B];
  int i,j;

       // Use the name or IP address of a machine in
       // your local network, or "" for current node
  for(j=0;j<B;j++) NODENAMES[j] = "localhost";

                               // Open connection to remote nodes
  for(j=0;j<B;j++)
    mosInst[j]=xprd.connect(NODENAMES[j]);

  for(j=0;j<B;j++)
    System.out.println("Submodel node: " +
       mosInst[j].getSystemInformation(XPRDMosel.SYS_NODE) + " on " +
       mosInst[j].getSystemInformation(XPRDMosel.SYS_NAME));

                               // Compile the model file on one instance
  mosInst[0].compile("", "rmt:rtparams3.mos", "rmt:rp3.bim");

  for(i=0;i<A;i++)
  {	                       // Load the bim file into remote instances
    modPar[i]=mosInst[i%B].loadModel("rmt:rp3.bim");
                               // Run-time parameters
    modPar[i].execParams = "PARAM1=" + i + ",PARAM2=" + (0.1*i) +
                          ",PARAM3='a string " + i + "',PARAM4=" + (i%2==0);
    modPar[i].run();           // Run the model
  }

  for(i=0;i<A;i++)
  {
    xprd.waitForEvent();       // Wait for model termination
    xprd.dropNextEvent();      // Ignore termination event message
  }

  for(j=0;j<B;j++)
    mosInst[j].disconnect();   // Disconnect remote instance

  new File("rp3.bim").delete();   // Cleaning up
 }
}

Job queue for parallel execution in a distributed architecture

The following Java program implements a job queue for coordinating the execution of a list of model runs on several remote Mosel instances, similarly to the Mosel model in Section Job queue for parallel execution in a distributed architecture. Each Mosel instance processes up to NUMPAR concurrent submodels. The job queue and the list of terminated jobs are represented by List structures. The program closely matches the previous implementation with a Mosel master model, including the use of the subroutine startNextJob to retrieve the next job from the queue and to start its processing.

import com.dashoptimization.*;
import java.lang.*;
import java.util.*;
import java.io.*;

public class runrtparqueued
{
 static final int J=10;             // Number of jobs to run
 static final int NUMPAR=2;         // Number of parallel model executions
                                    // (preferrably <= no. of processors)
 static int[] jobid;
 static int[] modid;
 static String[] modNode;

 public static void main(String[] args) throws Exception
 {
  XPRD xprd=new XPRD();
                            // Use the name or IP address of a machine in
                            // your local network, or "" for current node
  String[] NodeList={"localhost","localhost"};
  final int nbNodes=(NodeList.length<J?NodeList.length:J);
  XPRDMosel[] mosInst=new XPRDMosel[nbNodes];
  int[] MaxMod=new int[nbNodes];
  XPRDModel[] modPar=new XPRDModel[nbNodes*NUMPAR];
  int nct;
  List<Integer> JobList=new ArrayList<Integer>();
  List<Integer> JobsRun=new ArrayList<Integer>();
  int JobSize;
  XPRDEvent event;
  int lastId=0;

  //**** Setting up remote Mosel instances ****
  for(int n=0;n<nbNodes;n++)
  {
   mosInst[n]=xprd.connect(NodeList[n]);
   MaxMod[n]= NUMPAR;
         // Adapt this setting to number of processors and licences per node
  }
                                    // Compile the model file on first node
  mosInst[0].compile("", "rmt:rtparams.mos", "rmt:rtparams.bim");

  //**** Loading model instances ****
  nct=0;
  for(int n=0;(n<nbNodes) && (nct<J);n++)
   for(int m=0;(m<MaxMod[n]) && (nct<J);m++)
   {                               // Load the bim file
    modPar[nct]=mosInst[n].loadModel("rmt:rtparams.bim");
    if(modPar[nct].getNumber()>lastId) lastId=modPar[nct].getNumber();
    nct++;
   }

  jobid=new int[lastId+1];
  modid=new int[lastId+1];
  modNode=new String[lastId+1];
  for(int j=0;j<nct;j++)
  {
   int i=modPar[j].getNumber();
   modid[i]=j;                    // Store the model ID
   modNode[i]=modPar[j].getMosel().getSystemInformation(XPRDMosel.SYS_NODE);
  }

  for(int i=0;i<J;i++)            // Define the list of jobs (instances)
   JobList.add(new Integer(i));
  JobSize=JobList.size();         // Store the number of jobs
  JobsRun.clear();                // List of terminated jobs is empty

  //**** Start initial lot of model runs ****
  for(int j=0;j<nct;j++)
   startNextJob(JobList,modPar[j]);

  //**** Run all remaining jobs ****
  while(JobsRun.size()<JobSize)
  {
   xprd.waitForEvent();           // Wait for model termination
   event=xprd.getNextEvent();     // We are only interested in "end" events
   if(event.eventClass==XPRDEvent.EVENT_END)
   {                              // Keep track of job termination
    JobsRun.add(new Integer(jobid[event.sender.getNumber()]));
    System.out.println("End of job "+ jobid[event.sender.getNumber()] +
                              " (model "+ modid[event.sender.getNumber()]+ ")");
    if(!JobList.isEmpty())        // Start a new run if queue not empty
     startNextJob(JobList,event.sender);
   }
  }

  for(int n=0;n<nbNodes;n++)
   mosInst[n].disconnect();                 // Terminate remote instances

  new File("rtparams.bim").delete();        // Cleaning up
 }

//******** Start the next job in a queue ********
 static void startNextJob(List<Integer> jobList,XPRDModel model) throws Exception
 {
  Integer job;
  int i;

  job=jobList.remove(0);      // Retrieve and remove first entry from job list
  i=job.intValue();
  System.out.println("Start job "+ job + " (model " + modid[model.getNumber()]+
                          ") on "+ modNode[model.getNumber()] );
  model.execParams = "PARAM1=" + i + ",PARAM2=" + (0.1*i) +
                          ",PARAM3='a string " + i + "',PARAM4=" + (i%2==0);
  model.run();
  jobid[model.getNumber()]=i;
 }
} 

Finding available Mosel servers

In the preceding examples we have specified the names of the Mosel instances that we wish to connect to. It is also possible to search for Mosel servers on the local network and hence decide dynamically which remote instances to use for the processing of submodels. This search functionality is provided by the method findXsrvs of class XPRD that returns a set of at most M (second argument) instance names in its third argument. The first argument of this method selects the group number of the servers, further configuration options such as the port number are accessible through control paramters (please refer to the documentation of findXsrvs in the XPRD Javadoc for further detail).

import com.dashoptimization.*;
import java.lang.*;
import java.io.*;
import java.util.*;

public class findservers
{
 static final int M=20;

 public static void main(String[] args) throws Exception
 {
  XPRD xprd=new XPRD();
  XPRDMosel mosInst=null;
  Set<String> Hosts=new HashSet<String>();

  xprd.findXsrvs(1, M, Hosts);
  System.out.println(Hosts.size() + " servers found.");

  for(Iterator<String> h=Hosts.iterator(); h.hasNext();)
  {
   String i=h.next();

   try {
    mosInst=xprd.connect(i);     // Open connection to a remote node
                                 // Display system information
    System.out.println("Server " + i + ": " + mosInst.getSystemInformation());
    mosInst.disconnect();        // Disconnect remote instance
   }
   catch(IOException e) {
    System.out.println("Connection to " + i + " failed");
   }
  }
 }
}