Working with remote Mosel instances

The remote execution of submodels only requires few additions to models running submodels on a single Mosel instance. The examples in this section show how to extend a selection of the models we have seen so far with distributed computing functionality. Before you access Mosel on a remote host, or an additional local instance of Mosel, you need to start up the Mosel server (on the node you wish to use) with the command

xprmsrv

This command may take some options, such as the verbosity level, and the TCP port to be used by connections. The options are documented in the section on mmjobs in the Mosel Language Reference Manual, and you can also display a list with

xprmsrv -h

Figure 4: Remote submodel execution

Executing a submodel remotely

To execute a model remotely, we first need to create a new Mosel instance on the remote node. In this connect statement, the string that identifies the remote machine may be its local name, an IP address, or the empty string for the current node. In the example runrtdistr.mos below, we compile the submodel locally and after connecting, load the BIM file into the remote instance (notice the use of the rmt I/O driver to specify that the BIM is located on the root node). The load subroutine now takes an additional first argument indicating the Mosel instance we want to use. All else, including the run statement and the handling of events remain the same as in the single instance version of this model from Section Runtime parameters.

model "Run model rtparams remotely"
 uses "mmjobs"

 declarations
  modPar: Model
  mosInst: Mosel
 end-declarations
                                   ! Compile the model file
 if compile("rtparams.mos")<>0 then exit(1); end-if

 NODENAME:= ""                     ! "" for current node, or name, or IP address
                                   ! Open connection to a remote node
 if connect(mosInst, NODENAME)<>0 then exit(2); end-if
                                   ! Load the bim file
 load(mosInst, modPar, "rmt:rtparams.bim")
                                   ! Start model execution + parameter settings
 run(modPar, "PARAM1=" + 2 + ",PARAM2=" + 3.4 +
             ",PARAM3='a string'" + ",PARAM4=" + true)
 wait                              ! Wait for model termination
 dropnextevent                     ! Ignore termination event message

end-model 

Alternatively to the above, we might have compiled the submodel remotely after having established the connection, for instance using this code (the rmt prefix now appears in the compilation since we assume that the submodel is located at the same place as the master model).

 if connect(mosInst, NODENAME)<>0 then exit(2); end-if
 if compile(mosInst, "", "rmt:rtparams.mos", "rtparams.bim")<>0 then
  exit(1); end-if
 load(mosInst, modPar, "rtparams.bim")  

Compiling the submodel on the original (root) node is preferable if we wish to execute several model instances on different remote nodes, and it may also be necessary if write access is not permitted on the remote node.

By default, the rmt driver refers to the root node. It is also possible to prefix the filename by its node number enclosed in square brackets, the special node number -1 denotes the immediate parent node, such as in

 load(mosInst, modPar, "rmt:[-1]rtparams.bim")  

Note: In this example, we assume that we know of a specific Mosel instance that we wish to connect to. The example model in Section Finding available Mosel servers shows how to search for available xprmsrv servers on the local network.

Parallel submodels in distributed architecture

The following model runrtpardistr.mos is an extension of the example from Section Parallel submodels. The 10 model instances to be run are now distributed over 5 Mosel instances, assigning 2 models per instance.

model "Run model rtparams in distributed architecture"
 uses "mmjobs", "mmsystem"

 declarations
  A = 1..10
  B = 1..5
  modPar: array(A) of Model
  moselInst: array(B) of Mosel
  NODENAMES: array(B) of string
 end-declarations

                                 !!! Select the (remote) machines to be used:
				 !!! Use names, IP addresses, or empty string
				 !!! for the node running this model
 forall(i in B) NODENAMES(i):= ""

                                   ! Compile the model file locally on root
 if compile("rtparams3.mos")<>0 then exit(1); end-if

 instct:=0
 forall(i in A) do
  if isodd(i) then
   instct+=1                       ! Connect to a remote machine
   if connect(moselInst(instct), NODENAMES(instct))<>0 then exit(2); end-if
  end-if

  writeln_("Current node: ", getsysinfo(SYS_NODE),
           " submodel node: ", getsysinfo(moselInst(instct), SYS_NODE))

                                   ! Load the bim file (located at root node)
  load(moselInst(instct), modPar(i), "rmt:rtparams3.bim")
                                   ! Start remote model execution
  run(modPar(i), "PARAM1=" + i + ",PARAM2=" + 0.1*i +
                 ",PARAM3='string " + i + "'" + ",PARAM4=" + isodd(i))
 end-do

 forall(i in A) do
  wait                             ! Wait for model termination
  dropnextevent                    ! Ignore termination event message
 end-do

end-model

We work here with a slightly modified version rtparams3.mos of the submodel that has some added reporting functionality, displaying node and model numbers and the system name of the node.

model "Runtime parameters"
 uses "mmjobs", "mmsystem"
 parameters
  PARAM1 = 0
  PARAM2 = 0.5
  PARAM3 = ''
  PARAM4 = false
 end-parameters

 writeln_("Node: ", getparam("NODENUMBER"), " ", getsysinfo(SYS_NODE),
          ". Parent: ", getparam("PARENTNUMBER"),
          ". Model number: ", getparam("JOBID"),
          ". Parameters: ",
          PARAM1, "  ", PARAM2, "  ", PARAM3, "  ", PARAM4)
end-model 

Job queue for parallel execution in a distributed architecture

Let us now take a look at how we can extend the job queue example from Section Job queue for managing parallel submodels to the case of multiple Mosel instances. We shall work with a single queue (list) of jobs, from which we take the first available job as soon as another model has terminated and we start this new model on the corresponding Mosel instance. Each instance of Mosel processes at most NUMPAR submodels at any time.

model "Run model rtparams with job queue"
 uses "mmjobs", "mmsystem"

 parameters
   J=10                             ! Number of jobs to run
   NUMPAR=2                         ! Number of parallel model executions
 end-parameters                     ! (preferrably <= no. of processors)

 forward procedure start_next_job(submod: Model)

 declarations
   RM: range                        ! Model indices
   JOBS = 1..J                      ! Job (instance) indices
   modPar: array(RM) of Model       ! Models
   jobid: array(set of integer) of integer  ! Job index for model UIDs
   JobList: list of integer         ! List of jobs
   JobsRun: set of integer          ! Set of finished jobs
   JobSize: integer                 ! Number of jobs to be executed
   Msg: Event                       ! Messages sent by models
   NodeList: list of string         !
   nodeInst: array(set of string) of Mosel  ! Mosel instances on remote nodes
   nct: integer
   modNode: array(set of integer) of string ! Node used for a model
   MaxMod: array(set of string) of integer
 end-declarations
                                    ! Compile the model file locally
 if compile("rtparams.mos")<>0 then exit(1); end-if

!**** Setting up remote Mosel instances ****
 sethostalias("localhost2","localhost")
 NodeList:= ["localhost", "localhost2"]
         !!! This list must have at least 1 element.
         !!! Use machine names within your local network, IP addresses, or
         !!! empty string for the current node running this model.

 forall(n in NodeList) MaxMod(n):= NUMPAR
         !!! Adapt this setting to number of processors and licences per node

 forall(n in NodeList, nct as counter) do
  create(nodeInst(n))
  if connect(nodeInst(n), n)<>0 then exit(1); end-if
  if nct>= J then break; end-if     ! Stop if started enough instances
 end-do

!**** Loading model instances ****
 nct:=0
 forall(n in NodeList, m in 1..MaxMod(n), nct as counter) do
   create(modPar(nct))
   load(nodeInst(n), modPar(nct), "rmt:rtparams.bim")  ! Load the bim file
   modPar(nct).uid:= nct            ! Store the model ID as UID
   modNode(modPar(nct).uid):= getsysinfo(nodeInst(n), SYS_NODE)
 end-do


 JobList:= sum(i in JOBS) [i]       ! Define the list of jobs (instances)
 JobSize:=JobList.size              ! Store the number of jobs
 JobsRun:={}                        ! Set of terminated jobs is empty

!**** Start initial lot of model runs ****
 forall(m in RM)
   if JobList<>[] then
     start_next_job(modPar(m))
   end-if

!**** Run all remaining jobs ****
 while (JobsRun.size<JobSize) do
   wait                             ! Wait for model termination
  ! Start next job
   Msg:= getnextevent
   if Msg.class=EVENT_END then      ! We are only interested in "end" events
     m:= Msg.fromuid                ! Retrieve the model UID
     JobsRun+={jobid(m)}            ! Keep track of job termination
     writeln_("End of job ", jobid(m), " (model ", m, ")")
     if JobList<>[] then            ! Start a new run if queue not empty
      start_next_job(modPar(m))
     end-if
   end-if
 end-do

 fdelete("rtparams.bim")            ! Cleaning up
end-model 

A new function used by this example is sethostalias: the elements of the list NodeList are used for indexing various arrays and must therefore all be different. If we want to start several Mosel instances on the same machine (here: the local node, designated by localhost) we must make sure that they are addressed with different names. These names can be set up through sethostalias (see the Mosel Language Reference Manual for further detail).

The procedure start_next_job that starts the next job from the queue on remains exactly the same as before (in Section Job queue for managing parallel submodels) and its definition is not repeated here.

Finding available Mosel servers

In the preceding examples we have specified the names of the Mosel instances that we wish to connect to. It is also possible to search for Mosel servers on the local network and hence decide dynamically which remote instances to use for the processing of submodels. This search functionality is provided by the procedure findxsrvs that returns a set of at most M (second argument) instance names in its third argument. The first subroutine argument selects the group number of the servers, further configuration options such as the port number to be used are accessible through control paramters (see the documentation of mmjobs in the Mosel Language Reference for the full list).

model "Find Mosel servers"
 uses "mmjobs"

 parameters
   M = 20                  ! Max. number of servers to be sought
 end-parameters

 declarations
   Hosts: set of string
   mosInst: Mosel
 end-declarations

 findxsrvs(1, M, Hosts)
 writeln_(Hosts.size, " servers found: ", Hosts)

 forall(i in Hosts)        ! Establish remote connection and print system info
   if connect(mosInst, i)=0 then
     writeln_("Server ", i, ": ", getsysinfo(mosInst))
     disconnect(mosInst)
   else
     writeln_("Connection to ", i, " failed ")
   end-if
end-model