Running several submodels

Once we have seen how to run a parameterized model from a master model it is only a small step to the execution of several different submodel instances from a master model. The following sections deal with the three cases of sequential, fully parallel, and restricted (queued) parallel execution of submodels. For simplicity's sake, the submodels in our examples are all parameterized versions of a single model. It is of course equally possible to compile, load, and run different submodel files from a single master model.

Sequential submodels

Figure 2: Sequential submodels

Running several instances of a submodel in sequence only requires small modifications to the master model that we have used for a single model instance as can be seen from the following example (file runrtparamseq.mos)—to keep things simple, we now only reset a single parameter at every execution:

model "Run model rtparams in sequence"
 uses "mmjobs"

 declarations
  A = 1..10
  modPar: Model
 end-declarations
                                   ! Compile the model file
 if compile("rtparams.mos")<>0 then exit(1); end-if
 load(modPar, "rtparams.bim")      ! Load the bim file

 forall(i in A) do
  run(modPar, "PARAM1=" + i)       ! Start model execution
  wait                             ! Wait for model termination
  dropnextevent                    ! Ignore termination event message
 end-do

end-model 

The submodel is compiled and loaded once and after starting the execution of a submodel instance we wait for its termination before the next instance is started.

Parallel submodels

Figure 3: Parallel submodels

The parallel execution of submodel (instances) requires slightly more modifications. We still have to compile the submodel only once, but it now needs to be loaded as many times as we want to run parallel instances. The wait statements are now moved to a separate loop since we first want to start all submodels and then wait for their termination.

model "Run model rtparams in parallel"
 uses "mmjobs"

 declarations
  A = 1..10
  modPar: array(A) of Model
 end-declarations
                                   ! Compile the model file
 if compile("rtparams.mos")<>0 then exit(1); end-if

 forall(i in A) do
  load(modPar(i), "rtparams.bim")  ! Load the bim file
  run(modPar(i), "PARAM1=" + i)    ! Start model execution
 end-do

 forall(i in A) do
  wait                             ! Wait for model termination
  dropnextevent                    ! Ignore termination event message
 end-do

end-model 

The order in which the submodel output appears on screen is nondeterministic because the models are run in parallel. However, since the submodel execution is very quick, this may not become obvious: try adding the line wait(1) to the submodel rtparams.mos immediately before the writeln statement (you will also need to add the statement uses "mmjobs" at the beginning of the model) and compare the output of several runs of the master model. You are now likely to see different output sequences with every run.

Job queue for managing parallel submodels

The parallel execution of submodels in the previous section starts a large number of models at the same time. With computationally more expensive submodel runs this simple design might not be an appropriate choice: as a general rule, we recommend that the number of concurrent (sub)models should not exceed the number of processors available to avoid any negative impact on performance, and there might also be restrictions on the number of concurrent models imposed by your Xpress licence.

The following example therefore shows how to extend the master model from the previous section as to limit the number of parallel submodel executions. The model instances to be run are implemented as a job queue, a list of instance indices that is accessed in first-in-first-out order. There are now two sets of indices, the job indices (set A) for the instances we wish to process and the model indices (set RM) corresponding to the concurrently executing Mosel models. We save the model index information as user model ID with each model and use the mapping jobid that relates the job indices to the model indices.

As before, we compile the submodel and load the required number of model instances into Mosel. We then take the first few entries from the job list and start the corresponding model runs. As soon as a model run terminates, a new instance is started through the procedure start_next_job (see below).

The model takes a parameter NUMPAR that lets you change the limit on the number of submodels at run time.

model "Run model rtparams with job queue"
 uses "mmjobs"

 parameters
   NUMPAR=2                         ! Number of parallel model executions
 end-parameters                     ! (preferrably <= no. of processors)

 forward procedure start_next_job(submod:Model)

 declarations
   RM = 1..NUMPAR                   ! Model indices
   A = 1..10                        ! Job (instance) indices
   modPar: array(RM) of Model       ! Models
   jobid: array(set of integer) of integer  ! Job index for model IDs
   JobList: list of integer         ! List of jobs
   JobsRun: set of integer          ! Set of finished jobs
   JobSize: integer                 ! Number of jobs to be executed
   Msg: Event                       ! Messages sent by models
 end-declarations
                                    ! Compile the model file
 if compile("rtparams.mos")<>0 then exit(1); end-if

 forall(m in RM) do
   load(modPar(m), "rtparams.bim")  ! Load the bim file
   modPar(m).uid:= m                ! Store the model ID as UID
 end-do

 JobList:= sum(i in A) [i]          ! Define the list of jobs (instances)
 JobSize:=JobList.size              ! Store the number of jobs
 JobsRun:={}                        ! Set of terminated jobs is empty

!**** Start initial lot of model runs ****
 forall(m in RM)
   if JobList<>[] then
     start_next_job(modPar(m))
   end-if

!**** Run all remaining jobs ****
 while (JobsRun.size<JobSize) do
   wait                             ! Wait for model termination
   Msg:= getnextevent
   if getclass(Msg)=EVENT_END then  ! We are only interested in "end" events
     m:=getfromuid(Msg)             ! Retrieve the model UID
     JobsRun+={jobid(m)}            ! Keep track of job termination
     writeln("End of job ", jobid(m), " (model ", m, ")")
     if JobList<>[] then            ! Start a new run if queue not empty
      start_next_job(modPar(m))
     end-if
   end-if
 end-do

end-model 

In the while loop above we identify the model that has sent the termination message and add the corresponding job identifier to the set JobsRun of completed instances. If there are any remaining jobs in the list, the procedure start_next_job is called that takes the next job and starts it with the model that has just been released.

 procedure start_next_job(submod: Model)
   i:=getfirst(JobList)             ! Retrieve first job in the list
   cuthead(JobList,1)               ! Remove first entry from job list
   jobid(submod.uid):= i
   writeln("Start job ", i, " (model ", submod.uid, ")")
   run(submod, "PARAM1=" + i + ",PARAM2=" + 0.1*i +
               ",PARAM3='string " + i + "'" + ",PARAM4=" + isodd(i))
 end-procedure 

The job queue in our example is static, in the sense that the list of jobs to be processed is fixed right at the beginning. Using standard Mosel list handling functionality, it can quite easily be turned into a dynamic queue to which new jobs are added while others are already being processed.