Initializing help system before first use

Working with multiple models: remote connection, coordination, communication, and parallelization


Type: Programming
Rating: 3 (intermediate)
Description: The Mosel module mmjobs enables the user to work with several models concurrently. We show here a series of examples of basic tasks that typically need to be performed when working with remote models in Mosel:
  • Check for available remote Mosel servers: findservers.mos
  • Run a single model on a remote machine: runrtdistr.mos (master model executing rtparams.mos)
  • Run a single model on a remote machine with configuration options: runrtdistrconf.mos (master model executing rtparams.mos)
  • Running parallel submodels in a distributed architecture: runrtpardistr.mos (master model executing rtparams3.mos)
  • Queuing submodels for parallel execution in a distributed architecture with one or several models per node: runrtparqueued.mos (master model executing rtparams3.mos)
  • 3-level tree of (parallel) submodels: runrtpartree.mos (master model executing rtparams2.mos)
  • Running a submodel that detaches itself from the master: runrtdetach.mos (master model executing rtparams4.mos)
  • Using the shmem (shared memory) I/O driver for data exchange (bin format): runsubshmdistr.mos (master model executing testsubshm.mos)
File(s): findservers.mos, runrtdistr.mos, rtparams.mos, runrtdistrconf.mos, runrtpardistr.mos, rtparams3.mos, runrtparqueued.mos, runrtpartree.mos, rtparams2.mos, runrtdetach.mos, rtparams4.mos (submodel), runsubshmdistr.mos, testsubshm.mos (submodel)


findservers.mos
(!*******************************************************
   Mosel Example Problems 
   ======================

   file findservers.mos
   ````````````````````
   Find Mosel servers that are able to accept remote model runs

   This file only produces output on the local node, 
   it does not start any remote runs.
       
   (c) 2013 Fair Isaac Corporation
       author: S. Heipcke, Jan. 2013
*******************************************************!)

model "Find Mosel servers"
 uses "mmjobs"

 parameters
   M = 20                  ! Max. number of servers to be sought
 end-parameters  

 declarations
   Hosts: set of string
   NODELIST: list of string
   mosInst: Mosel
 end-declarations

 findxsrvs(1, M, Hosts)
 writeln(Hosts.size, " servers found: ", Hosts)

 forall(i in Hosts)        ! Establish remote connection and print system info
   if connect(mosInst, i)=0 then
     writeln("Server ", i, ": ", getsysinfo(mosInst))
     disconnect(mosInst)
   else
     writeln("Connection to ", i, " failed ")
   end-if    

 NODELIST:=[""]            ! Construct a list starting with the current node
 forall(i in Hosts) NODELIST += [i] 

end-model

runrtdistr.mos
(!*******************************************************
   Mosel Example Problems 
   ======================

   file runrtdistr.mos
   ```````````````````
   Running a model on a remote machine,
   passing runtime parameters to the submodel.

   Before running this model, you need to set up the 
   NODENAME with a machine name/address of your local network.
   The node that is used needs to have the same version of
   Xpress installed and suitably licensed, and the server 
   "xprmsrv" must have been started on this machine.
       
   (c) 2010 Fair Isaac Corporation
       author: S. Heipcke, May 2010
*******************************************************!)

model "Run model rtparams remotely"
 uses "mmjobs"

 declarations
  modPar: Model
  mosInst: Mosel
 end-declarations
                                   ! Compile the model file
 if compile("rtparams.mos")<>0 then exit(1); end-if

       !!! Use the name or IP address of a machine in
       !!! your local network, or "" for current node
 NODENAME:= ""
                                   ! Open connection to a remote node:
 if connect(mosInst, NODENAME)<>0 then exit(2); end-if
                                   ! Load the bim file into the remote instance
 load(mosInst, modPar, "rmt:rtparams.bim") 
                                   ! Start model execution
 run(modPar, "PARAM1=" + 2 + ",PARAM2=" + 3.4 + 
             ",PARAM3='a string'" + ",PARAM4=" + true)
 wait                              ! Wait for model termination
 dropnextevent                     ! Ignore termination event message

end-model 

rtparams.mos
(!*******************************************************
   Mosel Example Problems 
   ======================

   file rtparams.mos
   `````````````````
   Model with different runtime parameter types;
   can be run as submodel from models 'runrtpar*.mos'.
       
   (c) 2008 Fair Isaac Corporation
       author: S. Heipcke, May 2006
*******************************************************!)

model "Runtime parameters"
 uses "mmjobs"
 parameters
  PARAM1 = 0
  PARAM2 = 0.5
  PARAM3 = ''
  PARAM4 = false
 end-parameters

 wait(round(5*random))
 writeln(PARAM1, "  ", PARAM2, "  ", PARAM3, "  ", PARAM4)
end-model 

runrtdistrconf.mos
(!*******************************************************
   Mosel Example Problems 
   ======================

   file runrtdistrconf.mos
   ```````````````````````
   Running a model on a remote machine,
   setting configuration options for the remote instance.

   Before running this model, you need to set up the 
   NODENAME with a machine name/address of your local network.
   The node that is used needs to have the same version of
   Xpress installed and suitably licensed, and the server 
   "xprmsrv" must have been started on this machine.
       
   (c) 2010 Fair Isaac Corporation
       author: S. Heipcke, May 2010, rev. Feb. 2014
*******************************************************!)

model "Run model rtparams remotely"
 uses "mmjobs"

 declarations
  modPar: Model
  mosInst: Mosel
 end-declarations
                                   ! Compile the model file
 if compile("rtparams.mos")<>0 then exit(1); end-if


!!! Connection name string:
!!! ======================
!!! Use the name or IP address of a machine in your local network,
!!! or "" for current node. 
!!! The name may also include configuration options (e.g., new values for
!!! environment variables, or arguments for the Mosel command line), or, 
!!! with the 'rcmd' driver specify a command to be executed.
!!!
!!! Here are some examples:
! NODENAME:= ""             ! A new local instance with unchanged settings
! NODENAME:= "localhost"    ! A new Mosel instance on local host 
                            ! (similar to previous, but uses xsrv protocol)

! NODENAME:= "xsrv:localhost|MOSEL_CWD=c:\\mydir\\"   
                            ! Set working directory for local instance (Win)
! NODENAME:= "xsrv:localhost|MOSEL_CWD=/tmp/mydir"   
                            ! Set working directory for local instance (Unix)

! NODENAME:= "ABCD123"      ! Remote instance on machine "ABCD123"
! NODENAME:= "123.456.789"  ! Remote instance at address 123.456.789
! NODENAME:= "ABCD123|MOSEL_CWD=/tmp"
                            ! Set working directory for remote instance (Unix)

! NODENAME:= "xsrv:localhost|MOSEL_ARGS=-rv 10 -rl '/tmp/mylog.txt'"
                            ! Set Mosel command line arguments. 
                            ! Full list is obtained with:  mosel -h 
                            ! -rv 10      very verbose remote communication
                            ! -rl aFile   define logging file for rmt instance

! NODENAME:= "rcmd:mosel -r -rv 10 -rl '/tmp/mylog.txt'"
                            ! Similar to the above, more direct communication
                            ! as this does not use the network server
! NODENAME:= "rcmd:rsh ABCD123 mosel -r"
                            ! Start remote instance on "ABCD123" connecting
                            ! via rsh 
                            ! (Note: option -r is required for remote runs)

! NODENAME:= "xsrv:localhost|XPRESSDIR=c:\\xpress7\\"
                            ! Use some other Xpress version (must support
                            ! remote features!), e.g. 32bit remotely with 
                            ! this model running under 64bit  

!!! Aliases:
!!! =======
!!! You can also associate other connections than default with node names.
!!! For example:
! sethostalias("localhost","rcmd:mosel -r")
! sethostalias("ABCD123","rcmd:rsh ABCD123 mosel -r")


 NODENAME:= "" 
                                   ! Open connection to a remote node
 if connect(mosInst, NODENAME)<>0 then exit(2); end-if
                                   ! Load the bim file into the remote instance
 load(mosInst, modPar, "rmt:rtparams.bim") 
                                   ! Start model execution
 run(modPar, "PARAM1=" + 2 + ",PARAM2=" + 3.4 + 
             ",PARAM3='a string'" + ",PARAM4=" + true)
 wait                              ! Wait for model termination
 dropnextevent                     ! Ignore termination event message

end-model 

runrtpardistr.mos
(!*******************************************************
   Mosel Example Problems 
   ======================

   file runrtpardistr.mos
   ``````````````````````
   Running several instances of a model from another
   Mosel model.
   - Parallel submodels in distributed architecture -

   Before running this model, you need to set up the array
   NODENAMES with machine names/addresses of your local network.
   All nodes that are used need to have the same version of
   Xpress installed and suitably licensed, and the server 
   "xprmsrv" must have been started on these machines.
   
   All files are local to the root node, no write access is
   required at remote nodes.
       
   (c) 2010 Fair Isaac Corporation
       author: S. Heipcke, May 2010, rev. Jan. 2013
*******************************************************!)

model "Run model rtparams in distributed architecture"
 uses "mmjobs", "mmsystem"

 declarations
  NUMI = 5
  A = 1..2*NUMI
  B = 1..NUMI
  modPar: array(A) of Model
  moselInst: array(B) of Mosel
  NODENAMES: array(B) of string
 end-declarations

                                 !!! Select the (remote) machines to be used:
				 !!! Use names, IP addresses, or empty string
				 !!! for the node running this model
 forall(i in B) NODENAMES(i):= if(isodd(i), "localhost", "xssh:127.0.0.1")
! NODENAMES:: (1..5)["localhost","","","rcmd:mosel -r","rcmd:mosel -r"] 

 writeln("Node: ", getparam("NODENUMBER"))
 
                                   ! Compile the model file locally on root
 if compile("rtparams3.mos")<>0 then exit(1); end-if

 instct:=0
 forall(i in A) do            
  if isodd(i) then
   instct+=1                       ! Connect to a remote machine
   if connect(moselInst(instct), NODENAMES(instct))<>0 then exit(2); end-if
  end-if
  
  writeln("Current node: ", getsysinfo(SYS_NODE), 
          " submodel node: ", getsysinfo(moselInst(instct), SYS_NODE))
  
                                   ! Load the bim file (located at root node)
  load(moselInst(instct), modPar(i), "rmt:rtparams3.bim")  
                                   ! Start remote model execution
  run(modPar(i), "PARAM1=" + i + ",PARAM2=" + 0.1*i +
                 ",PARAM3='string " + i + "'" + ",PARAM4=" + isodd(i))
 end-do
 
 forall(i in A) do
  wait                             ! Wait for model termination
  dropnextevent                    ! Ignore termination event message
 end-do

 fdelete("rtparams3.bim")          ! Cleaning up
 
end-model 

rtparams3.mos
(!*******************************************************
   Mosel Example Problems 
   ======================

   file rtparams3.mos
   ``````````````````
   Model with different runtime parameter types;
   can be run as submodel from models 'runrtpar*.mos'.
       
   (c) 2010 Fair Isaac Corporation
       author: S. Heipcke, May 2010
*******************************************************!)

model "Runtime parameters"
 uses "mmjobs", "mmsystem"
 parameters
  PARAM1 = 0
  PARAM2 = 0.5
  PARAM3 = ''
  PARAM4 = false
 end-parameters

 wait(round(5*random))
 writeln("Node: ", getparam("NODENUMBER"), " ", getsysinfo(SYS_NODE), 
         ". Parent: ", getparam("PARENTNUMBER"),
	 ". Model number: ", getparam("JOBID"), 
         ". Parameters: ", 
         PARAM1, "  ", PARAM2, "  ", PARAM3, "  ", PARAM4)
end-model 

runrtparqueued.mos
(!*******************************************************
   Mosel Example Problems 
   ======================

   file runrtparqueued.mos
   ```````````````````````
   Running several instances of a model from another
   Mosel model.
   - Queuing submodels for parallel execution in a
     distributed architecture (one or several models per node) -

   Before running this model, you need to set up the list
   NodeList with machine names/addresses of your local network.
   All nodes that are used need to have the same version of
   Xpress installed and suitably licensed, and the server 
   "xprmsrv" must have been started on these machines.

   The maximum number of models per node in array MaxMod needs
   to be adapted to the number of executions licensed on 
   the corresponding nodes.
   
   All files are local to the root node, no write access is
   required at remote nodes.
       
   (c) 2010 Fair Isaac Corporation
       author: S. Heipcke, Apr. 2010, rev. Dec. 2017
*******************************************************!)

model "Run model rtparams with job queue"
 uses "mmjobs", "mmsystem"

 parameters
   J=10                             ! Number of jobs to run
   NUMPAR=2                         ! Number of parallel model executions
 end-parameters                     ! (preferrably <= no. of processors)
   
 forward procedure start_next_job(submod: Model)
 
 declarations
   RM: range                        ! Model indices
   JOBS = 1..J                      ! Job (instance) indices
   modPar: dynamic array(RM) of Model       ! Models
   jobid: array(set of integer) of integer  ! Job index for model UIDs
   JobList: list of integer         ! List of jobs
   JobsRun: set of integer          ! Set of finished jobs
   JobSize: integer                 ! Number of jobs to be executed
   Msg: Event                       ! Messages sent by models
   NodeList: list of string         !
   nodeInst: dynamic array(set of string) of Mosel  ! Mosel instances on remote nodes
   nct: integer
   modNode: array(set of integer) of string ! Node used for a model
   MaxMod: array(set of string) of integer
 end-declarations
                                    ! Compile the model file locally
 if compile("rtparams.mos")<>0 then exit(1); end-if

!**** Setting up remote Mosel instances ****
 sethostalias("localhost2","localhost")
 NodeList:= ["localhost", "localhost2"]
         !!! This list must have at least 1 element.
         !!! Use machine names within your local network, IP addresses, or 
         !!! empty string for the current node running this model.

 forall(n in NodeList) MaxMod(n):= NUMPAR
         !!! Adapt this setting to number of processors and licences per node

 forall(n in NodeList, nct as counter) do
  create(nodeInst(n))
  if connect(nodeInst(n), n)<>0 then exit(1); end-if
  if nct>= J then break; end-if     ! Stop if started enough instances
 end-do 

!**** Loading model instances ****
 nct:=0
 forall(n in NodeList, m in 1..MaxMod(n), nct as counter) do
   create(modPar(nct))
   load(nodeInst(n), modPar(nct), "rmt:rtparams.bim")  ! Load the bim file
   modPar(nct).uid:= nct            ! Store the model ID as UID
   modNode(modPar(nct).uid):= getsysinfo(nodeInst(n), SYS_NODE)
 end-do 


 JobList:= sum(i in JOBS) [i]       ! Define the list of jobs (instances)
 JobSize:=JobList.size              ! Store the number of jobs
 JobsRun:={}                        ! Set of terminated jobs is empty

!**** Start initial lot of model runs ****
 forall(m in RM) 
   if JobList<>[] then
     start_next_job(modPar(m))
   end-if

!**** Run all remaining jobs ****
 while (JobsRun.size<JobSize) do
   wait                             ! Wait for model termination
  ! Start next job
   Msg:= getnextevent
   if Msg.class=EVENT_END then      ! We are only interested in "end" events
     m:= Msg.fromuid                ! Retrieve the model UID
     JobsRun+={jobid(m)}            ! Keep track of job termination
     writeln("End of job ", jobid(m), " (model ", m, ")")
     if JobList<>[] then            ! Start a new run if queue not empty
      start_next_job(modPar(m))
     end-if
   end-if
 end-do 

 fdelete("rtparams.bim")            ! Cleaning up
 
!*************************************************************************
 procedure start_next_job(submod: Model)
   i:=getfirst(JobList)             ! Retrieve first job in the list
   cuthead(JobList,1)               ! Remove first entry from job list
   jobid(submod.uid):= i   
   writeln("Start job ", i, " (model ", submod.uid, ") on ",
           modNode(submod.uid))
   run(submod, "PARAM1=" + i + ",PARAM2=" + 0.1*i +
               ",PARAM3='string " + i + "'" + ",PARAM4=" + isodd(i))
 end-procedure

end-model 

runrtpartree.mos
(!*******************************************************
   Mosel Example Problems 
   ======================

   file runrtpartree.mos
   `````````````````````
   Running several instances of a model from another
   Mosel model.
   - Parallel submodels over 3 levels (submodel tree)
     possibly running in a distributed architecture -

   Before running this model, you need to set up the array
   NODENAMES with machine names/addresses of your local network.
   All nodes that are used need to have the same version of
   Xpress installed and suitably licensed, and the server 
   "xprmsrv" must have been started on these machines.
       
   (c) 2010 Fair Isaac Corporation
       author: S. Heipcke, May 2010
*******************************************************!)

model "Recursive model runs"
 uses "mmjobs", "mmsystem"

 parameters
  RMT="rmt:"
  SUBMODNAME="runrtpartree"
  LEVEL = 0
  BRANCH = 0
  MODCOUNT = 1
  PARAM1 = 0
  PARAM2 = 0.5
  PARAM3 = ''
  PARAM4 = false
 end-parameters 

 declarations
  A = 1..5
  modPar: array(A) of Model
  M:Mosel
  NODENAMES: array(range) of string
 end-declarations


!!! Select the (remote) machines to be used:
!!! Use names, IP addresses, or empty string for the node running this model
 NODENAMES:: (1..3)["", "", ""]     


 writeln(" "*LEVEL, LEVEL, "/", BRANCH, ": ", SUBMODNAME, "(", MODCOUNT, "), ",
         "Node: ", getparam("NODENUMBER"))

 
 NEWSUBNAME:=if(LEVEL>1, "rtparams2", SUBMODNAME)
 
                                   ! Compile the model file in first run
 if LEVEL=0 then
  if compile("runrtpartree.mos")<>0 then exit(1); end-if
  if compile("rtparams2.mos")<>0 then exit(1); end-if
 end-if
 
 r:=connect(M,NODENAMES(LEVEL+1))
 
 forall(i in A) do
  load(M,modPar(i), RMT+NEWSUBNAME+".bim")  ! Load the bim file
                                   ! Start model execution
  run(modPar(i), "SUBMODNAME=" + NEWSUBNAME + ",MODCOUNT=" + (10*MODCOUNT+i) + 
                 ",LEVEL="+ (LEVEL+1) + ",BRANCH="+ i +
                 ",PARAM1=" + i + ",PARAM2=" + 0.1*i +
                 ",PARAM3='string " + i + "'" + ",PARAM4=" + isodd(i))
!  wait(1)
 end-do
 
 forall(i in A) do
  wait                             ! Wait for model termination
  dropnextevent                    ! Ignore termination event message
 end-do

end-model 

rtparams2.mos
(!*******************************************************
   Mosel Example Problems 
   ======================

   file rtparams2.mos
   ``````````````````
   Model with different runtime parameter types;
   can be run as submodel from model 'runrtpartree.mos'.
       
   (c) 2010 Fair Isaac Corporation
       author: S. Heipcke, May 2010
*******************************************************!)

model "Runtime parameters"
 uses "mmjobs", "mmsystem"
 
 parameters
  LEVEL = 0
  BRANCH = 0
  MODCOUNT = 1
  PARAM1 = 0
  PARAM2 = 0.5
  PARAM3 = ''
  PARAM4 = false
 end-parameters

! wait(1)
 writeln(" "*LEVEL, LEVEL, "/", BRANCH, " (", MODCOUNT, ")", ": ", PARAM1, "  ", PARAM2, "  ", PARAM3, "  ", PARAM4)
 
end-model 

runrtdetach.mos
(!*******************************************************
   Mosel Example Problems 
   ======================

   file runrtdetach.mos
   ````````````````````
   Running a model on a remote machine,
   retrieve information about the submodels detaching
   itself from the master.

   Before running this model, you need to set up the 
   NODENAME with a machine name/address of your local network.
   The node that is used needs to have the same version of
   Xpress installed and suitably licensed, and the server 
   "xprmsrv" must have been started on this machine.
       
   (c) 2014 Fair Isaac Corporation
       author: S. Heipcke, Feb. 2014
*******************************************************!)

model "Run model rtparams4 remotely"
 uses "mmjobs", "mmsystem"

 declarations
  modPar: Model
  mosInst: Mosel
  event: Event
 end-declarations
                                   ! Compile the model file
 if compile("rtparams4.mos")<>0 then exit(1); end-if

       !!! Use the name or IP address of a machine in
       !!! your local network, or "" for current node
 NODENAME:= ""
                                   ! Open connection to a remote node:
 if connect(mosInst, NODENAME)<>0 then exit(2); end-if

                                   ! Redirect streams (elsewhere than master)
 setdefstream(mosInst, "null:", string(expandpath("rt4log.txt")), "sysfd:2")

                                   ! Load the bim file into the remote instance
 load(mosInst, modPar, "rmt:rtparams4.bim") 
 fdelete("rtparams4.bim")
                                   ! Start model execution
 run(modPar, "PARAM1=" + 2 + ",PARAM2=" + 3.4 + 
             ",PARAM3='a string'" + ",PARAM4=" + true)
 wait                              ! Wait for model termination

 event:=getnextevent               ! An event is available
 if getclass(event) = EVENT_END then
   writeln("Termination event received.")
 else
   writeln("Event received: ", event) 
 end-if
 
 if getstatus(modPar) = RT_DETACHED then
   writeln("Submodel 'rtparams4' detached.")
 end-if

end-model 

rtparams4.mos
(!*******************************************************
   Mosel Example Problems 
   ======================

   file rtparams4.mos
   ``````````````````
   Model with different runtime parameter types
   - Submodel detaches itself from its caller -

   *** Not intended to be run standalone - run from runrtdetach.mos ***
       
   (c) 2014 Fair Isaac Corporation
       author: S. Heipcke, Feb. 2014
*******************************************************!)

model "Runtime parameters"
 uses "mmjobs", "mmsystem"
 
 parameters
  PARAM1 = 0
  PARAM2 = 0.5
  PARAM3 = ''
  PARAM4 = false
 end-parameters

 starttime:=gettime

 detach                         ! Detach model from caller

 writeln("(", gettime-starttime, "sec) Detached from master.")

 wait(2+round(5*random))
 writeln(PARAM1, "  ", PARAM2, "  ", PARAM3, "  ", PARAM4)
 
 writeln("(", gettime-starttime, "sec) Terminated")

end-model 

runsubshmdistr.mos
(!*******************************************************
   Mosel Example Problems 
   ======================

   file runsubshmdistr.mos
   ```````````````````````
   Running a model on a remote machine.
   Communication of data from/to master model
   using 'shmem' IO driver with 'bin' and 'rmt'.
       
   (c) 2017 Fair Isaac Corporation
       author: S. Heipcke, June 2017
*******************************************************!)

model "Run model testsubshm remotely"
 uses "mmjobs"

 declarations
  mosInst: Mosel
  modSub: Model
  A = 30..40 
  B: array(A) of real
  nd: integer
 end-declarations
                                   ! Compile the model file
 if compile("testsubshm.mos")<>0 then exit(1); end-if

       !!! Use the name or IP address of a machine in
       !!! your local network, or "" for current node
 NODENAME:= ""
                                   ! Open connection to a remote node:
 if connect(mosInst, NODENAME)<>0 then exit(2); end-if
 nd:=mosInst.node
                                   ! Load the bim file into the remote instance
 load(mosInst, modSub, "rmt:[-1]testsubshm.bim")

                                   ! Write data onto the instance of the submod.
 initializations to "bin:rmt:["+nd+"]shmem:indata"
  A
 end-initializations

 run(modSub)                       ! Start model execution
 wait                              ! Wait for model termination
 dropnextevent                     ! Ignore termination event message

 initializations from "bin:rmt:["+nd+"]shmem:resdata"
  B
 end-initializations

 writeln(B)
 
end-model 

testsubshm.mos
(!*******************************************************
   Mosel Example Problems 
   ======================

   file testsubshm.mos
   ```````````````````
   Simple model printing some output.
   Communication of data from/to master model
   using 'shmem' IO driver with 'bin'.

   *** Not intended to be run standalone - run from runsubshmdistr.mos ***
       
   (c) 2011 Fair Isaac Corporation
       author: S. Heipcke, April 2011
*******************************************************!)

model "Test submodel"
 declarations
  A: range 
  B: array(A) of real
 end-declarations

 initializations from "bin:shmem:indata"
  A
 end-initializations
 
 forall(i in A) B(i):= i^2

 initializations to "bin:shmem:resdata"
  B
 end-initializations

end-model 

© 2001-2019 Fair Isaac Corporation. All rights reserved. This documentation is the property of Fair Isaac Corporation (“FICO”). Receipt or possession of this documentation does not convey rights to disclose, reproduce, make derivative works, use, or allow others to use it except solely for internal evaluation purposes to determine whether to purchase a license to the software described in this documentation, or as otherwise set forth in a written software license agreement between you and FICO (or a FICO affiliate). Use of this documentation and the software described in it must conform strictly to the foregoing permitted uses, and no other use is permitted.