Exchanging data between an application and a model

In the previous sections we have seen how to obtain solution information and other data from a Mosel model after its execution. For the integration of a model into an application a flow of information in the opposite sense, that is, from the host application to the model, will often also be required, in particular if data are generated by the application that serve as input to the model. It is possible to write out this data to a (text) file or a database and read this file in from the model, but it is clearly more efficient to communicate such data in memory directly from the application to the model.

In this section we show two versions of our Burglar example where all input data is loaded from the application into the model, using dense and sparse data format respectively. The same communication mechanism, namely a combination of the two I/O drivers (see Section Generalized file handling for further detail) raw and mem, is also used to write back the solution from the model to the calling application.

An alternative communication mechanism is presented in Section Dynamic data. Instead of working with blocks of predefined size as in the previous cases, here data is passed through flows, allowing for dynamic sizing on the application level, a feature that is particularly useful for solution output with sparse data structures.

A separate example (Section Scalars) shows how to input and output scalar data.

Dense arrays

In the first instance we are going to consider a version of the `Burglar' model that corresponds to the very first version we have seen in Section The burglar problem where all arrays are indexed by the range set ITEMS = 1..8. In our C program ugiodense.c below, this corresponds to storing data in standard C arrays that are communicated to the Mosel model at the start of its execution.

#include <stdio.h>
#include "xprm_mc.h"

double vdata[8]={15,100,90,60,40,15,10, 1};   /* Input data: VALUE */
double wdata[8]={ 2, 20,20,30,40,30,60,10};   /* Input data: WEIGHT */
double solution[8];               /* Array for solution values */

int main()
{
 XPRMmodel mod;
 int i,result;
 char vdata_name[40];             /* File name of input data 'vdata' */
 char wdata_name[40];             /* File name of input data 'wdata' */
 char solution_name[40];          /* File name of solution values */
 char params[144];                /* Parameter string for model execution */

 if(XPRMinit())                   /* Initialize Mosel */
  return 1;

/* Prepare file names for 'initializations' using the 'raw' driver */
 sprintf(vdata_name, "noindex,mem:%p/%d", vdata, (int)sizeof(vdata));
 sprintf(wdata_name, "noindex,mem:%p/%d", wdata, (int)sizeof(wdata));
 sprintf(solution_name, "noindex,mem:%p/%d", solution, (int)sizeof(solution));

                                  /* Pass file names as execution param.s */
 sprintf(params, "VDATA='%s',WDATA='%s',SOL='%s'", vdata_name, wdata_name,
         solution_name);

 if(XPRMexecmod(NULL, "burglar6.mos", params, &result, &mod))
  return 2;                       /* Execute a model file */

 if((XPRMgetprobstat(mod)&XPRM_PBRES)!=XPRM_PBOPT)
  return 3;                       /* Test whether a solution is found */

/* Display solution values obtained from the model */
 printf("Objective value: %g\n", XPRMgetobjval(mod));
 for(i=0;i<8;i++)
  printf(" take(%d): %g\n", i+1, solution[i]);

 XPRMresetmod(mod);               /* Reset the model */

 return 0;
} 

In this example we use the raw I/O driver for communication between the application and the model it executes. Employing this driver means that data is saved in binary format. File names used with the raw driver have the form rawoption[,...],filename. The option noindex for this driver indicates that data is to be stored in dense format, that is, just the data entries without any information about the indices—this format supposes that the index set(s) is known in the Mosel model before data is read in. The filename uses the mem driver, this means that data is stored in memory. The actual location of the data is specified by giving the address of the corresponding memory block and its size.

The program above works with the following version of the `Burglar' model where the locations of input and output data are specified by the calling application through model parameters. Instead of printing out the solution in the model, we copy the solution values of the decision variables take into the array of reals soltake that is written to memory and will be processed by the host application.

model Burglar6
 uses "mmxprs"

 parameters
  VDATA = ''; WDATA = ''         ! Locations of input data
  SOL = ''                       ! Location for solution data output
  WTMAX = 102                    ! Maximum weight allowed
 end-parameters

 declarations
  ITEMS = 1..8                   ! Index range for items

  VALUE: array(ITEMS) of real    ! Value of items
  WEIGHT: array(ITEMS) of real   ! Weight of items

  take: array(ITEMS) of mpvar    ! 1 if we take item i; 0 otherwise
  soltake: array(ITEMS) of real  ! Solution values
 end-declarations

 initializations from 'raw:'
  VALUE as VDATA  WEIGHT as WDATA
 end-initializations

! Objective: maximize total value
 MaxVal:= sum(i in ITEMS) VALUE(i)*take(i)

! Weight restriction
 sum(i in ITEMS) WEIGHT(i)*take(i) <= WTMAX

! All variables are 0/1
 forall(i in ITEMS) take(i) is_binary

 maximize(MaxVal)                 ! Solve the MIP-problem

! Output solution to calling application
 forall(i in ITEMS) soltake(i):= getsol(take(i))

 initializations to 'raw:'
  soltake as SOL
 end-initializations

end-model 

Sparse arrays

Let us now take a look at the case where we use a set of strings instead of a simple range set to index the various arrays in our model. Storing the indices with the data values makes necessary slightly more complicated structures in our C program for the input and solution data. In the C program below (file ugiosparse.c), every input data entry defines both, the value and the weight coefficient for the corresponding index.

#include <stdio.h>
#include "xprm_mc.h"

const struct
{                                 /* Initial values for array 'data': */
 const char *ind;                 /*   index name */
 double val,wght;                 /*   value and weight data entries */
} data[]={{"camera",15,2}, {"necklace",100,20}, {"vase",90,20},
          {"picture",60,30}, {"tv",40,40}, {"video",15,30},
          {"chest",10,60}, {"brick",1,10}};

const struct
{                                 /* Array to receive solution values: */
 const char *ind;                 /*   index name */
 double val;                      /*   solution value */
} solution[8];

int main()
{
 XPRMmodel mod;
 int i,result;
 char data_name[40];              /* File name of input data 'data' */
 char solution_name[40];          /* File name of solution values */
 char params[96];                 /* Parameter string for model execution */

 if(XPRMinit())                   /* Initialize Mosel */
  return 1;

/* Prepare file names for 'initializations' using the 'raw' driver */
 sprintf(data_name, "slength=0,mem:%p/%d", data, (int)sizeof(data));
 sprintf(solution_name, "slength=0,mem:%p/%d", solution, (int)sizeof(solution));

                                  /* Pass file names as execution param.s */
 sprintf(params, "DATA='%s',SOL='%s'", data_name, solution_name);

 if(XPRMexecmod(NULL, "burglar7.mos", params, &result, &mod))
  return 2;                       /* Execute a model file */

 if((XPRMgetprobstat(mod)&XPRM_PBRES)!=XPRM_PBOPT)
  return 3;                       /* Test whether a solution is found */

/* Display solution values obtained from the model */
 printf("Objective value: %g\n", XPRMgetobjval(mod));
 for(i=0;i<8;i++)
  printf(" take(%s): %g\n", solution[i].ind, solution[i].val);

 XPRMresetmod(mod);

 return 0;
} 

The use of the two I/O drivers is quite similar to what we have seen before. We now pass on data in sparse format, this means that every data entry is saved together with its index (tuple). Option slength=0 of the raw driver indicates that strings are represented by pointers to null terminated arrays of characters (C-string) instead of fixed size arrays.

Similarly to the model of the previous section, the model burglar7.mos executed by the C program above reads and writes data from/to memory using the raw driver and the locations are specified by the calling application through the model parameters. Since the contents of the index set ITEMS is not defined in the model we have moved the declaration of the decision variables after the data input where the contents of the set is known, thus avoiding the creation of the array of decision variables as a dynamic array.

model Burglar7
 uses "mmxprs"

 parameters
  DATA = ''                        ! Location of input data
  SOL = ''                         ! Location for solution data output
  WTMAX = 102                      ! Maximum weight allowed
 end-parameters

 declarations
  ITEMS: set of string             ! Index set for items
  VALUE: array(ITEMS) of real      ! Value of items
  WEIGHT: array(ITEMS) of real     ! Weight of items
 end-declarations

 initializations from 'raw:'
  [VALUE,WEIGHT] as DATA
 end-initializations

 declarations
  take: array(ITEMS) of mpvar      ! 1 if we take item i; 0 otherwise
 end-declarations

! Objective: maximize total value
 MaxVal:= sum(i in ITEMS) VALUE(i)*take(i)

! Weight restriction
 sum(i in ITEMS) WEIGHT(i)*take(i) <= WTMAX

! All variables are 0/1
 forall(i in ITEMS) take(i) is_binary

 maximize(MaxVal)                  ! Solve the MIP-problem

! Output solution to calling application
 forall(i in ITEMS) soltake(i):= getsol(take(i))

 initializations to 'raw:'
  soltake as SOL
 end-initializations

end-model 

Dynamic data

The two examples of in-memory communication of dense and sparse data in the preceding sections have in commun that all data structures in the application, and in particular the structures to receive output data, are of fixed size. We therefore now introduce an alternative communication mechanism working with flows, that enables dynamic sizing of data structures on the application level, a feature that is particularly useful for solution output where effective data sizes are not known a priori. This communication mechanism is based on the callback I/O driver cb (see also Section Redirecting the Mosel output). The main body of our C program now looks as follows.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "xprm_mc.h"

                                  /* Input values for data: */
char *ind[]={"camera", "necklace", "vase", "picture", "tv", "video",
             "chest", "brick"};              /* Index names */
double vdata[]={15,100,90,60,40,15,10, 1};   /* Input data: VALUE */
double wdata[]={ 2, 20,20,30,40,30,60,10};   /* Input data: WEIGHT */
int datasize=8;

struct SolArray
{                                 /* Array to receive solution values: */
 const char *ind;                 /*   index name */
 double val;                      /*   solution value */
};

struct SolArray *solution;
int solsize;

int main()
{
 XPRMmodel mod;
 int i,result;
 char data_name[40];              /* File name of input data 'data' */
 char solution_name[40];          /* File name of solution values */
 char params[96];                 /* Parameter string for model execution */

 if(XPRMinit())                   /* Initialize Mosel */
  return 1;

/* Prepare file names for 'initializations' using the 'cb' driver */
 sprintf(data_name, "cb:%p", cbinit_from);
 sprintf(solution_name, "cb:%p", cbinit_to);

                                  /* Pass file names as execution param.s */
 sprintf(params, "DATAFILE='%s',SOLFILE='%s'", data_name, solution_name);

 if(XPRMexecmod(NULL, "burglar13.mos", params, &result, &mod))
  return 2;                       /* Execute a model file */

 if((XPRMgetprobstat(mod)&XPRM_PBRES)!=XPRM_PBOPT)
  return 3;                       /* Test whether a solution is found */

/* Display solution values obtained from the model */
 printf("Objective value: %g\n", XPRMgetobjval(mod));
 for(i=0;i<solsize;i++)
  printf(" take(%s): %g\n", solution[i].ind, solution[i].val);

 XPRMresetmod(mod);

 return 0;
}

The function for dynamic output retrieval employs the Mosel library functions that we have already seen in Section Accessing modeling objects and solution values for models after their termination. The prototype of the function cbinit_to needs to be exactly as shown below.

int XPRM_RTC cbinit_to(XPRMcbinit cb, void *info, const char *label,
                       int type, XPRMalltypes *ref)
{
 XPRMarray solarr;
 XPRMset sets[1];
 int indices[1];
 XPRMalltypes rvalue;
 int ct;

 if(strcmp(label,"SOL")==0)
 {
  solarr=ref->array;

  solsize=XPRMgetarrsize(solarr);
  solution = (struct SolArray *)malloc(solsize * sizeof(struct SolArray));

  XPRMgetarrsets(solarr,sets);    /* Get the indexing sets
                                     (we know array has 1 dimension) */
  ct=0;
  XPRMgetfirstarrtruentry(solarr,indices); /* Get the first true index tuple */
  do
  {
   solution[ct].ind=XPRMgetelsetval(sets[0],indices[0],&rvalue)->string;
   XPRMgetarrval(solarr,indices,&rvalue);
   solution[ct].val=rvalue.real;
   ct++;
  } while(!XPRMgetnextarrtruentry(solarr,indices));
 }
 else
 {
  printf("Unknown output data item: %s %p\n", label, ref);
 }
 return 0;
}

The dynamic data input to a Mosel model uses a new set of dedicated library functions.

The format used to represent data is the same as the default text format used by initializations blocks. For example, the array definition

 mydata: [ ("ind1" 3) [5 1.2] ("ind2" 7) [4 6.5] ] 

is represented by the following sequence of function calls:

XPRMcb_sendctrl(cb, XPRM_CBC_OPENLST, 0);     ! [
XPRMcb_sendctrl(cb, XPRM_CBC_OPENNDX, 0);     !   (
XPRMcb_sendstring(cb, "ind1", 0);             !     "ind1"
XPRMcb_sendint(cb, 3, 0);                     !     3
XPRMcb_sendctrl(cb, XPRM_CBC_CLOSENDX, 0);    !   )
XPRMcb_sendctrl(cb, XPRM_CBC_OPENLST, 0);     !   [
XPRMcb_sendint(cb, 5, 0);                     !     5
XPRMcb_sendreal(cb, 1.2, 0);                  !     1.2
XPRMcb_sendctrl(cb, XPRM_CBC_CLOSELST, 0);    !   ]
XPRMcb_sendctrl(cb, XPRM_CBC_OPENNDX, 0);     !   (
XPRMcb_sendstring(cb, "ind2", 0);             !     "ind2"
XPRMcb_sendint(cb, 7, 0);                     !      7
XPRMcb_sendctrl(cb, XPRM_CBC_CLOSENDX, 0);    !   )
XPRMcb_sendctrl(cb, XPRM_CBC_OPENLST, 0);     !   [
XPRMcb_sendint(cb, 4, 0);                     !     4
XPRMcb_sendreal(cb, 6.5, 0);                  !     6.5
XPRMcb_sendctrl(cb, XPRM_CBC_CLOSELST, 0);    !   ]
XPRMcb_sendctrl(cb, XPRM_CBC_CLOSELST, 0);    ! ]         

The last argument '0' in these functions indicates that data is to be processed not immediately but only once the queue of tokens is full.

For our example, we thus have the following function definition (again, the prototype of the callback function must be defined exactly to the form expected by Mosel):

int XPRM_RTC cbinit_from(XPRMcbinit cb, void *info, const char *label,
                         int type, void *ref)
{
 int i;

 if(strcmp(label,"DATA")==0)
 {
  XPRMcb_sendctrl(cb, XPRM_CBC_OPENLST, 0);
  for(i=0;i<datasize;i++)
  {
   XPRMcb_sendctrl(cb, XPRM_CBC_OPENNDX, 0);
    XPRMcb_sendstring(cb, ind[i], -1, 0);
   XPRMcb_sendctrl(cb, XPRM_CBC_CLOSENDX, 0);
   XPRMcb_sendctrl(cb, XPRM_CBC_OPENLST, 0);
    XPRMcb_sendreal(cb, vdata[i], 0);
    XPRMcb_sendreal(cb, wdata[i], 0);
   XPRMcb_sendctrl(cb, XPRM_CBC_CLOSELST, 0);
  }
  XPRMcb_sendctrl(cb, XPRM_CBC_CLOSELST, 0);
  return 0;
 }
 else
 {
  fprintf(stderr,"Label `%s' not found.\n",label);
  return 1;
 }
}

The model file burglar13.mos receives through its run-time parameters the callback functions that are to be used for data input/output in the initializations sections. The definition of the mathematical model is the same as in the previous model version and left out in the listing below.

model Burglar13
 uses "mmxprs"

 parameters
  DATAFILE = ''                    ! Location of input data
  SOLFILE = ''                     ! Location for solution data output
  WTMAX = 102                      ! Maximum weight allowed
 end-parameters

 declarations
  ITEMS: set of string             ! Index set for items
  VALUE: array(ITEMS) of real      ! Value of items
  WEIGHT: array(ITEMS) of real     ! Weight of items
  soltake: array(ITEMS) of real    ! Solution values
 end-declarations

 initializations from DATAFILE
  [VALUE,WEIGHT] as "DATA"
 end-initializations

...

 initializations to SOLFILE
  soltake as "SOL"
 end-initializations

end-model

Scalars

Besides arrays one might also wish to simply exchange scalars between the calling application and a Mosel model. One way of passing the value of a scalar to a model is to define it as a model parameter and pass the new value as an execution parameter of the model (as shown in Section Parameters). Alternatively, we might read or write scalar values in initializations blocks similarly to what we have seen in the previous section for arrays.

Consider the following C program: there are three scalars, wmax, numitem, and objval. The value of the first should be read in by the Mosel model and the last two receive solution values from the optimization run in the model.

#include <stdio.h>
#include "xprm_mc.h"

int wmax=100;
int numitem;
double objval;

int main()
{
 XPRMmodel mod;
 int result;
 char wmax_name[40];              /* File name of input data 'wmax' */
 char num_name[40];               /* File name of output data 'num' */
 char sol_name[40];               /* File name of solution value */
 char params[160];                /* Parameter string for model execution */

 if(XPRMinit()) return 1;         /* Initialize Mosel */

/* Prepare file names for 'initializations' using the 'raw' driver */
 sprintf(wmax_name, "mem:%p/%d", &wmax, (int)sizeof(wmax));
 sprintf(num_name, "mem:%p/%d", &numitem, (int)sizeof(numitem));
 sprintf(solution_name, "mem:%p/%d", &objval, (int)sizeof(objval));

                                  /* Pass file names as execution param.s */
 sprintf(params, "WMAX='%s',NUM='%s',SOLVAL='%s'", wmax_name, num_name,
         sol_name);

 if(XPRMexecmod(NULL, "burglar12.mos", params, &result, &mod))
  return 2;                       /* Execute a model file */

 if((XPRMgetprobstat(mod)&XPRM_PBRES)!=XPRM_PBOPT)
  return 3;                       /* Test whether a solution is found */

/* Display solution values obtained from the model */
 printf("Objective value: %g\n", objval);
 printf("Total number of items: %d\n", numitem);

 XPRMresetmod(mod);

 return 0;
} 

The Mosel model takes as execution parameters the filenames (location in memory) of the three scalars. The value WTMAX is initialized from the data in the application and the two other locations are written to in the initializations to block at the end of the model.

model Burglar12
 uses "mmxprs"

 parameters
  NUM = ''                         ! Location for no. of items output
  SOLVAL = ''                      ! Location for objective value output
  WMAX = ''                        ! Maximum weight allowed
 end-parameters

 declarations
  WTMAX: integer                   ! Maximum weight allowed
  ITEMS = {"camera", "necklace", "vase", "picture", "tv", "video",
           "chest", "brick"}     ! Index set for items
  VALUE: array(ITEMS) of real      ! Value of items
  WEIGHT: array(ITEMS) of real     ! Weight of items
  soltake: array(ITEMS) of real    ! Solution values
 end-declarations

 VALUE :: (["camera", "necklace", "vase", "picture", "tv", "video",
            "chest", "brick"])[15,100,90,60,40,15,10,1]
 WEIGHT:: (["camera", "necklace", "vase", "picture", "tv", "video",
            "chest", "brick"])[2,20,20,30,40,30,60,10]

 initializations from 'raw:'
  WTMAX as WMAX
 end-initializations

 declarations
  take: array(ITEMS) of mpvar      ! 1 if we take item i; 0 otherwise
 end-declarations

! Objective: maximize total value
 MaxVal:= sum(i in ITEMS) VALUE(i)*take(i)

! Weight restriction
 sum(i in ITEMS) WEIGHT(i)*take(i) <= WTMAX

! All variables are 0/1
 forall(i in ITEMS) take(i) is_binary

 maximize(MaxVal)                  ! Solve the MIP-problem

! Print out the solution
 writeln("Solution:")
 forall(i in ITEMS)  writeln(" take(", i, "): ", getsol(take(i)))

! Output solution to calling application
 initializations to 'raw:'
  evaluation of getobjval as SOLVAL
  evaluation of round(sum(i in ITEMS) getsol(take(i))) as NUM
 end-initializations

end-model