Using IO drivers for data exchange

Mosel comes with a default set of IO drivers which are used as data source/destination. The selection of the driver is achieved via the file name in use: for instance file name "myfile" is a physical file handled by the operating system but "mem:myfile" is a block of memory managed by the mem driver. IO drivers are mainly used to interface specific data sources with Mosel (like odbc from the mmodbc module). In this context, each data source may require a dedicated driver that can be implemented in a user module through the Mosel NI (refer to the Mosel NI Reference Manual for further explanation). Drivers may also be employed to easily exchange information between the application running the Mosel Libraries and a model. In particular the predefined drivers cb, mem and raw are specifically designed for this purpose.

sysfd driver

Thanks to this driver, a file descriptor provided by the operating system may be used in place of a file. The general syntax of a file name for the sysfd driver is:

sysfd:OSfd

where OSfd is a numerical file descriptor (Posix) or a file handle (Windows). File descriptors are usually returned by C functions open or fileno (from a C-stream obtained with fopen) on Posix systems. Under Windows, file handles can be created using CreateFile or obtained with _get_osfhandle (from a C file descriptor) for instance. When a program starts, 3 files are automatically opened for input, output and errors; they are respectively associated to file numbers 0,1 and 2 (this applies to both Posix systems and Windows). Mosel uses these file decriptors as default streams.

Example:

XPRMsetdefstream(NULL,XPRM_F_ERROR,"sysfd:1"); /* redirect error to output stream */

cb driver

This driver allows using a function as a file. The general syntax of a file name for the cb driver is:

cb:funcaddr[/refval]

where funcaddr is the address of the callback function and the optional parameter refval is a pointer (both references must be expressed in hexadecimal). Depending on the type of stream to manage (i.e. a general stream or a for an initializations block) a specific function type as to be provided.

Handling of general streams

The expected function must have the following prototype:

long XPRM_RTC func(XPRMmodel model, void *ref, char *buf, unsigned long size);

Whenever data needs to be transferred, Mosel calls this function indicating the location (buf) and the size (size) of the buffer to use. The parameter ref is the information provided to Mosel during the opening of the file (refval above). The model reference may be NULL if the stream is used directly by Mosel (for instance for compilation). When the stream is open for writing, the return value of the function is ignored. If the corresponding output stream is open in text mode, the function is called at each end of line and the buffer can be seen as a NULL terminated character string (the size does not include the terminating character). When used for reading, the function should return the number of bytes actually copied into the buffer (0 means end of file).

Example:

long XPRM_RTC simpleout(XPRMmodel model, void *ref, char *buf,
              unsigned long size)
{
 printf("OUT: %.*s",(int)size,buf);
 return 0;
}

 ...

char fname[32];

sprintf(fname, "cb:%p", simpleout);
XPRMsetdefstream(NULL, XPRM_F_ERROR, fname);  /* redirect error str. to 'simpleout' */
 ...

Handling of initializations blocks

In the case of an initializations block, the expected function must be of the following form:

int XPRM_RTC func(XPRMcbinit cbinit, void *ref,const char *label, int type, XPRMalltypes *obj);

When executing an initializations from block, the function is called once for each label with the label to be initialized (label), its type (type) and a reference to the object (obj). The parameter ref is the information provided to Mosel during the opening of the file (refval above). The function must then send to Mosel the data to be used for the initialization of the object using routines XPRMcb_sendint, XPRMcb_sendreal, XPRMcb_sendstring and XPRMcb_sendctrl. The 3 first functions provide the basic type values while the last one is used to structure the data stream (i.e. delimit a list of array indices or a collection of values) in a similar fashion as in an ASCII initialization file.

Example:

int XPRM_RTC initfrom(XPRMcbinit cbinit, void *ref,const char *label,
              int type, XPRMalltypes *ref)
{
 int i;

 if(strcmp(label,"I")==0)      /* I:10 */
 {
  XPRMcb_sendint(cbinit,10,0)
 }
 else if(strcmp(label,"S")==0) /* S:[1 2 3] */
 {
  XPRMcb_sendctrl(cbinit,XPRM_CBC_OPENLST,0);
   for(i=1;i<=3;i++)
    XPRMcb_sendreal(cbinit,(double)i,0);
  XPRMcb_sendctrl(cbinit,XPRM_CBC_CLOSELST,0);
 }
 else if(strcmp(label,"A")==0) /* A:[(1) "a"] */
 {
  XPRMcb_sendctrl(cbinit,XPRM_CBC_OPENLST,0);
   XPRMcb_sendctrl(cbinit,XPRM_CBC_OPENNDX,0);
    XPRMcb_sendint(cbinit,1,0);
   XPRMcb_sendctrl(cbinit,XPRM_CBC_CLOSENDX,0);
   XPRMcb_sendstring(cbinit,"a",1,0);
  XPRMcb_sendctrl(cbinit,XPRM_CBC_CLOSELST,0);
 }
}
/* The associated Mosel code:
 declarations
  I:integer
  S:set of real
  A:array(range) of string
 end-declarations
 initializations from INITFILE
  I S A
 end-initialisations
*/

Similarly, when executing an initializations to block, the function is called once for each label with the object reference, its type and associated label (in this case the first parameter is NULL). The user function can then inspect the object using the usual routines of the Mosel Runtime Library.

mem driver

With this driver, a block of memory is used as data source. Three different types of blocks are supported: named blocks can be used only from a model during its execution, are identified by a label and their allocation is dynamic. The second type uses a block of memory already allocated: it is characterized by an address and a size. With the third form the file name corresponds to a reference to a dedicated data structure to hold the properties of a memory block managed by Mosel.

The general syntax of a file name for the mem driver accessing a named block is:

mem:label[/minsize[/incstep]]

where label is an identifier whose first character is a letter and minsize an optional initial amount of memory to be reserved (size is expressed in bytes, in kilobytes with suffix "k" or in megabytes with suffix "m"). The memory block is allocated dynamically and resized as necessary. By default the size of the memory block is increased by pages of 4 kilobytes: the optional parameter incstep may be used to change this page size (i.e. the default setting is "label/0/4k"). The special value 0 modifies the allocation policy: instead of being increased of a fixed amount, the block size is doubled. In all cases unused memory is released when the file is closed.

When a named memory block is used in a model, it is possible to access the block of memory allocated by the driver by searching for the label in the model's dictionary: the function XPRMfindident returns a reference to an object of structure XPRM_STR_MEM that describes the location and size of the memory block.

The general syntax of a file name for the mem driver accessing a fixed block is:

mem:addr/size[/actualsize]

where addr and size identify the memory block (the pointer must be expressed in hexadecimal). Optionally a pointer to a size_t value may be provided (actualsize expressed in hexadecimal): when the stream is open for writing, this variable receives the size actually used by the operation (its value thus ranges between 0 and size). Moreover, if the stream is open in append mode, writing starts after the location indicated by this value. When the stream is open for reading, the value is used in place of size if it is smaller than this upper limit.

Example:

char blk[2048];
char fname[40];
size_t actualsize;

sprintf(fname, "mem:%p/%u/%p", blk, (int)sizeof(blk), &actualsize);
XPRMcompmod(NULL, "mymodel", fname, NULL);   /* compile model to memory */
printf("BIM data uses %u bytes.\n", actualsize);
mod=XPRMloadmod(fname, NULL);                /* load BIM file from memory */

The last form is similar to first one except that the name of the block is replaced by an address prefixed by the & symbol:

mem:&addr[/minsize[/incstep]]

where addr is a reference (expressed in hexadecimal) to an XPRMmemblk data structure:

typedef struct
   {
    void *ref;    /* Base address of the block */
    size_t size;  /* Size of the block */
   } XPRMmemblk;

When the block is used for the first time, fields of the structure must be cleared: Mosel handles (re)allocation of the memory block when writing to the corresponding memory file. The function XPRMfreememblk can be used to release the memory allocated through this IO driver (deleting the file from the Mosel code has the same effect).

Example:

XPRMmemblk memblk;
char fname[40];

sprintf(fname, "mem:&%p", &memblk);
memset(&memblk,0,sizeof(XPRMmemblk));
XPRMcompmod(NULL, "mymodel", fname, NULL);   /* compile model to memory */
mod=XPRMloadmod(fname, NULL);                /* load BIM file from memory */
XPRMfreememblk(&memblk)                      /* release BIM file memory */

raw driver

The raw driver provides an implementation of the ``initializations blocks'' in binary mode: instead of translating information from/to text format, data is kept in its raw representation. Typically this driver will be combined with the mem driver in order to exchange arrays of data between the model and an application through memory without translation. The general syntax of a file name for the raw driver is:

raw:[noindex,align,noalign,append,all,slength=#]

When using the raw driver as a file for an initializations block, no actual data location is provided at the beginning of the block. The driver uses each label as a file name for locating data.

Example:

initializations from "raw:noindex"
 t as "datafile.bin"
 r as "mem:0x1234/456"
end-initializations

Data transfer is achieved without conversion: 4 bytes for an integer, 8 bytes for a real, 1 byte for a Boolean, strings are of fixed size or just an address, external types are translated to strings (if ``tostr'' is available for the type) and anything else has the size of an address that is 4 or 8 bytes depending on the architecture. The option slength specifies the fixed length of strings, default value for this parameter is 16 (shorter strings are padded with 0 characters, longer strings are cut). The special value 0 implies that the address of the string is used.

If option append is specified, files open for writing are open in append mode.

Transfer of scalar is straightforward and sets are treated as a collection of consecutive scalars. The handling of arrays varies depending on the options: by default, each array element is preceded by its indices (for instance t(1,2) is stored or read as 1,2,t(1,2)). If option noindex is in use, only values are listed and if option all has been given, all elements of dynamic arrays are listed (by default: only existing elements).

The driver aligns data according to the processor architecture requirements assuming the starting address provided is aligned properly (for instance on Sparc processors real values [or doubles] are aligned on 8 bytes boundaries). Thanks to this property, it is safe to map data exchanged using this driver with the corresponding structure in the C language.

Example:

declarations
 a: array(integer,boolean) of real
end-declarations
! the above declaration can be mapped to the following C-structure:
! struct {
!    int ndx1
!    char ndx2
!    double a_ndx1_ndx2 };
! This structure uses 13 bytes with an Intel processor and 16 on a Sparc

This behavior may be changed by using the align and noalign options (for instance for saving binary data to physical files, alignment is not necessary and uses more memory).

Options may be specified for each label individually: they have to be given as a list preceding the actual filename.

Example: the following model:

parameters
 DAT=""
 RES=""
end-parameters
declarations
 d:array(string) of real
 r:array(1..10) of real
end-declarations
initializations from "raw:"
 d as "slength=0,mem:"+DAT  ! load data from memory location defined by DAT
end-initializations
...
initializations to "raw:"
 r as "noindex,mem:"+RES  ! save results in memory location defined by RES
end-initializations

can be used with the following C-source:

char params[128];
struct { const char *ndx; double v; } d[]={{"one",10}, {"two",0.5}};
double r[10];

sprintf(params, "DAT='%p/%u', RES='%p/%u'", d, sizeof(d), r, sizeof(r));
XPRMrunmod(mod, &result, params);

bin driver

Like the raw driver, the bin driver provides an implementation of the ``initializations blocks'' in binary mode. However, thanks to a structured and architecture independent data format the bin driver can handle cross platform files containing all records of an initialisations block. An application can generate and decode files using this format with the help of the bindrv library. Refer to the documentation of this library for further explanation.