Defining IO drivers

Mosel accesses files through IO drivers: a driver provides a set of functions implementing basic IO operations (open, close, read a block, write a block). As a consequence, any kind of data source may be exposed as a file (or stream) in the Mosel environment as long as it can be accessed through the basic operations listed above.
IO drivers are declared using the XPRM_SRV_IODRVS service which points to a table of drivers. Each driver is identified by its name and a table of functions. This table of functions is of the form ( operation code, function ) which associates to each code a function performing the operation (except for operation XPRM_IOCTRL_INFO which is only a text string describing the driver). The last record of this table must be the pair ( 0,NULL ).

Example:

static XPRMiofcttab mydrv_fcts[]=
        {
         {XPRM_IOCTRL_OPEN, mydrv_open},
         {XPRM_IOCTRL_CLOSE, mydrv_close},
         {XPRM_IOCTRL_READ, mydrv_read},
         {XPRM_IOCTRL_WRITE, mydrv_write},
         {XPRM_IOCTRL_INFO, "mydrv description"},
         {0,NULL}
        };

static XPRMiodrvtab iodrivers[]=
        {
         {"mydrv", mydrv_fcts},
         {NULL, NULL}
        };

static XPRMdsoserv tabserv[]=
        {
         {XPRM_SRV_IODRVS, iodrivers}
        };

In addition to basic operations, a driver may provide implementations for the initializations block, file removal and file renaming. The ``initializations from/to'' routines get direct access to Mosel internal data and can therefore work at the binary level. The other file operations may be required for resource management.

All functions performing IO operations are sent a Mosel execution context. However, this context may be NULL if the stream is used outside of the execution of a model (for instance for compiling a model source). It is not necessary to provide an implementation for all of the possible operations. However, the open function and one of the data transfer routines are mandatory (read or write or initfrom or initto).

Operation ``Open Stream''

XPRM_IOCTRL_OPEN: void *open(XPRMcontext ctx, int *mode, const char *fname,
                            unsigned int *enc, int *bufsize)

This function is called to open a stream associated to the given file name (the provided file name does not include driver name). Parameter mode is a pointer to the open mode. This value is bit encoded and the following bits may be set:

XPRM_F_BINARY

open in binary mode (default is text mode)

XPRM_F_WRITE

open for writing (default is for reading)

XPRM_F_APPEND

when open for writing, do not reset file but append data to the end of the original file

XPRM_F_ERROR

when open for writing, stream will be used as an error stream.

XPRM_F_LINBUF

when open for writing, stream is line buffered: buffer is flushed after each line (by default streams are flushed when full or when an explicit flush is executed)

XPRM_F_INIT

stream open for an `initialisations' block

XPRM_F_SILENT

stream open in silent mode (error messages are not displayed)

XPRM_F_DELCLOSE

delete file after the stream has been closed

These modes may be combined (for instance XPRM_F_WRITE|XPRM_F_LINBUF) so testing the mode should be done using masks. The open function may alter the mode by changing some bits (LINBUF, INIT and SILENT; others are ignored) and by using bits 16 to 31 that are not used by Mosel although saved with the stream's mode (which can be retrieved later using function fgetinfo and is provided to the close function). Note the particular meaning of INIT: if the function is called with this bit set, the stream will be used for an initialisations block. Resetting this bit indicates that the driver provides an handler for this operation and expects that Mosel will use this handler. Otherwise, even if the driver publishes the operation, the default procedures are used.
Mosel saves the current input and output streams after execution of the open function if new files have been opened using fopen. These current streams are restored when calling operations ``close'', ``read'', ``skip'', ``write'', ``init from'' and ``init to''. This is useful when the stream implements a filter (i.e. it preprocesses some data actually stored in another file).

The parameter enc is the encoding used when the file is open in text mode (i.e. flag XPRM_F_BINARY is not set). If no encoding has been explicitly stated in the file name (using the "enc:" prefix) the value of this variable is XPRM_FE_ENCDEF (that is equivalent to UTF-8). Otherwise this value consists in 2 16-bits words: the first part is the encoding ID as used by the XPRNLS library (please refer to the XPRNLS Reference Manual for further explanation), it can be extracted using the mask XPRM_FE_MSK_ENC. The second part of the value is used to record the encoding options (XPRM_FE_STRICT, XPRM_FE_UNIX, XPRM_FE_DOS, XPRM_FE_BOM, XPRM_FE_NOBOM). The function may modify the encoding or disable it entirely by switching to binary mode.

The parameter bufsize is the size (in kilobytes) of the internal buffer allocated for the stream. A driver can change this setting that must remain between 2 (the default value) and 64.

The return value of this function will be used as input for other IO functions. A return value of NULL indicates a failure: in this case, the function setioerrmsg may be used to set a descriptive text for the error that is displayed after the function returns (if the stream is not open in silent mode).

Operation ``Close Stream''

XPRM_IOCTRL_CLOSE: int close(XPRMcontext ctx, void *stream, int mode)

This optional function is called to close a stream previously open using the corresponding open function (after it has been flushed if it is an output stream). The second parameter is the pointer returned by a successful execution of the ``open'' function and the third parameter is the current mode of the stream. If an error has been detected on this stream, the bit XPRM_F_IOERR is set.

This function must return 0 in case of success, all other values are interpreted as error codes. In the later case, the function setioerrmsg may be used to set a descriptive text for the error that is displayed after the function returns (if the stream is not open in silent mode).

Operation ``Read Block''

XPRM_IOCTRL_READ: long read(XPRMcontext ctx, void *stream, void *buffer,
                            unsigned long size)

This function is used to load the stream buffer open for reading. Parameters buffer and size define location and size of the buffer associated to the stream. This function returns the number of bytes actually copied into the buffer. A value of 0 characterises an end of file and a negative value is interpreted as an error. In the later case, the function setioerrmsg may be used to set a descriptive text for the error that is displayed after the function returns (if the stream is not open in silent mode).

Operation ``Skip Block''

XPRM_IOCTRL_SKIP: int skip(XPRMcontext ctx, void *stream, int size)

This optional function is used to skip a block of data from an input stream. If the operation is successful a positive value has to be returned; the special value -2 indicates that the corresponding stream does not support the operation. In this case the system will read (and discard) the number of bytes to be skipped. A negative value is interpreted as an error and the function setioerrmsg may be used to set a descriptive text for the error that is displayed after the function returns (if the stream is not open in silent mode).

Operation ``Write Block''

XPRM_IOCTRL_WRITE: long write(XPRMcontext ctx, void *stream, void *buffer,
                              unsigned long size)

This function is used to flush the stream buffer open for writing. Parameters buffer and size define location and size of the buffer associated to the stream. This function must return a positive value if successful - any negative value or zero is interpreted as an error status. In the later case, the function setioerrmsg may be used to set a descriptive text for the error that is displayed after the function returns (if the stream is not open in silent mode).

Operation ``Initializations From''

XPRM_IOCTRL_IFROM: int initfrom(XPRMcontext ctx, void *stream, int nbrec,
                                const char **labels, int *types,
                                XPRMalltypes **adrs, int *nbread)

This function implements an initializations from block. For this function to be called, the open function must reset the INIT bit of the open mode. The information provided describes the block to be processed: number of records nbrec, for each record i its label labels[i], its type types[i] and its address adrs[i] (direct address for Mosel objects and indirect address for objects of external types). If a label is associated to a tuple of arrays, the corresponding address is a list of arrays.

If auto finalization is in use (this is indicated by the control parameter "autofinal", see getparam), sets must be finalized using fnlset just after they have been initialized. Also, in order to handle the case of index sets of implicit dynamic arrays, it is necessary to call beginarrinit before the initialization of each array and endarrinit after the process has completed.

During its execution the function should set to NULL each entry i of the adrs array for which reading has been completed and store in nbread[i] the number of successfully input lines (i.e. when reading a tuple of arrays, 1 line means 1 value for each array). The return value of initfrom is interpreted as the number of records that have not been successfully read in (i.e. 0 for success). In case of an IO error the function should return -1.

Operation ``Initializations To''

XPRM_IOCTRL_ITO: int initto(XPRMcontext ctx, void *stream, int nbrec,
                            const char **labels, int *types, XPRMalltypes **adrs)

This function implements an initializations to block. For this function to be called, the open function must reset the INIT bit of the open mode. The information provided describes the block to be processed: number of records nbrec, for each record i its label labels[i], its type types[i] and its address adrs[i] (direct address for Mosel objects and indirect address for objects of external types). If a label is associated to a tuple of arrays, the corresponding address is a list of arrays.

During its execution the function should set to NULL each entry i of the adrs array for which saving has been completed. The return value of initto is interpreted as the number of records that have not been successfully saved (i.e. 0 for success). In case of an IO error the function should return -1.

Operation ``Remove File''

XPRM_IOCTRL_RM: int remove(XPRMcontext ctx, const char *todel)

This function is called to remove (or delete) a file. Parameter todel is the name of the file to be deleted (IO driver name is not included). Mosel does not check whether the file is currently open: depending on the driver this may make the operation impossible and should be checked by the function if necessary. The convention for return values is as follows: 0 indicates a success, 1 if the file cannot be accessed and 4 if the operation is not possible (e.g. file open or protected).

Operation ``Move File''

XPRM_IOCTRL_MV: int move(XPRMcontext ctx, const char *src, const char *dst)

This function is called to move (or rename) a file. Parameters src and dst are names of the source and destination files (IO driver name is not included). Mosel does not check whether the files are currently open: depending on the driver this may make the operation impossible and should be checked by the function if necessary. The convention for return values is as follows: 0 indicates a success, 1 if the source file cannot be accessed, 2 if destination file cannot be open for writing, 3 if the copy failed and 4 if the source cannot be removed after copy. The special value -1 can be used to tell Mosel to perform the operation by deleting the file after having made a copy of it.

Operation ``File Size''

XPRM_IOCTRL_SIZE: size_t fsize(XPRMcontext ctx, const char *file)

This function is called to retrieve the size of a file. Parameter file is the name of the file to be considered (IO driver name is not included). Mosel does not check whether the file is currently open: depending on the driver this may make the operation impossible and should be checked by the function if necessary. On error (size_t)-1 must be returned.