Example: compression

With the gzip I/O driver for file compression implemented by the example module zlib, compressed files may be used for Mosel input and output. For example, we may generate and run a compressed BIM file with the following command:

mosel comp mymodel.mos -c "example of compression" -o zlib.gzip:mymodel.gz
mosel run zlib.gzip:mymodel.gz

To use compressed data files in a model we may write:

initializations from "zlib.gzip:burglar.dat.gz"

Another use of this driver may be to generate compressed matrix files:

exportprob("zlib.gzip:prob.mat.gz")

The I/O driver gzip implements an interface to the gzip compression format of the ZLIB library. The complete example module zlib also implements a second driver, compress, for a different compression format defined by this library. The driver gzip works with physical files, compress is a more general driver that works with streams. The ZLIB library is required to execute this example: it can be found at http://www.zlib.org.

Implementation

A C program defining new I/O drivers has the following components

• Interface structures: the DSO interface table, the table of services, the table of drivers, and a table of functions per driver.
• Module initialization function
• Implementation of driver access functions: opening and closing streams, standard reading and writing, special reading and writing for initializations blocks, error handling, deleting and moving files.

A driver must implement at least the stream open functionality and one of the standard or special reading or writing functions; everything else is optional.

The definition of the interface structures of module zlib is the following. The structure of all tables and the function prototypes are defined by the Mosel NI.

                                /* Functions of the 'gzip' driver */
static void *gzip_open(XPRMcontext ctx, int *mode, const char *fname);
static int gzip_close(XPRMcontext ctx, gzFile gzf, int mode);
static long gzip_read(XPRMcontext ctx, gzFile gzf, void *buffer,
                                                        unsigned long size);
static long gzip_write(XPRMcontext ctx, gzFile gzf, void *buffer,
                                                        unsigned long size);
static XPRMiofcttab iodrv_gzip[]=
        {
         {XPRM_IOCTRL_OPEN, gzip_open},
         {XPRM_IOCTRL_CLOSE, gzip_close},
         {XPRM_IOCTRL_READ, gzip_read},
         {XPRM_IOCTRL_WRITE, gzip_write},
         {XPRM_IOCTRL_INFO, "filename"},
         {0,NULL}
        };
                                /* Drivers of the zlib module */
static XPRMiodrvtab iodrv_zlib[]=
        {
         {"gzip", iodrv_gzip},
         {NULL,NULL}
        };
                                /* Table of services: only IO drivers */
static XPRMdsoserv tabserv[]=
        {
         {XPRM_SRV_IODRVS, iodrv_zlib}
        };
                                /* DSO interface: only services */
static XPRMdsointer dsointer=
        {
         0, NULL,
         0, NULL,
         0, NULL,
         sizeof(tabserv)/sizeof(mm_dsoserv), tabserv
        };

static XPRMnifct mm;            /* For storing Mosel NI function table */

The previous code extract is best read from bottom to top. It declares an object of type XPRMnifct to store the NI function table (to be retrieved during the initialization of the module). The main DSO interface table dsointer lists the functionality provided by this module: it only defines services. The table of services, tabserv, indicates which services are defined, namely new I/O drivers. The list of I/O drivers is given in the table iodrv_zlib. The functionality implemented by the I/O driver gzip is listed in the table iodrv_gzip. The function bodies of gzip_open, gzip_close, etc., are printed below. The string set with INFO will be used by the `help' command of Mosel to indicate the usage of this driver (it must be used with the name of a physical file, not with extended filenames).

The module initialization function performs the standard initialization of Mosel modules: retrieving the Mosel NI function table and the NI version number and returning the module DSO interface table to Mosel. In addition, we check the version of the ZLIB library.

DSO_INIT zlib_init(XPRMnifct nifct, int *interver, int *libver,
                     XPRMdsointer **interf)
{
 const char *zlv;

 mm=nifct;                      /* Save the table of Mosel NI functions */
 *interver=XPRM_NIVERS;         /* The interface version we are using */

                                /* We check the ZLIB library version here */
 zlv=zlibVersion();
 if((zlv==NULL)||(zlv[0]!=ZLIB_VERSION[0]))
 {
  mm->dispmsg(NULL,"ZLIB: wrong version (expect %s got %s).\n",ZLIB_VERSION,
                                                          (zlv!=NULL)?zlv:"0");
  return 1;
 }
 else
 {
  *libver=XPRM_MKVER(0,0,1);    /* The version of the module: 0.0.1 */
  *interf=&dsointer;            /* Our interface */
  return 0;
 }
}

Below are printed the functions implementing the gzip driver. Besides the obligatory `open' function the gzip driver implements the functions `closing a file', `reading' (= uncompress), and `writing' (= compress).

/**** Open a gzip-file for (de)compression ****/
static void *gzip_open(XPRMcontext ctx, int *mode, const char *fname)
{
 char cmode[16];
 int cml;

 cml=2;
 cmode[0]=((*mode)&XPRM_F_WRITE)?'w':'r';
 cmode[1]=((*mode)&XPRM_F_BINARY)?'b':'t';
 if((*mode)&XPRM_F_APPEND) cmode[cml++]='a';
 cmode[cml]='\0';
 return gzopen(fname,cmode);
}

/**** Close the gzip-file ****/
static int gzip_close(XPRMcontext ctx, gzFile gzf, int mode)
{ return gzclose(gzf); }

/**** Uncompress a block of data ****/
static long gzip_read(XPRMcontext ctx, gzFile gzf, void *buffer,
                                                        unsigned long size)
{ return gzread(gzf, buffer, size); }

/**** Compress a block of data ****/
static long gzip_write(XPRMcontext ctx, gzFile gzf, void *buffer,
                                                        unsigned long size)
{ return gzwrite(gzf, buffer, size); }

As mentioned earlier, the full zlib example module in the module examples of the Mosel distribution defines a second compression driver, compress, in a very similar way to what has been shown here for the gzip driver.