Alternative text data file formats

In the previous section we have introduced the I/O driver mmsheet.csv that reads and writes text files in CSV format. With only minimal changes to the previous model we now switch to a different text data format, namely the diskdata file format that is accessed via the diskdata I/O driver.

model Burglar2dd
 ...
 initializations from "mmetc.diskdata:sparse;noq"
  [VALUE,WEIGHT] as "burglardd.dat"
 end-initializations
 ...
 initializations to "mmetc.diskdata:append,sparse"
  SOLTAKE as "burglarout.txt"
 end-initializations
end-model

This will result in exactly the same behaviour as when using the procedure diskdata of the module mmetc instead of the initializations blocks.

model Burglar2ddb
 uses "mmetc", "mmxprs"
 ...
! Reading data from file
 diskdata(ETC_SPARSE+ETC_NOQ, "burglardd.dat", [VALUE,WEIGHT])
 ...
! Solution output
 diskdata(ETC_OUT+ETC_APPEND+ETC_SPARSE, "burglarout.txt", SOLTAKE)
end-model

The data file burglardd.dat used in both cases has the following contents.

! Data file for `burglar2dd.mos'
camera,    15,  2
necklace, 100, 20
vase,      90, 20
picture,   60, 30
tv,        40, 40
video,     15, 30
chest,     10, 60
brick,      1, 10

The result file burglarout.txt is now formatted as shown here.

"camera",1
"necklace",1
"vase",1
"picture",1
"tv",0
"video",1
"chest",0
"brick",0

The diskdata I/O driver and the procedure diskdata are documented in the section `mmetc' in the `Mosel Language Reference Manual'.

XML and JSON format files

The data for our example problem could also be provided in the form of the following XML file burglar.xml.

<!-- Data file for `burglar2x.mos' -->
<BurgData>
  <Item name="camera">   <Value> 15</Value> <Weight> 2</Weight> </Item>
  <Item name="necklace"> <Value>100</Value> <Weight>20</Weight> </Item>
  <Item name="vase">     <Value> 90</Value> <Weight>20</Weight> </Item>
  <Item name="picture">  <Value> 60</Value> <Weight>30</Weight> </Item>
  <Item name="tv">       <Value> 40</Value> <Weight>40</Weight> </Item>
  <Item name="video">    <Value> 15</Value> <Weight>30</Weight> </Item>
  <Item name="chest">    <Value> 10</Value> <Weight>60</Weight> </Item>
  <Item name="brick">    <Value>  1</Value> <Weight>10</Weight> </Item>
</BurgData> 

Given that XML files do not use any fixed format that would make it possible to define a suitable I/O driver, we cannot read or write XML files through initializations blocks. XML files need to be parsed or queried for their contents using the routines of the mmxml module. The data file burglar.xml is read into our Mosel data structures using the following code:

 declarations
  ITEMS: set of string           ! Index set for items
  VALUE: array(ITEMS) of real    ! Value of items
  WEIGHT: array(ITEMS) of real   ! Weight of items

  BurgData, ResData: xmldoc      ! XML document
  Root, Node: integer            ! XML nodes
  NodeList: list of integer
 end-declarations

! Reading data from an XML file
 load(BurgData, "burglar.xml")

! Retrieve all 'Item' nodes
 getnodes(BurgData, "BurgData/Item", NodeList)

! Retrieve 'Value' and 'Weight' information
 forall(i in NodeList) do
   VALUE(getstrattr(BurgData,i,"name")):=
     getrealvalue(BurgData, getnode(BurgData, i, "Value") )
   WEIGHT(getstrattr(BurgData,i,"name")):=
     getrealvalue(BurgData, getnode(BurgData, i, "Weight") )
 end-do   

For generating solution output in XML format we need to build up an XML document following standard XML design rules (all elements are forming a tree under a single root element).

! Create solution representation in XML format
 Root:=addnode(ResData, 0, XML_ELT, "SolTake")    ! Create root node "SolTake"
 setattr(ResData, Root, "objective", getobjval)   ! ... with attr. "objective"

 forall(i in ITEMS) do
   Node:=addnode(ResData, Root, XML_ELT, "Item") ! Add a node to "SolTake"
   setattr(ResData, Node, "name", i)             ! ... with attribute "name"
   setvalue(ResData, Node, SOLTAKE(i))           ! ... and solution value
 end-do

 save(ResData, "burglarout.xml")    ! Save solution to XML format file 

This Mosel code will result in the following output (the appearance can be influenced via different controls and settings of mmxml, see the corresponding chapter of the `Mosel Language Reference Manual'):

<?xml version="1.0" encoding="iso-8859-1"?>
<SolTake objective="280">
  <Item name="camera">1</Item>
  <Item name="necklace">1</Item>
  <Item name="vase">1</Item>
  <Item name="picture">1</Item>
  <Item name="tv">0</Item>
  <Item name="video">1</Item>
  <Item name="chest">0</Item>
  <Item name="brick">0</Item>
</SolTake>   

The module mmxml also provides routines for reading and writing text files in JSON format, transforming them internally to an XML-based representation where JSON 'values' are encoded as jsv elements. The value of the attribute 'jst' indicates the object type ( 'num'—number, 'boo'—boolean, 'str'—string, 'nul'—null, 'obj'—object, or 'arr'—array). Elements representing object members have the name of the member (instead of jsv).
Data from the following JSON file

{
  "Item": [
    {"Name":"camera",   "Value": 15, "Weight": 2},
    {"Name":"necklace", "Value":100, "Weight":20},
    {"Name":"vase",     "Value": 90, "Weight":20},
    {"Name":"picture",  "Value": 60, "Weight":30},
    {"Name":"tv",       "Value": 40, "Weight":40},
    {"Name":"video",    "Value": 15, "Weight":30},
    {"Name":"chest",    "Value": 10, "Weight":60},
    {"Name":"brick",    "Value":  1, "Weight":10} ]
}

can be read with this Mosel code (all declarations remain the same as in the previous model version reading XML format data):

! Reading data from a JSON file
 jsonload(BurgData, "burglar.json")

! Retrieve all 'Item' nodes
 getnodes(BurgData, "jsv/Item/jsv", NodeList)

! Retrieve 'Value' and 'Weight' information
 forall(i in NodeList) do
   VALUE(getstrvalue(BurgData, getnode(BurgData,i,"Name"))):=
     getrealvalue(BurgData, getnode(BurgData, i, "Value") )
   WEIGHT(getstrvalue(BurgData, getnode(BurgData,i,"Name"))):=
     getrealvalue(BurgData, getnode(BurgData, i, "Weight") )
 end-do
 

After loading the contents of a JSON file, it may be helpful to display the resulting XML representation on screen since we parse this representation to retrieve the input data:

 save(BurgData, "")        ! Display the XML representation on screen

The results output in JSON format is constructed using XML routines:

! Create solution representation in JSON format
 Root:=addnode(ResData, 0, XML_ELT, "jsv")            ! Create root node "jsv"
 Node:=addnode(ResData, Root, "Objective", getobjval) ! Add a node to "jsv"
 Node:=addnode(ResData, Root, XML_ELT, "Items")       ! Add a node to "jsv"
 forall(i in ITEMS)
   n:=addnode(ResData, Node, i, SOLTAKE(i))           ! Add a node to "Items"

 jsonsave(ResData, "burglarout.json")    ! Save solution to JSON format file

This results in the following output in file burglarout.json:

{
  "Objective":280,
  "Items":{
    "camera":1,
    "necklace":1,
    "vase":1,
    "picture":1,
    "tv":0,
    "video":1,
    "chest":0,
    "brick":0
  }
}

Free format files

If none of the provided text file access methods fits a particular data file format, you can always use read/readln in combination with fopen and fclose. For example, a data file of the form

! Data file for `burglar2ff.mos'
camera [v=15, w=2]
necklace [v=100, w=20]
...

can be read by the following Mosel code (model file burglar2ff.mos)

 fopen("burglarff.dat", F_INPUT) ! Open file for reading
 while (not iseof) do            ! Read up to end-of-file
   fskipline("!")                ! Skip lines starting with '!'
   readln(j, " [v=", VALUE(j), ", w=", WEIGHT(j), "]")
   if getparam("nbread") < 6 then
     writeln("Error reading data for index '", j, "'")
   end-if
 end-do
 fclose(F_INPUT)                 ! Close the input file 

And similarly, for redirecting output to a file instead of displaying it on the default output (screen) we simply surround it by fopen and fclose, this time selecting the output stream in fopen:

 fopen("burglarsol.txt", F_OUTPUT)
 writeln("Solution:\n Objective: ", getobjval)
 forall(i in ITEMS)  writeln(" take(", i, "): ", getsol(take(i)))
 fclose(F_OUTPUT)

Multiline text or string

It is also possible to include data in the form of text blocks directly in a model. A multiline text (or string) must be surrounded by backquotes and an optional marker text. For example, the text data file read by the initial version of our model can be included and read thus (model version burglar2t.mos):

 public declarations
  default_data: text            ! Data input values
 end-declarations

 default_data:=
`--------------- Input data ----------------
           ! Item      Value  Weight
BurgData: [(camera)    [ 15   2]
           (necklace)  [100  20]
           (vase)      [ 90  20]
           (picture)   [ 60  30]
           (tv)        [ 40  40]
           (video)     [ 15  30]
           (chest)     [ 10  60]
           (brick)     [  1  10]]
--------------- Input data ----------------`

 initializations from "text:default_data"
  [VALUE,WEIGHT] as "BurgData"
 end-initializations

And similarly, for redirecting output in Mosel's text format to a text block instead of writing it to a physical file we can use the following Mosel code:

 public declarations
  burglarout: text              ! Solution output
 end-declarations

 initializations to "text:burglarout"
  SOLTAKE
 end-initializations

! Display contents of output file
 writeln("Solution file:\n", burglarout)