Procedures and functions

HTTP client

The HTTP requests GET, HEAD, POST, PUT, PATCH and DELETE can be sent to a web service using functions httpget, httphead, httppost, httpput, httppatch and httpdel respectively. Each of these functions takes at least two parameters: the URL of the resource and a file name where to store the result of the operation. POST, PUT and PATCH requests require an additional file, namely the data source to be sent to the web service.
HTTP requests are either processed synchronously or asynchronously.

When a request is sent in synchronous mode (the default), the HTTP function call returns after the processing has completed and the return value corresponds to the status of the request (a successful request will have a status value between 200 and 299). The following example uses www.bing.com to search for 'FICO' using a synchronous request:

 status:=httpget("http://www.bing.com/search?q=FICO", "result.html")
 if status div 100=2 then
  writeln("Found FICO!")
 else
  writeln("Request failed with code :", status, " (", httpreason(status), ")")
 end-if

If the asynchronous mode is active (that is, the parameter http_async is set to true) the HTTP functions return just after the request has been sent, without waiting for the reply by the server. The processing continues in a separate thread of execution (up to http_maxasync requests can be handled at the same time) and the function returns a request identifier (or an error code in case of failure during the connection phase). Once the request has completed (i.e. the server has replied) an event of class EVENT_HTTPEND is raised (please refer to the documentation of the module mmjobs for further explanation on how to handle events). The associated value of this event is request_id+status/1000. For instance if request number 1300 succeeded with status 204 ('no data') the corresponding event value is 1300.204. An asynchronous request can be cancelled using httpcancel: in this case an event is still generated but its status is 998.
In the following example, search for 'FICO' is sent to BING, Yahoo and Ask at the same time. A loop is then started to wait for answers from each of the search engines.

 setparam("http_async",true)                ! Switch to asynchronous mode
 reqyahoo:=httpget("http://us.search.yahoo.com/search/?p=FICO", "resyahoo.html")
 writeln("Request ", reqyahoo, " sent to Yahoo")
 reqbing:=httpget("http://www.bing.com/search?q=FICO", "resbing.html")
 writeln("Request ", reqbing, " sent to BING")
 reqask:=httpget("http://uk.ask.com/web?q=FICO", "resask.html")
 writeln("Request ", reqask, " sent to Ask")
 if reqbing<1000 or reqyahoo<1000 or reqask<1000 then
  writeln("A request has failed!")
 else
  nbdone:=0
  repeat
   wait                                      ! Wait for an event
   evt:=getnextevent
   if evt.class = EVENT_HTTPEND then         ! One of the requests completed
    reqnum:=floor(evt.value)                 ! Get request number
    write("Request ", reqnum, " done: ")
    status:=round((evt.value-reqnum)*1000)   ! Get HTTP status
    if status div 100=2 then                 ! 200<=status<300 is success
     writeln("Found FICO!")
    else                                     ! Any other value is an error code
     writeln("Failed with code :", status, " (", httpreason(status), ")")
    end-if
    nbdone+=1
   end-if
  until nbdone=3                             ! Finished when all requests are done
 end-if

By default the module performs direct TCP connections to the servers but a proxy may be specified using the http_proxy and http_proxyport parameters.

It is possible to set a limit on the time spent for connecting to a server by using http_maxcontime. The parameter http_maxreqtime defines a time limit on the entire request (i.e. connection and retrieval of result). mmhttp will wait undefinitely for each request if none of these parameters is defined.

When requests are sent to a secure server (i.e. URL starting with "https://") the trusted certificates file https_cacerts must be available such that authenticity of servers can be verified. If this verification is not required, the control parameter https_trustsrv has to be set to true. If the requested secure server requires client authentication, client certificate https_cltcrt and associated private key https_cltkey must be defined. Note that these parameters are published by mmssl: this module has to be used when a secure requests have to be sent.

HTTP client functions:

HTTP server

The mmhttp module integrates an HTTP server that is started using the procedure httpstartsrv and stopped with httpstopsrv (the server is stopped in any case when the execution of the model terminates). The server behaviour may be changed using these module parameters: http_defport defines the TCP port on which the server is listening (by default a random port is selected); http_defpage indicates which page or label the server has to consider when no path is specified in a request (by default this is "index.html"); http_srvconfig defines the set of request types supported by the model (for instance only GET and POST) as well as whether a secure server is to be started; http_maxreq sets a limit on the number of simultaneous connections that are kept active.

When a secure server (HTTPS) is requested (the server config includes flags HTTP_SSL or HTTP_SSLONLY) besides the optional basic settings similar to those used for the standard server (like https_defport) additional parameters have to be set. The server certificate https_srvcrt as well as its private key https_srvkey are required. Moreover, if the clients are requested to authenticate themselves (server option HTTP_CLTAUTH), the authorised certificate file https_cacerts must include the expected certificates. Note that these parameters are published by mmssl: this module has to be used when a secure server is started.

The server runs in the background and notifies the model of incoming connections through events of class EVENT_HTTPNEW (please refer to the documentation of mmjobs for further explanation on how to handle events). The value associated with this event is a request number: the connection to the client is kept open and the model has to reply to the request in order to complete the operation. Any data associated with the incoming request (query in the case of a GET or data sent via POST, PUT or PATCH) is saved into a temporary file before the event is sent. URL encoded information is automatically decoded and converted to a format compatible with initialisations from blocks. The function httppending may also be used to retrieve the list of requests currently waiting for a reply.

Request properties can be obtained through a set of dedicated routines: httpreqfrom is the IP address of the client; httpreqtype is the request type (i.e. GET, POST, PUT, PATCH or DELETE); httpreqheader is the request header; httpreqstat reports the status associated to a request number (for instance whether it is active, or has associated data); httpreqlabel is the label of the request; httpreqcookies returns the cookies found in the header. The label of a request is its URL after having removed server reference and the query data (for example, the label returned for "http://srv/some/path?a=10" is "some/path"); httpreqfile is the name of the temporary file holding data associated to the request. The connection status of a request might be inspected with httpreqconstat that indicates whether the cleint is still waiting for a reply.

Three different methods can be used to reply to a request: httpreplycode will only return a status code associated with a short (error) message; httpreply takes as input a file to be sent back to the client (with a success status code) and httpreplyjson converts its input parameter into JSON data that is sent back to the client.

The following example shows how to implement a simple file server. This program expects GET (download a file) and PUT (upload a file) requests sent to the port 2533. The URI of the request is interpreted as the file name: for example, the URL "http://srv:2533/myfile.txt" could be used to access file "myfile.txt" stored on host "srv" running this example.

setparam("http_defport", 2533)                ! Set server port (2533)
setparam("http_srvconfig",HTTP_GET+HTTP_PUT)  ! Only GET and PUT requests
setparam("workdir", getparam("tmpdir"))       ! Move to temporary directory
httpstartsrv                                  ! Server now running
repeat
 wait                                         ! Wait for an event
 ev:=getnextevent
 if ev.class=EVENT_HTTPNEW then               ! Request pending
  r:=integer(ev.value)                        ! Get request ID
  fname:=httpreqlabel(r)                      ! File name will be the URI
  if httpreqtype(r)=HTTP_GET then             ! Client wants to get the file
   if bittest(getfstat(fname), SYS_TYP) = SYS_REG then
    httpreply(r,fname)                        ! If available: send it
   else
    httpreplycode(r,404)                      ! Otherwise: reply "Not Found"
   end-if
  elif httpreqtype(r)=HTTP_PUT and            ! Client wants to put a file
       httpreqstat(r)>=2 then                 ! File must be non-empty
   fmove(httpreqfile(r), fname)               ! Try to save it
   if getsysstat=0 then
    httpreplycode(r,204)                      ! If success: reply "No Content"
   else
    httpreplycode(r,403)                      ! Otherwise: reply "Forbidden"
   end-if
  else
    httpreplycode(r,400)                      ! Empty files are refused
  end-if
 end-if
until false

In the above example all requests are handled by the same model but it is also possible to dispatch the processing of the requests to several submodels running concurrently to improve the efficiency of the service. To implement a multithreaded server a queue of requests of type reqqueue (See Section New functionality for the Mosel language) has to be declared as a global shared entity and each of the submodels must be clones of the server model (in order to have access to this shared queue). The master model can then start all of its submodels and initialise the HTTP server as in the preceding example but it can move the HTTP requests it receives to the queue using httpreqpush. The submodels enter a loop starting with a call to httpreqpop (to indicate that they are ready to handle a request) followed by a wait: requests coming from the master model are notified via an event of class EVENT_HTTPNEW and exposed just as in a single-model server. The general structure of the server looks like the following:

parameters
 MASTER=true                  ! Running the master or a worker?
 NBW=5                        ! Number of submodels to start
end-parameters

declarations
 queue: shared reqqueue       ! The shared queue of requests
 procedure run_master
 procedure run_worker
 procedure process_request(req:integer)
end-declarations

if MASTER then
 run_master
else
 run_worker
end-if

! The master model runs the HTTP server
procedure run_master
 declarations
  workers:array(1..NBW) of Model
 end-declarations
 forall(i in 1..NBW) do
  load(workers(i))                    ! Each worker is a clone of the server
  run(workers(i),"MASTER=false")
 end-do
 setparam("http_defport", 2533)                ! Set server port (2533)
 setparam("http_srvconfig",HTTP_GET+HTTP_PUT)  ! Only GET and PUT requests
 setparam("workdir", getparam("tmpdir"))       ! Move to temporary directory
 httpstartsrv                                  ! Server now running
 repeat
  wait                                         ! Wait for an event
  ev:=getnextevent
  if ev.class=EVENT_HTTPNEW then               ! Request pending
   r:=integer(ev.value)                        ! Get request ID
   httpreqpush(r,queue)                        ! Move it to the queue
  end-if
 until false
end-procedure

! Each worker waits for requests sent by the master model and processes them
procedure run_worker
 setparam("workdir", getparam("tmpdir"))       ! Move to temporary directory
 repeat
  httpreqpop(queue)                            ! Model ready
  wait                                         ! Wait for an event
  ev:=getnextevent
  if ev.class=EVENT_HTTPNEW then               ! Request pending
   r:=integer(ev.value)                        ! Get request ID
   process_request(r)                          ! Actual processing
  end-if
 until false
end-procedure

HTTP server functions: