Insight REST API client

Topics covered in this chapter:

Introduction

The mminsightrestapi package implements a client for making calls to the Insight v5 REST webservice interfaces. You can use mminsightrestapi to create/update/delete apps, folders and scenarios; to read or write entity data, and to perform any other operation that is supported by the Insight REST webservices.

mminsightrestapi can be used both inside and outside of Insight apps. When used within an Insight scenario, there is no relationship between the calls made through mminsightrestapi and the currently running scenario (any reads/writes behave as though they came from outside of the scenario).

The mminsightrestapi subroutines and data structures implement a direct mapping to the Insight v5 REST webservice operations; it's recommended this package is only used where the required operation is not supported by the mminsightscenario package.

Initialization

Before making any calls to the webservices, you must initialize an insightapi2~insightconfig value with the URL of the Insight server, for example:

parameters
  INSIGHT_URL="http://localhost:8080/"
end-parameters
declarations
  cfg: insightapi2~insightconfig
end-declarations
insightapi2~insightconfig_init(cfg, INSIGHT_URL)

Provided that Mosel restrictions are not active, the package will automatically read your Insight REST API credentials from the Windows Credential Manager, or the Mac Keychain, or an encrypted file on Linux - consult the below section Providing REST API credentials for instructions on how to do this.

If you are not able to provide credentials securely using the Credential Manager / Keychain / etc, you can pass them to the insightapi2~insightconfig_init procedure directly, for example:

parameters
  INSIGHT_URL="http://localhost:8080/"
  CLIENT_ID=""
  SECRET=""
end-parameters
declarations
  cfg: insightapi2~insightconfig
end-declarations
insightapi2~insightconfig_init(cfg, INSIGHT_URL, CLIENT_ID, SECRET)

If your code is executing within an Insight 5 app on DMP, you can initialize the insightapi2~insightconfig using a populated dmpresource value, e.g.

declarations
  cfg: insightapi2~insightconfig
  comp: dmpresource
end-declarations

! Populate dmpresource for current Insight component
dmpinitcomp(comp, getdmpcompid, "Xpress Insight", "", getdmplifecycleenv)
if comp.status<>DMP_OK then
  writeln('Error initializing dmpresource: ', comp.lasterror)
  exit(1)
end-if

! Pass component details to insightconfig_init
insightapi2~insightconfig_init(cfg, comp)

Once the insightapi2~insightconfig value is initialized, it should be passed to all requests to the Insight server, e.g.:

declarations
  apps: insightapi2~page_of_app
end-declarations
apps := insightapi2~get_apps(cfg, 0, 1000)

The credentials will not be validated until the first call to an Insight webservice operation - it is not possible for the call to insightapi2~insightconfig_init to fail.

Design Patterns

The mminsightrestapi package is derived directly from the public Insight OpenAPI specification and therefore is not aligned with normal Mosel design patterns in terms of type and function naming and record behaviour. For each operation supported by the Insight webservice interface, the package offers several subroutines that differ in what arguments are required. Each takes an insightapi2~insightconfig value containing details of the Insight webservices to use, and returns the expected return type of a 'successful' call (e.g. insightapi2~scenario in the case of insightapi2~create_scenario). The first subroutine variant takes the operation's input parameters and a insightapi2~httpresponse record into which details of the response will be written; for example to rename a scenario using this pattern:

declarations
  SCENARIO_ID='2926e9a1-9568-4116-84ed-2e75190bc651'
  fields_to_update: insightapi2~scenario
  updated_scenario: insightapi2~scenario
  resp: insightapi2~httpresponse
end-declarations
fields_to_update.name := 'new name'
updated_scenario := insightapi2~update_scenario(cfg, SCENARIO_ID, fields_to_update, resp)
if not resp.success then
  writeln_('ERROR renaming scenario: ',resp.errormsg)
end-if

The second variant omits the insightapi2~httpresponse argument; when called in this way, the model will terminate with an error message if the operation fails, e.g.:

declarations
  SCENARIO_ID='2926e9a1-9568-4116-84ed-2e75190bc651'
  fields_to_update: insightapi2~scenario
  updated_scenario: insightapi2~scenario
end-declarations
fields_to_update.name := 'new name'
updated_scenario := insightapi2~update_scenario(cfg, SCENARIO_ID, fields_to_update)
! Model will automatically terminate on failure

The final variant wraps the input parameters in an insightapi2~httprequest value; this is intended mostly for advanced use cases where you want to build up the parameters programmatically. To rename the scenario using this pattern, you would do this:

declarations
  SCENARIO_ID='2926e9a1-9568-4116-84ed-2e75190bc651'
  fields_to_update: insightapi2~scenario
  updated_scenario: insightapi2~scenario
  resp: insightapi2~httpresponse
  req: insightapi2~httprequest
end-declarations
fields_to_update.name := 'new name'
req.params('id') := SCENARIO_ID
req.body := fields_to_update
updated_scenario := insightapi2~update_scenario(cfg, req, resp)
if not resp.success then
  writeln_('ERROR renaming scenario: ',resp.errormsg)
end-if

When making a call to an operation that returns a binary file, you should set the bodyfilename attribute of the insightapi2~httpresponse record to the path of the file to write to.

The data models of the Insight REST API have been converted into Mosel types which are described in the Data Model Types section of this document. These behave similarly to Mosel records, but have several important differences:

• All the fields are optional — unpopulated fields will return their default values if you try to access them but you can check if a field is present using the has<name> functions (e.g. to see if a insightapi2~attachment named att has a populated id field, call has_id(att)).
• All values stored in the fields are read-only — if a field contains a structured type, you must create that value in a local variable before assigning it to the field.
• You cannot use a record constructor to initialize or create new instances of any of the data model types.

The insightapi2~dictionary type (or a similar type such as insightapi2~dictionary_of_text) is used to represent a collection of named values. There is a standard pattern for accessing dictionaries where you can call insightapi2~get_value to access an individual named value but access a list of the name-value pairs in the entries field (or values if you want the values without the names). To edit the values in a dictionary, you use the insightapi2~put_value procedure.

Internally, requests to the Insight webservices are made through the dmp module and you can use the dmp_max_retries and dmp_retry_error_codes parameters to control how many times and which failed requests should be automatically retried, before returning an error to the calling code.

Providing REST API credentials

Overview

The recommended way to pass the Insight REST API credentials to the mminsightrestapi or mminsightscenario package is using your system's local secure storage. This section describes how to do this for each platform. Before adding your REST API credentials to the secure storage, you must obtain them from Insight for your account — consult the Insight 5 documentation for instructions.

You will not be able to provide credentials in this way if your Mosel will be executing in restricted mode, or on another server, or within an Insight execution worker, or within a role account. Consult the mminsightrestapi or mminsightscenario documentation for alternate ways to pass in credentials, but always remember that the client ID and secret values allow access to your account on the Insight 5 server, so must be handled securely.

Note that for Windows and Mac, this process is identical to providing REST API credentials to Workbench or to the Insight 5 Compute Interface; if you have already configured either, you do not need to repeat these steps again.

Windows

1. Open the Credential Manager, by typing credential manager in the search box on the taskbar and selecting Credential Manager Control Panel.
2. Select Windows Credentials to access the manager.
3. In the Generic Credentials list, click Add a generic credential
4. In the Internet or network address field, enter ficoxpress:<insight URL>, e.g. ficoxpress:http://localhost:8080
5. In the User name field, enter the Client ID value from Insight.
6. In the Secret field, enter the Client Secret value from Insight.
7. Click OK, then close the Credential Manager.

Mac

1. In Applications > Utilities, open the Keychain Access app on your Mac.
2. Select the login keychain in the Keychains list.
3. Select File > New Password Item.
4. In the Keychain Item Name field, enter ficoxpress:<insight URL>, e.g. ficoxpress:http://localhost:8080
5. In the Account Name field, enter the Client ID value from Insight.
6. In the Password field, enter the Client Secret value from Insight.
7. Click Add, then close the Keychain Access application.

Linux

On Linux, openssl tools are used to encrypt a file containing the Client id and Client secret.

To start, choose a location to save a keyfile. Set environment variable XPRESS_CREDENTIALS_DSO_AUTH_KEYFILE to the keyfile location. From the Linux bash shell, enter the following command and press Enter:

XPRESS_CREDENTIALS_DSO_AUTH_KEYFILE = keyfile_path

Note: For security, the <keyfile_path> should be on a different drive to the home directory.

Populate the key file with secure random data. Enter the following command and press Enter:

openssl rand -base64 -out $XPRESS_CREDENTIALS_DSO_AUTH_KEYFILE 48

To encrypt the Client ID and Client secret values into a file named .ficoxpress in your home directory, enter the following command, replacing CLIENTID and SECRET with the actual Client ID and Client secret values from the Insight Server:

(echo "CLIENTID" ; echo "SECRET" ) | openssl enc -aes-256-cbc -salt -md sha256 -out ~/.ficoxpress -pass file:$XPRESS_CREDENTIALS_DSO_AUTH_KEYFILE

In your Insight 5 execution worker configuration, you must configure the XPRESS_CREDENTIALS_DSO_AUTH_KEYFILE environment variable to be the same value used above.

If you wish to store the .ficoxpress file somewhere other than your home directory, you can set the XPRESS_CREDENTIALS_DSO_AUTH_DIR to the path of the alternate location.

Example: Creating a scenario

This example code demonstrates how to create a new scenario within an existing app. It assumes that cfg is an initialized insightapi2~insightconfig value and APP_ID is the ID of the app in which to create the scenario.

declarations
  appref: insightapi2~reference_app
  newscenreq: insightapi2~scenario_creation_request
  newscen: insightapi2~scenario
  resp: insightapi2~httpresponse
end-declarations

appref.id := APP_ID
appref.object_type := 'APP'
newscenreq.parent := appref
newscenreq.name := 'my scenario'

newscen := insightapi2~create_scenario(cfg, newscenreq, resp)
if not resp.success then
  writeln_('ERROR creating scenario: ',resp.errormsg)
  exit(1)
end-if
writeln('Created new scenario with ID: ',newscen.id)

Example: Updating an array

This example code demonstrates how to fetch the an entity 'prices' of type array(regions:set of string,items:set of integer) of real, double each value, and save this change back to the server. It assumes that cfg is an initialized insightapi2~insightconfig value and SCENARIO_ID is the ID of the scenario to modify.

declarations
  query: insightapi2~scenario_data_query
  data: insightapi2~scenario_data
  modifications: insightapi2~scenario_data_modification
  delta: insightapi2~entity_delta
  arrdelta: insightapi2~array_delta
  elem: insightapi2~array_element
  resp: insightapi2~httpresponse

  prices: dynamic array(regions:set of string,items:set of integer) of real
end-declarations

! Fetch array
query.entity_names := ['prices']
data := insightapi2~get_scenario_data(cfg, SCENARIO_ID, query, resp)
if not resp.success then
  writeln_('ERROR downloading "prices" entity: ',resp.errormsg)
  exit(1)
end-if

! Now copy the downloaded 'prices' entity into a local Mosel array.
! Start by finding the 'prices' entity in the scenario_data value
with prices_entity=insightapi2~get_value(data.entities, 'prices') do
  ! The array data will have been fetched in a nested set of dictionaries, the names in each
  ! being the index set values
  ! Iterate through the outer dictionary, which will be indexed by region name
  forall (region_entry in prices_entity.insightapi2~array.array.entries) do
    ! 'region_entry' value will be dictionary from region
    ! name to (dictionary from item ID as string to price value)
    forall (item_entry in region_entry.value.insightapi2~dictionary.entries) do
      ! 'item_entry' value will be value from prices array
      prices( string(region_entry.name), parseint(item_entry.name) ) := item_entry.value.real
    end-do
  end-do
end-do

! Next we update populate the arrdelta structure with an array_element for each element we
! want to update
forall (region in regions, item in items | exists(prices(region,item))) do
  reset(elem)
  elem.key := [region,item]
  elem.value := prices(region,item)*2
  arrdelta.add := arrdelta.add + [elem]
end-do
! Save the arraydelta describing the changes to this array into a scenario_data_modification
! record
delta.entity_name := 'prices'
delta.array_delta := arrdelta
modifications.deltas := [delta]
! Send the modification to the server
insightapi2~modify_scenario_data(cfg, SCENARIO_ID, modifications)
if not resp.success then
  writeln_('ERROR updating "prices" entity: ',resp.errormsg)
  exit(1)
end-if
writeln('Prices have been doubled.')

Be aware that if you are fetching an entity of type integer, the value arriving in Mosel will always be of type real - you will need to cast these to integer if the type is significant.

Example: Downloading an attachment

This example code demonstrates how to download a scenario attachment named 'alldata.xls' to a local file. It assumes that cfg is an initialized insightapi2~insightconfig value and SCENARIO_ID is the ID of the scenario.

declarations
  resp: insightapi2~httpresponse
  attach: insightapi2~attachment
  page: insightapi2~page_of_attachment
end-declarations

! Fetch the list of attachments and look for one with the name we want
page := insightapi2~get_scenario_attachments(cfg, SCENARIO_ID, 0, 1000, resp)
if not resp.success then
  writeln_('ERROR fetching list of scenario attachments: ',resp.errormsg)
  exit(1)
end-if
forall (att in page.content) do
  if att.filename='alldata.xls' then
    attach := att
    break
  end-if
end-do
if not has_id(attach) then
  ! Attachment record never populated
  writeln('Attachment "alldata.xls" not found')
  exit(1)
end-if

! Set the filename into which to save the attachment
resp.bodyfilename := 'alldata.xls'
! Download the attachment
insightapi2~get_scenario_attachment_file(cfg, SCENARIO_ID, attach.id, resp)
if not resp.success then
  writeln_('ERROR downloading attachment: ',resp.errormsg)
  exit(1)
end-if
writeln('Attachment downloaded')

Example: Uploading an attachment

This example code demonstrates how to create a scenario attachment named 'username.txt'. Note we specify the attachment content using a insightapi2~file_data record, specifying a local source filename containing the attachment content that has a different name from the attachment filename in the Insight server - the filename field can be left unpopulated if these are the same. We assume that cfg is an initialized insightapi2~insightconfig value and SCENARIO_ID is the ID of the scenario.

declarations
  public username='william shakespeare'
  resp: insightapi2~httpresponse
  attach: insightapi2~attachment
end-declarations

attach := insightapi2~upload_scenario_attachment(cfg, SCENARIO_ID,
  insightapi2~file_data(.source:='text:username', .filename:='username.txt'), true, "mytag", resp)
if not resp.success then
  writeln_('ERROR uploading attachment: ',resp.errormsg)
  exit(1)
end-if
writeln('Attachment uploaded')