Data Management and Scalability

The following rules are used to identify entities for input and results data:

• Any entity of basic type is included in the schema as an input data entity.
• Any entity of MP type is identified in the schema as a results data entity.

Note Entities are included in the schema only if declared prior to the insightpopulate call.

!@insight.manage input
!@insight.update.afterexecution
state: string

In the example above, the state entity is identified as input data that should be updated at the end of the model. Normal behavior is to ignore any modifications to the input data that the model itself makes during a scenario execution. However this can be overridden by using mosel annotations. Please see the Xpress Insight Mosel Reference for details.

Result data is always deleted when an index set is modified. This is to ensure that result arrays remain consistent after such a modification.

If the input data is modified without the results data being deleted, then the results data will be flagged as dirty by the system until the scenario is executed again. This dirty status can be checked by the API.

A custom view typically checks scenario.isResultsAvailable before deciding whether to do anything with the results data for that scenario. A custom view that is expected to cater to deferred results deletion may also want to check this in conjunction with the scenario.isDirty flag in order to decide whether to visualize the data (and if so, how to indicate that the results are out of date).

Each loaded scenario holds an independent (compressed) copy of the input data, and each executed scenario holds an independent copy of the results. For very large models, performance can be improved by removing from the schema entities that do not strictly need to be persisted. These should be marked in the Mosel annotations as !@insight.manage ignore for the corresponding entity element.

For the input data, this would include input data that does not change from scenario to scenario. Such input data could be loaded from a model resource or a data source during the RUN mode of model execution. The entities in this fixed data should be marked as ignored entities by the companion file.

Note Unmanaged (ignored) entities are not persisted or made available to a client through the server API. This drawback can be remedied by calculating in the model useful aggregated metrics from the fixed data and recording these in the results data.

Depending on the requirements of the application, it may be possible to improve the performance for handling large volumes of results data by persisting calculated metrics rather than the results data in its entirety. It is possible to engineer the model to make the selection between persisting metrics or the full results data based on a runtime parameter that could be set per scenario. The actual Mosel mpvar/linctr entities would be optionally excluded from the results and copied to an equivalent set of basic type entities that are included in the schema as results. If the copy step is skipped, then the empty entities will occupy negligible space for most scenarios.