Migrating from Xpress Insight 4 to Xpress Insight 5

Major changes to Insight 5

• Internet Explorer 11 support is replaced by MS Edge support. Custom views should be validated against MS Edge.
• Custom views should use VDL and JS API version 5.0 or later. VDL and JS API version 4.8 is supported in Insight 5.4 to aid migration but is deprecated and will be removed in 5.5.
• Mosel models must be compiled with Mosel 5 or later.
• App models that make custom REST API calls will need to be modified to call the v2 REST API. See the section below for more details.
• Custom views that make custom REST API calls will need to be modified to call the v2 REST API. See the section below for more details.
• Tableau views are now integrated as components in custom VDL views. See the section below for more details.
• The onQueue and onExecute results deletion settings (configured in the cfile) are not used. Data will be replaced when the execution completes.

Minor changes to Insight 5

• Insight 5 apps must declare a minimal companion file.
• System authorities PROJECT* have been renamed to APP*.
  • Custom code that tests against the system authorities assigned to a user must be updated to use the new names.
  • Users must check all assigned authorities are still valid and any custom authorities they have created do not clash.
• Apps that rely on parameters the model does not explicitly declare will need to be modified as such parameters are excluded from the app schema.

Changes to using the REST API

• App models that make custom REST API calls will need to be modified to call the v2 REST API.
• Custom views that make custom REST API calls will need to be modified to call the v2 REST AP.
• Custom views that make custom REST API calls should adopt resolveRestEndpoint to add a prefix to the url path if the Insight server is running in an environment where path-based routing is used. The JavaScript methodinsight.resolveRestEndpoint is documented in the Making REST Requests with the JavaScript API.
• REST clients will need to authenticate with API credentials not username and password. This includes app models making custom REST calls. For more see the Accessing the Xpress Insight Server Using the REST API.
• Common changes required for REST API v2 compatibility are:
  • API base path change from /insightservices to /api.
  • Resource names in paths are plural e.g., /api/scenarios.
  • Requests need to specify the API version in Content-Type and Accept content negotiation headers.
  • The precise http codes returned may have changed.
  • Endpoint that return collections are now paginated.
  • Code which extracts an error message from a response needs to be updated to handle the new error structure with error and optional inner error properties.
  • REST endpoints for import and export behave substantially differently in API v2. Import and export processing is a background task and the response to the initial request indicates that the task has been accepted, not that it has been completed.

Changes to integrating Tableau Views

The changes to Tableau integration that are most significant for the migration of existing Tableau views are:

• Tableau visualisations are now embedded in VDL views as vdl-tableau components.
• Tableau workbooks are not declared in the companion file, they are discovered in the app archive.
• The mirror table dependencies of a Tableau view are defined by the vdl-tableau component, not extracted from the workbook definition.
• All mirroring is on demand. There is no longer the option to configure post-execution mirroring in the companion file.
• Each copy of an app hosted by Insight gets a dedicated database schema and database access credentials (provided to the developer via the Web Client cog menu Mirror Database option for users with DEVELOPER authority). The mirror table prefix is no longer needed to avoid table name collisions and has been removed. The previous prefix could be copy-pasted into each mirror table name to preserve backwards compatibility with existing workbooks if necessary.
• The schema of the system tables in each mirror database has changed, invalidating data source definitions in existing workbooks and data source files.

Changes to the companion file:

Changes to the mirror database schema:

See the section Configuring how Data is Exposed to BI Software for details of the individual table schemas.

Migrating an app with Tableau content to Insight 5

The following sequence describes the recommended way to migrate existing standard Tableau views to Insight 5. It does not cover all possible situations.

Update the companion file (CFile)

With the app open in Workbench, open the CFile and make the following changes:

1. Update the CFile to version 5.0.
2. In the database-mirror section
   1. Remove the following attributes of database-mirror (if they have been set):
      • sync-after-execution
      • include-index-groupings
      • insert-rows-batch-size
      • table-prefix

      Note: In the case where existing Tableau workbooks will be migrated to Insight 5, it is advisable to change the mirror table names to include the table-prefix. This will aid the manual migration of the Tableau workbooks. This is also advisable for server data sources if they have been defined.

   2. Remove the following attributes of mirror-table (if they have been set):
      • dimension
      • sync-after-execution
   3. Rename the colname attribute of mirror-table to alias if it has been set.
   4. Remove any sync-for-tableau-view elements and its contents if set.
3. In the client section.
   1. Comment out the tableau element if it exists (the comment syntax for XML is ( <!-- --> ). We will use this as a reminder of the global filter configuration later.
   2. Comment out any tableau-workbook elements. We will use these as a reminder of the views that need replacing later.
4. If your app source contains the Tableau workbooks, then rename the Tableau source folder to Working and create a new empty Tableau folder. If your app source doesn't yet contain the Tableau workbooks, then create Working and Tableau source folders and copy across the workbooks from the Insight 4 app source to the Working folder. Placing the workbooks in the Working folder rather than the Tableau folder avoids Insight 5 attempting to publish them whilst they are not compatible.
5. Publish the app to the Insight 5 development server.

Author the VDL views to host the Tableau visualizations

Each workbook that was commented out in step 3 of the Update the companion file (CFile) section would contribute its worksheets/dashboards (those that aren't marked as hidden) as app views in Insight 4. In Insight 5, each of those worksheets/dashboards needs a new VDL view added to the app. For example: if the app included 2 workbooks and each workbook was comprised of 3 worksheets then 6 new VDL views would need to be added for the Insight 5 app.

For each Tableau worksheet/dashboard not marked as hidden:

1. Add a new view in Workbench with the New View wizard. The title of the view should be the name of the workbook worksheet/dashboard, as this is what Insight 4 would have labeled the view in the view navigation.
2. Select the Basic View template and finish.
3. Remove the heading on the section.
4. Remove the second row.
5. Add the vdl-tableau component to vdl-column. Set following attributes:
   • workbook to an empty string
   • view to an empty string
   • mirror-tables to a comma separated list of the mirror tables that the Tableau sheet is dependent on.
6. Switch to the code view in View Designer and add the following class to the vdl-tableau component to replicate the way in which the Tableau view occupies the entire Web Client view with no margins in Insight 4.

   <vdl-tableau class=”vdl-tableau-full-page” ..

7. Update the app and republish it to XI5.
8. In the Insight 5 Web Client, open the app, click on the cog menu and click on the Mirror Database link. Record the mirror database credentials information as this will be needed to create the new workbook data sources.

Modify the Tableau workbook data sources

For each Tableau workbook:

1. In the Insight 5 web client, navigate to the VDL view corresponding to the first sheet in the workbook. Place scenarios containing the required data for the view on the shelf. Wait until the data is mirrored and keep the browser page open.

   The unconfigured (no workbook or view attributes set yet) displays the selection_id value. This value will be needed in Tableau Desktop to filter in the scenario data for validation of the sheets.

2. Open the Tableau workbook in Tableau Desktop.

   Tip: You may want to have another copy of the original Insight 4 workbook open to refer to as it will help when creating the joins between the tables.

3. For each of the existing PostgreSQL data sources in the workbook
   1. Create a new data source
      For Embedded Data Sources

      • Create a MySQL data source using the mirror database information obtained earlier.
      • Add the insight_selection table and then the app mirror tables declared in the PostgreSQL data source (the insight_projects, insight_scenarios and insight_security tables are replaced by the insight_selection table in Insight 5).
      • When joining on the insight_selection table, join using the scenario_id.
      • Add a username filter to data source (once you add the username filter to the data source you must also sign into Tableau Server from Tableau Desktop using Server - Sign in, in order to see the data).

      For Server Data Sources (defined in the Insight 5 CFile data-sources section)

      • Create a Server data source and select the correct source from the list. This will have been published to Tableau server by Insight 5 when the app was uploaded.
   2. Compare the existing PostgreSQL data source with the new data source. For each field
      • Rename the field if it differs
      • Change the data type if it differs
      • Add any calculations defined in the PostgreSQL datasource

        Tip: The easiest way of doing this is via the data source left hand side panel. This lists all the fields and gives their type and remote field name.

        

4. Go to each Tableau worksheet in the workbook
   1. Swap the data source with the replacement Insight MySQL data source you created using the "Replace Data Source" option from the Data menu.

      

      

      Note: with Blended data sources any linkages will need to include the selection id and scenario id

   2. Remove the scenario_id filter and replace it with a selection_id filter.

      Note: place selection_id as the first item on the filters. Add any 'Apply to worksheets' and 'Apply to context' settings that were applied to scenario id.

   3. Set the selection_id filter to the value show by the unconfigured vdl-tableau component in step 1 of the Modify the Tableau workbook data sources section.
   4. Check that there are no calculations shown in red.
   5. Check any binnings, data formatting, colouring, size, labels, tooltips, etc.

      Note: This is where a deep understanding of the construction of the worksheet/dashboard is needed.

5. Test the worksheets/dashboards using Tableau Desktop.
6. Once satisfied, close the PostgreSQL data sources, and save and close the workbook.
7. Move the workbook from the Working folder in the app source to the Tableau folder.
8. For each worksheet/dashboard (that isn't hidden) in the workbook, configure the vdl-tableau component in the corresponding VDL view in View Designer
   1. Set the workbook attribute to the workbook name.
   2. Set the view attribute to the workbook sheet name.
   3. Set the global filters attribute to any of the global filters commented out in step 3 Update the companion file (CFile)which are relevant for this sheet.
9. Republish the app to Insight 5.
10. Test that the views corresponding to the workbook, function as expected.