Using the Tuner

For a given optimization problem, setting suitable control parameters frequently results in improved performance such as solution time reduction. The Xpress Optimizer built–in tuner can help a user to identify such set of control settings that allows the Xpress Optimizer or Xpress SLP to solve problems faster than by using defaults.

Basic Usage

With a loaded problem, the tuner can be started by simply calling TUNE from the console, or XPRStune from a user application. The tuner will then search for better control settings from a list of controls (called the tuner method). To achieve this, the tuner will solve the problem with its default baseline control settings and then solve the problem multiple times with each individual control and certain combinations of these controls.

As the tuner works by solving a problem mutiple times, it is important and recommended to set time limits. Setting MAXTIME will limit the effort spent on each individual solve and setting TUNERMAXTIME will limit the overall effort of the tuner.

The tuner works with LP and MIP problems. It automatically determines the problem type by examining the characteristics of the current problem. It is possible to tune a MIP problem as an LP or vice versa by passing the flag l or g to XPRStune or TUNE.

The tuner can also work with SLP and MISLP problems when Xpress Nonlinear is available. Note that for SLP or MISLP problems, the time limit is set with XSLP_MAXTIME.

The Tuner Method

A tuner method consists of a list of controls for the tuner to try with. It is possible to run the tuner with different pre–defined lists of controls, so–called factory tuner methods, or with a user–defined list of controls. When using the tuner, it will automatically choose a default factory tuner method according to the problem type. A non–default factory tuner method can be selected by setting the TUNERMETHOD control. There are several choices available for factory tuner methods, among them:

• A simple MIP method, which only features a few controls and can be used in situations where tuning with the default method would take too long, e.g., because the instance to be tuned takes a long time for each individual solve
• A comprehensive MIP method, which features a larger list of controls (and control settings) and can be used when individual instance solves are relatively fast or the default method could not reveal a satisfying improvement
• A root–focus method, which only considers controls that affect the root node processing of the MIP solve. It can either be used when root and tree behavior should be tuned independently, in a two stage process, or when it is evident that improvements have to come from root node processing. When tuning with a root–focus, it might make sense to choose minimizing the primal dual integral as a tuner target.
• A tree–focus method, which only considers controls that affect the tree search behavior of the MIP solve. It can either be used when root and tree behavior should be tuned independently, in a two stage process, or when it is evident that improvements have to come from the tree search, e.g., because the dual bound needs better branching.
• A method for tuning primal heuristics, which should be used when finding a better MIP solutions is the only focus and improving the best bound can be neglected. In this case, it might make sense to choose improvement of the primal bound also as a tuner target.

Please also refer to the documentation of the TUNERMETHOD control.

A tuner method can be written out using XPRStunerwritemethod. This function will create a file in XTM format, that is effectively a list of Xpress Optimizer controls, each with a set of possible settings to try in tuning. When writing out one of the factory methods, it is recommended to first select the tuner method by setting TUNERMETHOD, or to load a targeting problem, so that the tuner can write out suitable tuner methods for the respective problem types.

Users can provide their own method to the tuner by setting up an XTM file (or editing one that has been written out). This can be read into the tuner with XPRStunerreadmethod. An alternate way to load a user–defined tuner method is to set the TUNERMETHODFILE control to the file name. This will only work when no tuner method has been loaded by explicitly calling XPRStunerreadmethod. If a user–defined method is successfully loaded, the tuner will use it and not load any factory tuner method.

Please refer to Appendix The Tuner Method (.xtm) File for the format of tuner method file.

The Tuner Output

While the tuner examines various control settings, it prints a progress report to the console. At the same time, it writes out the result and individual logs to the file system.

On the console, the tuner will print a one–line summary for each finished run. When a new better control setting is identified, it will be highlighted with an asterisk (*) at the begining of its log line, and followed by details of the control setting and its log file name. The console progress logging can be switched off by disabling the OUTPUTLOG control. Please refer to Appendix The Tuner Log for a more detailed description of the tuner logging.

In the background, the tuner will output the result and individual logs to the file system. By default, all the output files will be stored in the directory tuneroutput/probname/. The root folder path can be changed by setting the TUNEROUTPUTPATH control. This is the central folder in which all subfolders for the results and logs of different problems will be stored. The subfolders themselves are automatically named using the current problem name. They can be manually given a different name by setting the TUNERSESSIONNAME control. The subfolder contains one result file in XML format, and many log files, one for each evaluated control setting. The XML result file consists of the control settings, solution results and pointers to the log files of all finished tuner runs.

The file output can be turned off completely by disabling the TUNEROUTPUT control.

The Tuner Target

A tuner target defines how to compare two finished runs with different control settings.

A common usage of the tuner is to pursue a solution time reduction, where two runs will be compared by their solution time, the faster one is considered the better. However, when both of the runs time out, it will be more meaningful to compare other attributes of the two runs, for example the final gap or the best integer solution for MIP problems.

The tuner will choose a default tuner target according to problem types. For instance, comparing the time firstly and then the gap is the default tuner target for MIP problems. A user can select a different target by setting the TUNERTARGET control. Please refer to the documentation of TUNERTARGET for a list of supported tuner targets.

Restarting the Tuner

When tuning the same problem again, the tuner will attempt to pick up results from previous tuner runs so that it can avoid testing with the same control settings again. For this, it checks whether an XML result file is available in the directory tuneroutput/probname/, see Section The Tuner Output. Reusing of the history results even works when a user changes the baseline settings or uses a different tuner method. In this case, the tuner will only pick up history results which match the new control combinations. By default, when a new control setting is evaluated, the result will be appended to the existing result file from the previous tuner session.

This feature of reusing and appending to previous results can be switched off by setting the TUNERHISTORY control. This control has the default value 2, which allows both, reusing and appending. Setting it to 1 will switch off reusing of the results, while still allowing to append new result to the XML result file. Setting it to 0 will switch off appending as well; consequently, the old result file will be overwritten. Note that all log files from previous tuner session will always be kept even if they run with identical settings. This is realized by having a time stamp and a unique number in the file name. Log files can only be removed manually.

Tuner with Multiple Threads

The tuner can work in parallel, i.e., it can run several evaluations of different control settings simultaneously. When setting TUNERTHREADS larger than 1, the tuner will start in parallel mode with the given number of threads. Setting the tuner threads won't affect the number of threads used by each individual run. However, it is natural that, when solving different control settings in parallel, each of the runs may slow down.

When using the parallel tuner, it is worth considering to set the THREADS control as well; ideally such that the product of THREADS and TUNERTHREADS is at most the number of system threads.

Tuner with Problem Permutations

For a certain problem, there may exist several "lucky" controls, that show a better performance by coincidence and not due to structural reasons. Such lucky controls will typically not work with other problems of the same type, or when a user modifies the problem slightly or updates Xpress. They can be thought of as false positives of tuning.

To address this issue, the tuner can exploit a phenomenon known as performance variability and solve the problem with multiple random permutations. When setting TUNERPERMUTE to a positive number, for each control setting, the tuner will solve the original problem and the corresponding number of permuted problems and finally aggregate their results as one. Generally, tuner results with permutations are expected to be more stable.

Tuning a Set of Problems

The tuner can tune a set of problems to search for an overall best control setting for all the problems in the set. Tuning a problem set can be started from the optimizer console with the command

tune probset problem.set,

where the problem.set is a plain text file which contains a list of problem files in MPS or LP format.

The tuner starts by checking all the problems defined in the problem set file. It will read in each problem to find out its type (one of LP, MIP, SLP and MISLP) and optimization direction. When there are mixed problem types, the tuner will quit with a warning message. The tuner can work with mixed optimization directions and it will treat the whole problem set as a minimization problem. For a given problem set, it is possible to force the tuner to tune the problem set as LP or MIP problems with the command

tune lpset problem.set or tune mipset problem.set

respectively.

For a problem set, the tuner works by solving each individual problem in the set for each specific combination of control settings separately. When all the problems in the set are solved for a specific control setting, the tuner combines the individual problem results into a consolidated one and reports it on the console. During the solve, for each problem in the set, the tuner will output its result and log files to a path defined by TUNEROUTPUTPATH/PROBLEMNAME. For the main problem set, the tuner will write the consolidated results to the main output path, together with a concatenated copy of all the individual problem logs.

When tuning a problem set again, the tuner can pick up the result of existing runs for the main problem set and for each separate problem in the set as well. If the full problem set can be recovered from the existing tuning records, the tuner will omit solving them as usual. Otherwise, the tuner will go through all the problems in the set. For each problem in the set, the tuner will also check whether it is possible to pick up an existing result with the specific control setting and omit solving for existing ones when possible.

Advanced Topics

Besides explicitly calling TUNE or XPRStune, the tuner can also be started by enabling the TUNERMODE control. When enabling this control (setting to 1), all the optimization such as XPRSmipoptimize or XPRSlpoptimize will be carried out as a tuned optimization. The Optimizer will first use the tuner to find the best setting and then apply the best setting to solve the problem. On the other hand, a user can disable this control (setting to 0) to always disable the tuner, such that a call to XPRStune will have no effect. This TUNERMODE has a default value of -1, which won't affect the behaviour of any of the above mentioned functions.

When using the tuner from a user application with callbacks, the callbacks will also be passed on to each individual runs. A user needs to keep in mind that these callbacks may be called mutiple times from the tuner, as the tuner will solve the problem mutiples times. Moreover, when using the parallel tuner, it is the user's responsibility to ensure that callbacks are thread–safe.

Though the tuner is built–in with the Xpress Optimizer, it can tune nonlinear problems when Xpress Nonlinear is available. Currently, parallel tuning and permutations will be disabled in this case.