Introduction

This module implements functionality for exchanging data between a Mosel model and Python 3 (CPython) and for calling Python 3 scripts.

The python3 module defines an I/O driver for exchanging data using the initializations from and initializations to Mosel constructs.

It is the Mosel run-time library that loads and runs the Python interpreter, not the other way round.

The purpose of the module is to make the extensive scientific and numeric capabilities of Python available from Mosel. This module does not implement an interactive Python shell. However, the interaction of the Mosel model and the Python interpreter is similar to an interactive shell: writing to the I/O driver and executing Python scripts changes the state of the interpreter.

Prerequisites

This module does not include Python binaries. In order to use Python you need a working installation of Python, version 3.4.3 or newer and targeting the same platform as Mosel (you won't be able to use, e.g., the Windows 32-bit version of Python from the Windows 64-bit version of Mosel). The most recent supported Python version is 3.7.x. It is recommended to download and install the Anaconda Python distribution from www.anaconda.com. Alternatively, Python binaries for Windows and Mac OS are also available at www.python.org. On Linux, Python 3 is most likely part of your distribution and provided in a package called "python3" that can be installed via the package manager. Note that Python 3 is not part of the Red Hat Enterprise Linux 7 standard repository. In order to use the latest Python release on Linux, we recommend to download the latest Anaconda Python distribution for Linux from the Anaconda website.

The Mosel module python3 tries to automatically locate the correct Python libraries on your system, applying the following rules. If the environment variable PYTHONHOME is specified, it will load the libraries of the Python installation in that directory. Otherwise, it searches for the Python executable in the directories specified in the PATH environment variable. If the Python executable has been found, the module will try to load the libraries of the Python installation of that Python executable. If the libraries could not be loaded with the help of the environment variables, then on Windows they are loaded from the latest Python installation specified in the registry (keys: HKEY_CURRENT_USER and HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\3.*\InstallPath). If the previous steps failed, then the python3 module will load the most recent libraries from the standard library search paths. On Windows it looks for python3*.dll and python3.dll, on Linux for libpython3.*m.so and libpython3.so, and on Mac OSX for libpython3.*m.dylib and libpython3.dylib.

If you have multiple installations of Python, or if Python could not be located automatically, or if the initialization of Python fails, you will need to set the environment variable PYTHONHOME to point to your Python installation directory or set PATH to include the location of the Python executable.

Note that the loading of Python is not influenced by Mosel statements like setparam("workdir",...) or setenv("PYTHONHOME",...) in the Mosel model that uses the Python module since these don't affect the process environment used for Python loading. The environment variables must be set before launching the Mosel instance that serves for executing Python scripts in order to influence the loading of Python.

As an example, if Python 3.7.0 is installed on Windows in "C:\opt\python370" then this directory is also the correct value for the PYTHONHOME environment variable or alternatively, add this directory to the PATH environment variable.

Python initialization

The Python environment is automatically initialized at the point where a Mosel model uses for the first time any function that requires it. So we can have the following small example that just executes a Python script:

model "Python script example"
  uses "python3"
  pyrun("my-python-script.py")
end-model

Alternatively it is possible to explicitly initialize Python using the pyinit function. If the initialization fails, then try setting the PYTHONHOME environment variable to the path of your Python installation.

At the end of the model execution, the Python environment will automatically be released. It is also possible to explicitly release the environment using pyunload. This can be useful for resetting the interpreter state.

It is only possible to initialize one Python interpreter per Mosel instance. For that reason it is not possible to initialize and use the Python interpreter in two models in parallel if both models are run in the same Mosel instance. However, you can initialize and use multiple interpreters in concurrent models if each model is run in a separate Mosel instance.

Data types

The types of data that can be exchanged with Python are the four Mosel elementary types boolean, integer, real and string, plus arrays, lists and sets of these. Nested compositions are supported. Both static and dynamic Mosel arrays are supported and mapped to dictionaries of the element type. Mosel lists and sets are exported to Python lists and sets.

There is no direct mapping to other Python types such as NumPy arrays, however these can be exchanged after conversion from and to supported types. For example, a Mosel array will be exported to a Python dictionary, which can be transformed to a NumPy array in Python:

  declarations
    I, J: range
    A, A_inverse: array(I, J) of real
  end-declarations

  writeln('Run Python script that defines invert_matrix function.')
  pyrun('invert_matrix.py')

  I := 0..2
  J := 0..2
  A :: [1,0,3,
        0,1,2,
        0,0,1]

  initializations to PY_IO_GLOBAL_VAR
    I
    J
    A
  end-initializations

  writeln("Invert matrix with NumPy.")
  pyexec('A_inverse = invert_matrix(A, I, J)')

In the Python environment, the exported matrix A and its index sets I and J are available as global variables A, I and J. In the pyexec procedure call, those variables are passed as input parameters to the invert_matrix Python function. This function is defined in the example script invert_matrix.py and has been imported by the pyrun procedure call. It transforms the Python dictionary to a NumPy array, before inverting the matrix:

  # Convert dictionary to NumPy array.
  np_a = np.array([[dictionary[i, j] for j in indices2] for i in indices1])

See the model invert-matrix.mos for a full example. Note that sparse Mosel arrays are exported to sparse Python dictionaries. In this example, the Mosel matrix A is dense, hence the Python dictionary is also dense, that is, for each key tuple (i, j) in the cross product of I and J the dictionary returns a value.

For the opposite operation, that is, initializing a Mosel array from a NumPy array, it is necessary to transform the NumPy array to a Python dictionary:

  # Convert NumPy array to dictionary.
  a_inverse = {(i, j): np_a_inverse[i, j] for j in indices2 for i in indices1}

where np_a_inverse is a NumPy array and a_inverse the dictionary, which is returned by the function invert_matrix. That returned dictionary is stored in the global Python variable A_inverse and can be imported to a Mosel array with:

  initializations from PY_IO_GLOBAL_VAR
    A_inverse
  end-initializations

Note that the initializations from drivers are additive, which means that the elements of the Python dictionary are added to the existing Mosel array. If the Python dictionary is dense as in the example above, then all elements of the Mosel array will be overwritten. However, if the Python dictionary is sparse and the Mosel array is non-zero, it is necessary to manually clear its contents before initializing it with values from the sparse dictionary. In that situation, the Mosel array should be declared as dynamic and be cleared with delcell. See Section Driver python for an example with a dynamic array and delcell: io_example.mos.