Authenticating

Using s3bucket Properties

The simplest way to initialize the s3bucket with your S3 bucket's URL and access credentials is by directly setting properties of the s3bucket object within the model. For example:

model DirectInitExample
  uses "s3"
  declarations
    mybucket: s3bucket
  end-declarations

  mybucket.url := "https://s3-us-west-2.amazonaws.com/nameofmybucket/"
  mybucket.region := "us-west-2"
  mybucket.keyprefix := "myprefix/"                                          ! Optional
  mybucket.accesskeyid := "JKHGDMNAYGWEnbbsJGDI"
  mybucket.secretkey := "jhdfusJasfui;SVFYSIAVS++siufgsuUISNISOWJ"
  mybucket.sessiontoken := "kHUFGBSUjbfusbuioUHDFSIngudblincxubhxop0szofbv"  ! Optional
  ! mybucket now initialized and can be used
end-model

The Bucket URL, Region, Access Key ID and Secret Key must always be specified. If the credentials you are given include a Session Token (sometimes referred to as a Security Token), then you must also specify this. Giving a Key Prefix is optional.

Note that the S3 credentials will not be verified until you make a request to the S3 service. The values you give for the url, region and keyprefix can be read back out of the s3bucket, but for security reasons the accesskeyid, secretkey and sessiontoken properties may not be read.

If you are using server-side encryption with AWS-managed keys, you can configure this by setting additional fields on the s3bucket object, as follows:

mybucket.sse := "aws:kms"
  mybucket.ssekmskeyid := "keyid"                           ! Replace 'keyid' with ID of yourkey in the AWS key management service
  mybucket.ssecontext := "x=1"                              ! Encryption context; optional

Using a JSON Configuration File

As an alternative to setting credentials in the model, you can specify them in a JSON document, the contents of which which you assign to the s3_buckets parameter. The document should be in the following format:

{
  "<bucket-id>": {
    "url": "<URL of bucket>",
    "region": "<AWS Region>",
    "keyPrefix": "<Key Prefix, optional>",
    "accessKeyId": "<AWS Access Key ID>",
    "secretKey": "<AWS Secret Key>",
    "sessionToken": "<AWS Session Token, optional>"
  }
}

The "<bucket-id>" string is a key that you use to refer to the bucket definition in the JSON and has no other meaning. You can specify multiple buckets in the same JSON file so long as they have different "<bucket-id>" strings, e.g.:

{
  "firstbucket": {
    "url": "https://s3-us-west-2.amazonaws.com/nameofmybucket/",
    "region": "us-west-2",
    "keyPrefix": "myprefix/",
    "accessKeyId": "JKHGDMNAYGWEnbbsJGDI",
    "secretKey": "jhdfusJasfui;SVFYSIAVS++siufgsuUISNISOWJ",
    "sessionToken": "kHUFGBSUjbfusbuioUHDFSIngudblincxubhxop0szofbv"
  },
  "secondbucket": {
    "url": "https://s3-us-east-2.amazonaws.com/nameofmyotherbucket/",
    "region": "us-east-2",
    "accessKeyId": "IHFSUGFOSFUHFJSYIFSG",
    "secretKey": "nusduoUHf;sufbuOFUGSFHRHAFFAbvubddsa=jfb",
    "sessionToken": "UHFSUOFIhfushfglhoFGSiguosnoahusfppgjoUFSUFINM"
  }
}

Then you can initialize an s3bucket in the model with the credentials from the JSON by calling the s3init procedure. For example, if you save the above sample JSON in a file called "buckets.json":

model InitExample
  uses "s3","mmsystem"
  declarations
    public bucketcfg: text
    mybucket: s3bucket
  end-declarations

  ! Load buckets.json into a variable so it can be passed to a parameter
  fcopy("buckets.json","text:bucketcfg")
  setparam("s3_buckets",string(bucketcfg))

  ! Initialize mybucket using the 'firstbucket' set of credentials
  s3init(mybucket, "firstbucket")
  if s3status(mybucket)<>S3_OK then
    writeln("Bucket initialization error: ", s3getlasterror(mybucket))
    exit(1)
  end-if
  ! mybucket now initialized and can be used
end-model

As before, the S3 credentials will not be verified until you make a request to the S3 service.

The s3_buckets parameter is special in that it has a single value shared by all models within the Mosel instance - this means that if, for example, you set it for your master model, then the same value will be used for all submodels that you start in the same Mosel process.

After calling s3init, the only property on the s3bucket that may be modified is keyprefix, which must always start with the value it was given by s3init.

If you are using server-side encryption with AWS-managed keys, you can configure this by setting 3 additional properties on the JSON object: sse should be the string "aws:kms", sseKmsKeyId the identifier of your key stored in the AWS key-management service, and sseContext is the (optional) encryption context string.

Using the DMP Solution Bucket

When using a DMP component such as Xpress Insight, Xpress Workbench or Xpress Executor, in a DMP 2.0+ environment, the Mosel instance will automatically be configured to access an S3 bucket that is shared by all components in the solution. To use this, simply call s3init with the string "solutionData", or the constant S3_DMP_SOLUTIONDATA which evaluates to "solutionData":

model DmpInitExample
  uses "s3"
  declarations
    mybucket: s3bucket
  end-declarations

  ! Initialize mybucket using the 'solutionData' set of credentials
  s3init(mybucket, S3_DMP_SOLUTIONDATA)
  if s3status(mybucket)<>S3_OK then
    writeln("Bucket initialization error: ", s3getlasterror(mybucket))
    exit(1)
  end-if
  ! mybucket now initialized and can be used
end-model

By default you will access the solutionData folder matching your component's current DMP lifecycle stage (design, staging or production). In Xpress Insight only, you can also initialize your s3bucket with the folder of a different lifecycle stage by passing one of the constants S3_DMP_DESIGN, S3_DMP_STAGING and S3_DMP_PRODUCTION, as follows:

model DmpInitExample
  uses "s3"
  declarations
    mybucket: s3bucket
  end-declarations

  ! Initialize mybucket using the 'solutionData' credentials for the 'staging' lifecycle
  s3init(mybucket, S3_DMP_SOLUTIONDATA, S3_DMP_STAGING)
  if s3status(mybucket)<>S3_OK then
    writeln("Bucket initialization error: ", s3getlasterror(mybucket))
    exit(1)
  end-if
  ! mybucket now initialized and can be used
end-model