Dynamic Configurations

Typically when creating integration configurations, most fields will be static; that is, the options for the field will be the same for every tenant. However, it can be useful to include dynamic configurations, where the integration populates the options with values from an API or database, so they are different for each tenant.

To see an example of an integration that benefits from a dynamic config, and what it looks like for users on the Integration Hub or the In-App marketplace, review this article.

An Integration requires three key changes to add dynamic configurations, each of which will be explained in more detail below:

1. The PANDIUM.yaml schema property for a dynamic config needs to be given a reference to a list of options in the schema’s definitions.

2. The PANDIUM.yaml schema needs to have a definitions section with an entry for each of those references.

3. The integration’s code must have an init sync flow which fetches the tenant specific options for each dynamic config’s definition and prints them to the standard out.

Pandium will save the init sync’s standard out, and use it to provide the options on the Connection Settings page for each dynamic config.

Adding a Definitions Reference to a Schema Property

Every configuration needs an entry under schema properties. If a configuration has multiple options, those options can be listed under a definitions entry which is referenced by the property’s $ref.

A configuration with options that are dynamically populated based on an API or database fetch must include the $ref prop in its schema properties entry.

Here are some examples of schema properties with the $ref prop:

 properties:
      log_level:
        $ref: 'definitions/log_level_options'
        default: INFO
        type: string
      mailchimp_audience:
        $ref: '#/definitions/audiences'
        type: string 
      klaviyo_list:
        $ref: '#/definitions/lists' 

Notice that the type prop (usually required for all properties entries) can sometimes be omitted when the $ref prop is provided. This works when the property’s type can be inferred from the definition referenced by $ref.

Adding Schema Definitions

A schema definitions section needs to be added to the PANDIUM.yaml when there are properties that reference definitions. Like the schema properties, definitions is a collection of string to object mappings.

The string is the definition’s name which can be referenced by a property’s $ref.

The object to which each definition entry is mapped specifies a list of options. If the options will be dynamically populated based on an API or database fetch, then the list will just have a placeholder in PANDIUM.yaml. That placeholder will be replaced with tenant specific options once they are fetched during an init sync.

Here is an example of a schema definition for a static config. Notice that the options are specified here in the PANDIUM.yaml because they do not need to be fetched, so the options for this config are the same for every tenant.

   properties:
      log_level:
        $ref: '#/definitions/log_level_options'
        default: INFO
        type: string
    definitions:
      log_level_options:
        enum:
          - INFO
          - DEBUG
          - WARNING
          - ERROR

Static config options like these can also be listed within the property without referencing a definition. To see an example like that, look at the enum code samples of this article (link to PANDIUM.yaml reference article on schema properties, specifically the string section)

Here is an example of a schema definition for a dynamic config. The list of options only has one entry- the placeholder. The actual options will be fetched during an init sync so they can be different for each tenant.

    properties:
      mailchimp_audience:
        $ref: '#/definitions/audiences'
        default: INFO
        type: string
    definitions:
      audiences:
        enum:
          - placeholder

Here is another example of a dynamic config’s schema definition. Notice that the type for these options is oneOf rather than strings in an enum.

    properties:
      klaviyo_list:
        $ref: '#/definitions/lists
        default: INFO
        type: string
    definitions:
      lists:
        oneOf:
          - const: placeholder
            Title: Placeholder

Adding an Init Sync to the Integration Script

Once a schema is prepared to receive dynamic data, the integration script provides that data when there is an init sync that prints a standard out.

In the script, the appropriate run mode can be found by accessing the context environment variable: PAN_CTX_RUN_MODE . If the value of this environment variable is 'init', then the script can add logic to populate the dynamic configs.

When Pandium displays the Connection Settings page for an integration with dynamic configs it will populate the options for those configs using information loaded from the init sync’s last line output to stdout.

Script Example:

if (pandium.context.runMode === 'init') {
    const mcAudiences = await mcClient.getMany('audience');
    const kLists = await kClient.getMany('audience');
    const stdout = {
        // If a key in the standard out matches one of the PANDIUM.yaml's configs.schema.definitions its values will populate the dropdown for configs that reference that definition.
        'audiences': mcAudiences.map(({ name }) => name)
        'lists': kLists.map((list) => {const: list.id, title: list.name})    };
    // Print json string to stdout and end the script.
    console.log(JSON.stringify(stdout));
    return
}

In the above example, a developer has added logic to do the following when PAN_CTX_RUN_MODE is init:

1. Gather a list of Mailchimp audiences and Klaviyo Lists by calling each of their APIs.

2. Put those list and audience options into an object.

3. Print that object to the stdout as a JSON string at the end of the script.

Pandium saves the init sync’s standard out, and uses it to provide the options on the Connection Settings page for the dynamic configurations.

Putting it All Together

Here is what a full PANDIUM.yaml could look like if it had all of the properties from the examples above:

version: 0.4
base: node:20.9.0
build: npm install --production && npm i --save-dev @types/node && npm run build
run: node .
configs:
 schema:
   type: object
   
   definitions:
     log_level_options:
       enum:
         - INFO
         - DEBUG
         - WARNING
         - ERROR  
     audiences:
       enum:
         - placeholder
     lists:
       oneOf:
         - const: placeholder
           title: Placeholder

   properties:
     log_level:
       $ref: '#/definitions/log_level_options'
       default: INFO
       type: string
     mailchimp_audience:
       $ref: '#/definitions/audiences'
       type: string 
     klaviyo_list:
       $ref: '#/definitions/lists'

 uischema:
   type: VerticalLayout
   elements:

   - type: Section
     label: First Section
     elements:
     - type: Control
       label: Log Level
       scope: "#/properties/log_level"
     - type: HorizontalLayout
       elements:
       - type: Control
         label: MailChimp Audience
         scope: "#/properties/mailchimp_audience"
       - type: Control
         label: Klaviyo List
         scope: "#/properties/klaviyo_list"

After the init sync has been run, this .yaml would produce the following connections settings page on the Integration Hub:

To achieve the options in the screenshot above, the last line of stdout from the integration’s init sync would have printed:

[OUT] {"audiences":["Friends","Family","Acquaintances"],"lists":[{"title":"Prospective Customers","const":1},{"title":"One Time Customers","const":2},{"title":"Loyal Customers","const":3}]}

For more detailed instructions on how to implement dynamic configurations review this tutorial.