DevOps

How to Add Custom Actions in Mistral

openstack-mini

Introduction

Mistral is a workflow service and part of OpenStack, a free open standard cloud computing platform. Mistral provides APIs and tools to write and execute workflows. Mistral workflows can be written in YAML with help of YAQL or Jinja2 templating. Each workflow consists of a series of tasks. Each such task is performed using actions. Mistral comes with a good set of standard actions also know as system actions.

Need of custom actions

Even though Mistral is bundled with standard actions, these actions are not always enough. Take an example of standard HTTP action known as std.HTTP. This action triggers error-sequence if HTTP response status code not in range of 200 to 307. But in real life not all APIs follow standards making std.HTTP obsolete. To handle such scenarios, Mistral provides capabilities to roll out our own custom actions.

Writing a custom action

Writing custom action is easy. We just need to write a class inherited from mistral.actions.base.Action and implement run method. Lets write a custom action similar to std.HTTP. But this action will trigger error-sequence if only status code not in (200, 201, 409). First clone Mistral repo from github and create a new python packages called plugins in root directory of mistral.

# clone repo
git clone https://github.com/openstack/mistral.git

# create plugins package. you can use any name instead of plugins
cd mistral
mkdir -p plugins/__init__.py

Now create a customactions.py file in this plugins package and add below content in it. Alternatively you can download this file directly from this link. Final directory structure should look like this.

mistral
├── mistral
├── ... 
├── ... 
├── ... 
└── plugins
    ├── __init__.py
    └── customactions.py
from oslo_log import log as logging
import requests
import six
import json as json_lib
from mistral import exceptions as exc
from mistral_lib import actions


LOG = logging.getLogger(__name__)


class HTTPAction(actions.Action):
    def __init__(self,
                 url,
                 method="GET",
                 params=None,
                 body=None,
                 json=None,
                 headers=None,
                 cookies=None,
                 auth=None,
                 timeout=None,
                 allow_redirects=None,
                 proxies=None,
                 verify=None):
        super(HTTPAction, self).__init__()

        if auth and len(auth.split(':')) == 2:
            self.auth = (auth.split(':')[0], auth.split(':')[1])
        else:
            self.auth = auth

        if isinstance(headers, dict):
            for key, val in headers.items():
                if isinstance(val, (six.integer_types, float)):
                    headers[key] = str(val)

        if body and json:
            raise exc.ActionException(
                "Only one of the parameters 'json' and 'body' can be passed"
            )

        self.url = url
        self.method = method
        self.params = params
        self.body = json_lib.dumps(body) if isinstance(body, dict) else body
        self.json = json
        self.headers = headers
        self.cookies = cookies
        self.timeout = timeout
        self.allow_redirects = allow_redirects
        self.proxies = proxies
        self.verify = verify

    def run(self, context):
        LOG.info(
            "Running HTTP action "
            "[url=%s, method=%s, params=%s, body=%s, json=%s,"
            " headers=%s, cookies=%s, auth=%s, timeout=%s,"
            " allow_redirects=%s, proxies=%s, verify=%s]",
            self.url,
            self.method,
            self.params,
            self.body,
            self.json,
            self.headers,
            self.cookies,
            self.auth,
            self.timeout,
            self.allow_redirects,
            self.proxies,
            self.verify
        )

        try:
            url_data = six.moves.urllib.parse.urlsplit(self.url)
            if 'https' == url_data.scheme:
                action_verify = self.verify
            else:
                action_verify = None

            resp = requests.request(
                self.method,
                self.url,
                params=self.params,
                data=self.body,
                json=self.json,
                headers=self.headers,
                cookies=self.cookies,
                auth=self.auth,
                timeout=self.timeout,
                allow_redirects=self.allow_redirects,
                proxies=self.proxies,
                verify=action_verify
            )
        except Exception as e:
            LOG.exception(
                "Failed to send HTTP request for action execution: %s",
                context.execution.action_execution_id
            )
            raise exc.ActionException("Failed to send HTTP request: %s" % e)

        LOG.info(
            "HTTP action response:\n%s\n%s",
            resp.status_code,
            resp.content
        )

        # Represent important resp data as a dictionary.
        try:
            content = resp.json(encoding=resp.encoding)
        except Exception:
            LOG.debug("HTTP action response is not json.")
            content = resp.content
            if content and resp.encoding not in (None, 'utf-8'):
                content = content.decode(resp.encoding).encode('utf-8')

        _result = {
            'content': content,
            'status': resp.status_code,
            'headers': dict(resp.headers.items()),
            'url': resp.url,
            'history': resp.history,
            'encoding': resp.encoding,
            'reason': resp.reason,
            'cookies': dict(resp.cookies.items()),
            'elapsed': resp.elapsed.total_seconds()
        }

        if resp.status_code not in range(200, 201, 409):
            return actions.Result(error=_result)

        return _result

Adding custom action in Mistral

To add this new plugin in Mistral, we have to first publish it in namespace. Open setup.cfg file from root directory of mistral source code. Look for line starting with mistral.actions and append below line in that section.

    custom.http = plugins.customactions:HTTPAction

What exactly we are doing here is, binding our new action with namespace called custom.http. You can use any name instead of this. Remember this name as it will be used in workflows to invoke action. So finally setup.cfg will look similar to this.

mistral.actions =
    std.async_noop = mistral.actions.std_actions:AsyncNoOpAction
    .
    .
    .
    custom.http = plugins.customactions:HTTPAction

Now if you are using Mistral in single node configuration with docker container build it again using below command.

docker-compose -f tools/docker/docker-compose/infrastructure.yaml \
             -f tools/docker/docker-compose/mistral-single-node.yaml \
             -p mistral up -d --build

In case of multi-node configuration, build container with help of below command.

docker-compose -f tools/docker/docker-compose/infrastructure.yaml \
             -f tools/docker/docker-compose/mistral-multi-node.yaml \
             -p mistral up -d -- build

Alternatively, if Mistral is installed in your system under virtualenv, you just need to update DB. If it is not under virtualenv, then first re-install Mistral and then update DB. DB can be updated using below command.

$ mistral-db-manage --config-file <path-to-config> populate

Mostly config file is present at /etc/mistral/mistral.conf .

Validate new custom action

Now it is time to see if our new shiny action is added successfully or not. This can be done with help of mistral client. Install python packages called mistral-pythonclient.

pip install python-mistralclient

Now pull the list of all actions available in Mistral with help of client.

mistral action-list

If custom.HTTP is in output, we are ready to use it.

| ID                                   | Name             | Namespace | Is system | Input                        | Description                  | Tags   | Created at          | Updated at |
+--------------------------------------+------------------+-----------+-----------+------------------------------+------------------------------+--------+---------------------+------------+
.
.
.
| 959cae1b-055e-4d1d-815c-520510fb9ce6 | custom.http      |           | True      | url, method="GET", params... | Custom HTTP action. Defau... | <none> | 2020-03-19 21:08:49 | None       |
.

Hope you like this article ! Stay Awesome !

Leave a Reply