Recording Ansible runs with ARA

IDSTower performs every remote operation on your Suricata hosts with ansible-playbook. ARA (Ansible Run Analysis) is a third party project that records those runs and exposes them through a web interface and a REST API, giving you a searchable history of every cluster setup, rule deployment, configuration change, service operation and health check: the tasks that ran, their duration, and the result on each host. ARA is not part of IDSTower and the ara package is not included in the IDSTower installation or Docker image, this page describes how to add it.

How it works

IDSTower runs ansible-playbook as a child process that inherits the environment of the IDSTower service and only adds the few variables it needs on top, so any variable set on the service reaches every playbook run. Enabling ARA is therefore purely a matter of setting its environment variables, no IDSTower configuration is involved.

IDSTower sets the below variables itself, do not override them:

  • ANSIBLE_STDOUT_CALLBACK=json - IDSTower parses the JSON output of every run, changing it breaks all cluster operations. ARA is a notification callback and records runs alongside this one.

  • ANSIBLE_HOST_KEY_CHECKING=False and ANSIBLE_DEPRECATION_WARNINGS=False

  • ANSIBLE_SSH_CONTROL_PATH and ANSIBLE_SSH_ARGS, and the ansible_ssh_args variable - set for the operations that log in with the SSH credentials you supplied (setting up a cluster, adding hosts, resetting SSH trust), so that each of them checks those credentials. IDSTower passes ansible_ssh_args as an extra variable, which takes precedence over any value in an inventory, group_vars or ansible.cfg.

Every other variable, including ANSIBLE_CALLBACK_PLUGINS and all ARA_* variables, is passed through untouched.

Prerequisite: an ARA API server

Run the ARA API server somewhere reachable from the IDSTower server, for example:

$ docker run -d --name ara -p 8000:8000 \
    -e ARA_ALLOWED_HOSTS='["ara.example.com","127.0.0.1","localhost"]' \
    quay.io/recordsansible/ara-api:latest

The host name IDSTower uses in ARA_API_SERVER must be listed in ARA_ALLOWED_HOSTS, otherwise the ARA server rejects the recordings with Bad Request (400).

Docker

  1. Build a derived image that adds the ARA callback plugin. ara must be installed with the same Python interpreter that runs Ansible (/usr/bin/python3 in the IDSTower image), a virtual environment does not work because the callback plugin is imported by Ansible’s own interpreter, and --break-system-packages is required because the image’s Python is externally managed:

FROM idstower/idstower:<tag>

USER root
RUN apt-get update \
 && apt-get install -y --no-install-recommends python3-pip \
 && rm -rf /var/lib/apt/lists/* \
 && python3 -m pip install --no-cache-dir --break-system-packages ara
# shaped like /usr/local/lib/python3.<minor>/dist-packages/ara/plugins/callback,
# take the real value from step 2 rather than copying one from this page
ENV ANSIBLE_CALLBACK_PLUGINS=<path printed by python3 -m ara.setup.callback_plugins>
USER idstower

  1. Build it and ask the image where the plugin actually landed. python3 -m ara.setup.callback_plugins is the only source for that path, it follows the Python version of the image’s base and moves whenever that base is updated. Put what it prints in the ENV line and build once more, the value has to be baked in because Docker does not evaluate shell substitutions in -e arguments:

$ docker build -t idstower-ara:<tag> .
$ docker run --rm --entrypoint /bin/bash idstower-ara:<tag> -c 'python3 -m ara.setup.callback_plugins'

  1. Run the derived image with the ARA client variables added to the environment you already pass to IDSTower:

$ docker run -d --name idstower -p 80:80 -p 443:443 \
    -e "ARA_API_CLIENT=http" \
    -e "ARA_API_SERVER=http://ara.example.com:8000" \
    ... your usual IDSTower variables ... \
    idstower-ara:<tag>

Packaged installation

  1. Install ara with the same interpreter Ansible uses, which is the one reported by ansible --version, then print the callback plugin path. On distributions whose Python is externally managed, add --break-system-packages to the pip command:

$ ansible --version | grep "python version"
$ sudo python3 -m pip install ara
$ python3 -m ara.setup.callback_plugins

  1. Add the variables to the IDSTower service with a systemd drop-in, using the path the last command in step 1 printed, it depends on the distribution’s Python version:

$ sudo systemctl edit idstower.service

[Service]
Environment="ANSIBLE_CALLBACK_PLUGINS=<path printed by python3 -m ara.setup.callback_plugins>"
Environment="ARA_API_CLIENT=http"
Environment="ARA_API_SERVER=http://ara.example.com:8000"

  1. Reload systemd and restart IDSTower:

$ sudo systemctl daemon-reload
$ sudo systemctl restart idstower.service

If the ARA server requires authentication, add ARA_API_USERNAME and ARA_API_PASSWORD, use ARA_API_INSECURE=true to accept a self signed certificate and ARA_API_TIMEOUT to change the default 30 seconds timeout.

Verifying

Trigger any operation from IDSTower, such as deploying rules or running a health check, then list the recorded runs or browse the ARA web interface at http://ara.example.com:8000:

$ ARA_API_CLIENT=http ARA_API_SERVER=http://ara.example.com:8000 ara playbook list

Notes

  • ARA is a customer side extension, the ara package is not shipped with IDSTower and is not covered by IDSTower support.

  • Upgrading IDSTower to a newer Docker image requires rebuilding the derived image from the new tag.

  • If the ARA server is unreachable, Ansible logs a callback warning and the IDSTower operation still completes normally, the recordings are lost for the duration of the outage.

  • Check what your ARA version stores before pointing it at a production IDSTower. When IDSTower connects to a host that is not registered yet, it hands ansible-playbook the SSH and privilege escalation password on the command line, as --extra-vars "ansible_ssh_pass=..." and --extra-vars "ansible_become_password=...", so anything that records a playbook’s arguments is in a position to record those credentials. Whether they are actually stored is decided by ARA’s own ARA_IGNORED_ARGUMENTS setting, which is ARA’s behaviour and not something IDSTower controls or can promise on its behalf, so check that setting in the documentation for the ARA version you deploy, and inspect one recorded playbook, before you trust it.