Upgrade Guide from v3.1.x to v3.2.0

This guide explains how to upgrade from IDSTower v3.1.0 or v3.1.1 to v3.2.0, to upgrade please follow the below steps.

Note

What this upgrade changes

  • Database schema: v3.2.0 includes a database schema change. It is applied automatically the first time the upgraded service starts — no action is required. If you prefer to apply it yourself, see Applying the database schema upgrade manually below.

  • Host configuration: every host is flagged for a configuration update, which you run on every cluster right after the upgrade — see step 11. Until it has run on a host, IDSTower does not start, stop or restart that host’s services.

  • Rules: each rule’s action, priority, target and threshold now keep the value the rule text says apart from your own change to it. The upgrade keeps every value you changed, so your sensors keep running the same rules — see Rule-text values and your overrides.

  • Feeds: the first update of each rules, IOC and MISP Feed (exported MISP events) feed after the upgrade downloads and imports it in full, even if its content has not changed. Rules and indicators that v3.1.x failed to store are imported then, and the values and tags IDSTower derives from each rule text are refreshed. MISP API and TAXII feeds keep downloading only what is new since their last update, so indicators v3.1.x lost from them do not come back on their own: to download such a feed again from its initial range, disable it and enable it again. It costs what an update of a changed feed always costs. An indicator feed’s import also sets every Expired indicator the feed still lists back to Enabled, so list yours before you upgrade — see step 3.

  • AWS Connector: review your reject rules before you upgrade — see step 2.

  • Custom roles: changing the Indicators Alerting settings now needs a permission — see step 13.

  • Configuration profiles: unchanged, no profile upgrade is needed.

  • Prerequisite: the IDSTower server needs ansible-core 2.15 or later — see step 1.

  1. Check the Ansible version on your IDSTower server

$ ansible --version

The first line must read ansible [core 2.15 or a later version. With an older version, deployments to hosts that use apt fail at the repository step, with a message naming the installed version. Upgrade ansible-core before you continue; Requirements lists the supported version.

  1. AWS Connector only: review your reject rules before you upgrade

Skip this step if you do not use the AWS Connector. Otherwise do it now, on v3.1.x: every AWS export runs within 15 minutes of the upgraded service starting, and every 15 minutes after that, and an export cannot be paused on its own, so there is no time to review afterwards.

v3.1.x leaves every rule with a reject, rejectsrc, rejectdst or rejectboth action out of the AWS export, and exports a rule whose action you changed to one of them with its original action. v3.2.0 exports both with the reject action: a reject rule on tcp as it is, every other one as drop. The first AWS export after the upgrade therefore makes AWS Network Firewall block traffic it did not block before, and the added rules also count toward the rule group’s capacity.

  1. On the Rules Management page, open the Tags Filter menu and select reject, rejectsrc, rejectdst and rejectboth under Action. v3.1.x stores an action you set for a rule in the same place as the rule’s own action, so the list also holds the rules whose action you changed to one of these.

  2. Decide for each of these rules whether AWS Network Firewall should block its traffic.

  3. Change the action of the rules it should not block, for example to alert, before you continue.

Note

The Action row offers only the actions that some rule text uses. If you set rules to a reject action that no rule text uses, list them with a rules export whose filter is Overriding Rule Action equals that action (Settings → Exports).

  1. Indicator feeds: list your Expired indicators before you upgrade

Skip this step if you use no indicator feeds (IOC, MISP or TAXII). On v3.1.x, enabling, disabling, reviewing or whitelisting an indicator on the Indicators Management page set an expiry date that expired it again within the hour, so your Expired indicators can include false positives you had disabled, put in review or whitelisted. The first update of each IOC and MISP Feed feed after the upgrade imports it in full, and any import, of those or of a MISP API or TAXII feed, sets every Expired indicator the feed lists back to Enabled, so those false positives would alert again. An indicator that is still Disabled stays disabled.

On the Indicators Management page, open the Status Filter menu, tick Expired alone, and keep a list of the indicators it shows that you had disabled, put in review or whitelisted. Step 13 sets their status again after the upgrade.

Note

With Delete Expired Indicators on, the default, the nightly cleanup of v3.1.x has already deleted the indicators that expired before its last run, and a feed that lists one of them adds it again as Enabled. Those can only be found again among the Enabled indicators after the upgrade.

  1. Stop your IDSTower service

$ sudo systemctl stop idstower.service

  1. Take a full backup of your current IDSTower database

$ sudo mysqldump -B [DatabaseName] -u [DatabaseUsername] -p > IDSTower_backup_`date +%F_%H_%M_%S`.sql

Note

Replace [DatabaseName] and [DatabaseUsername] with your actual database name and username. You can find them in your IDSTower configuration file, /opt/idstower/appsettings.json.

  1. Take a backup of your IDSTower directory (including settings)

$ sudo cp -Rp /opt/idstower /opt/idstower_3.1.x_backup_`date +%F_%H_%M_%S`

  1. Upgrade IDSTower to Version 3.2.0.

on CentOS/RHEL/RockyLinux/AlmaLinux/OracleLinux:

$ sudo yum update idstower-3.2.0-1

on Ubuntu/Debian:

$ sudo apt update
$ sudo apt install idstower=3.2.0-1

  1. Start the IDSTower service.

$ sudo systemctl daemon-reload
$ sudo systemctl start idstower.service

  1. Verify IDSTower service has started correctly and database schema has been migrated

$ sudo systemctl status idstower.service

  1. Login to the IDSTower web interface and confirm the version shown is 3.2.0.

Post-Upgrade Steps

11. Update the configuration on every host

Open each cluster in the web interface. A Config Update Required banner at the top of the cluster page shows how many hosts have configuration changes that are not applied yet; after this upgrade, that is every host:

  1. Click Update Config on the banner

  2. Review the list of hosts in the confirmation dialog and click Update Config

  3. Wait for the update to complete on all hosts

The update deploys the v3.2.0 host scripts and writes an identity marker into each Suricata instance’s configuration and log directories. From then on IDSTower uses the marker to tell a renamed instance from a deleted one when it cleans up a host. The update restarts the Suricata instances IDSTower records as running and leaves the ones it records as stopped stopped.

Important

Run the update right after the upgrade, on every host. Until it has run on a host, IDSTower refuses to start, stop or restart that host’s services, and refuses a Suricata or Filebeat upgrade that includes the host.

Do not rename a Suricata instance before the update has run on its host. Without the marker, IDSTower cannot tell the renamed instance from a deleted one: it removes the old instance’s service and configuration, but leaves its log directory in place and names it in the deployment output, so you can remove it by hand.

Warning

Instances added to a Fresh cluster on v3.1.x may be running while IDSTower records them as stopped. On a Fresh cluster, v3.1.x recorded an added instance as stopped, but started every instance of a configuration group whenever it applied the configuration and found the group’s first instance running. v3.2.0 follows each instance’s recorded state, so the first update stops such instances. After the update, check the instances’ states on the cluster page and start the ones that should run. Onboarded (Existing) clusters are not affected: their added instances are recorded as running.

12. Review the rule values the upgrade kept as overrides

The upgrade keeps every stored action, priority, target and threshold that differs from the rule’s own text as your override, so the rules your sensors run do not change (one exception is described in Rule-text values and your overrides). Some of those values were never set by anyone: they can be left over from an older revision of the rule, or from a vendor editing a rule in place. To review them:

  1. On the Rules Management page, open the status filter menu and tick the values under Overridden Filter: it lists the rules that have an override for that value.

  2. To return rules to the values in their rule text, select them, or select all rules in the search result, and use Reset To Default in the bulk action menus.

  3. Each rule’s history (Professional and Enterprise licenses) shows what the upgrade did to it, recorded as System: Upgrade to v3.2.0.

The IDSTower log of the first start also lists the overrides that no user change accounts for, grouped by probable origin.

13. Review your indicator statuses and custom roles

Three fixes in this release change what you may have to put right:

  • Indicators: on v3.1.x, enabling, disabling, reviewing or whitelisting an indicator on the Indicators Management page set an expiry date that expired it again within the hour, and a false positive you had disabled or whitelisted came back as Enabled on the next feed import. v3.2.0 fixes this, but cannot know which statuses were lost, and the first update of each indicator feed after the upgrade sets the Expired indicators it still lists back to Enabled. After each feed’s first update, set the status again of the indicators you listed in step 3, and of any other indicator you had disabled, put in review or whitelisted that is back as Enabled. Indicators that no feed lists any more stay Expired: set those again too.

  • Custom roles: on v3.1.x any signed-in user could change the Indicators Alerting settings, whatever their role. v3.2.0 requires the Update Indicators Alerting Settings permission (IndicatorsAlertingSettings.Update), which the built-in Admin and Detection Engineer roles have and the Analyst and Auditor roles do not. If users of a custom role should keep changing these settings, add the permission to their role under Settings → Users.

  • AWS Network Firewall exports: the rules with a reject action are now exported — see step 2, which you do before the upgrade.

14. Verify cluster status

  1. Navigate to Cluster → Summary

  2. Verify all hosts show as connected and their Suricata services are running

  3. Verify that rules and IOCs are still being deployed correctly

Rule-text values and your overrides

In v3.2.0 each rule’s action, priority, target and threshold has two parts: the value its rule text says, and your override. Sensors and exports apply your override where you set one, and the rule text’s value everywhere else, so a rule you never changed is deployed exactly as its vendor wrote it. The rule editor marks each override with an Overridden badge, and its Use the rule text’s value button removes it.

What the upgrade does to your rules

  • A stored value that differs from the rule text is kept as an override, so deployed rules do not change.

  • The exception is a rule action whose latest change on v3.1.x was the bulk Reset To Default. From now on it follows the rule text, so its deployed action can change. The rule’s history records it, and the log of the first start lists those rule IDs.

One-time changes after the upgrade

  • Every host receives one reformatted rules file: priority, target and threshold are written as the vendor wrote them instead of in IDSTower’s former format. Suricata reads both the same way.

  • Exports change in spelling, not in values, for the same rules: threshold is written as in the rule text, and target: loses the space IDSTower used to add. On the ET Open ruleset this concerns about 18% of the rules (9,462 of 51,764 in one deployment), most of them for target:.

  • In STIX exports those rules get new object ids, because an object’s id is derived from the rule it holds. A consumer that keys on STIX ids sees them as new objects once.

  • The first feed import after the upgrade refreshes the tags IDSTower derives from the rule text for many rules, which can narrow a saved export filter on metadata tags.

Saved export filters keep selecting the same rules, with new names for the fields they use:

  • Rule Action (Applied) and Rule Priority (Applied), formerly the Overriding fields, match your override, otherwise the rule text’s value.

  • Rule Action (Rule Text) and Rule Priority (Rule Text), formerly the Original fields, match the rule text’s own value.

  • The new Rule Action Override and Rule Priority Override fields match your override alone.

Two exceptions: a rule whose last action change was the bulk Reset To Default now follows its text, as above; and the (Rule Text) priority is the priority IDSTower parses from the rule rather than the first priority:N; found anywhere in its text, so a rule whose priority keyword comes before a msg containing priority:1; no longer counts as priority 1.

Thresholds are now checked the way Suricata loads them

  • A stored threshold override that Suricata 8 rejects never loaded on your sensors — for example backoff without a multiplier, or threshold, limit or both without seconds. Where the rule text has a threshold of its own, the upgrade removes such an override, so the rule now loads with the vendor’s threshold, and records the removal in the rule’s history. Where the rule text has none, the override is kept and the log of the first start names the rule’s SID; it can no longer be saved again unchanged.

  • backoff and track by_flow work on Suricata 8 only; Suricata 7 rejects them.

Hosts that use apt

v3.2.0 manages the Elastic and Debian backports apt repositories as deb822 .sources files.

  • Healthy hosts need nothing. On Ubuntu, Debian and other apt hosts, the Elastic .list file or /etc/apt/sources.list block written by v3.1.x becomes /etc/apt/sources.list.d/idstower-elastic-<major>.sources on the next Filebeat install or upgrade. On Debian the backports block becomes 00-idstower-debian-backports.sources on the next Suricata deployment, validation or upgrade, or is dropped when the host defines backports itself.

  • Target hosts get the python3-debian package installed, which the repository step needs. A host restricted to a local mirror needs that package available there.

Warning

A file in /etc/apt/sources.list.d that defines the Elastic repository for the Filebeat major version IDSTower installs is deleted whole, other repositories in the same file included. Files for other Elastic major versions are left alone. Move any other repository out of such a file before you upgrade Filebeat.

Hosts whose apt is already broken — every package operation fails with Conflicting values set for option ... regarding source, typically because v3.1.x duplicated an Elastic line you had edited — are repaired by the next Filebeat install or upgrade on that host. Debian backports conflicts are repaired by the next Suricata upgrade, or by the validation when the host is added to a cluster. Until then, deploying a cluster or upgrading Suricata and Filebeat together still fails there, so upgrade Filebeat on its own first.

On a cluster whose apt hosts are all broken, the link offering a newer Filebeat version can be missing, because it comes from the versions the hosts report and broken apt cannot report any. Either remove the duplicate line by hand (apt-get update names both files in its error) and then upgrade Filebeat, or call the API’s POST /api/Clusters/{clusterId}/upgradeTargetSoftware with the Filebeat version given explicitly in NewTargetSoftwareVersion.

Reverse proxies in front of IDSTower

While IDSTower compiles the rules, a host’s request for them now waits up to 30 seconds for the compilation to finish before IDSTower answers that it is busy and the host retries later. If your hosts reach IDSTower through a reverse proxy, make sure its read timeout is longer than 30 seconds; otherwise the proxy times out first and the hosts log a failed rules update instead.

Applying the database schema upgrade manually

This section is optional — it is only for installations where the database user IDSTower runs as is not permitted to change the schema, or where schema changes must be applied by a DBA. Everyone else should skip it: step 8 applies the schema automatically.

The schema upgrade script for this release is published at:

https://download.idstower.com/packages/3.2.0/sql_scripts/IDSTower_upgrade_database_schema_from_3.1.x_to_3.2.0.idempotent.sql

Apply it against your IDSTower database between step 7 and step 8:

$ sudo mysql -u [DatabaseUsername] -p [DatabaseName] < IDSTower_upgrade_database_schema_from_3.1.x_to_3.2.0.idempotent.sql

The script is idempotent — re-applying it is safe.

Important

After a manual schema upgrade you must still start IDSTower and let it finish.

An IDSTower upgrade has two parts: the schema change above, and a data-migration step that adapts your existing data to the new schema — in this release, separating each rule’s text values from your overrides. The script performs only the first part, and deliberately does not record the new schema version. Your database will still report 3.1.0 until the service has started and completed the data migration — that is expected, and it is what causes the outstanding step to run.

Do not insert the version row into database_version_history by hand: doing so tells IDSTower the upgrade is complete and the data migration will never run.

Once the service has started successfully, you can confirm the upgrade finished with:

$ sudo mysql -u [DatabaseUsername] -p -e "SELECT * FROM [DatabaseName].database_version_history;"

The most recent row should read 3.2.0.

  1. done.