Skip to content

fix: keep correlation event_type unquoted with pySigma >= 1.5.1 - #9

Merged
wayne-csrs merged 1 commit into
wayne-csrs:mainfrom
cristianchiriac:fix/correlation-ruleid-quoting
Oct 6, 2026
Merged

wayne-csrs merged 1 commit into
wayne-csrs:mainfrom
cristianchiriac:fix/correlation-ruleid-quoting

Conversation

@cristianchiriac

Copy link
Copy Markdown
Contributor

pyproject.toml allows pysigma>=1.0.0, so a fresh pip install pysigma-backend-signals now pulls pySigma 2.0.0. With pySigma 1.5.1 or later, temporal correlations come out like this:

subsearch (( fieldA='value1' AND fieldB='value2' | set event_type="'base_rule_1'" | set field=fieldC ))
subsearch (( fieldA='value3' AND fieldB='value4' | set event_type="'base_rule_2'" | set field=fieldD ))

| temporal window=15min eventtypes=base_rule_1,base_rule_2 by fieldC

event_type is set to 'base_rule_1' with the single quotes included, but eventtypes= refers to base_rule_1, so the two no longer match. test_temporal_correlation_rule_stats_query and test_temporal_ordered_correlation_rule_stats_query fail on pySigma 1.5.1 and 2.0.0. CI doesn't catch it because poetry.lock pins pySigma 1.3.3.

Cause: SigmaHQ/pySigma#534 (released in 1.5.1) passes {ruleid} through the new convert_correlation_ruleid(). That method quotes the id with str_quote (') and only removes the quotes again when the template itself wraps {ruleid} in str_quote. The Signals template uses double quotes (set event_type="{ruleid}"), so the single quotes stay.

Fix: SignalsBackend overrides convert_correlation_ruleid() and strips the surrounding str_quote. Output is the same as before 1.5.1. On older pySigma versions the method is never called, so nothing changes there.

Testing: pytest, 18 passed with each of pySigma 1.3.3 (lock file), 1.5.1 and 2.0.0. Before this change, the two temporal correlation tests fail on 1.5.1 and 2.0.0.

pySigma 1.5.1 added TextQueryBackend.convert_correlation_ruleid(), which
converts the rule id like a string value and wraps it in str_quote ('). The
subsearch template already wraps {ruleid} in double quotes, so multi-rule
correlations produced set event_type="'base_rule_1'" while the temporal
aggregation still referenced eventtypes=base_rule_1, and the two no longer
matched.

Strip the single quotes in an override so the output is the same as with
earlier pySigma versions.
@wayne-csrs
wayne-csrs merged commit e2a325f into wayne-csrs:main Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants