flooding-of-alerts-deduplication.json (104.1 KB)
When a large number of identical or related alerts arrive in rapid succession, such as repeated failed login attempts, there is a race condition where multiple workflow executions can begin before the first execution has finished creating a case.
Without additional controls, each workflow execution attempts to look up an existing case before one has been created. As a result, multiple workflows conclude that no case exists and each creates a new case, rather than creating a single case with subsequent alerts appended as case notes.
The approach described here prevents duplicate case creation while minimising any impact on normal alert processing.
In this workflow, the deduplication action is not used to suppress or discard duplicate alerts. Instead, it is configured with Emit Duplicates = True, allowing every alert to continue through the workflow. The deduplication result is simply used to determine whether an alert is the first alert in a flood or a subsequent alert.
The first alert received within the deduplication window is marked as unique and immediately continues through the workflow to create the case.
Any subsequent matching alerts received during that same deduplication window are marked as non-unique. Rather than attempting to create another case, these alerts are delayed for a short period before continuing through the workflow.
The delay duration should match the amount of time required for the first alert to progress from ingestion to successful case creation. In this example, both the deduplication window and the delay are configured to 10 seconds.
When the delayed alerts resume, the initial workflow has already completed case creation. Their case lookup therefore succeeds, allowing them to append their information as case notes instead of creating duplicate cases.
Configuring the deduplication window and delay to match the case creation time keeps the delay to the absolute minimum required. Only alerts arriving as part of the initial flood are delayed, and only long enough for the first workflow to create the case.
Once the deduplication window has expired, any new matching alerts are no longer considered part of the flood. They bypass the delay entirely, immediately locate the existing case, and append their information as case notes.
This approach ensures that:
-
Normal alert processing remains as fast as possible.
-
Only alerts that arrive during an alert flood incur a delay.
-
The delay is only as long as required to allow the initial case to be created.
-
Duplicate case creation is avoided while ensuring every alert is still processed and recorded.