Dataset set postmatch 5576 v9 #11834
Conversation
Will be useful for dataset, when it needs to find a transaction buffer again.
|
ERROR: ERROR: QA failed on build_asan. Pipeline 22812 |
The set operation of dataset keyword was done even if signature did not fully match, which is not the expected behavior. We want dataset to behave like flowbits for instance. This patch changes the behavior of the dataset keyword to do a match and a post match for the set operation. The postmatch retrieves the data, using the list identifier associated to the buffer for this signature. This avoids to store the buffer(s), when we do not have a dedicated storage (per signature and per tx) that can own and clean arbitrary buffers over multiple packets, in the case the transaction spans over multiple packets with different tx progresses for instance. If detection runs on one packet, the InspectionBuffer are cached and fast to get. The most expensive case if for multi buffers, where we need to run detection again, to see which occurences match all payload keywords and should be added in the dataset. Ticket: OISF#5576
catenacyber
force-pushed
the
dataset-set-postmatch-5576-v9
branch
from
e9282b2 to
3eea1e0
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #11834 +/- ##
==========================================
- Coverage 82.60% 82.56% -0.04%
==========================================
Files 912 912
Lines 249351 249413 +62
==========================================
- Hits 205965 205918 -47
- Misses 43386 43495 +109
Flags with carried forward coverage won't be shown. Click here to find out more. |
|
Information: QA ran without warnings. Pipeline 22853 |
|
I would like to avoid rerunning transforms and detection. One alternative way could be to explore having a immediate dataset operation that stores the data in a temporary location (det_ctx for example) and then uses that from post-match. |
Ok...
This is problematic as https://redmine.openinfosecfoundation.org/issues/7197 shows it for flowvar |
|
Next in #11962 |
Link to ticket: https://redmine.openinfosecfoundation.org/issues/
https://redmine.openinfosecfoundation.org/issues/5576
Describe changes:
SV_BRANCH=OISF/suricata-verify#2000
#11719 with rebase after unset operation has been added
So, now Victor, you will ask me for more tests
Right ?