Lua xform/v21 #12425
Lua xform/v21 #12425
Conversation
This commit makes the detection engine thread context available for transforms to use. The Lua transform requires this value. Issue: 2290
Issue: 2290 This commit extends the hash table logic with an alternate free function that provides the detection engine context. Users that wish to use the next functionality must use the HashListTableInitWithCtx function when initializing the hash table. Using this interface will result in the hash table "free with context" function (new) being used instead.
Issue: 2290 Defer freeing the keyword hash table until the engine context has been freed. This eliminates a double-free from occurring. For the unittests ONLY, clear the keyword_hash to prevent a double free attempt.
Adds a new lua script capability to use a script as a buffer transform keyword. It uses a `transform` lua function that returns the input buffer after modifying it. Issue: 2290
Issue: 2290
git grep -A 1 -w InspectionBufferSetup shows numbers cases of the pattern:
- InspectionBufferSetup
- InspectionBufferApplyTransforms
Refactor the implementations of those functions into
InspectionBufferSetupAndApplyTransforms to reduce function call count.
Issuer: 2290
So transforms can access them through DetectEngineThreadCtx
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #12425 +/- ##
==========================================
- Coverage 80.63% 80.62% -0.01%
==========================================
Files 918 919 +1
Lines 258696 258900 +204
==========================================
+ Hits 208598 208746 +148
- Misses 50098 50154 +56
Flags with carried forward coverage won't be shown. Click here to find out more. |
|
Information: QA ran without warnings. Pipeline 24263 |
| int count = 0; | ||
| char *saveptr = NULL; | ||
| char *token = strtok_r(lua->copystr, ",", &saveptr); | ||
| while (token != NULL && count < LUAXFORM_MAX_ARGS) { |
There was a problem hiding this comment.
@jlucovsky is there a point in having a list of predefined keys that have a specific meaning, like bytes and offset in some of your examples? We'll have to decide now to avoid collisions with scripts that may be in use in the future.
There was a problem hiding this comment.
@victorjulien Although the examples show bytes and offsets, there are no predefined options. The current logic accepts up to 10 comma-separated values and passes those in args to the script.
| @@ -984,7 +986,7 @@ static void DetectBufferTypeFreeFunc(void *data) | |||
| static int DetectBufferTypeInit(void) | |||
| { | |||
| BUG_ON(g_buffer_type_hash); | |||
| g_buffer_type_hash = HashListTableInit(256, DetectBufferTypeHashNameFunc, | |||
| g_buffer_type_hash = HashListTableInitWithCtx(256, DetectBufferTypeHashNameFunc, | |||
There was a problem hiding this comment.
is this never freed?
There was a problem hiding this comment.
I don't think it is. I checked master and I'm not seeing the free happen by the time exit is called
$ git diff
diff --git a/src/util-hashlist.c b/src/util-hashlist.c
index 085a988afe..0b340ea606 100644
--- a/src/util-hashlist.c
+++ b/src/util-hashlist.c
@@ -108,7 +108,9 @@ void HashListTableFree(HashListTable *ht)
if (ht->array != NULL)
SCFree(ht->array);
- SCFree(ht);
+ void *ptr = ht;
+ memset(ht, 0, sizeof(*ht));
+ SCFree(ptr);
}
int HashListTableAdd(HashListTable *ht, void *data, uint16_t datalen)
And via gdb
Thread 1 "Suricata-Main" hit Breakpoint 1, __GI_exit (status=status@entry=0) at ./stdlib/exit.c:137
warning: 137 ./stdlib/exit.c: No such file or directory
(gdb) p *g_buffer_type_hash
$1 = {array = 0x55555636a240, listhead = 0x555556295de0, listtail = 0x5555563e8680, array_size = 256, Hash = 0x5555557b3680 <DetectBufferTypeHashNameFunc>,
Compare = 0x5555557b3700 <DetectBufferTypeCompareNameFunc>, Free = 0x5555557b3770 <DetectBufferTypeFreeFunc>}
If it was freed, the dump from gdb should've shown 0's.
Am I missing something?
#12251, with following changes/updates:
SV_BRANCH=OISF/suricata-verify#2240