Added check and cleanup for stale Suricata PID file upon service startup

[jasper-vdhoven]

An issue that I have run into is that at times the Suricata service will produce the following message in suricata.log: mpm-ac: Just ran out of space in the queue. Please file a bug report on this. At this point the service will wait for a bit and attempt to restart the Suricata process inside the container. However, as Suricata has run before it leaves behind a (now) stale PID file under /usr/local/var/run/suricata/suricata.pid which will cause Suricata to instantly exit:

[32 - Suricata-Main] 2024-11-07 10:39:44 Error: pidfile: pid file '/usr/local/var/run/suricata/suricata.pid' exists but appears stale. Make sure Suricata is not running and then remove /usr/local/var/run/suricata/suricata.pid. Aborting! 

This will then repeat a few times until the container is killed for not starting, and the cycle repeats indefinitely. A potential fix I came up with during debugging, is that by removing the stale PID file Suricata will then correctly start.

From my own testing this appears to fix the startup issue with the Suricata service itself. I have run into a separate issue where the service doesn't appear to load any rules, but I have not been able to deduct whether this is due to my fix or a broader issues with our setup. The relevant error produced by the Suricata service is as follows:

{"@timestamp": "2024-11-18 13:45:58,043", "event": { "module": "assemblyline", "dataset": "assemblyline.service.devsuricata" }, "host": { "ip": "x.x.x.x", "hostname": "807845ce0fd8" }, "log": { "level": "ERROR", "l
ogger": "assemblyline.service.devsuricata" }, "process": { "pid": "1" }, "message": "Error occurred while updating signatures: RetryError[<Future at 0x7f1a9385bbd0 state=finished returned bool>]. Reverting to the former signature set."}
{"@timestamp": "2024-11-18 13:45:58,043", "event": { "module": "assemblyline", "dataset": "assemblyline.service.devsuricata" }, "host": { "ip": "x.x.x.x", "hostname": "807845ce0fd8" }, "log": { "level": "WARNING", 
"logger": "assemblyline.service.devsuricata" }, "process": { "pid": "1" }, "message": "No valid suricata ruleset found. Suricata will run without rules..."}

The name devsuricata is the name of the self-built Suricata service using the standard Suricata service files + my own fix applied to suricata_.py.

Feedback, improvements or critique are welcome!

[cccs-rs]

Thanks for the fix! I believe this may address CybercentreCanada/assemblyline#217 🚀