Alternate JSON implementation adding json-log output format

[blkmajik]

This basically does the same thing as PR #478 does but in what I think is a cleaner way. This implementation keeps all of the JSON output in the one file and just allows you to choose between variations in white space between records and fields.

I can't get my version to compile on master but I think that is due to the ongoing work for ja3/ja4. I have an alternate implementation that is working against 1.7.4.

The gist of the differences between json and json-log formats is that json prints out a human readable version of a JSON array of objects. The "json-log" format is more designed for log processors such as logstash, fluent-bit and things like that. This format presents one record per line where each line is a wholly contained JSON document rather than the whole output being one JSON document.

[phaag]

Thanks for the input on json-log format. Before we continue, I would like to consolidate this with #478 and @markzz. I want to avoid, that we have countless json output formats. Finally all of them need to be maintained.
If we all could agree on your ( @blkmajik ) proposal, I could fit that into master. I am not an expert on the various forms of json formats, and therefore would appreciate your input @blkmajik @markzz. Does #511 work for both of you?

If so I could add that after the master branch gets a bit more stable. As you have noticed, I currently work on ja3/ja4.

Please comment!

[phaag]

@blkmajik - Maybe you could post an example record to make it a bit easier.

[blkmajik]

@blkmajik - Maybe you could post an example record to make it a bit easier.

json-log format is basically:

{key="value",key2="value"}
{key="value",key2="value"}
{key="value",key2="value"}

vs the existing json format of:

[
{
key="value",
key2="value"
},
{
key="value",
key2="value"
},
{
key="value",
key2="value"
}
]

[blkmajik]

I believe that the format that @markzz was proposing is the same as what I have. Looking through the code it looks like markzz just copied the existing output_json.c and stripped the newlines and tabs out of it. His is the same format with an extra space between key/value pairs within each object. I believe his would look more like this:

{ key="value", key2="value" }
{ key="value", key2="value" }

The difference being the irrelevant (to JSON) extra spaces. The key to what we are both looking for is the one complete JSON document per line vs one JSON document per run of nfdump. I believe both formats have value as they address two different consumer types of the data.

My version is a competing implementation for what amounts to be the same output format. I believe my version is more maintainable as there is just one file for JSON output and what you choose is the variations in the "irrelevant" white space for the different output formats.

[phaag]

Approved

[phaag]

Thanks!

[phaag]

@blkmajik

I would like to reference issue #553 and would like to ask, if ndjson would work for you. So I would propose to change the format json-log to that standard:

{key="value",key2="value"}
{key="value",key2="value"}
{key="value",key2="value"}

@blkmajik - Maybe you could post an example record to make it a bit easier.