-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path_notes.txt
402 lines (282 loc) · 15.3 KB
/
_notes.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
Wiki - Syslog
https://en.wikipedia.org/wiki/Syslog
Image
https://commons.wikimedia.org/wiki/File:Syslog.png
What is Syslog and How it Works
https://sematext.com/glossary/syslog/
https://www.paessler.com/it-explained/syslog
https://www.auvik.com/franklyit/blog/what-is-syslog/
Is Syslog Useful?
https://www.dnsstuff.com/is-syslog-useful
RFC 5424
https://tools.ietf.org/html/rfc5424
6.1. Message Length
Syslog message size limits are dictated by the syslog transport
mapping in use. There is no upper limit per se. Each transport
mapping defines the minimum maximum required message length support,
and the minimum maximum MUST be at least 480 octets in length.
Any transport receiver MUST be able to accept messages of up to and
including 480 octets in length. All transport receiver
implementations SHOULD be able to accept messages of up to and
including 2048 octets in length. Transport receivers MAY receive
messages larger than 2048 octets in length. If a transport receiver
receives a message with a length larger than it supports, the
transport receiver SHOULD truncate the payload. Alternatively, it
MAY discard the message.
Transmission of Syslog Messages over UDP
https://www.rfc-editor.org/rfc/rfc5426.txt
"...When network MTU is not known in advance, the safest assumption is
to restrict messages to 480 octets for IPv4 and 1180 octets for IPv6...."
UDP (om_udp) NXLog
https://docs.nxlog.co/refman/current/om/udp.html
UDP
https://en.wikipedia.org/wiki/User_Datagram_Protocol
Length
This field specifies the length in bytes of the UDP header and UDP data.
The minimum length is 8 bytes, the length of the header.
The field size sets a theoretical limit of 65,535 bytes
(8-byte header + 65,527 bytes of data) for a UDP datagram.
However, the actual limit for the data length, which is imposed by the
underlying IPv4 protocol, is 65,507 bytes
(65,535 bytes − 8-byte UDP header − 20-byte IP header).
Analyze syslog messages
https://blog.datalust.co/seq-input-syslog/
Understanding Syslog Formats: A Comprehensive Guide
https://www.atatus.com/blog/syslog-formats/
Priority (PRIVAL): A numerical value that combines the facility and severity level of the message. It is enclosed in angle brackets (< >).
Timestamp: Indicates the time when the message was generated. It is typically in the format of MMM DD HH:MM:SS (month, day, hour, minute, second).
Hostname: Identifies the origin of the syslog message, indicating the system or device that generated the log.
Tag: Tag or Process ID represents the application or process that generated the message. It is often user-defined.
Message: The actual log information or event description.
<PRIVAL>VERSION TIMESTAMP HOSTNAME APP-NAME PROCID MSGID [STRUCTURED-DATA] MESSAGE
<165>1 2003-10-11T22:14:15.003Z myhostname myapp 1234 ID47 - [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] An application event log entry...
<165>: Priority value, where 165 is the combination of facility (16) and severity (5).
1: Version number of the syslog protocol being used.
2003-10-11T22:14:15.003Z: Timestamp in the format of ISO 8601.
myhostname: Hostname identifying the system or device that generated the log.
myapp: Tag indicating the application or process that generated the message.
1234: Process ID (PID) of the application.
ID47: Message ID.
-: Structured data field is optional and absent in this example.
[exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"]: Structured data providing additional information in a key-value pair format.
An application event log entry...: The actual log message or event description.
Timestamp
A formalized timestamp that denotes the date and time when the event is logged and includes the syslog generation time with the year and milliseconds with respect to the time zone.
The following example shows the date and time format in RFC 5424.
2020-08-13T22:14:15.003Z represents August 13, 2020 at 10:14 PM and 15 seconds, 3 milliseconds into the next second. The timestamp is in UTC. The timestamp provides millisecond resolution.
Note: The suffix "Z", when applied to a time, denotes a Coordinated Universal Time (UTC) offset of 00:00.
time-stamp is the ISO 8601 compatible standard timestamp format (yyyy-mm-ddThh:mm:ss+-ZONE)
RFC5424 syslog Message Format introduction
https://hackmd.io/@njjack/syslogformat
Numerical Severity
code
0 Emergency: system is unusable
1 Alert: action must be taken immediately
2 Critical: critical conditions
3 Error: error conditions
4 Warning: warning conditions
5 Notice: normal but significant condition
6 Informational: informational messages
7 Debug: debug-level messages
syslog messages examples
with no STRUCTURED-DATA
<34>1 2003-10-11T22:14:15.003Z mymachine.example.com su - ID47 - BOM'su root' failed for lonvick on /dev/pts/8
PRI: <34>, Facility: 4, Severity: 2
VERSION: 1
timestamp:2003-10-11T22:14:15.003Z
HOSTNAME: mymachine.example.com.
APP-NAME: su
PROCID: unknown, displayed as NULVALUE -
MSGID: ID47.
MSG is 'su root' failed for lonvick..., encoded in UTF-8.
There is no STRUCTURED-DATA, displayed as NULVALUE -
with STRUCTURED-DATA
<165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID@32473 iut="3" eventSource= "Application" eventID="1011"] BOMAn application event log entry...
with no MSG
<165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID@32473 iut="3" eventSource= "Application" eventID="1011"][examplePriority@32473 class="high"]
Syslog sidecar
https://github.com/g41797/syslogsidecar
THE SYSLOG HELL
https://techblog.bozho.net/the-syslog-hell/#:~:text=RFC5424%20defines%20a%20key-value,define%20their%20own%20message%20formats.
syslog.h
- https://sites.uclouvain.be/SystInfo/usr/include/sys/syslog.h.html
Severity Levels:
#define LOG_EMERG 0 /* system is unusable */
#define LOG_ALERT 1 /* action must be taken immediately */
#define LOG_CRIT 2 /* critical conditions */
#define LOG_ERR 3 /* error conditions */
#define LOG_WARNING 4 /* warning conditions */
#define LOG_NOTICE 5 /* normal but significant condition */
#define LOG_INFO 6 /* informational */
#define LOG_DEBUG 7 /* debug-level messages */
"debug", "info", "notice", "warning", "err", "crit", "alert", "emerg"
Facility codes:
#define LOG_KERN (0<<3) /* kernel messages */
#define LOG_USER (1<<3) /* random user-level messages */
#define LOG_MAIL (2<<3) /* mail system */
#define LOG_DAEMON (3<<3) /* system daemons */
#define LOG_AUTH (4<<3) /* security/authorization messages */
#define LOG_SYSLOG (5<<3) /* messages generated internally by syslogd */
#define LOG_LPR (6<<3) /* line printer subsystem */
#define LOG_NEWS (7<<3) /* network news subsystem */
#define LOG_UUCP (8<<3) /* UUCP subsystem */
#define LOG_CRON (9<<3) /* clock daemon */
#define LOG_AUTHPRIV (10<<3) /* security/authorization messages (private) */
#define LOG_FTP (11<<3) /* ftp daemon */
/* other codes through 15 reserved for system use */
#define LOG_LOCAL0 (16<<3) /* reserved for local use */
#define LOG_LOCAL1 (17<<3) /* reserved for local use */
#define LOG_LOCAL2 (18<<3) /* reserved for local use */
#define LOG_LOCAL3 (19<<3) /* reserved for local use */
#define LOG_LOCAL4 (20<<3) /* reserved for local use */
#define LOG_LOCAL5 (21<<3) /* reserved for local use */
#define LOG_LOCAL6 (22<<3) /* reserved for local use */
#define LOG_LOCAL7 (23<<3) /* reserved for local use */
"kern" 0 kernel messages
"user" 1 random user-level messages
"mail" 2 mail system
"daemon" 3 system daemons
"auth" 4 security/authorization messages
"syslog" 5 messages generated internally by syslogd
"lpr" 6 line printer subsystem
"news" 7 network news subsystem
"uucp" 8 UUCP subsystem
"cron" 9 clock daemon
"authpriv" 10 security/authorization messages (private)
"ftp" 11 ftp daemon
"local0" 16 local use 0
"local1" 17
"local2" 18
"local3" 19
"local4" 20
"local5" 21
"local6" 22
"local7" 23 local use 7
Priority = Facility * 8 + Severity Level
Syslog client
Arduino C++
https://github.com/arcao/Syslog
https://github.com/arduino-libraries/Ethernet/tree/master/src
C
https://code.google.com/archive/p/eventlog-to-syslog/
Go
https://github.com/RackSec/srslog
https://cs.opensource.google/go/go/+/refs/tags/go1.22.6:src/log/syslog/
https://pkg.go.dev/log/syslog
search: https://pkg.go.dev/search?q=syslog
State of datetime handling in Zig?
https://ziggit.dev/t/state-of-datetime-handling-in-zig/2149
Zig Datetime
https://github.com/frmdstryr/zig-datetime
formatISO8601: https://github.com/frmdstryr/zig-datetime/blob/master/src/datetime.zig#L1394
zig-time
https://github.com/nektro/zig-time
Format timestamp into ISO 8601 strings
https://ziggit.dev/t/format-timestamp-into-iso-8601-strings/3824
Timezone-aware datetime - zdt
https://ziggit.dev/t/timezone-aware-datetime-zdt/2618
https://github.com/FObersteiner/zdt
Zig network lib
https://github.com/ikskuh/zig-network
build.zig.zon
https://git.musuka.dev/paoda/zba-gdbstub/src/commit/e5c1d4d2b7ff4c2f1876ca40bd79ba2c06b38a5d/build.zig.zon
build.zig
https://git.musuka.dev/paoda/zba-gdbstub/src/commit/e5c1d4d2b7ff4c2f1876ca40bd79ba2c06b38a5d/build.zig
Zig log
https://github.com/ziglang/zig/blob/master/lib/std/log.zig#L7
Examples of network code
https://github.com/malcolmstill/zig-amqp ++++++++
https://github.com/taskforcesh/bullmq-redis
https://github.com/nmeum/zoap
https://github.com/marler8997/netpunch
https://github.com/marler8997/dbus-zig
https://github.com/karlseguin/smtp_client.zig
https://github.com/karlseguin/websocket.zig
https://github.com/karlseguin/mqttz
Creating UDP server from scratch in Zig
https://blog.reilly.dev/creating-udp-server-from-scratch-in-zig
https://github.com/rofrol/udp-server-from-scratch-in-zig
String Builder / Buffer For Zig
https://github.com/karlseguin/buffer.zig
Unicode in Zig
https://zig.news/dude_the_builder/series/6
Zig Strings
https://www.reddit.com/r/Zig/comments/rxmg9p/zig_strings_in_5_minutes/
https://www.huy.rocks/everyday/01-04-2022-zig-strings-in-5-minutes
https://rguiscard.medium.com/zig-tutorial-2-b0a027c4c652
https://github.com/JakubSzark/zig-string
Get a tarball of any repo directly
wget -O full-path-to-result.tar.gz https://github.com/User/repo/archive/master.tar.gz
and master can be any ref, e.g. a tag.
e.g. wget -O /tmp/sputniik.tar.gz https://github.com/g41797/sputnik/archive/master.tar.gz
zig fetch --save https://github.com/frmdstryr/zig-datetime/archive/master.tar.gz
.....
zig build test -freference-trace --summary all
IDE:
/usr/bin/zig test --color on /home/g41797/CLionProjects/github.com/g41797/syslog/src/root.zig -O Debug
/usr/bin/zig build test --color on $PWD/src/root.zig -O Debug -freference-trace --summary all
Interface
https://github.com/mitchellh/libxev/blob/main/src/main.zig
https://www.openmymind.net/Zig-Interfaces/
How to test several zig files etc
https://github.com/ZystemOS/pluto
Allocators explained
https://dayvster.com/blog/zig-allocators-explained/
zig-msgpack
https://github.com/zigcc/zig-msgpack
Neovim client
https://github.com/jinzhongjia/znvim
Telnet client
https://github.com/michidk/telnet-zig/
Reader&Writer patterns
https://zighelp.org/chapter-2/#readers-and-writers
A tagged union interface
https://codeberg.org/dude_the_builder/zig_in_depth/src/branch/main/35_interface/src/to_string_tagged.zig
Practices
https://www.reddit.com/r/Zig/comments/sanpzf/zig_best_practices_emerging_patterns/
Examples of "linking"
https://github.com/karlseguin/pg.zig/blob/master/build.zig
https://github.com/karlseguin/pg.zig/blob/master/build.zig.zon
https://github.com/karlseguin/log.zig
Simple Scalable Unbounded Queue
https://zig.news/kprotty/simple-scalable-unbounded-queue-34c2
WTF is Build.Zig.Zon and Build.Zig
https://zig.news/edyu/zig-package-manager-wtf-is-zon-2-0110-update-1jo3
https://github.com/edyu/wtf-zig-zon-2
https://github.com/beachglasslabs/zig-duckdb
Zig-DNS
https://github.com/dantecatalfamo/zig-dns
Example of sub-modules
https://github.com/dantecatalfamo/zig-dns/blob/master/.gitmodules
Use git submodule
https://zig.news/fuzhouch/use-git-submodule-local-path-to-manage-dependencies-24ig
Once manually:
git submodule add https://github.com/g41797/mailbox src/deps/mailbox
git submodule add https://github.com/frmdstryr/zig-datetime src/deps/zig-datetime
git submodule add https://github.com/ikskuh/zig-network src/deps/zig-network
Periodically :-(
git submodule update --remote
Mastering project management in Zig
https://blog.orhun.dev/zig-bits-03/
@import() and Packages
https://zig.news/mattnite/import-and-packages-23mb?comments_sort=oldest#toggle-comments-sort-dropdown
Import
https://github.com/rofrol/zig-postgres-template
https://github.com/rofrol/zig-postgres
Heap Memory and Allocators
https://www.openmymind.net/learning_zig/heap_memory/#heap_memory
Zig loops
https://gencmurat.com/en/posts/zig-loops/
Listening ports
sudo netstat --tcp --udp --listening --programs --numeric
Import
zig fetch --save-exact https://github.com/g41797/mailbox/archive/master.tar.gz
zig fetch --save=zig-datetime git+https://github.com/frmdstryr/zig-datetime
zig fetch --save=network git+https://github.com/ikskuh/zig-network
Github MD
https://gist.github.com/nikhilnayyar002/7a35e653d3d590e317c829243e73b110
VSCode configuration
variables
https://code.visualstudio.com/docs/editor/variables-reference
launch.json & tasks.json
https://gist.github.com/amantuladhar/ee210628d55161ba3046b1ac9ff419a1