Add configurable logging system #407

enigbe · 2024-11-20T16:01:23Z

Overview

This PR introduces a flexible logging system for LDK Node by implementing a LogWriter interface that supports writing logs to different destinations.

What this PR does

Introduces a LogWriter interface, allowing Writer variants to handle log output destinations. The supported Writer variants can now:
- Write logs to the filesystem,
- Forward to a log implementer,
- Relay logs to a custom logger.
Exposes LogWriter to bindings.
Test logging to different destinations with:
- In-memory log logger,
- In-memory LogWriter logger.

Related Issue(s)

Expose logger interface #309
Cleaner implementation of closed draft feat: add customizable logging system #401

tnull

Cool, thank you for looking into this! This already looks pretty good, but I have some comments after the first round of review.

bindings/ldk_node.udl

tnull · 2024-11-27T12:38:22Z

...in/ldk-node-android/lib/src/androidTest/kotlin/org/lightningdevkit/ldknode/AndroidLibTest.kt

@@ -28,13 +28,11 @@ class AndroidLibTest {
        config1.storageDirPath = tmpDir1
        config1.listeningAddresses = listOf(listenAddress1)
        config1.network = Network.REGTEST
-        config1.logLevel = LogLevel.TRACE


Should we still set TRACE level, just by the new means?

Yes we should. I created a new CustomLogWriter class via which we can set the log level and also test LogWriter. Ran into some issues with NoPointer/Pointer.NULL when testing this (due to unfamiliarity with Kotlin) and would appreciate any pointers to resolve.

Cargo.toml

src/builder.rs

src/logger.rs

tnull · 2024-11-27T13:37:30Z

tests/common/mod.rs

 }

 /// Simple in-memory mock `log` logger for tests.
+#[derive(Debug)]


Why is this Debug?

NodeBuilder is Debug, and it fields an optional LogWriterConfig. The config object has to be Debug and custom log writer variant needs to implement it for this reason.

tests/common/mod.rs

tests/integration_tests_rust.rs

tnull · 2024-12-09T08:14:56Z

@enigbe Is there any update on this? Please let me know if you're hitting any blockers.

This seems to need a minor rebase by now.

enigbe · 2024-12-10T03:50:34Z

No blockers on this. I'll be pushing updates later today.

tnull · 2024-12-18T09:32:43Z

@enigbe Please let me know if/when this is ready for the next round of review!

enigbe

@tnull I believe this is ready for another review. I have addressed the majority of the concerns you raised in the first pass.

Regarding your concerns about testing the logging to custom loggers, I agree that the necessary refactor would be extensive and could detract from the purpose of this PR. As such, I plan to address these test-related changes in a follow-up PR.

Additionally, I encountered some challenges testing a custom logger in Kotlin and would greatly appreciate your guidance or suggestions.

tnull

Did another round of review, already looks pretty good.

I took a quick look at the Kotlin failures, but also couldn't immediately spot what's up, let me know if you want me to have a closer look though.

Btw, you could consider rebasing on #426 (or on main after it lands) which generally fixes pre-existing CI failures.

bindings/ldk_node.udl

src/config.rs

src/builder.rs

src/logger.rs

tnull · 2025-01-06T13:39:38Z

src/logger.rs

+
+impl LogWriter for Writer {
+	fn log(&self, record: LogRecord) {
+		let log = format!(


Hmm, so format! allocates a string on the heap. As we try to avoid allocations (to reduce heap fragmentation where possible), can we a) only create this string when we're sure we need it to log (i.e, after passing the log level filtering), and b) can we avoid it altogether for the CustomWriter case, and possibly even the LogFacadeWriter case? In the latter case we should be able to give the arguments to the respective macros directly, no?

src/logger.rs

...in/ldk-node-android/lib/src/androidTest/kotlin/org/lightningdevkit/ldknode/AndroidLibTest.kt

src/logger.rs

tnull · 2025-01-17T12:35:46Z

@enigbe Sorry, this needs a rebase now that #426 landed (which however also should fix the pre-existing CI failures, finally)., Let me know if this is ready for another round of review!

tnull · 2025-01-21T13:30:27Z

src/payment/unified_qr.rs

Feel free to drop this commit and rebase cleanly on main.

* Add flexible log writer interface for multiple destinations * Implement filesystem writing capability via FilesystemLogger * Prefix LDK-based objects with 'Ldk' for consistency * Add configuration options for log file path and log level

* Add support for user-provided custom logger to write logs to, allowing users to provide any logger that implements LogWriter * Add test to cover this use case, implementing Log- Writer for the mock, in-memory MockLogger.

This commit includes several improvements aimed at simplifying code, improving readability, and optimizing performance: - Inline enum struct fields to reduce indirection and clutter. - Rename structs to improve clarity and better reflect their purpose. - Enable specific feature flags in dependencies for a more streamlined build. - Eliminate unnecessary data allocations to optimize memory usage. - Correct minor grammatical error in documentation.

- allocates Strings only when necessary with `uniffi` feature, otherwise, keeps LogRecord fileds as lifetime references.

This commit adds a CustomLogWriter class to the kotlin library test, configuring the writer via the exposed node builder and tests the ability to log to the custom writer destination.

…ehavior - Removed trait name in description. - Expanded the description of the `LogWriter` trait to clarify its behavior when the `uniffi` feature is enabled or disabled. - Improved explanation of the trait’s responsibilities, including handling log messages and forwarding to outputs.

- Add a custom Debug implementation for LogWriterConfig to work around NodeBuilder's Debug constraints. - Remove the Debug trait from LogWriter, as it is no longer needed due to the custom Debug impl on LogWriterConfig. - Remove the Debug implementation from Writer for consistency.

- Revert MockLogger configuration and assertions - Remove clutter from setting up LogWriter across all tests

- Remove remnants of TestLogWriter and MockLogger that will be reintroduced in a later PR to add a more structured TestConfig object.

Removes the default_log_* functions that were hitherto public functions used to share private constants. These constants (DEFAULT_LOG_LEVEL and the renamed DEFAULT_LOG_FILE_PATH) have been made public instead.

With this, we avoid cloning the log writer config before calls to setup_logger. The default behaiour is also handled at a single location.

Additionally, clone Arc to custom_log_writer idiomatically.

Replacing the FilesystemLoggerConfig previously used by the NodeBuilder to configure the logging to the filesystem.

…`log` macros

…ew_custom_writer

Removes them from LogWriterConfig and new_custom_writer

enigbe mentioned this pull request Nov 20, 2024

feat: add customizable logging system #401

Closed

7 tasks

G8XSU requested a review from tnull November 21, 2024 18:37

tnull reviewed Nov 27, 2024

View reviewed changes

enigbe force-pushed the 2024-11-configurable-logging-system branch from 1d57cab to d8eb0e1 Compare December 11, 2024 12:18

G8XSU mentioned this pull request Dec 12, 2024

Ldk-Server Milestone 3 lightningdevkit/ldk-server#11

Open

9 tasks

enigbe force-pushed the 2024-11-configurable-logging-system branch 2 times, most recently from 7fa2928 to 283fe85 Compare December 16, 2024 00:32

enigbe force-pushed the 2024-11-configurable-logging-system branch 4 times, most recently from 0660dfa to 6766c60 Compare December 19, 2024 21:54

enigbe commented Dec 20, 2024

View reviewed changes

tnull reviewed Jan 6, 2025

View reviewed changes

tnull reviewed Jan 13, 2025

View reviewed changes

src/logger.rs Outdated Show resolved Hide resolved

enigbe force-pushed the 2024-11-configurable-logging-system branch 2 times, most recently from 322f97f to 3b71223 Compare January 21, 2025 10:01

tnull reviewed Jan 21, 2025

View reviewed changes

src/payment/unified_qr.rs Outdated

Copy link

Collaborator

tnull Jan 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to drop this commit and rebase cleanly on main.

enigbe added 9 commits January 21, 2025 16:00

feat(logger): add & impl LogWriter for Writer

b8e4e95

fix(bindings): remove config log level from kotlin tests

973f74e

feat: forward logs to log facade

ba8ab5d

test(logwriter): forward logs to mock in-memory log logger

263de29

fix: correct "Forwards" spelling error

c675fd7

feat(custom): forward logs to a custom logger

0f6bd79

* Add support for user-provided custom logger to write logs to, allowing users to provide any logger that implements LogWriter * Add test to cover this use case, implementing Log- Writer for the mock, in-memory MockLogger.

fix(test): fix setting log's global logger twice

5bbdb17

chore: revert the renaming of LogLevel to LdkLevel

a5c2bac

enigbe added 23 commits January 21, 2025 16:00

feat: address costly log record re-allocation

940ecc8

- allocates Strings only when necessary with `uniffi` feature, otherwise, keeps LogRecord fileds as lifetime references.

test(kotlin-android): set custom log writer via builder method

d1d5f54

This commit adds a CustomLogWriter class to the kotlin library test, configuring the writer via the exposed node builder and tests the ability to log to the custom writer destination.

test: clean up MockLogger and LogWriter setup

1e812e5

- Revert MockLogger configuration and assertions - Remove clutter from setting up LogWriter across all tests

test(kotlin-jvm): set custom logging via builder method

8aad671

test: setup node with filesystem logger at gossip log level

a215c25

- Remove remnants of TestLogWriter and MockLogger that will be reintroduced in a later PR to add a more structured TestConfig object.

chore: drop file path clone when writing to file writer

7feaf8c

chore: group log-related objects in bindings

abb604f

refactor: remove default_log_* functions

e63cc1d

Removes the default_log_* functions that were hitherto public functions used to share private constants. These constants (DEFAULT_LOG_LEVEL and the renamed DEFAULT_LOG_FILE_PATH) have been made public instead.

docs: provide link to the logging facade crate

0f55727

refactor: setup logger to accept ref to optional log writer config

02e8b41

With this, we avoid cloning the log writer config before calls to setup_logger. The default behaiour is also handled at a single location.

refactor: clean up match arm in setup_logger

84a7dde

Additionally, clone Arc to custom_log_writer idiomatically.

docs: simplify log_level description in FilesystemLoggerConfig

03319d5

refactor: set filesystem logger with optional file_path & log_level

4852c63

Replacing the FilesystemLoggerConfig previously used by the NodeBuilder to configure the logging to the filesystem.

refactor: remove Send + Sync from CustomWriter

ee7d3a6

refactor: optimize log string alloc & pass record fields directly to …

c63fb09

…`log` macros

refactor: change Result return type for infallible new_log_facade & n…

731916d

…ew_custom_writer

chore: rename Record as LdkRecord

143e396

refactor: remove Android CustomLogWriter

6df3e95

chore: remove leftover Send + Sync with LogWriter

1656072

Removes them from LogWriterConfig and new_custom_writer

fix: allocation of default LogWriterConfig to reference

a61455a

enigbe force-pushed the 2024-11-configurable-logging-system branch from 3b71223 to a61455a Compare January 21, 2025 15:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add configurable logging system #407

Add configurable logging system #407

enigbe commented Nov 20, 2024

tnull left a comment

tnull Nov 27, 2024

enigbe Dec 20, 2024 •

edited

Loading

tnull Nov 27, 2024

enigbe Dec 17, 2024 •

edited

Loading

tnull commented Dec 9, 2024

enigbe commented Dec 10, 2024

tnull commented Dec 18, 2024

enigbe left a comment

tnull left a comment

tnull Jan 6, 2025

tnull commented Jan 17, 2025

tnull Jan 21, 2025

Add configurable logging system #407

Are you sure you want to change the base?

Add configurable logging system #407

Conversation

enigbe commented Nov 20, 2024

Overview

What this PR does

Related Issue(s)

tnull left a comment

Choose a reason for hiding this comment

tnull Nov 27, 2024

Choose a reason for hiding this comment

enigbe Dec 20, 2024 • edited Loading

Choose a reason for hiding this comment

tnull Nov 27, 2024

Choose a reason for hiding this comment

enigbe Dec 17, 2024 • edited Loading

Choose a reason for hiding this comment

tnull commented Dec 9, 2024

enigbe commented Dec 10, 2024

tnull commented Dec 18, 2024

enigbe left a comment

Choose a reason for hiding this comment

tnull left a comment

Choose a reason for hiding this comment

tnull Jan 6, 2025

Choose a reason for hiding this comment

tnull commented Jan 17, 2025

tnull Jan 21, 2025

Choose a reason for hiding this comment

enigbe Dec 20, 2024 •

edited

Loading

enigbe Dec 17, 2024 •

edited

Loading