Eliminate Duplicate Logs in RP2 input plugin

[macanudo527]

I'm not sure if this is just an issue I'm having with my current configuration or if this is in the stable version, but I get duplicate logs when processing RP2 input:

INFO: Processing ADA
INFO: Processing ADA
INFO: Processing BETH
INFO: Processing BETH

Is anyone else getting this same bug?

Possible causes and What has been Attempted

I think this is being caused by two references pointing to the input plugin or the logger being instantiated twice? I'm not sure. I tried to make the RP2 plugin singleton and got the same issue.

[eprbell]

Strange, I'm not seeing it. Here's what I get with test_data.ods:

2024-05-21 06:42:52,567/rp2/INFO: Processing B1
2024-05-21 06:42:52,571/rp2/INFO: Processing B2
2024-05-21 06:42:52,576/rp2/INFO: Processing B3
2024-05-21 06:42:52,580/rp2/INFO: Processing B4

[macanudo527]

Thanks. There must be a bug in my current PR. I'll have to iron it out.

[macanudo527]

@eprbell It seems to happen when I use the rp2 input plugin twice, with different descriptors, in the same config.ini file. I guess the rp2 input module is designed to only be used once?

[eprbell]

@eprbell It seems to happen when I use the rp2 input plugin twice, with different descriptors, in the same config.ini file. I guess the rp2 input module is designed to only be used once?

Sorry, just so I understand: do you mean you're using the same data loader plugin twice in the same DaLI .ini config file? If so, that's supported: e.g. if you have multiple Trezor wallets you would have multiple Trezor sections in the config file (with different qualifier to distinguish them). See https://github.com/eprbell/dali-rp2/blob/main/config/test_config.ini.

[macanudo527]

Yeah, it should be supported, but there seems to be a bug for the rp2 data loader where if you use the data loader twice you get duplicate logs for both.

[eprbell]

I'm not seeing it when running RP2 on the DaLI outputs of https://github.com/eprbell/dali-rp2/blob/main/config/test_config.ini. Here is the RP2 output I get:

INFO: Country: us
INFO: Generation Language: en
INFO: Accounting method: fifo
INFO: Configuration file: output/crypto_data.ini
INFO: Input file: output/crypto_data.ods
INFO: Processing BTC
INFO: Generating output for plugin 'rp2.plugin.report.open_positions'
INFO: Plugin 'rp2.plugin.report.open_positions' output: /Users/<snip>/prj/dali-rp2/output/fifo_open_positions.ods
INFO: Generating output for plugin 'rp2.plugin.report.rp2_full_report'
INFO: Plugin 'rp2.plugin.report.rp2_full_report' output: /Users/<snip>/prj/dali-rp2/output/fifo_rp2_full_report.ods
INFO: Generating output for plugin 'rp2.plugin.report.us.tax_report_us'
INFO: Plugin 'rp2.plugin.report.us.tax_report_us' output: /Users/<snip>/prj/dali-rp2/output/fifo_tax_report_us.ods
INFO: Log file: ./log/rp2_2024_05_22_17_17_19_686632.log
INFO: Generated output directory: output/
INFO: Done

Can you attach a small repro file?

[macanudo527]

I'm not sure if I can I think it has something to do with my latest PR. It looks like the CCXT converter is actually creating duplicate logs as well. I'm not sure why that is spilling out into the RP2 input processor, though.

[eprbell]

Can you try it on a freshly checked out tree? That'll help determine if the issue is with your PR or with the rest of the code.

[macanudo527]

OK, it is duplicating the logs on pretty much everything except main. It is very odd problem.

I don't think I can run my pull on the regular branch. I'd have to make up data, but let me see what i can do.

[macanudo527]

There is no duplicate logs on the main branch. It is definitely a bug in the new PR. I'll close this and continue the conversation there.

EDIT: I lied. There are no duplicate logs in ccxt on the main branch, but the RP2 input plugin still generates double logs. Let me put together a simple file that reproduces it.

[macanudo527]

Here are two simple RP2 ODS files and a config.ini. You should be able to unzip in the root and test.

dupe_log_test.zip

[macanudo527]

I have this problem a few other times. I think it is due to the logger being assigned twice to the plugin for some reason, so the logging is handled by two loggers at the same time.