Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add example script for loading historical data #9

Merged
merged 9 commits into from
Oct 16, 2024
Merged

Conversation

defanator
Copy link
Owner

@defanator defanator commented Oct 15, 2024

For the record, here's the 1st iteration of fetching 11 months data without filtering duplicate datapoints:

2023-11/export.out:2024-10-15 17:22:44,481 [INFO] export_timeseries.py: ========[ 36 metrics, 1407174 values, oldest @ 1698929928 2023-11-02 16:58:48, newest @ 1701374340 2023-11-30 23:59:00 ]========
2023-12/export.out:2024-10-15 17:25:42,472 [INFO] export_timeseries.py: ========[ 29 metrics, 1494495 values, oldest @ 1701374401 2023-12-01 00:00:01, newest @ 1704052740 2023-12-31 23:59:00 ]========
2024-01/export.out:2024-10-15 17:27:00,738 [INFO] export_timeseries.py: ========[ 30 metrics, 1261557 values, oldest @ 1704052800 2024-01-01 00:00:00, newest @ 1706731141 2024-01-31 23:59:01 ]========
2024-02/export.out:2024-10-15 17:28:54,672 [INFO] export_timeseries.py: ========[ 30 metrics, 834969 values, oldest @ 1706731200 2024-02-01 00:00:00, newest @ 1709236741 2024-02-29 23:59:01 ]========
2024-03/export.out:2024-10-15 17:30:14,342 [INFO] export_timeseries.py: ========[ 30 metrics, 1214853 values, oldest @ 1709236800 2024-03-01 00:00:00, newest @ 1711915140 2024-03-31 23:59:00 ]========
2024-04/export.out:2024-10-15 17:31:16,346 [INFO] export_timeseries.py: ========[ 29 metrics, 1437306 values, oldest @ 1711915200 2024-04-01 00:00:00, newest @ 1714507141 2024-04-30 23:59:01 ]========
2024-05/export.out:2024-10-15 17:32:32,869 [INFO] export_timeseries.py: ========[ 29 metrics, 1483783 values, oldest @ 1714507200 2024-05-01 00:00:00, newest @ 1717185541 2024-05-31 23:59:01 ]========
2024-06/export.out:2024-10-15 17:33:25,777 [INFO] export_timeseries.py: ========[ 31 metrics, 532012 values, oldest @ 1717185600 2024-06-01 00:00:00, newest @ 1719777540 2024-06-30 23:59:00 ]========
2024-07/export.out:2024-10-15 17:35:04,258 [INFO] export_timeseries.py: ========[ 31 metrics, 486524 values, oldest @ 1719777600 2024-07-01 00:00:00, newest @ 1722455940 2024-07-31 23:59:00 ]========
2024-08/export.out:2024-10-15 17:36:11,679 [INFO] export_timeseries.py: ========[ 30 metrics, 490345 values, oldest @ 1722456000 2024-08-01 00:00:00, newest @ 1725134340 2024-08-31 23:59:00 ]========
2024-09/export.out:2024-10-15 17:36:59,481 [INFO] export_timeseries.py: ========[ 30 metrics, 481736 values, oldest @ 1725134400 2024-09-01 00:00:00, newest @ 1727726340 2024-09-30 23:59:00 ]========

vs filtered sets:

2023-11/export.out:2024-10-15 19:26:30,850 [INFO] export_timeseries.py: ========[ 36 metrics, 460609 values, oldest @ 1698929928 2023-11-02 16:58:48, newest @ 1701374340 2023-11-30 23:59:00 ]========
2023-12/export.out:2024-10-15 19:27:53,125 [INFO] export_timeseries.py: ========[ 29 metrics, 508032 values, oldest @ 1701374401 2023-12-01 00:00:01, newest @ 1704052740 2023-12-31 23:59:00 ]========
2024-01/export.out:2024-10-15 19:28:26,852 [INFO] export_timeseries.py: ========[ 30 metrics, 502793 values, oldest @ 1704052800 2024-01-01 00:00:00, newest @ 1706731141 2024-01-31 23:59:01 ]========
2024-02/export.out:2024-10-15 19:29:22,498 [INFO] export_timeseries.py: ========[ 30 metrics, 473055 values, oldest @ 1706731200 2024-02-01 00:00:00, newest @ 1709236741 2024-02-29 23:59:01 ]========
2024-03/export.out:2024-10-15 19:30:32,888 [INFO] export_timeseries.py: ========[ 30 metrics, 501935 values, oldest @ 1709236800 2024-03-01 00:00:00, newest @ 1711915140 2024-03-31 23:59:00 ]========
2024-04/export.out:2024-10-15 19:31:29,204 [INFO] export_timeseries.py: ========[ 29 metrics, 483872 values, oldest @ 1711915200 2024-04-01 00:00:00, newest @ 1714507141 2024-04-30 23:59:01 ]========
2024-05/export.out:2024-10-15 19:32:00,079 [INFO] export_timeseries.py: ========[ 29 metrics, 498261 values, oldest @ 1714507200 2024-05-01 00:00:00, newest @ 1717185541 2024-05-31 23:59:01 ]========
2024-06/export.out:2024-10-15 19:32:36,189 [INFO] export_timeseries.py: ========[ 31 metrics, 486113 values, oldest @ 1717185600 2024-06-01 00:00:00, newest @ 1719777540 2024-06-30 23:59:00 ]========
2024-07/export.out:2024-10-15 19:32:56,900 [INFO] export_timeseries.py: ========[ 31 metrics, 486422 values, oldest @ 1719777600 2024-07-01 00:00:00, newest @ 1722455940 2024-07-31 23:59:00 ]========
2024-08/export.out:2024-10-15 19:33:49,521 [INFO] export_timeseries.py: ========[ 30 metrics, 490328 values, oldest @ 1722456000 2024-08-01 00:00:00, newest @ 1725134340 2024-08-31 23:59:00 ]========
2024-09/export.out:2024-10-15 19:34:14,550 [INFO] export_timeseries.py: ========[ 30 metrics, 481736 values, oldest @ 1725134400 2024-09-01 00:00:00, newest @ 1727726340 2024-09-30 23:59:00 ]========

Closes #8.

@defanator defanator merged commit 8f73c71 into main Oct 16, 2024
6 checks passed
@defanator defanator deleted the load_data_examples branch October 16, 2024 08:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tests and example(s) for load_data API
2 participants