Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish_L2 / RID LUT Manager #385

Merged
merged 8 commits into from
Mar 22, 2024
Merged

Publish_L2 / RID LUT Manager #385

merged 8 commits into from
Mar 22, 2024

Conversation

nicHoch
Copy link
Collaborator

@nicHoch nicHoch commented Mar 13, 2024

Preparations for publishing l2 products

also

use of a singelton RidLutManager that pulls in data from the platform api and persiets it into a local csv file

@nicHoch nicHoch requested a review from samaloney March 13, 2024 14:12
@nicHoch
Copy link
Collaborator Author

nicHoch commented Mar 13, 2024

end2end is looking good complains about the added header keyword in all fits files

@codecov-commenter
Copy link

codecov-commenter commented Mar 13, 2024

Codecov Report

Attention: Patch coverage is 42.17391% with 133 lines in your changes are missing coverage. Please review.

Project coverage is 75.97%. Comparing base (96fedb6) to head (0a071aa).

Files Patch % Lines
stixcore/processing/find.py 0.00% 74 Missing ⚠️
stixcore/io/RidLutManager.py 58.33% 35 Missing ⚠️
stixcore/products/level0/scienceL0.py 60.00% 12 Missing ⚠️
stixcore/processing/pipeline.py 70.00% 3 Missing ⚠️
stixcore/processing/pipeline_status.py 0.00% 3 Missing ⚠️
stixcore/products/level2/housekeepingL2.py 0.00% 2 Missing ⚠️
stixcore/io/fits/processors.py 66.66% 1 Missing ⚠️
stixcore/processing/LBtoL0.py 50.00% 1 Missing ⚠️
stixcore/processing/pipeline_cli.py 0.00% 1 Missing ⚠️
stixcore/products/product.py 83.33% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #385      +/-   ##
==========================================
- Coverage   77.10%   75.97%   -1.13%     
==========================================
  Files          66       68       +2     
  Lines        6906     7069     +163     
==========================================
+ Hits         5325     5371      +46     
- Misses       1581     1698     +117     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@samaloney samaloney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this depend on #382 as that would need to be merged first if so?

Looks ok to me few small queries. What the find.py thing for?

stixcore/io/RidLutManager.py Outdated Show resolved Hide resolved
stixcore/io/RidLutManager.py Show resolved Hide resolved
stixcore/io/RidLutManager.py Outdated Show resolved Hide resolved
stixcore/io/fits/processors.py Show resolved Hide resolved
stixcore/version_conf.py Show resolved Hide resolved
stixcore/io/RidLutManager.py Show resolved Hide resolved
@nicHoch
Copy link
Collaborator Author

nicHoch commented Mar 20, 2024

the find script is a helper CLI tool that scans the given fits folder for files with certain properties. it is not used within the pipeline but will help us to find dedicated fits files for reprocessing/review ...

not many properties implemented so fare but some. Level, version, product_type, OBT_BEG/END ...

Comment on lines +74 to 77
class ScienceProduct(GenericProduct, EnergyChannelsMixin, FitsHeaderMixin):
"""Generic science data product class composed of control and data."""
def __init__(self, *, service_type, service_subtype, ssid, control, data, **kwargs):
"""Create a generic science data product composed of control and data.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@samaloney here the L0 files getting the mixing

Comment on lines +535 to +536
prod.add_additional_header_keywords(additional_header_keywords)
return prod
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@samaloney here a list of keywords is added
there is also a prod.add_additional_header_keyword() if you would like to add a single one
but most likely as the product is created as last step you have to store the keyword in a list during processing

Comment on lines +620 to 621
additional_header_keywords.append(("TRIG_SCA", factor, 'used trigger descale factor'))
triggers, triggers_var = unscale_triggers(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@samaloney here i keep track of one additional keyword

@samaloney samaloney merged commit f5b77dc into i4Ds:master Mar 22, 2024
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants