Add 2nd set of Datacite field methods #181

ehanson8 · 2024-05-29T15:56:26Z

Purpose and background context

Continue refactoring Datacite transform to use field methods.

How can a reviewer manually see the effects of these changes?

Run the following command to see that the Datacite transform still transforms a source file:

pipenv run transform -i tests/fixtures/datacite/datacite_records.xml -o output/datacite-transformed-records.json -s jpal

Includes new or updated dependencies?

NO

Changes expectations for external applications?

NO

What are the relevant tickets?

https://mitlibraries.atlassian.net/browse/TIMX-284

Developer

All new ENV is documented in README
All new ENV has been added to staging and production environments
All related Jira tickets are linked in commit message(s)
Stakeholder approval has been confirmed (or is not needed)

Code Reviewer(s)

The commit message is clear and follows our guidelines (not just this PR message)
There are appropriate tests covering any new functionality
The provided documentation is sufficient for understanding any new functionality introduced
Any manual tests have been performed and verified
New dependencies are appropriate or there were no changes

Why these changes are being introduced: * Continue refactoring Datacite to use field methods How this addresses that need: * Add field methods and associated private methods for dates, edition, file_formats, format, funding_information, identifiers, languages, links, and locations * Update related_items code block to generate related_identifiers list that was moved into get_identifiers * Add unit tests for new field methods Side effects of this change: * None Relevant ticket(s): * https://mitlibraries.atlassian.net/browse/TIMX-284

ehanson8 · 2024-05-29T15:57:45Z

tests/sources/xml/test_datacite.py

+    assert Datacite.get_file_formats(source_record) is None
+
+
+def test_get_format_success():


The usual tests were deliberately omitted given the method's guaranteed output

tests/sources/xml/test_datacite.py

ghukill

Overall, looking real good. Left a couple of comments, but nothing major. I'm finding the file, each time, easier and easier to scan.

tests/sources/xml/test_datacite.py

transmogrifier/sources/xml/datacite.py

ghukill · 2024-05-29T19:09:49Z

transmogrifier/sources/xml/datacite.py

+            )
+
+    @classmethod
+    def _get_dates(cls, source_record: Tag) -> Iterator[timdex.Date]:


Forgive me if we've already discussed this, but I keep tripping on this private method _get_dates() and it's relationship to get_dates(). What about a rename to something like _get_other_dates()? I realize that "other" isn't very exact, but I think it definitively differentiates from the higher level field method get_dates().

What about get _get_datacite_dates which I think is more accurate? I could also update _get_contributors to _get_datacite_contributors since that's what originally prompted the conversation

Ah thanks for the reminder, I thought it had come up.

I don't mean to be difficult, but even adding datacite feels odd... like we know it's datacite already. But I don't really have a better solution.

I suppose one unique property is that we're setting the date kind dynamically by the attribute dateType; wonder if that could come into play naming wise? Maybe something like _get_dates_by_datetype_attribute()? It's long... but it tells you precisely what's going on?

Or we could focus on the element used, like _get_dates_by_date_element()? or _parse_dates_from_date_elements().

Open to pushback here. If folks are okay with _get_dates() and _get_contributors() I could be too.

I like _get_dates_by_date_element, I think I'll go with that

transmogrifier/sources/xml/datacite.py

* Rename methods for clarity

ehanson8 · 2024-05-30T14:25:53Z

@ghukill @jonavellecuerdo Pushed a new commit based on Graham's comments

ghukill

I dig it! Thanks for the changes. At this stage, they feel really understandable. Maybe in "phase 2" we take another pass at naming, once we've seen a lot, but appreciate the changes now.

jonavellecuerdo

One small change request and a question for both of you (@ghukill and @ehanson8).

transmogrifier/sources/xml/datacite.py

jonavellecuerdo · 2024-05-30T17:19:47Z

transmogrifier/sources/xml/datacite.py

+    @classmethod
+    def get_languages(cls, source_record: Tag) -> list[str] | None:
+        languages = []
+        if language := source_record.metadata.find("language", string=True):


[Non-blocking] When do we include the "_element" suffix to a variable?

Some cases, we just use the field name:

get_funding_information

get_languages

get_edition

Other cases, we include the suffix "_element":

get_identifier

_get_alternate_identifier

This is something I've personally been getting tripped up on in the transforms I've been working on --naming conventions assigned to find/find_all method calls. These differences don't cause any problems but consistency would be nice...

@ghukill @ehanson8

I was using the _element convention where element attributes are accessed in addition to element.string so it probably should be used for get_funding_information as well. To me, it doesn't feel as necessary when we're just grabbing element.string. Interested in consistency as well so I want to hear everyone's thoughts

I see the thought process there @ehanson8, and I agree that consistency would be nice, but it doesn't feel like a deal breaker to me with some variation. Given how common the convention is to a) find an element, and then b) use that element variable to extract a string or an attribute value, I think the _element could get pretty chatty.

It might be worth an issue? Then we could keep moving through this refactor, but make a global pass at some point.

In summary: I'm personally okay with variations, given different times in codebae, but an issue to revist could be nice.

Thanks for sharing! I'll apply that thought process as I work on other transforms, @ehanson8 .

Good call, created an issue

ehanson8 requested review from ghukill and jonavellecuerdo May 29, 2024 15:56

ehanson8 commented May 29, 2024

View reviewed changes

tests/sources/xml/test_datacite.py Show resolved Hide resolved

ghukill reviewed May 29, 2024

View reviewed changes

This was referenced May 29, 2024

Evaluate class vs instance methods #183

Open

Evaluate logging for usefulness #180

Open

Create global list dedupe function to run after source record processing #184

Closed

Updates based on discussion in PR #181

bb3613b

* Rename methods for clarity

ghukill self-requested a review May 30, 2024 15:03

ghukill approved these changes May 30, 2024

View reviewed changes

jonavellecuerdo reviewed May 30, 2024

View reviewed changes

jonavellecuerdo self-requested a review May 30, 2024 18:14

jonavellecuerdo approved these changes May 30, 2024

View reviewed changes

ehanson8 mentioned this pull request May 30, 2024

Evaluate variable naming conventions #186

Open

ehanson8 merged commit ff1aaeb into main May 30, 2024
5 checks passed

ehanson8 deleted the TIMX-284-datacite-fmr-strikes-back branch May 30, 2024 18:54

ehanson8 mentioned this pull request Jun 4, 2024

Add final set of Datacite field methods #189

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add 2nd set of Datacite field methods #181

Add 2nd set of Datacite field methods #181

ehanson8 commented May 29, 2024

ehanson8 May 29, 2024

ghukill left a comment

ghukill May 29, 2024

ehanson8 May 29, 2024

ghukill May 29, 2024 •

edited

Loading

ehanson8 May 29, 2024

ehanson8 commented May 30, 2024

ghukill left a comment

jonavellecuerdo left a comment

jonavellecuerdo May 30, 2024

ehanson8 May 30, 2024

ghukill May 30, 2024

jonavellecuerdo May 30, 2024

ehanson8 May 30, 2024

		assert Datacite.get_file_formats(source_record) is None


		def test_get_format_success():

Add 2nd set of Datacite field methods #181

Add 2nd set of Datacite field methods #181

Conversation

ehanson8 commented May 29, 2024

Purpose and background context

How can a reviewer manually see the effects of these changes?

Includes new or updated dependencies?

Changes expectations for external applications?

What are the relevant tickets?

Developer

Code Reviewer(s)

Choose a reason for hiding this comment

ghukill left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghukill May 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ehanson8 commented May 30, 2024

ghukill left a comment

Choose a reason for hiding this comment

jonavellecuerdo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghukill May 29, 2024 •

edited

Loading