Implement timelines for annotation and transcription 186955197 #544

aaron-plahn · 2024-02-01T02:06:19Z

No description provided.

* use media item IDs instead of URLs on photographs * use media IDs on songs * add comment about spatial feature properties image URL * fix build * remove time range context for songs * update test setup for seed-test-data CLI command * WIP[media-item-ids]: fix additional tests * seed related audio item in translate song lyrics test

* docs(coscrad-frontend): add instructions for Redux DevTools * WIP address PR comments --------- Co-authored-by: yaanahuu2 <[email protected]>

* implement Term.fromEventHistory * WIP Event source terms and use event sourcing for term tests * fix command tests that involve terms * opt out of auto-generated tests for event sourced terms (and vocabulary lists) * support publication flow for terms * support query access grants for terms * rewrite grant resource read access to user test case with non-event sourced instance * fix additional tests with term event sourcing * skip obsolete test * WIP[event source terms] fix additonal tests * WIP[event source terms] fix a couple additional tests * update base event payload interface * fix base event interace * add invalid cases to Term.fromEventHistory test * refactor based on PR (#511)

* WIP: happy path for page content method * WIP finalize validation and tests for translatePageContent * add integration test for command * finished fuzz test * refactor based on PR (#512) * update text file to use new base event api --------- Co-authored-by: Blake Sellars <[email protected]>

* WIP implement Term.addAudio * finish implementing Term.addAudio * validate happy path * WIP: integration test * update valid test case * finish integration test for add-audio-for-term * add payload type test for add-audio-for-term * update stale snapshots * update stale snapshots * include langauge code on add audio for term * expose URL from media item in queries --------- Co-authored-by: Aaron Plahn <[email protected]>

* WIP implement media ingester * satisfy happy-path test * remove legacy bilingual text property from media items * fix some failing tests

* WIP prototype serving of protected static assets * add happy path test * ensure that url lines up with endpoint in media imports * add comprehensive test coverage for binary media asset queries * remove filepath from query response * gitkeep the static asset directory * create static assets dir if necessary in tests * increase test timeout for **ingest-media-items** test * fix test data setup * fix removal of the filepath property from media item queries * refactor based on PR (#516)

* WIP add failing test for **export-media-annotations** * feat(coscrad-cli): introduce **export-media-annotations**

…ion (#520) * feat(coscrad-cli): discover audio-visual media item lengths on ingestion * update stale snapshot * update another stale snapshot * refactor: break out a separate MediaProber class * refactor based on PR (#520) * fix MediaItem.validateInvarints * fix CreateVideo * update default lengthMillisecond in CREATE_MEDIA_ITEM

* WIP feat(api): introduce CREATE_PHOTOGRAPH * finish CREATE_PHOTOGRAPH integration test * add payload type test for CREATE_PHOTOGRAPH * update snapshot to include CREATE_PHOTOGRAPH schema * import IDGenerationModule into PhotographModule * fix invalid references in photograph test data * refactor based on PR (#521)

* feat(coscrad-cli): create resoruces for media items in **ingest-media-items** * feat(coscrad-cli): create resources in **ingest-media-items**

…tems** (#524) * feat(coscrad-cli): discover photograph dimensions in **ingest-media-items** * update stale snapshots * refactor based on PR (#524) * fix media probe for duration

* add multilingual audio * introduce audio for digital text pages * WIP happy path for add audio for page * fix happy path test for ADD_AUDIO_FOR_DIGITAL_TEXT_PAGE * add invalid test cases for add audio to digital text page * refactor based on PR (#525) * update test data with empty multilingual audio for pages * fix stale snapshots --------- Co-authored-by: Aaron Plahn <[email protected]>

…ges (#514)

…citations (#527) * feat: update all (including user-facing) references to bibliogrpahic citations * fix test setup for bibliographic citations * refactor based on PR (#527)

* refactor(api): leverage multilingual audio for terms * opt-out of stale migration tests * relax types in stale migration code * update CREATE_PROMPT_TERM handler to support multilingual audio * update stale snapshot

…sourcing logic (#531) * implement simpler pattern for organizing domain event sourcing logic * append event to history automatically when applying an event

* feat(api): remove redundant book resource * test(coscrad-frontend): remove reference to old book model in test * loosen interface and fix build * fix some types * update stale snapshots and fix test data * fix additional tests * remove identity context * fix some tests * update tests to support changes to category test data

…ting command streams (#538) * updated execute-event-stream to support streams for multiple aggregate roots * refactor based on PR (#538) --------- Co-authored-by: yaanahuu2 <[email protected]> Co-authored-by: Aaron Plahn <[email protected]>

* WIP: create menu component and api call * WIP: pull API url from config * WIP: update api url

* refactor(api): use new pattern to event source digital texts * implement event handler for digital text title translated * implement update method * implement invalid test cases for translateTitle update method * add happy integration test for translate digital text title * added invalid test cases for translte digital text title * added validation and fuzz test for translate digital text title * removed comments * refactor based on PR (#537) * fix some tests * use event sourcing for digital text test data setup * refactor TestEventStream and fix duplicate append event calls for Term * fix reference errors in updated digital text test data * update stale snapshots * update additional snapshots * update additional stale snapshots --------- Co-authored-by: Aaron Plahn <[email protected]>

* WIP introduce create audio annotation note * WIP submit annotation note immersive form * WIP begin adding cypress test and troubleshoot * fix issue with media item test data in cypress test * WIP re-work cypress test to add login * WIP expand test and format context * WIP update audio annotator appearance * WIP remove testData.json from git index * WIP fix cypress user issue and command stream * repair git history * test for time range values in annotation * add ui feedback and test * break out component for annotator and form * fix max depth issue with nested annotator component * show acknowledge button in immersive form * modify keydown to exclude meta keys * remove unnecessary hook and disable command panel for audio item page * update component test, break out types, and git ignore test data --------- Co-authored-by: yaanahuu2 <[email protected]> Co-authored-by: Aaron Plahn <[email protected]>

* handle VocabularyListCreated * WIP Add vocabulary list test events to TestEventStream * finalize VocabularyList.fromEventHistory * update API for VocabularyList.registerFilterProperty

* WIP: implement carousel with alphabet cards * WIP: changes from PR * WIP: add detox tests

* add happy path test for DigtalText.addAudioForTitle * add invalid path test for DigtalText.addAudioForTitle * WIP: digital text from event history test * implement event sourcing for AUDIO_ADDED_FOR_DIGITAL_TEXT_TITLE * add happy path test for add-audio-for-digital-text-title * add invalid test cases for add-audio-for-digital-text-title * add payload type and fuzz test for add-audio-for-digital-text-title * Fixed mistake for add-audio-for-digital-text-title * add second valid test case for audio that targets title translation * refactor for add-audio-for-digital-text-title * create test data directory if necessary before writing * fix issue with creating test data directory --------- Co-authored-by: Aaron Plahn <[email protected]>

* implement DigitalText.addPhotographToPage * add handler for PHOTOGRAPH_ADDED_TO_DIGITAL_TEXT_PAGE * add happy path test ADD_PHOTOGRAPH_TO_DIGITAL_TEXT_PAGE * add invalid test cases * add type validation test ADD_PHOTOGRAPH_TO_DIGITAL_TEXT_PAGE * fix tests for ADD_PHOTOGRAPH_TO_DIGITAL_TEXT_PAGE

yaanahuu2

Looks great, awesome to see this take shape

yaanahuu2 · 2024-02-01T02:38:43Z

apps/api/src/domain/models/context/edge-connection.entity.ts

@@ -158,6 +160,22 @@ export class EdgeConnection extends Aggregate {
        );
    }

+    isAudioVisualAnnotation(): boolean {


yaanahuu2 · 2024-02-01T02:40:51Z

apps/coscrad-frontend-e2e/src/e2e/resources/audio-items/audio-annotation-flow.e2e.cy.ts

@@ -202,7 +213,13 @@ describe('the audio annotation process', () => {

            describe(`before the successful command is acknowledged`, () => {
                beforeEach(() => {
-                    steps.apply([setTimeRange, fillTextForm, chooseLanguage, submitForm]);
+                    steps.apply([


Love the steps!

yaanahuu2 · 2024-02-01T02:41:22Z

...scrad-frontend-e2e/src/e2e/resources/audio-items/audio-annotation-timeline-queries.e2e.cy.ts

+
+const nubmerOfNotes = 5;
+
+const langaugeCodeForNotes = LanguageCode.Chilcotin;


(sp) langaugeCodeForNotes

yaanahuu2 · 2024-02-01T02:45:43Z

...scrad-frontend/src/components/resources/audio-item/audio-item-detail.full-view.presenter.tsx

+                   * video annotation?
+                   */
+                  return [
+                      buildTimelineMark({


I would imagine this as just a vertical line that then becomes the left hand side of a rectangle when the time range is fully selected. That way you can see the full selection and like we talked about it could have a transparency.

yaanahuu2 · 2024-02-01T02:51:26Z

apps/coscrad-frontend/src/components/timeline/timeline.tsx

+    name: string;
+}
+
+export const Timeline = ({ marks, defaultValue, step, min, max, name }: TimelineProps) => {


So here, if I switch out the MUI slider for a custom timeline component, this API would change

yaanahuu2 · 2024-02-01T02:52:46Z

.../src/lib/aggregate-views/view-models/resources/audio-item/audio-item.view-model.interface.ts

+    /**
+     * Just the notes that use time range context.
+     */
+    annotations: INoteViewModel[];


yaanahuu2 · 2024-02-01T02:55:23Z

libs/media-player/src/lib/audio-annotator.tsx

+     * freedom to syncronize state between the audio element and ad-hoc command
+     * forms.
+     */
+    audioRef: RefObject<HTMLAudioElement>;


from my reading and prototyping the custom timeline, useRef can also be used for grabbing information about the layout of a div. I'll be interested to see if that will work in our context.

sonarqubecloud · 2024-02-06T01:48:18Z

Quality Gate failed

Failed conditions

B Reliability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

aaron-plahn and others added 30 commits December 20, 2023 12:11

docs(coscrad-frontend): add instructions for Redux DevTools (#497)

6f8bc88

* docs(coscrad-frontend): add instructions for Redux DevTools * WIP address PR comments --------- Co-authored-by: yaanahuu2 <[email protected]>

feat(cli): introduce **ingest-media-items** (#513)

929bec6

* WIP implement media ingester * satisfy happy-path test * remove legacy bilingual text property from media items * fix some failing tests

feat(coscrad-cli): introduce **export-media-annotations** (#518)

e15365f

* WIP add failing test for **export-media-annotations** * feat(coscrad-cli): introduce **export-media-annotations**

feat(coscrad-cli): create resources in **ingest-media-items** (#522)

bc02871

* feat(coscrad-cli): create resoruces for media items in **ingest-media-items** * feat(coscrad-cli): create resources in **ingest-media-items**

feat(coscrad-cli): discover photograph dimensions in **ingest-media-i…

ff85978

…tems** (#524) * feat(coscrad-cli): discover photograph dimensions in **ingest-media-items** * update stale snapshots * refactor based on PR (#524) * fix media probe for duration

feat(media-player): introduce UX support for selecting audio time ran…

376d6ab

…ges (#514)

feat: include tags in annotation queries (#526)

db27f95

test: add test for media probe duration discovery (#528)

8d9e807

feat: update all (including user-facing) references to bibliogrpahic …

bcba12b

…citations (#527) * feat: update all (including user-facing) references to bibliogrpahic citations * fix test setup for bibliographic citations * refactor based on PR (#527)

refactor(api): leverage multilingual audio for terms (#529)

e5affae

* refactor(api): leverage multilingual audio for terms * opt-out of stale migration tests * relax types in stale migration code * update CREATE_PROMPT_TERM handler to support multilingual audio * update stale snapshot

refactor(api): implement simpler pattern for organizing domain event …

9ad6d7b

…sourcing logic (#531) * implement simpler pattern for organizing domain event sourcing logic * append event to history automatically when applying an event

WIP: create menu component and api call (#517)

de24ca8

* WIP: create menu component and api call * WIP: pull API url from config * WIP: update api url

feat(api): introduce VocabularyList.fromEventHistory (#542)

f75448e

* handle VocabularyListCreated * WIP Add vocabulary list test events to TestEventStream * finalize VocabularyList.fromEventHistory * update API for VocabularyList.registerFilterProperty

WIP: implement carousel with alphabet cards (#539)

9b7250c

* WIP: implement carousel with alphabet cards * WIP: changes from PR * WIP: add detox tests

add draft timeline component

622b8fe

WIP add annotation track to audio-item detail view

dce4660

WIP implement annotation timeline and e2e test

da1a80f

aaron-plahn requested review from jbambrick and yaanahuu2 February 1, 2024 02:06

aaron-plahn added the work in progress label Feb 1, 2024

yaanahuu2 reviewed Feb 1, 2024

View reviewed changes

jbambrick approved these changes Feb 2, 2024

View reviewed changes

aaron-plahn added 4 commits February 5, 2024 14:35

WIP test annotation timeline

923ca82

refresh annotations when notes are updated

a99abde

WIP enable full annotation

63ebafd

WIP deal with unwanted re-renders

8fe57ce

aaron-plahn force-pushed the integration branch from 3a56466 to 07209a6 Compare March 5, 2024 17:43

yaanahuu2 mentioned this pull request May 27, 2024

prototype for immersive ux for Audio annotation and transcription 186640224 #593

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement timelines for annotation and transcription 186955197 #544

Implement timelines for annotation and transcription 186955197 #544

aaron-plahn commented Feb 1, 2024

yaanahuu2 left a comment

yaanahuu2 Feb 1, 2024

yaanahuu2 Feb 1, 2024

yaanahuu2 Feb 1, 2024

yaanahuu2 Feb 1, 2024

yaanahuu2 Feb 1, 2024

yaanahuu2 Feb 1, 2024

yaanahuu2 Feb 1, 2024

sonarqubecloud bot commented Feb 6, 2024


		const nubmerOfNotes = 5;

		const langaugeCodeForNotes = LanguageCode.Chilcotin;

Implement timelines for annotation and transcription 186955197 #544

Are you sure you want to change the base?

Implement timelines for annotation and transcription 186955197 #544

Conversation

aaron-plahn commented Feb 1, 2024

yaanahuu2 left a comment

Choose a reason for hiding this comment

yaanahuu2 Feb 1, 2024

Choose a reason for hiding this comment

yaanahuu2 Feb 1, 2024

Choose a reason for hiding this comment

yaanahuu2 Feb 1, 2024

Choose a reason for hiding this comment

yaanahuu2 Feb 1, 2024

Choose a reason for hiding this comment

yaanahuu2 Feb 1, 2024

Choose a reason for hiding this comment

yaanahuu2 Feb 1, 2024

Choose a reason for hiding this comment

yaanahuu2 Feb 1, 2024

Choose a reason for hiding this comment

sonarqubecloud bot commented Feb 6, 2024

Quality Gate failed