MIMIC-III OMOP ETL errors

[justin13601]

I'm getting the following:

(meds) justin@dreamer-1-spot:~/justin-dev/ACES/mimic-iii-meds$ meds_etl_omop /home/justin/justin-dev/ACES/mimic-iii-omop /home/justin/justin-dev/ACES/mimic-iii-meds
Generating metadata from OMOP `concept` table
Generating metadata from OMOP `concept` table: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:06<00:00,  6.87s/it]
Generating metadata from OMOP `concept_relationship` table: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:06<00:00,  6.19s/it]
Extracting dataset metadata: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 308.72it/s]
Decompressing OMOP tables, mapping to MEDS Unsorted format, writing to disk...
 18%|████████████████████████████████████████▏                                                                                                                                                                                    | 2/11 [40:53<3:03:58, 1226.54s/it]
Traceback (most recent call last):
  File "/home/justin/miniconda3/envs/meds/bin/meds_etl_omop", line 8, in <module>
    sys.exit(main())
  File "/home/justin/miniconda3/envs/meds/lib/python3.10/site-packages/meds_etl/omop.py", line 770, in main
    process_table_csv(task)
  File "/home/justin/miniconda3/envs/meds/lib/python3.10/site-packages/meds_etl/omop.py", line 388, in process_table_csv
    write_event_data(
  File "/home/justin/miniconda3/envs/meds/lib/python3.10/site-packages/meds_etl/omop.py", line 191, in write_event_data
    assert len(options) > 0, f"Could not find the time column {schema.names()}"
AssertionError: Could not find the time column ['visit_detail_id', 'person_id', 'visit_detail_concept_id', 'visit_start_date', 'visit_start_datetime', 'visit_end_date', 'visit_end_datetime', 'visit_type_concept_id', 'provider_id', 'care_site_id', 'admitting_source_concept_id', 'discharge_to_concept_id', 'preceding_visit_detail_id', 'visit_source_value', 'visit_source_concept_id', 'admitting_source_value', 'discharge_to_source_value', 'visit_detail_parent_id', 'visit_occurrence_id', 'visit_detail_source_value', 'visit_detail_source_concept_id', 'admitting_concept_id', 'discharge_to_source_concept_id']

I am currently passing MIMIC-III tests, but not MIMIC-III-OMOP tests (same errors as MIT-LCP/mimic-omop#64, MIT-LCP/mimic-omop#72, and MIT-LCP/mimic-omop#73)). So I'm unsure if this error is on the OMOP side or the actual ETL side - have not had a chance to check in detail yet, but it seems like this is related: psql:etl/StandardizedClinicalDataTables/VISIT_DETAIL/etl.sql:332: NOTICE: table "visit_detail_assign" does not exist, skipping

Tagging: @mmcdermott
cc: @EthanSteinberg

[mmcdermott]

@justin13601 was this ever resolved? And @EthanSteinberg does it look to you like this is an issue with the OMOP ETL pipeline or just with the MIMIC-III converted data? I suspect the latter, but am not an expert. If it is the latter, we can close as it is outside this scope.

[justin13601]

It seems like a MIMIC-III OMOP issue as Synthea works fine - closing