This repository has been archived by the owner on Oct 12, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 18
/
Copy pathCHANGELOG
1009 lines (898 loc) · 61 KB
/
CHANGELOG
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
CHANGELOG
=========
2018-10-15 v0.15.5
* Update badges in README (Taeber Rapczak)
* Merge pull request #230 from PFWhite/feature/rediRunDebugDocs (Christopher P. Barnes)
* Update redi-run-debug.md (Patrick White)
* added docs for debuging redi runs (Patrick White)
* Update README-example-files.md (Michael Buchholz)
* Improve configuration examples for use in vagrant installs (Roy Keyes)
* Merge pull request #212 from ctsit/develop (Nicholas Rejack)
* Merge pull request #211 from nrejack/fix_getemrdata_unittest (Nicholas Rejack)
* Fixing broken unittest TestGetEMRData. Skipping problematic test. See issue #210. (Nicholas Rejack)
2016-07-12 v0.15.4
* Updating .gitignore. (Nicholas Rejack)
* Adding new test to verify behavior when keep_all_events is set to True. (Nicholas Rejack)
* Adding new logic to properly update event tree when keep_all_events is True. (Nicholas Rejack)
* Updating new test to test when keep_all_events is True (Nicholas Rejack)
* Modifying redi.py to correctly increment events in the case where we are keeping all. (Nicholas Rejack)
* Improving error logging when preprocessors fail. (Nicholas Rejack)
* Adding setuptools_scm to install_requires in setup.py. (Nicholas Rejack)
* fixed merge messages left in TODO (Christopher Barnes)
2016-05-19 v0.15.3
* modify TestSortElementTree to test for new commandline switch for keeping all results that bypasses the compress events functions (Christopher Barnes)
* Updating GetTheDocs URL in README.md (Nicholas Rejack)
* Allow setuptools_scm to set the version when it's importing redi.py (Nicholas Rejack)
* docs/conf.py needs to get the version from one directory up. (Nicholas Rejack)
* Adding 2 necessary elements to validate formEvents with HCV RNA forms. Fixing whitespace. (Nicholas Rejack)
* Adding setuptools_scm to docs/conf.py. Now using dynamically-set version number (based on Git tag) (Nicholas Rejack)
* Adding setuptools_scm to redi.py. Now using dynamically-set version number (based on Git tag) (Nicholas Rejack)
* Adding setuptools_scm to setup.py. Now using dynamically-set version number (based on Git tag) (Nicholas Rejack)
* Fixing broken test by reordering XML statements so they validate. (Nicholas Rejack)
* Modifying redi/redi.py to add validation for XSD for formEvents.xml (Nicholas Rejack)
* Added redi/utils/formEvents.xsd to validate formEvents.xml (Nicholas Rejack)
* add info on tests needed before I can make a pull request, to TODO (Christopher Barnes)
* comment out test for --keep-all, add TODO to fix this later (Christopher Barnes)
* Add 2 TODOs that cover issues with the timestamp and how it affects conmeds. (Buck72)
* take the -keep-all logic out of get_key_date (Christopher Barnes)
* take the -keep-all logic out of get_key_date (Christopher Barnes)
* add keep-all-results logic to get_key_dates (Christopher Barnes)
* add keep-all-results logic to get_key_dates (Christopher Barnes)
* mod TestSortElementTree.py to have test keep-all results inputs (Christopher Barnes)
* mod TestSortElementTree.py to have correct number of args(3) to handle new input keep-all (Christopher Barnes)
* fixed incorrect ref to input_file to input_file_path on line 400 in _run (Christopher Barnes)
* set flase correctly in sort elements to turn OFF compression (Christopher Barnes)
* add keep all to create person xml func (Christopher Barnes)
* add keep all to create person xml func (Christopher Barnes)
* add keep all to create person xml func (Christopher Barnes)
* change all refs to --kepp all to use true and false (Christopher Barnes)
* fix inputs to sort_element_tree, pass in keep all results (Christopher Barnes)
* fix inputs to _run, change inputfilename to input_file_path (Christopher Barnes)
* fix inputs to _run, added -K and -f args and set defaults to none (Christopher Barnes)
* process raw file correctly as part of keep all results (Christopher Barnes)
* process raw file correctly as part of keep all results (Christopher Barnes)
* cleanup -K keep all switch (Christopher Barnes)
* cleanup -K keep all switch (Christopher Barnes)
* cleanup -K keep all switch (Christopher Barnes)
* add switch to keep all results (Christopher Barnes)
2016-03-17 v0.15.2
* Summary: Hotfix to fix logging behavior when input file is not present.
* Fixing check for input file in redi.py. Now log the file's location only if it's there. (Nicholas Rejack)
* Update README.md (Nicholas Rejack)
2016-03-17 v0.15.1
* Summary: RED-I now accepts a file as input from the command line, using -f FILENAME.
* add tests to validate the input file used with the commandline switch -f or --file (Christopher Barnes)
* move logger setup and config to start sooner since it is needed by other functions (Christopher Barnes)
* add testdata.csv to use as test input for commandline options test and development (Christopher Barnes)
* added .coverage, cover/, coverage.xml and nosettest.xml to .gitignore. The are local to my test runs (Christopher Barnes)
* add TODO items (Christopher Barnes)
* add todo item for adding a switch to allow another sorting behavior for data points and events (Christopher Barnes)
* Fixing broken test. (Nicholas Rejack)
* added raw text file to def _run and function signature (Kevin Steven Hanson)
* moved raw txt exception out of run function (Kevin Steven Hanson)
* adding option to bypass raw.txt with an if else block that makes more sense than my previous one (Kevin Steven Hanson)
* Fixing bad operator. (Nicholas Rejack)
* Adding code to catch if user tries to use -e and -f at same time (Nicholas Rejack)
* added ability to use input file and bypass hard coded raw.txt (Kevin Steven Hanson)
* i called logger in my agruement before it was defined, my bad (Kevin Steven Hanson)
* adding info statement to work with logger for -f argument (Kevin Steven Hanson)
* Added the missing usage for the file input. (Buck72)
* Add the flag for input file for REDI to use. (Buck72)
* created input file path to hanlde -f file handle with a basic print statement for now (Kevin Steven Hanson)
* add .eggs as correction for directory name to .gitignore (Christopher Barnes)
* add .egg and virtualenvironment folder venv from instructions to .gitignore (Christopher Barnes)
* Adding a new exception to watch for in redcapClient.py: ConnectionError (Nicholas Rejack)
2016-02-12 v0.15.0
* Summary: Improved handling of network issues while sending to REDCap.
* Adding exception handling for NewConnectionError in redi/utils/redcapClient.py (Nicholas Rejack)
* Adding parenthesis to REDCAP_RECORDS_PROGRAM in vagrant/Makefile (Nicholas Rejack)
2015-12-03 v0.14.4
* Summary: RED-I now has built-in log rotation.
* Removing whitespace in docs/redi_usage.rst (Nicholas Rejack)
* Adding note in documentation about log rotation. (Nicholas Rejack)
* Adding log file rotation. Stores 31 days, after which the files are rotated. (Nicholas Rejack)
* Moving logic of preprocessor sorting to run_preproc. Run on a sorted list of preprocessors and fetch module name from original dict. Add comments to clarify logic of running preproc. (Nicholas Rejack)
* Log the preprocessors as they are run. Fix a syntax error. (Nicholas Rejack)
* Move redcap_records.py to its own package, redcap_cli, already published in the Python Package Index (Philip Chase)
2015-11-25 v0.14.3
* Summary: Now RED-I can fetch more files initially and run pre-processors in a specific order.
* Runs the dictionary of preprocessors in order sorted by their keys (module names). This is necessary when using multiple preprocessors that have ordered rules. Unsorted dictionaries have unpredictable ordering. (Nicholas Rejack)
* make clean: remove `.eggs` folder (Andrei Sura)
* Fixes #168 - add option to display version number (Andrei Sura)
* Moving logic for creating dictionary of files to download to __init__ method. Improved logging. (Nicholas Rejack)
* Updating TestGetEMRData.py to match new variable names in GetEmrData.py. (Nicholas Rejack)
* Updated GetEmrData.py to rename download_file variable to download_list and download_file function to download_files. (Nicholas Rejack)
* Fixing TestGetEMRData.py now that a new variable has been added to the EmrFileAccessDetails. (Nicholas Rejack)
* UpdatingGetEmrData.py with new functionality. Now, in settings.ini a dictionary of files to be downloaded can be specified for emr_data_file. (Nicholas Rejack)
* Changing variables in EmrFileAccessDetails for readability. (Nicholas Rejack)
* Update README.md (Christopher P. Barnes)
* Update README.md (Christopher P. Barnes)
* use travis in container (Andrei Sura)
* Add makefile tasks related to pypi (Andrei Sura)
* added RedHat/Fedora installation documentation. (Nicholas Rejack)
* bumping version number on docs/conf.py (Nicholas Rejack)
* repaired some broken text in about.rst in documentation. (Nicholas Rejack)
2015-05-14 v0.14.2
* Summary: Preprocessing has been improved to use the rewritten raw CSV file after processing it. This relocates some of the XML processing from GetEMRData.
* Update about.rst (Buck at UF)
* updated Windows install instructions to simplify the procedure (Nicholas Rejack)
* added expanded test for TestGetDBPath (Nicholas Rejack)
* updated TestSuite to run new test (Nicholas Rejack)
* fixed function name in comment in TestCreateImportDataJson.py (Nicholas Rejack)
* added new test for TestGetDBPath to increase coverage. (Nicholas Rejack)
* fixing version number in redi.py (Nicholas Rejack)
* fixed broken tests after relocating processing from GetEmrData (Nicholas Rejack)
* made csv to xml preprocessing dependent on getting the EMR file (Nicholas Rejack)
* moved XML conversion of raw file out of GetEmrData. (Nicholas Rejack)
* `make test` runs all tests (Andrei Sura)
* Update README.md (Christopher Barnes)
2015-04-23 v0.14.1
* Summary: PreProc and PostProc Hooks! RED-I has added the ability to hook scripts for post processing and preprocessing along with other minor fixes. These features now allow for the writing of filters and the emailing of log messages, for example, at the begining and end of a RED-I run.
* removed project-specific code from preproc.py (Nicholas Rejack)
* removing project-specific preprocessing test file (Nicholas Rejack)
* removed line from settings.ini. REDI-I runs out of the box without any preprocessors required (Nich
* removing test-scenarios bash script (Nicholas Rejack)
* removed scenario data files. These were for project-specific testing. (Nicholas Rejack)
* removed non-informative comments in redi.py (Nicholas Rejack)
* removed renamed config dir (Nicholas Rejack)
* renaming config to previous config-example (Nicholas Rejack)
* added scenario 9 to test scenarios. (Nicholas Rejack)
* added scenario 7. bash script now shows test number. (Nicholas Rejack)
* added scenario 6 (Nicholas Rejack)
* updated test-scenarios with scenario 6 : (Nicholas Rejack)
* Add Scenario 9 (Taeber Rapczak)
* Correct the date of a result in Scenario 5 (Taeber Rapczak)
* Add Scenario 6 (Taeber Rapczak)
* Add 5 test scenarios and test-scenarios.bash (Taeber Rapczak)
* Do not filter rows without a known "panel" (Taeber Rapczak)
* Correct spelling error (Ruchi Vivek Desai)
* Fix order of expected output in test_fetch_panels (Taeber Rapczak)
* Fix the numbering (Ruchi Vivek Desai)
* Add missing tags (Ruchi Vivek Desai)
* Replace hard-coded fetch_panels() (Taeber Rapczak)
* Renumber the steps (Ruchi Vivek Desai)
* Fix indentation briken by commit id b8431f26ec459daf1cc1efe9ebf7c4583dacc572 (Ruchi Vivek Desai)
* Fix indentation of Windows installation instructions (Ruchi Vivek Desai)
* Add instructions for installing RED-I on Windows (Ruchi Vivek Desai)
* Fix email address (Ruchi Vivek Desai)
* Minor code cleanup (Taeber Rapczak)
* Add more context for preprocessors to use (Taeber Rapczak)
* Implement filter_old_labs() (Taeber Rapczak)
* Implement filter_old_labs() (Taeber Rapczak)
* Implement group_rows_by_panel (Taeber Rapczak)
* fetch_consent_dates function now implemented and working. (Nicholas Rejack)
* Resolve preproc import issue (Taeber Rapczak)
* updated preproc.py. Only fails on NotImplementedError now. (Nicholas Rejack)
* updated preproc.py to fix variable names when fetching consent date. (Nicholas Rejack)
2015-04-09 v0.14.0
* Summary: RED-I has multiple functional and other improvements in this release: licensing has been added in each file. RED-I is now available from PyPI as 'redi'. Most importantly,
when multiple lab results come in on the same day, RED-I now takes the first lab of the day, unless it has a value of 'canceled'. RED-I also auto-resumes in the case of a network timeout. Reporting has been improved with more information.
* Update README.md (Andrei Sura)
* removed loinc_code from get_key_timestamp sorting function. data should only be sorted by study_id, form, and date_time_stamp initially. (Nicholas Rejack)
* Add license to all .py files (Ruchi Vivek Desai)
* Remove bug in pull request for github issue #104 (Ruchi Vivek Desai)
* Add initial changes to code for fixing bug in Github issue #104 (Ruchi Vivek Desai)
* Move check for file existence from GetEMRData to SimpleConfigParser (Ruchi Vivek Desai)
* Log errors related to EMR server to facilitate debugging of redi run from cron (Ruchi Vivek Desai)
* Add bad Research IDs to the Error block in the report that gets emailed (Ruchi Vivek Desai)
* #133 Add actual site name to subject line in the emailed report (Ruchi Vivek Desai)
* Add tools for creating input xml files from csv + new profiling task (Andrei Sura)
* Call `compress_data_using_study_form_date` inside `sort_element_tree` This is one example of pre-processing data before using translation table... (Ruchi Vivek Desai)
* Update the unit test test/TestSortElementTree.py (Ruchi Vivek Desai)
* Add pycrypto as a required library for installation in setup.py (Ruchi Vivek Desai)
* `make clean` - Remove generated pysftp files during testing (Ruchi Vivek Desai)
* Save partial fix for filtering out multiple results on same day (Ruchi Vivek Desai)
* Minor corrections to Makefile due rename of `bin` to `redi` (Ruchi Vivek Desai)
* removed line in setup.py. README.md already included in MANIFEST. In addition, defining 'redi' package twice in package_data overwrites the previous definition, causing .xsd and .xsl files to not be included (Nicholas Rejack)
* Uncomment raise MaxRetryError and change the factor by which sleep time increases to 6 (Ruchi Vivek Desai)
* Correct the function name for retry number 10 (Ruchi Vivek Desai)
* Fix test case broken by commit id adf6f96764f319b41027e1423ea85862e0f3162f (Ruchi Vivek Desai)
* Make changes suggested in the review of pull request (Ruchi Vivek Desai)
* Move import sys to top of the code (Ruchi Vivek Desai)
* Correct concatenation of Attempt no and retry_count (Ruchi Vivek Desai)
* Add comment for dummy_init function (Ruchi Vivek Desai)
* Add code for github issue #108 auto resume (Ruchi Vivek Desai)
* Add working test for send_data_to_redcap function (Ruchi Vivek Desai)
* Add test for send_data_to_redcap function (Ruchi Vivek Desai)
* Add code for auto-resume on network timeout (Ruchi Vivek Desai)
* Change the email in setup.py to `[email protected]` because the old email was not reachable (Andrei Sura)
2014-12-19 v0.13.2
* hotfix to change behavior in updateStatusFieldValueInPersonFormEventree. Status field no longer
marked "NOT_DONE" if units for that field are missing.
2014-12-04 v0.13.1
* Fixed major flaw in the Throttle class introduced in v0.13.0
* Added --bulk-send-blanks, which will send all blank form-events together
* Added support for redcap_plugins to our vagrant testing and sample project
* Added new Makefile targets for various helper tasks like profiling and clearing the IP ban list
* Installed mcrypt package for PHP to enable plugins in vagrant
* Added a REDCap plugin to generate URLs for enhanced error reporting
* Fixed #34: Data Import Report totals only reflect most recent run
* Added execution time to the report
2014-11-20 v0.13.0
The focus of this release has been clean up. Related functions were moved into
their own files for easier maintenance and comprehension. Reports were enhanced
and bugs related to them were fixed. Throttling and uploading were both
re-examined and new algorithms implemented which, during testing, run-times
decreased anywhere between 12 and 57%.
* Add new REDCap Rate Limit throttling algorithm
* Improve speed of marking events as sent
* Add script that generates enrollment records
* Rename "bin" to "redi"
* Write report.html to data directory instead of current directory
* Write report.xml to data directory instead of configuration directory
* Added helper class `bin/utils/rawxml.py` to store information about the
input file (can add more info if needed)
* Extract report creation and sending from _run()
* Add more useful aliases for vagrant box
* Updated documentation
* Enhanced report
* Add the `report.html` as attachment
2014-10-07 v0.12.0
* Summary: Converted documentation to .rst format, enable key-based access to SFTP servers to fetch data, enable installation via fetching from the PyPI repository.
* remove parse_args() function (Radha Kandula)
* Add docopt to redi (Radha Kandula)
* Small fix for the README links (Andrei Sura)
* Use the `[email protected]` email for consistency with `setup.py` (Andrei Sura)
* Document installation from binaries: `pip install redi` (Andrei Sura)
* Restore the short README.md because it is referenced by `setup.py` (Andrei Sura)
* Remove unused `doc` folder All `*.rst` files are in now in the `docs` folder (Andrei Sura)
* Move `doc/flowcharts` to `docs/flowcharts` (Andrei Sura)
* Fix links in `README.rst` (Andrei Sura)
* Replace `README.md` by a shorter `README.rst` because we have the full content in `docs/about.rst` (Andrei Sura)
* Add `help` tasks in the the main `Makefile` (Andrei Sura)
* Fix some misspellings of REDCap (Taeber Rapczak)
* Fix configuration files to allow uploadig to the pypi.python.org repo @see https://pypi.python.org/pypi/redi/ (Andrei Sura)
* Fix broken documentation links during conversion to `*.rst` (Andrei Sura)
* Remove files `setup.rst and test.rst` from `docs/api` to avoid unnecessary document creation (Andrei Sura)
* added default emr password of None (Nicholas Rejack)
* removed emr sftp server password from required parameters (Nicholas Rejack)
* removed author name (Radha Kandula)
* Make `GetEmrData.py` immune to `string` passed as port number (Andrei Sura)
* Add `requirements.txt` file for RTD (Andrei Sura)
* Use default theme in `docs/conf.py` (Andrei Sura)
* Edit one method doc to verify if sphinx can pick it (Andrei Sura)
* Save important files for docs `sphix-ification` (Andrei Sura)
* Save converted files from `doc/*.md` to `docs/*.rst` (Andrei Sura)
* Copy images from `doc/images` to `docs/images` for the ReST migration (Andrei Sura)
* implemented review comments for pull request #84 (Radha Kandula)
* Add contact info to `README.md` (Andrei Sura)
* Update AUTHORS.md (Andrei Sura)
* Expand documentation about `Installation Steps Using Source Code` + Add short section `How to Use RED-I` (Andrei Sura)
* One line documentation fix for deprecated `--keep=yes` (Andrei Sura)
* Reformat `describing_a_redcap_form_to_red-i.md` for shorter lines + Moved `screenshot-field-name-lookup.png` to the `images` folder (Andrei Sura)
* add fields() function to Event class to get all the fields of it (Radha Kandula)
* Rules for clearing values using keyword 'cancel' (Radha Kandula)
* Fix anchors for document sections in README.md (Andrei Sura)
* Link to Philip's RED-I presentation from the main README.md (Andrei Sura)
* Link to our code review guidelines `doc/code-review-checklist.md` from the main README.md (Andrei Sura)
* Add the removed pieces from README.md as seprate files in the `doc/` folder (Andrei Sura)
* Add more links to existing files in the `doc/` folder from README.md (Andrei Sura)
* Add support for --record parameter in redcap_records.py (Philip Chase)
* Fix issue with download file name when connecting to sftp (Andrei Sura)
* Remove developer specific details from main README.md (Andrei Sura)
* Add dependency on `sftpserver` to `.travis.yml` (Andrei Sura)
* Add unit test for connecting to sftp with a private key (Andrei Sura)
* Use new `sfpt_server_*` params when connecting to sftp with a private key (Andrei Sura)
* Use `EmrFileAccessDetails` object for passing new sftp params: * sftp_server_port * sftp_server_private_key * sftp_server_private_key_pass (Andrei Sura)
* Correct names for two tasks in `vagrant/Makefile` (Andrei Sura)
* Improve documentation in `config-example/settings.ini` Closes issue #63 (Andrei Sura)
* Remove unused script: `exportForms.sh` (Andrei Sura)
* Use boolean args `keep` and `emrdata` Closes issue #53 with dedicated unit test class `TestArgs` (Andrei Sura)
2014-09-23 v0.11.3
* Summary: Update config-example to work with new sample project
* Fixed #73: Change use of redcap_server and redcap_uri settings
* Fixed #72: Remove circular dependency redi <--> redi_lib
* Fixed #68: Sample REDCap project is tied to a specific REDCap version number
* Fixed #64: Create a RED-I config to match the example REDCap project.
* Fixed #63: Add documentation within config-example/settings.ini to explain the use of each parameter
* Fixed #62: Regroup settings within settings.ini
* Fixed #60: Make a test data set of CBC and Chemistry data
* Fixed #59: Revise report sending code to note the locations of reports/email bodies in the console output
* Fixed #58: Remove project specific details from vagrant/Makefile
* Fixed #49: "Exceeded event list... Event count of 11 exceeds maximum of 11"
* Fixed #47: Do not hardcode smtp host/port
* Fixed #27: Update /vagrant/README.md to match new features
* Fixed #4: Create a working example REDCap Project suitable for RED-I Demonstration
* Fixed #3: No config-example folder
* Rename README_Travis_CI_Setup.md to `setup_travis_ci.md` Also move the related images to `images/setup_travis_ci`
* Cut the section about adding a new REDCap project from vagrant/README.md This resulted in creation of a new document: doc/add_new_redcap_project.md and associated files in dedicated folder: `images/add_new_redcap_project/`
* Remove deprecated document: `doc/README_test_against_redcap.md`
* Delete empty file: `vagrant/README-projects.md`
* Added scripts/compare_settings.sh to help find differences Example usage: ./compare_settings.sh ../config/settings.ini ../config-vcu/settings.ini
* Add clinical-commit-to-loinc.xml helper scripts
* Add synthetic_data tool, makefakedata.R
* Improve `bin/utils/redcap_records.py` - now works with json/xml
* Log information about rules loading
2014-09-10 v0.11.2
* Summary: performance improvements to speed up run time.
* Closes #42 - "redcap_records.py lost the events and fields arguments" (Andrei Sura)
* Remove implied assert of a formImportedFieldName (Taeber Rapczak)
* Add test showing formImportedFieldName is required (Taeber Rapczak)
* Refactor TestCopyDataToPersonFormEventTree.py (Taeber Rapczak)
* additional fix for #39 (Ruchi Vivek Desai)
* fixes #39 (Ruchi Vivek Desai)
* Update README with --skip-blanks and --verbose (Taeber Rapczak)
* Increase pylint score (Taeber Rapczak)
* Add suport for skipping blanks to generate_output (Taeber Rapczak)
* Add --skip-blanks flag (Taeber Rapczak)
* Add unit test to check for --skip-blanks (Taeber Rapczak)
2014-09-03 v0.11.1
* Replace clinicalComponentId with loinc_code in config-example
* BUG: Interpret `send_email` as bool - fixes #35
* Add verify_ssl switch to tell PyCap when to verify the ssl certificate of the REDCap server
* Add missing help text and fix typographical errors in vagrant/Makefile
* Add additional exported forms to rc_get target of vagrant/Makefile
* Remove demographic test data from vagrant folder and add Makefile target to copy this project-specific data from the config folder
2014-08-25 v0.11.0
* Introduced Data Directories (see documentation for --datadir)
* Resumable RED-I (see documentation for --resume)
* Serialize Collection Date Summary
* Included Debian 7.4 installation script (see scripts/install_deb_7_4.sh)
* Updated settings: removed emr_log_file since it wasn't used
* Removed dependency on Python3 (converted csv2xml.py)
* Significantly sped up the data pre-processing step when getting EMR data.
* Changed behavior of combining --emrdata and --dryrun; **will** get EMR data.
* BUG: Fixed SQLite issue
* BUG: event count 15 exceeds maximum of 74
* BUG: Closed hanging SFTP connection in GetEmrData.py
* Other minor bug fixes
* Updated documentation and example configuration
* Cleaned up comments in code
* DEV: Added coverage and lint goals to Makefile
* DEV: Added the generated files to the list of files to remove by `make clean`
2014-08-11 v0.10.1
* Add Travis CI configuration and documentation for activating Travis in forks of this repo.
2014-08-07 v0.10.0
* Revised configuration architecture
* Updated code to better conform to our Logging Guidelines
* Added PyCap-based import-export tool
* Moved temporary files to dynamically-generated, secure temporary folder
* Added tests for REDCap connection and alerts when it fails
* Added check for data freshness
* Added example configuration content
* Improved Python packaging
* Cleaned up code to PEP-8 standards
* Added configuration file documentation
* Added dry-run functionality
* Added default configuration values where appropriate
2014-08-05 v0.9.1
* HOTFIX to replace smtp.ufl.edu with configurable option in setup.json
2014-08-01 v0.9.0
* Testing: Added Vagrant-based testing VM and documentation for how to use it
* Post-processing: Added support for loading and running custom Python modules
* LOINC: Added translation for clinical lab components to LOINC
* Change default overwrite behavior to overwite
* Added configureable send rate to prevent triggering REDCap request limits
* Updated config-example to reflect the new features in v0.8.0
* More robust handling of data with missing collection date
* Fixed bug in create_eav_output()
* Remove unused files from repository
2014-06-16 v0.8.0
* Erase data in missing fields to assure a complete overwrite across all form event for each subject
* Send data to REDCap one form instance at a time
* Isolate data model from REDCap output
* Verify REDCap API responses and pass errors to report
* Added test cases to improve test code coverage
2014-06-13 v0.7.0
* Add feature to translate research subject IDs to REDCap IDs based on queries of the target redcap system
* Add config-examples with sample configuration files
* Update README.md
* Update data specification
* Add export_from_redcap.py, a tool to read data from a REDCap project using the REDCap API
* Update AUTHORS.md file
* Update LICENSE
2014-03-10 v0.6.2
* Summary: Completed commits required to share repo at UF
* Parameterize and add file tests in README-install.md (Philip Chase)
* Fix PROJECT_ROOT environment variable (Philip Chase)
* Revise 'Manually run REDI' section of README-install.md to reflect change in getEmrData.sh (Philip Chase)
* Added more credits to the AUTHORS file (Philip Chase)
* Update README.md to reflect the current project state (Philip Chase)
* Add README file (Mohan Das Katragadda)
* add manifest file for setup distribution generation (Radha Kandula)
* add setup.py (Radha Kandula)
* Add authors file (Philip Chase)
* Add config directory to gitignore (Philip Chase)
2014-03-07 v0.6.1
* Summary: Add location independence to tests. Add BSD 3-Clause license
* Add loction independence and config folder creation to TestCreateSummaryReport.py (Philip Chase)
* Add location independence and tearDown to TestSendDatatoRedcap.py (Philip Chase)
* removed config folder dependencies as much as possible (Mohan Das Katragadda)
* removed the code to create temporary files (Radha Kandula)
* added updateFormCompletedFieldName to the TestSuite (Radha Kandula)
* Revise TestUpdateFormDateField.py to be independent of external inputs (Philip Chase)
* cleared dependancy on external files and cleaned up some temp file creations (Radha Kandula)
* Add license file with BSD 3-clause license (Philip Chase)
* made few changes to Test files to enable them run independent of its locations (Mohan Das Katragadda)
2014-03-07 v0.6.0
* Summary: De-identified sample and test data and removed unneeded filed and references to HCV Target
* Removing the init.sh (Mohan Das Katragadda)
* Collapse features updateRawXml.sh and rawTxt2rawXml.sh into getEmrData.sh (Philip Chase)
* Move result dates in sample data files to be shortly after specimen taken time (Philip Chase)
* Remove old method of data deidentification (Philip Chase)
* Finish date deidentification in sample data (Philip Chase)
* Add date formats to the list of those dithered (Philip Chase)
* Deidentify remaining date info in tests (Philip Chase)
* Add ditherDates.pl to de-identify dates (Philip Chase)
* added the pdf file for redi-run-control-flow (Radha Kandula)
* renamed flow.graffle to redi-run-control-flow.graffle (Radha Kandula)
* made changes to changeYearScript and moved it to the Config folder (Mohan Das Katragadda)
* Remove 'hcv' from comments in redi.py (Philip Chase)
* Update README.md to reflect the current REDI structure (Philip Chase)
* remove unused parameters from setup.json (Philip Chase)
* changed name to just porject (Mohan Das Katragadda)
* remove file no longer needed in ./config (Philip Chase)
* updated the names from rx2ra to redi (Radha Kandula)
* Deleted config/units_found_in_ufshands_clinical_lab_values.csv (Mohan Das Katragadda)
* Remove translationTable.csv becuase it is an old unused format (Philip Chase)
* deleted the unnecessary files in docs (Radha Kandula)
* made changes to getEMR script and updateRawXML script so they can take entire URI as an input argument (Mohan Das Katragadda)
* Fixed the failure in TestReadConfig.py (Mohan Das Katragadda)
* Made changes to the Test files so they can find project root relative to their locations (Mohan Das Katragadda)
* made changes to redi.py so it can get the correct path of the project root all by itself (Mohan Das Katragadda)
* changed path of the subjectMap.csv (Mohan Das Katragadda)
* moved the global variable proj_root to includes section at the top (Radha Kandula)
* Making changes to test files to use the proj root environment varibale (Mohan Das Katragadda)
* added the modifications for env variable PROJ_HCV_ROOT (Radha Kandula)
* Made changes to redi.py to use the environment variable (Mohan Das Katragadda)
* init.sh to set environment variables for the project and made changes to changeYearScript.sh to use the newly created environment variable (Moha
* moved changeyear script to bin/utils (Radha Kandula)
* Parameterize ./bin/utils/getEMRData.sh (Mohan Das Katragadda)
* removed rediJSLog module as redi handles its own logger (Radha Kandula)
* removed the rediJS module (Radha Kandula)
* deleted the personIndex directory for De-identification (Radha Kandula)
* changes the REAME according to new directory layout (Radha Kandula)
* moved subjectMap.csv to doc directory and made changes to the data (Mohan Das Katragadda)
* moved bin/utils/export_records_from_unc.py to config folder (Mohan Das Katragadda)
* removed raw-original.txt (Mohan Das Katragadda)
* Replaced all year values in sample and test data with the year 1905. (Mohan Das Katragadda)
2014-02-25 v0.5.0
* Summary: Add support for configured date formats and make HCVTarget-specific features optional
* Update README-install.md with a warning to mac users to use GNU Utilities (Philip Chase)
* Fix debugging lines in redi.py (Philip Chase)
* Add support for missing form_imported and field_status fields in generate_redcap_eav (Philip Chase)
* Add support for output_date_format in generate_redcap_eav function (Philip Chase)
* Add configurable date formats for input and output dates (Philip Chase)
2014-02-25 v0.4.1
* Add second component element for INR to accommodate second component ID (Philip Chase)
* Update redeployment procedure in README-install.md and add run procedure to same (Philip Chase)
2014-02-25 v0.4.0
* Summary added CBC and Chemistry form fields, INR form, and email summary report
* Fixed createSummaryReport test issue (Mohan Das Katragadda)
* Fixed issue with new forms (Mohan Das Katragadda)
* Fixed createSummaryReport test issue (Mohan Das Katragadda)
* Fixed issue with new forms (Mohan Das Katragadda)
* Commented out the Import Alerts (Mohan Das Katragadda)
* Add support for missing redcapFieldNameUnits in translationTable.xml (Philip Chase)
* updated generate redcap as per new changes in function parameters (Radha Kandula)
* deleted file mailreport.py (Mohan Das Katragadda)
* updated to remove the temporary xml file (Radha Kandula)
* updated the new schema definition (Radha Kandula)
* added comments for debugging (Radha Kandula)
* updated according to new schema (Radha Kandula)
* added INR component to translationalTable (Radha Kandula)
* all code review changes implemented including code for handling new forms (Mohan Das Katragadda)
* added INR to xml files (Nicholas Rejack)
* create summary report test added (Radha Kandula)
* Formatting date in the XSL (Mohan Das Katragadda)
* Reverting changes (Mohan Das Katragadda)
* made changes to xsl (Mohan Das Katragadda)
* Add documentation for the guessComponents.pl tool that helps identify desired components in lab data (Philip Chase)
* updated installation instructions (Nicholas Rejack)
* changes made during the code review (Mohan Das Katragadda)
* xsd updated as per new structure requirement (Radha Kandula)
* clean up changes (Mohan Das Katragadda)
* changed the sender email to [email protected] (Radha Kandula)
* added alert update (Radha Kandula)
* modified send email code (Mohan Das Katragadda)
* Fix the redeployment procedure and embrace adhoc temporary directories (Philip Chase)
* added null check for alerts in xsl (Radha Kandula)
* Renamed installation README to README-install.md (Philip Chase)
* added summary and subject details sections to the xsl (Mohan Das Katragadda)
* updated import alerts section of stylesheet (Radha Kandula)
* xsl file (Mohan Das Katragadda)
* Populating report details is complete (Mohan Das Katragadda)
* Maximum event list and multiple values alerts added (Radha Kandula)
* few more changes (Mohan Das Katragadda)
* Add REDI data flow diagrams with swimlanes (Philip Chase)
* few more changes related to the summary report user story (Mohan Das Katragadda)
* commiting changes (Mohan Das Katragadda)
* setup changes (Mohan Das Katragadda)
* merged develop branch into feature (Mohan Das Katragadda)
* Add componentMapping data for glucose, potassium, and sodium (Philip Chase)
* Add chem form fields Glucose, Potassium, and Sodium to translationTable.xml (Philip Chase)
* Add Lymphocytes % to the translation table and to the test TestGenerateRedcapEAV.py (Philip Chase)
* Add basic config/report.xsd (Philip Chase)
2014-02-18 v0.3.6
* Correct name of log file in getEmrData.sh (Philip Chase)
2014-02-10 v0.3.5
* Summary update generate_redcap_eav to handle forms with only one line of data and add personIndex REDCap project
* Mark mrn as an identifier in personIndex data dictionary (Philip Chase)
* updated generate_redcap_eav to handle forms with only one line of data (Nicholas Rejack)
* Update new installation and redeployment procedures (Philip Chase)
* Create basic README.install.md (Philip Chase)
* Update person index data dictionary and add instructions on how to configure the REDCap Project. (Philip Chase)
* Update person index data dictionary (Philip Chase)
* Add Person Index data dictionary and README (Philip Chase)
2014-02-07 v0.3.4
* Add lftp in Requirements section of README.install.md (Philip Chase)
* Fix bad references to the upload/download log file (Philip Chase)
2014-02-07 v0.3.3
* Update README.install.md to reflect installed instance (Philip Chase)
2014-02-07 v0.3.2
* updates to Generate redcap eav test (Radha Kandula)
2014-02-07 v0.3.1
* added code for last iteration (Radha Kandula)
2014-02-06 v0.3.0
* Summary: Set formImported fields, lab status fields, and omit labs without dates
* Update test data sets to be shorter and use redcap record_IDs (Philip Chase)
* Eliminate event that have no timestamp (Philip Chase)
* Replace subjectIDs with record IDs in subjectMap.csv (Philip Chase)
* fixed the duplicate entries bug (Radha Kandula)
* fixed the test for sort_element_tree for the new changes (Radha Kandula)
* fixed the test for Generate Redcap eav according to new changes (Radha Kandula)
* added work around comments (Radha Kandula)
* added timestamp check (Radha Kandula)
* changed the date format and sort_tree func to reflect new changes (Radha Kandula)
* updated .gitignore with the config/report.txt (Radha Kandula)
* added the changes to form update fields with correct status names (Radha Kandula)
* added new form field status changes (Radha Kandula)
* Changes related to Story Create emailed summary report (Mohan Das Katragadda)
* debugging (Radha Kandula)
* added some debugs (Radha Kandula)
* updated the individual form dones (Radha Kandula)
* Integrated send email with redi.py (Mohan Das Katragadda)
* added 75 events for cbc and chemistry forms (Radha Kandula)
* added rule enginer (Radha Kandula)
* Fix aserted value in TestGenerateRedcapEAV.py (Philip Chase)
* Removed unnecessary comments. (Mohan Das Katragadda)
* Changes made to Adapt REDI inputs to new EMR data format (Mohan Das Katragadda)
* Made changes to adapt REDI inputs to new EMR data format (Mohan Das Katragadda)
* Update token and examples in export_records_from_unc.py. Change binary from python to python3 and added execute permission (Philip Chase)
* Put UNC URI and REDCap Token into setup.json to streamline testing (Philip Chase)
* Add export_records_from_unc.py (Philip Chase)
* Deactivate the setting of 'redcap_form_status' = 'null' (Philip Chase)
* Fix value set for form_imported_field_name (Philip Chase)
* Adjust token and examples in export_records.py (Philip Chase)
* Update raw.xml with content from the new report style, processed by the revised rawTxt2rawXml.sh (Philip Chase)
* Change delimeter used in rawTxt2rawXml.sh from tab to comma (Philip Chase)
* Revise rawTxt2rawXml.sh to apply character sets to the input and output of csv2xml.py (Philip Chase)
* form done changes added (Radha Kandula)
* Update the test data files from the UF Shands EMR team with the the 2014-01-15 update (Philip Chase)
* Retroactively update CHANGELOG for the 0.2.1 release (Philip Chase)
* remove rawDataWithFormStatus.xml and rawDataWithFormImported.xml from repo (Philip Chase)
* Reduce the size of the form events list to match that of the dev instance at UNC. (Philip Chase)
* Adjust log message in update_event_name to refer to last_record_group (Philip Chase)
* added form status output file (Radha Kandula)
* 1.added function for updating the done status 2. udpated the tests for add_elements_to_tree 3. updated generate redcap eav test (Radha Kandula)
* Pylint report updated (Radha Kandula)
* updated some pylint changes after modifications to functions (Radha Kandula)
* changed the fieldImportedField name for each name to yes (Radha Kandula)
* added code to delete the test file generated by testing generateRedcapEAV (Radha Kandula)
* modified the generate redcap EAV for the formImportedField (Radha Kandula)
* added test for update_form_imported function (Radha Kandula)
* modified add_elements_to_tree and its test (Radha Kandula)
* added function to update formImportedFieldName (Radha Kandula)
* added formImportedFieldName and its value (Radha Kandula)
2014-01-13 v0.2.1
* Summary: Merge rx2ra.py and rediJS.py in to redi.py
* removed comments from setup.json (Radha Kandula)
* added function get_emr_data() (Radha Kandula)
* added documentation for plan of the log levels in configure_logging() function (Radha Kandula)
* added new variables for ftp server settings (Radha Kandula)
* updated the old variables in setup.json to python naming standards (Radha Kandula)
* added prototype to convert_clinical_data_to_xml function (Radha Kandula)
* Add spreadsheet showing the frequency of non-numeric values in lab data relevant to HCV Target (Philip Chase)
* Add the file units_found_in_ufshands_clinical_lab_values.csv which includes just that with frequency data (Philip Chase)
* Roll raw.txt back to the orignal format as the new format is not ready for primetime (Philip Chase)
* Add component mapping hints for hep.rnaHcv.rnaHeptimax and LOINC details for possible LOINC mapping (Philip Chase)
* pylint changes to logger (Radha Kandula)
* pylint changes on the new functions added (Radha Kandula)
* added params validation in the POSTed string (Radha Kandula)
* added buffer value to main to supress warnings from TestSuite (Radha Kandula)
* added comments to functions (Radha Kandula)
* added the mock listener function for response comparision (Radha Kandula)
* updated to fix test results (Nicholas Rejack)
* added logging to parse_raw_xml function (Nicholas Rejack)
* added logging to parse_translation_table function (Nicholas Rejack)
* added logging to parse_form_events function (Nicholas Rejack)
* added the send_data_to_redcap to testsuite (Radha Kandula)
* sending data to redcap added in test (Radha Kandula)
* created test for send_data_to_redcap (Radha Kandula)
* new flow updated (Radha Kandula)
* Add redi-dataFlow-forSecurityEval.graffle and its pdf (Philip Chase)
* Renamed REDI-JS to REDI in redi-js-dataFlow.* (Philip Chase)
* Add a specification for the clinical data file that embraces EPIC Clarity defaults (Philip Chase)
* changes done to email and pwd (Radha Kandula)
* report mail initial commit (Radha Kandula)
* no more rx2ra.py.. rebranded to redi.py (Radha Kandula)
* changes to test files for rx2ra to redi changes (Radha Kandula)
* merged redi JS with rx2ra (Radha Kandula)
* bin/rx2ra.py (Radha Kandula)
* integrated rediJS and rx2ra into a single flow (Radha Kandula)
* added test for generate redcap eav (Radha Kandula)
* added test for generate redcap eav (Radha Kandula)
* Adapt getEmrData.sh and subjectMap.csv to EMR processes (Philip Chase)
* updated function name changes for pylint: configure_logging (Radha Kandula)
* more pylint changes & logger usage changes (Radha Kandula)
* PylintReport updated (Radha Kandula)
* pylint changes & TestSuite updates (Radha Kandula)
* made TestWriteToFile class independent of file IO (Radha Kandula)
* added logger configuration in testwritefile (Radha Kandula)
* added null check for form_name in update_event_name function (Radha Kandula)
* added the None checks before joining the strings in UpdateEventName function (Radha Kandula)
* logger function called inappropriately. so, updated it (Radha Kandula)
* logger function accessed without configuring logging. so corrected it (Radha Kandula)
2013-12-18 v0.2.0
* Summary: Add rx2ra.py, getEmrData.sh
* Add documentation for getEmrData.sh (Philip Chase)
* adding logic puzzle image (Nicholas Rejack)
* updated powerpoint and report (Nicholas Rejack)
* Update SIPOC of data flow from emr to redcap (Philip Chase)
* added screenshots of draft report and agenda (Radha Kandula)
* added screenshots of draft report and agenda (Radha Kandula)
* Add presentation of data flow (Philip Chase)
* Add sample report to send to data abstractors (Philip Chase)
* Add counting of distinct values. Also removed HH:MM from timestamp (Philip Chase)
* Add balance of chemistry form fields to translationTable.xml (Philip Chase)
* Fix formDateField for chemistry form in formEvents.xml (Philip Chase)
* cast integer as string in log message in update_event_name (Philip Chase)
* Add and improve log output in generate_redcap_eav and update_event_name (Philip Chase)
* Move data tree sort before event name update (Philip Chase)
* Replace dummy subjectMap.csv with legit data (Philip Chase)
* Change call to sort_element_tree in main and exclude the output file created (Philip Chase)
* Fix sort_element_tree and its test (Philip Chase)
* Add eventName output in generate_redcap_eav (Philip Chase)
* Add timestamp_group concept in update_event_name to fix application of eventNames (Philip Chase)
* Revise input and output data in TestUpdateEventName class (Philip Chase)
* removed testing print statment in rx2ra (Nicholas Rejack)
* updated sort function. (Nicholas Rejack)
* Replace logException references with logger.level references (Philip Chase)
* Add event_name output in generate_redcap_eav (Philip Chase)
* Add call to update_event_name in rx2ra.py main (Philip Chase)
* Add event_list length error handling, logging, and comments in update_event_name (Philip Chase)
* Remove unused code from test/TestUpdateEventName.py (Philip Chase)
* updated docstring for sorting (Nicholas Rejack)
* updated sorted function (Nicholas Rejack)
* updated sorting function (Nicholas Rejack)
* added sorting helper function to rx2ra (Nicholas Rejack)
* Add update_event_name, its test and references in TestSuite class (Philip Chase)
* Remove redcap.eav output file from source control (Philip Chase)
* Add output for redcap eav event date form complete records (Philip Chase)
* Add output for redcap eav datum record and units record (Philip Chase)
* Add calls to update_formcompletedfieldname and update_redcap_field_name_value_and_units in rx2ra.py main (Philip Chase)
* Change variable and class references in TestSuite to 'rx2ra' (Philip Chase)
* Add update_redcap_field_name_value_and_units, its test and references to test in TestSuite (Philip Chase)
* Fix element references in function prototype comments of generate_redcap_eav (Philip Chase)
* Add new elements redcapFieldNameValue and redcapFieldNameUnits to add_elements_to_tree and its test (Philip Chase)
* Add TestAddElementsToTree class and references to it in TestSuite (Philip Chase)
* Add TestUpdateDataFromLookup class and references to it in TestSuite (Philip Chase)
* Fix indentation levels in TestSortElementTree (Philip Chase)
* Revert "fixed path in import for TestLog.py" (Philip Chase)
* Revise generate_redcap_eav to have right structure, but only dummy output records (Philip Chase)
* incremental update on TestSortElementTree, still not working. (Nicholas Rejack)
* incremental update on TestSortElementTree, still not working. (Nicholas Rejack)
* add TestSortElementTree to TestSuite (Nicholas Rejack)
* added test data to TestSortElementTree.py (Nicholas Rejack)
* fixed path in import for TestLog.py (Nicholas Rejack)
* added additional fields to sorting function (Nicholas Rejack)
* redcap.eav file (Radha Kandula)
* added generate_redcap_eav function (Radha Kandula)
* resolved bug in TestSuite (Radha Kandula)
* function names changed as per pylint standards (Radha Kandula)
* removed trailing whitespace (Nicholas Rejack)
* Exclude log and scratch files in .gitignore (Philip Chase)
* Delete scratch log file (Philip Chase)
* added docstring to add_elements_to_tree in rx2ra.py (Nicholas Rejack)
* Replace hcvtarget name for log file with rx2ra in test/TestLog.py (Philip Chase)
* Fix parameter names in test/TestReadConfig.py (Philip Chase)
* Replace hcvtarget name in log file with rx2ra. Prep for reading log file name parameter. (Philip Chase)
* Fix update_redcap_form (Philip Chase)
* Add generic function update_form_field and revise update_formcompletedfieldname to use it (Philip Chase)
* Add commented out code to call update_formcompletedfieldname (Philip Chase)
* Add update_formcompletedfieldname function, tests, and improved formEvents.xml (Philip Chase)
* added docstring for my parse_form_events function. (Nicholas Rejack)
* fixed indentation in my docstring (Nicholas Rejack)
* added docstring for my parse_raw_xml function. (Nicholas Rejack)
* fixed write method (Nicholas Rejack)
* changed redcapFormName to redcap_form_name in rx2ra.py (Nicholas Rejack)
* changed rawXmlFile in config/setup.json to raw_xml_file (Nicholas Rejack)
* changed form_events_file to formEventsFile in config/setup.json (Nicholas Rejack)
* changed translationTableFile to translation_table_file in config/setup.json (Nicholas Rejack)
* changed formName to form_name, component_id_text in rx2ra.py (Nicholas Rejack)
* changed formName to form_name, component_id_text in rx2ra.py (Nicholas Rejack)
* changed translationTable to translation_table, undefinedForm to undefined_form, formName to form_name in rx2ra.py (Nicholas Rejack)
* changed dtString and dateTime to dt_string and date_time in rx2ra.py (Nicholas Rejack)
* changed fileName to file_name in rx2ra.py (Nicholas Rejack)
* changed elementTree argument to element_tree (Nicholas Rejack)
* changed variable names translationTableRoot, translationTableDict, dataRoot, componentID to valid python variable names in rx2ra.py (Nicholas Rejack)
* changed parse_raw_XML to parse_raw_xml in rx2ra.py (Nicholas Rejack)
* changed formEventsTree to form_events_tree in rx2ra.py (Nicholas Rejack)
* changed collectionTime to collection_time in rx2ra.py (Nicholas Rejack)
* changed rawXmlFile to raw_xml_file in rx2ra.py (Nicholas Rejack)
* changed collectionDate to collection_date in rx2ra.py (Nicholas Rejack)
* removed 'file' (a python builtin reserved name) and replaced it with 'item' (Nicholas Rejack)
* reverted previous docstring change. (Nicholas Rejack)
* moved docstring inside module in rx2ra.py (Nicholas Rejack)
* wrapped line 91 in rx2ra.py (over 80 character limit) (Nicholas Rejack)
* changed translationTableFile to translation_table_file in rx2ra.py (Nicholas Rejack)
* changed translationalTableTree to translational_table_tree in rx2ra.py (Nicholas Rejack)
* changed requiredParameters to required_parameters in rx2ra.py (Nicholas Rejack)
* changed formEventsFile to form_events_file in rx2ra.py (Nicholas Rejack)
* changed setupJson to setup_json in rx2ra.py (Nicholas Rejack)
* changes to comply with pylint (Radha Kandula)
* added update formDatefield test function (Radha Kandula)
* updated important usage notes at the top of the file (Radha Kandula)
* changed function name update_formdatefield to comply with standards (Radha Kandula)
* added documentation of how to add the imports for adding testcases (Radha Kandula)
* added default value to failed lookup for update formDateField (Radha Kandula)
* Fix case in formEvents so that formname lookups will work (Philip Chase)
* created update formDatefield functions (Radha Kandula)
* changes made in test files to reflect the changed method names (Radha Kandula)
* Created a test suite (Radha Kandula)
* Add bin/utils/getEmrData.sh, a basic script to moved data to and from the SFTP host. (Philip Chase)
* updated a few file names in rx2ra.py: added test for logging: updated TestUpdateTimestamp indentation changes (Radha Kandula)
* changed updateRedcapForm to update_redcap_form in rx2ra.py (Nicholas Rejack)
* changed updateTimeStamp to update_time_stamp in rx2ra.py (Nicholas Rejack)
* changed writeElementTreeToFile to write_element_tree_to_file in rx2ra.py (Nicholas Rejack)
* changed addElementsToTree to add_elements_to_tree in rx2ra.py (Nicholas Rejack)
* changed parseTranslationTable to parse_translation_table in rx2ra.py (Nicholas Rejack)
* changed parseFormEvents to parse_form_events in rx2ra.py (Nicholas Rejack)
* changed parseRawXml to parse_raw_XML in rx2ra.py (Nicholas Rejack)
* changed readConfig to read_config in rx2ra.py (Nicholas Rejack)
* added logger functionality and put some test logs in functions (Radha Kandula)
* assigned 1 function to NR (Nicholas Rejack)
* Pylint report created and added (Radha Kandula)
* replaced tabs with 4 spaces (Nicholas Rejack)
* PYLINT: updated the code for coding standards & warnings (Radha Kandula)
* updated the correct class name (Radha Kandula)
* cleaning up timestamp function (Nicholas Rejack)
* updated timestamp format for correct separators (Nicholas Rejack)
* modified working updateTimeStamp function (Nicholas Rejack)
* commented out unneeded lines in timeStamp (Nicholas Rejack)
* modified rx2ra.py to output REDCap-style timestamp (Nicholas Rejack)
* added TestSortElementTree.py. incomplete. (Nicholas Rejack)
* Add test/TestUpdateRedcapForm.py (Philip Chase)
* created TestSortElementTree.py (Nicholas Rejack)
* rename makeHcvtargetTranslationTableXML.sh (Philip Chase)
* rename input and output files in makeHcvtargetTranslationTableXML.sh (Philip Chase)
* Add updateRedcapForm function (Philip Chase)
* unit test for the function updateTimeStamp (Radha Kandula)
* changed the access of data from global to local in func timestamp (Radha Kandula)
* update the func updatetimestamp (Radha Kandula)
* updated sorting on study_ID (Nicholas Rejack)
* Remove initialization of data and pointless copy of data (Philip Chase)
* Make data a non-global variable (Philip Chase)
* commented func call to updatetimestamp() (Radha Kandula)
* added updatetimestamp function (Radha Kandula)
* added updatetimestamp function (Radha Kandula)
* Change class name in test/TestReadConfig.py (Philip Chase)
* updated whitespace and comments for legibility (Nicholas Rejack)
* updated whitespace and comments for legibility (Nicholas Rejack)
* adds blank elements to data tree (Nicholas Rejack)
* Add line wrap in test/TestReadConfig.py (Philip Chase)
* deleting the temporarily created test file and added description (Radha Kandula)
* Make all .py files executable (Philip Chase)
* Add file creation and deletion so file error chekcing in readConfig will not raise an exception in testing (Philip Chase)
* Revise readConfig to raise logException on error (Philip Chase)
* Replace each tab with 4 spaces in testwrite.py (Philip Chase)
* Replace each tab with 4 spaces in rx2ra.py (Philip Chase)
* Exclude serialized test data files in .gitignore (Philip Chase)
* deleted aux function (Radha Kandula)
* global variable change (Radha Kandula)
* global variable change (Radha Kandula)
* updated add func call (Nicholas Rejack)
* etree changes (Radha Kandula)
* Add escaping of & in raw.txt input (Philip Chase)
* changed indentation (Radha Kandula)
* added missing tabs in addElementsToTree (Nicholas Rejack)
* Escape < and > symbols in raw.txt input (Philip Chase)
* implemented main,write,auxilaryhandler functions (Radha Kandula)
* Add gitignore lines for .pyc and .pyo files (Philip Chase)
* Add function readConfig, test, and associated config changes (Philip Chase)
* added addElementsToTree function (Nicholas Rejack)
* added XML file reading functions (Nicholas Rejack)
* Fix comments in setup.json (Philip Chase)
* Rename files referenced in rx2ra-dataFlow.graffle and pdf (Philip Chase)
* updated assignments (Radha Kandula)
* updated prototypes (Radha Kandula)
* Rename formElements to formEvents (Philip Chase)
* Add input parameter rawXmlFile in setup.json (Philip Chase)
* added function prototypes (Radha Kandula)
* Add sample input data files. (Philip Chase)
* Consolidate json configuraton data into one file. (Philip Chase)
2013-12-01 v0.1.1
* Created new project from redi-js and redi-js-config repos
2013-11-22 ( REDI-JS 70d9eaade3e1ca1a0d8b19fef3c92ec32d4f341d no release )
* added prototyping.txt with control flow and functions for program. (Nicholas Rejack)
* prototype new functions (Nicholas Rejack)
* Update rx2ra-dataFlow graffle and pdf files (Philip Chase)
* initial commit of rx2ra.py (Philip Chase)
* Add study_id to dts-event flow. Add study_id to Result_Value flow. (Philip Chase)
* Add study_id/Record to rx2ra.py flowchart (Philip Chase)
* Add data flowchart for rx2ra.py (Philip Chase)
* Add major update of ./src/redi/config and minor update to rediJS.py (Philip Chase)
* Move README.md to doc/README-install.md and add doc/README-development.md (Philip Chase)
* Add a proposed process flow chart (Philip Chase)
2013-08-27 REDI-JS version 0.1.0
* Relocate work folder and ignore and remove log folder (Philip Chase)
* Create separate logger for each jobname, make python3 rediMain.py work from command line (Yang Li)
* add work folder, only clean up data file after importing done (Yang Li)
* Loop jobnames and process single job, need to complete single log for each job (Yang Li)
* Restore the logging code, add some try except blocks in code (Yang Li)
* Add flow charts of redi-js workflows and data flows to redi-js docs (Philip Chase)
* Move extra README files to a doc subdirectory (Philip Chase)
* Improve spelling, grammar, and flow in README-newJob.md (Philip Chase)
* Fleshed out REDCap Form Design Concerns section of REDI-JS README-newJob.md (Philip Chase)
* Fleshed out Human Resources section of REDI-JS README-newJob.md (Philip Chase)
* Fleshed out Technical Requirements section of REDI-JS README-newJob.md (Philip Chase)
* Add Testing sections to redi-js README-newJob.md (Philip Chase)
* Fleshed out Process section of redi-js README-newJob.md (Philip Chase)
* Fix bad wording in config.example/README.md concerning sampleData (Philip Chase)
* Added document outline in README-newJob.md (Philip Chase)
* Create python logging (Yang Li)
* Add REDI-JS documentation (Philip Chase)
* Complete README.md in config.example (Philip Chase)
* Add example job myProject_myForm to config.example (Philip Chase)
* Use configuration files Philip provided, re-coding the python module (Yang Li)
* Start config.example configuration (Philip Chase)
* Add Git repo setup instructions in README.md (Philip Chase)
* Add basic redi-js README.md (Philip Chase)
* Add functions and unit tests for ExportRecord and ImportRecord, also modify code to use config.json (Yang Li)
* Commit code before update submodule (Yang Li)
* Add red-js-config as a submodule at src/redi/config (Philip Chase)
* add .gitignore (Yang Li)
* spelling correction and comment update (Yang Li)
* Create python project with download-API calls and download meta data unittest (Yang Li)
* Initial commit to redi-js (Philip Chase)
2013-11-21 ( redi-js-config 522afd8dbd39f4c86b7dbd92e09ce6cb33ad619b no release )
* Move rx2ra.py to redi-js repo (Philip Chase)
* Add sql to delete sample data from REDCap (Philip Chase)
* Add sample data for separate weeks (Philip Chase)
* Fix out-of-range values in CBC data in hcvtarget.redcapAPISampleData.csv (Philip Chase)
* Fix out of range data. (Philip Chase)
* updated 5 events for 5 subjects -m (Nicholas Rejack)
* Change subjects 101 and 103 to 100 to get multiple events on subject 100 (Philip Chase)
* updated sham data in redcapAPISampleData (Nicholas Rejack)
* convert cbc_lbdtc from date to DTS. Add cbc_complete with value of completed (Philip Chase)
* updated rx2ra.py (Nicholas Rejack)
* added main start (Nicholas Rejack)
* initial rx2ra.py commit (Nicholas Rejack)
* Round out cbc form with hemoglobin and lymphocyte data, translation, and component reports (Philip Chase)
* Add componentReporting tool, source dta and output data files (Philip Chase)
* Restore files in job-config to make rediMain.py happy (Philip Chase)
* Fix event names in hcvtargetFormEvents.xml (Philip Chase)
* Add hcvtargetFormEvents.xml (Philip Chase)
* Add hcvtarget.raw.xml, a sample input file for development of rx2ra.py (Philip Chase)
* Add rawTxt2rawXml.sh to convert a raw xml file coming from the UF IDR into a raw XML file for HCV Target (Philip Chase)
* in csv2xml.py, change incorrectly named 'declaration' parameter to 'xml-declaration' (Philip Chase)
* updated event name to match target data (Nicholas Rejack)
* Relocate old style job configuration files to deprecatedConfigFiles (Philip Chase)
* Relocate makeHcvtargetTranslationTableXML.sh to the tools directory (Philip Chase)
* Add tools/csv2xml.py (Philip Chase)
* update hcvtarget.translationTable.xml that was out of sync with hcvtarget.translationTable.csv (Philip Chase)
* Add hcvtarget.raw.txt, a tab delimited data export from the IDR delivered 2013-11-13 (Philip Chase)
* Add sample data and translation table for the cbc form for one study subject (Philip Chase)
* Add hcvtarget.translationTable.xml derived from hcvtarget.translationTable.csv (Philip Chase)
* Add hcvtarget.translationTable.csv with data for PLATELET COUNT, BILIRUBIN TOTAL, and BILIRUBIN DIRECT (Philip Chase)
* Add .gitignore to ignore work/ (Philip Chase)
* remove comma from the last entry of config.json (Yang Li)
* remove host and path from this file (Yang Li)
* Add gitMode parameter to config.json (Philip Chase)
* Fix bad parameter name in config.json (Philip Chase)
* Set parameter names and test values in config.json (Philip Chase)
* Add screenshots of REDCap forms for hcvtarget* (Philip Chase)
* Add subjects and and additional records to hcvtarget*.sampleData files (Philip Chase)
* Remove formName_complete fields from hcvtarget* sample files (Philip Chase)
* Update sampleData files for hcvtarget jobs based on new excludedFieldPattern files (Philip Chase)
* Removed calculated fields and added a second subject to hcvtarget_chemistry_coags_log.sampleData (Philip Chase)
* Add redcapDataImport.py and enumerateCalculatedFields.sh (Philip Chase)