Importing Legacy Data

Some notes about importing legacy data into the database. These notes are primarily for using the python slurpper to crawl through the folder structure in the ArcProject data folder to find the water quality and shapefiles for both the transects as well as the vertical profiles.

Rough folder structure:

• Mon_YEAR
• Arc_MMDDYR
  • Arc_MMDDYR_GPS
  • Arc_MMDDYR_WQ
• Arc_MMDDYR
• Datasheets
• SummaryFiles

File patterns

Transect water quality files usually contain a string "WQT" in the filename Vertical water quality files usually have a string "WQP" and the gain setting GPS points for the transects follow a pattern of "PosnPnt.shp" GPS points for the stationary water quality locations follow a pattern of "ZoopChlW.shp"

Parsing Site and Gain from the Filename

The slurper tool trys to parse the site code and gain setting (if applicable) using the underscores in the filename to parse the various information. All of the files in a given date folder should have the same pattern. If a file doen't match the rest of the files in the folder, it should be renamed manually.

Ex: ARC_012915_WQP_LH2_10. The fourth underscore part contains the site name and the fifth part contains the gain setting. The gain setting can be a mixture of text and numbers as the script strips out any letters. It must be a valid gain setting (0, 1, 10, 100). The site name can be any short string (but can't have any periods or other types of punctuation).