-
Notifications
You must be signed in to change notification settings - Fork 0
Importers
The majority of the data contained within this application is imported, either from public government source or private UCF specific data sets. The various importers and processes below will describe how to initialize and update the various datasets using the provided management commands.
The teledata import requires that the teledata
database be setup, configured in the settings_local.py
and be filled with data provided by UCF Information Technology. For the purposes of local development, obtain a backup of the production database from Communications and Marketing IT and create it locally.
Once the teledata
database is created, and configured in the settings_local.py
file, the import-teledata
command can be used to import the data into the Search Service.
Command: python manage.py import-teledata
.
This will output the total number of buildings, organizations, departments and staff that have been created, updated, skipped, could not be created due to error or deleted.
The units
application describes various organizational units and employees at UCF. Currently, only UCF Faculty are represented in the Employee
data, and the organizations, divisions, departments and colleges they belong to.
We receive an export from our ERP monthly with a list of faculty. The spreadsheet should have consistent field names, which are accounted for within the importer.
Prior to importing the file, the first row of the spreadsheet will need to be removed. Optionally, the line with the column names can also be removed, if desired.
To import the spreadsheet, use the following command:
python manage.py import-units
<filepath>
[--skip-first-row]
If the first line of your spreadsheet has the column names, be sure to add the --skip-first-row
flag so the column names are not processed.
An important follow up process is our name sanitization process. This runs the names of the organizational units through a process that corrects capitalization, expands abbreviations, and maps known unit names to the desired display version of the name. In addition, it attempts to associate existing programs
departments and colleges with their units
counterparts.
To process the records, use the following command:
python manage.py sanitize-unit-names
[--do-not-associate]
If you do not want to association logic to run, add the --do-not-associate
flag, and that step will be skipped.
The research
application describes researchers and research conducted by them, along with additional meta relating to both researchers and their research. There are two primary systems we import from to obtain this information: Academic Analytics and ORCID.
NOTE: Only researchers who have an employee ID matching a record in the units
application will be imported, so it is important you run the units
importers described above prior to importing research information.
There are two settings or parameters that are required to run the Academic Analytics importer:
RESEARCH_MAX_THREADS
ACADEMIC_ANALYTICS_API_URL
ACADEMIC_ANALYTICS_API_KEY
There is a default value provided for the API_URL
and RESEARCH_MAX_THREAD
settings that, in most cases, should not be changed. The API_KEY
will need to be obtained from Web Communications or the Office of Research, depending on the situation.
To import researchers and their research, run the following command:
python manage.py acad-analytics-import-researchers
[--api-url=<api_url>]
[--api-key=<api_key>]
[--force-update]
The --api-url
and --api-key
parameters can be used if those values are not set in the settings_local.py
or need to point to different values. The --force-update
flag will delete all existing researchers and research prior to importing, and can be useful when making changes to the importer or the research
models.
In addition to the research pulled from Academic Analytics, we pull researcher's educational background from ORCID if they have an ORCID associated with their researcher record.
There is one setting that must be in the settings_local.py
file prior to running this command:
ORCID_BASE_API_URL
The ORCID_BASE_API_URL
has a default value in the settings template, and should remain unchanged.
To import the educational information, run the follow command:
python manage.py orcid-meta-import
Coming Soon!