Skip to content
This repository has been archived by the owner on May 24, 2022. It is now read-only.

HowTo: Ingest a Data Model with a given, existing Schema

Bo Ferri edited this page Oct 5, 2016 · 2 revisions
  1. retrieve data resource
  2. go to data view in d:swarm back office ui
  3. select a data resource (in the table for "available for configuration")
  4. click "show details"
  5. copy the data resource identifier ("uuid"; usually starts with "Resource-")
  6. fetch the metadata via HTTP API request, e.g.,
curl http://[IP OR HOSTNAME OF YOUR D:SWARM INSTANCE]/dmp/resources/[RESOURCE IDENTIFIER] > [RESOURCE IDENTIFIER].json
  1. retrieve configuration
  2. copy the configuration identifier ("uuid") from the configuration in the configurations array of the fetched data resource
  3. fetch the metadata via HTTP API request, e.g.,
curl http://[IP OR HOSTNAME OF YOUR D:SWARM INSTANCE]/dmp/configurations/[CONFIGURATION IDENTIFIER] > [CONFIGURATION IDENTIFIER].json
  1. retrieve (existing) schema that should be (re-)used to ingest a data resource
  2. go to data view in d:swarm back office ui
  3. select a project that contains the schema that should be utilized to ingest the data resource (i.e. for data model creation)
  4. open this project in the modelling view
  5. copy the project identifier (last part of the URL)
  6. fetch the metadata via HTTP API request, e.g.,
curl http://[IP OR HOSTNAME OF YOUR D:SWARM INSTANCE]/dmp/projects/[PROJECT IDENTIFIER] > [PROJECT IDENTIFIER].json
  1. open the retrieved project in an editor
  2. search for "input_data_model" (or "output_data_model", if it it the schema of the output data model) key
  3. go to "schema" key in this sub tree
  4. copy the schema identifier ("uuid") of this schema
  5. fetch the metadata via HTTP API request, e.g.,
curl http://[IP OR HOSTNAME OF YOUR D:SWARM INSTANCE]/dmp/schemas/[SCHEMA IDENTIFIER] > [SCHEMA IDENTIFIER].json
  1. create data model JSON for the request to create a new data model (ingest data)
  2. fill in the missing parts in this data model JSON stub
{
  "name": "[INSERT NAME HERE]",
  "description": "[INSERT DESCRIPTION HERE]",
  "configuration": INSERT CONFIGURATION HERE,
  "schema": INSERT SCHEMA HERE,
  "deprecated": false,
  "data_resource": INSERT DATA RESOURCE HERE
}
 1. i.e. replace "INSERT DATA RESOURCE HERE" with the retrieved metadata of the data resource, replace "INSERT CONFIGURATION HERE" with the retrieved configuration, replace "INSERT SCHEMA HERE" with the retrieved schema and last (but not least) add a name and a description
  1. create a new data model via HTTP API with the completed data model JSON, e.g.,
curl -H "Content-Type:application/json" -H "Accept:application/json" --data-binary @[NAME OF YOUR NEW DATA MODEL JSON FILE].json -X POST http://[IP OR HOSTNAME OF YOUR D:SWARM INSTANCE]/dmp/datamodels -i -v
 1. you should get a "200" as response and the full data model JSON as response body
Clone this wiki locally