Skip to content

Pipelines

Pipelines are used to transform a resource into one or several entries. A pipeline is an entry of ResourceType.Pipeline that defines several properties that are used in the transformation process. A pipeline is executed by submitting its entry URI to the context's execution resource.

Currently only synchronous execution is supported.

Pipeline properties

The following properties (in the EntryStore namespace) are supported in the pipeline's metadata:

  • transform: points to the root-node of a transform
    • transformPriority: every pipeline can chain together several transforms, the order is determined by the transforms' priorities
    • transformType: determines which type of transform should be carried out
    • transformArgument: points to one or more transformArgumentKey-transformArgumentValue pairs to provide transform-specific parameters
    • pipelineToEntry: the destination entry - this optional property can be overriden by the destination setting in the execution request

Execution

Request

POST https://domain/store/context-id/execute

HTTP header: Content-type: application/json

Request body

{
  "pipeline": "https://domain/store/context-id/entry/20",
  "source": "https://domain/store/context-id/entry/21",
  "destination": "https://domain/store/context-id/entry/22"
}

The destination parameter is optional.

Response

HTTP status if successful: 201 CREATED

Response body

The response body of a successful request returns a list of entries that were created or updated during the pipelining process.

{
  "result": [
    "https://domain/store/context-id/entry/22",
    "https://domain/store/context-id/entry/23",
    "..."
  ]
}

CSV2RowStore Pipeline

The CSV2RowStore pipeline ("transformType": "rowstore") supports the following arguments:

  • action: determines what action to perform; one of the following values:
    • create (default if action parameter is not provided): creates a new dataset in RowStore
    • replace: replaces the data of an already existing dataset, requires the parameter datasetURL
    • append: appends data to an already existing dataset, requires the parameter datasetURL
    • setalias: sets or removes an alias of an existing dataset. Requires the parameters datasetURL and alias. The parameter alias is used to provide the alias to be set (should consist of alpha-numeric unicode characters only), must be omitted or left empty to remove an active alias.

All actions return a reference to exactly one result entry in the response body. In the case of create this will be a newly created entry and for replace and append the previously existing result entry is returned.