Skip to content

Traversal

In addition to regular requests to fetch entries one-by-one there are requests to fetch several entries and/or their metadata with a single request. Performing a search request is one possibility, another one is to perform a graph traversal based on the statements in the entry or metadata graphs. This page explains how to traverse graphs using the REST API.

Traversal of metadata graphs

Parameters:

  • The recursive parameter triggers the traversal and may contain a comma-separated list of URL-encoded URIs and/or traversal profiles (see below for how to configure profiles). The most widely used namespaces (dc, dct, foaf, vcard) are expanded automatically.
  • The scope parameter is optional and determines whether the traversal will be limited to the context of the starting point entry. By default, the traversal will happen across context boundaries in the whole repository, to avoid this the value of scope can be set to context.
  • The format parameter is optional (normal content negotiation works through the Accept-header) and works the same as with normal metadata requests.
  • The depth parameter is optional and determines how many levels the algorithm should traverse. The default depth is 10, and so is the maximum depth unless a profile configures a different max-depth (see below). A depth of 0 or less is treated like an omitted depth.

If the recursive parameter contains neither a configured profile nor a predicate that can be expanded, the request is answered with 400.

The traversal algorithm works as follows:

  1. The metadata of the requested entry is the starting point. a. If the requesting user does not have sufficient rights to read metadata, the request fails immediately.
  2. The recursive parameter is parsed and resolved into a list of URIs. This list forms the list of predicates that are matched in order to detect which objects to follow. Also, an eventually existing list of blacklisted predicate/object combinations is loaded from the configuration.
  3. The entry's metadata graph is loaded and checked against the blacklist. If at least one tuple of the blacklist matches the graph (it is a simple string match, so it does not matter whether the object is a literal or resource), the traversal is aborted for this entry and the currently loaded entry is not included in the result. The traversal continues with the other remaining entries of the same traversal level (if level > 0).
  4. The statements in the metadata graph are iterated over and matched against the list of predicates.
  5. If a predicate matches and the object is a URI that belongs to an entry, the object is fetched as entry and its metadata is traversed. This happens recursively. a. If the traversal is limited to the context, the object URI is tried as resource URI first, and if no entry matches it is tried as entry URI. Otherwise the object URI is looked up as an entry or resource URI of the repository first, and then as resource URI of link entries (which may be external URIs).
  6. Circular dependencies are detected. Also, at most depth levels are traversed (see above).
  7. A merged graph containing metadata (local and external) of all matching and traversed entries is returned.

Example: {base-uri}/1/metadata/99?recursive=dcat,foaf:knows&format=text/turtle

Traversal of entry graphs

This functionality remains to be implemented.

Configuration of traversal profiles

In addition to sending a list of URIs as URL parameter for the request, so called traversal profiles can be predefined in entrystore.properties. A profile consists basically of a name and a list of URIs, see below for DCAT as an example:

entrystore.traversal.dcat.1=http://www.w3.org/ns/dcat#contactPoint
entrystore.traversal.dcat.2=http://purl.org/dc/terms/publisher
entrystore.traversal.dcat.3=http://www.w3.org/ns/dcat#dataset
entrystore.traversal.dcat.4=http://www.w3.org/ns/dcat#distribution
entrystore.traversal.dcat.5=http://schema.theodi.org/odrs#copyrightHolder

The name is provided implicitly through the part of the key after entrystore.traversal. There is no upper limit for the amount of traversal profiles or its containing predicate URIs.

Number the predicates contiguously from .1. In 6.x only this indexed form defines a profile: a value without index (entrystore.traversal.dcat=...) does not register a profile, so a request naming it is answered with 400. Gaps in the numbering are tolerated in 6.x (.1 and .3 yield both predicates). Write sub-keys in the hyphenated form shown below (e.g. max-depth, not maxDepth); a profile name with characters other than lowercase letters, digits and - may not be bound as written in 6.x; write such keys in brackets to keep them verbatim, e.g. entrystore.traversal[my_profile.1]=.... See also indexed lists.

EntryStore 5.x

5.x also accepts a value without index as a profile with a single predicate, and stops reading the predicate list at the first gap in the numbering.

Additional settings

Max depth

The default max traversal depth of 10 can be changed with the following setting:

entrystore.traversal.<profile>.max-depth=20

The setting changes the maximum, not the default depth: without the URL parameter depth a traversal still covers 10 levels, so a client has to send e.g. depth=20 to traverse deeper. If parameter depth is higher than the configured max-depth then depth will be automatically limited to max-depth.

EntryStore 5.x

5.x only applies a max-depth lower than 10; higher values are ignored and the depth is limited to 10.

Limit

By default, at most 1000 entries will be handled during traversal. This can be changed with the following setting:

entrystore.traversal.<profile>.limit=5000

A value of 0 (or less) means unlimited.

Scope

By default, traversal happens across the whole repository. To change this behavior to limit traversal to the context of the starting entry the following setting can be used:

entrystore.traversal.<profile>.repository-scope=false (default: true)

In 6.x the value accepts true/on/yes/1 and false/off/no/0; any other value prevents startup (see value syntax).

May be overridden by URL-parameter scope.

Blacklisting predicate/object combinations

Blacklists can be defined per traversal profile. Despite its configuration per traversal profile, a restriction of the current implementation is that blacklists are applied globally. See the algorithm description above for information on how blacklists work.

A blacklist is configured as a list of tuples which are separated with a comma according to the pattern predicate,object. Common namespaces (see class org.entrystore.repository.util.NS) are expanded.

Example:

entrystore.traversal.dcat.blacklist.1=dcterms:accrualPeriodicity,http://publications.europa.eu/resource/authority/frequency/BIMONTHLY
entrystore.traversal.dcat.blacklist.2=dcterms:format,text/html

This exludes all entry's of which the metadata graphs contain the above predicate/object combinations, and it stops traversal from the matching entry's metadata downwards.

Which settings are applied if both profile(s) and URIs are provided in the traversal request?

If a request lists multiple profiles and/or predicate URIs in the recursive parameter, then the first detected profiles settings will be applied for the whole request.

Example: {base-uri}/1/metadata/99?recursive=dcat,skos,foaf:knows

If the profile dcat is configured with a limit of 0 then this will apply to the whole request, including traversals that will happen based on predicates listed in the skos profile or the foaf:knows predicate.