![\"GitHub\"](\"https://fiware.github.io/tutorials.Getting-Started/icon/GitHub-Mark-32px.png\"){kind=icon}
FIWARE-LD 102: Working with @context This tutorial examines the interaction between NGSI-LD and JSON-LD @context files. The @context files generated in the previous tutorial are used as the underlying data model for inputing context data and context information is queries and read back in different formats.
The docker-compose file for this tutorial can be found on GitHub:
FIWARE-LD 102: Working with @context
@context files\n\n“Some quotations are greatly improved by lack of context.”
\n― John Wyndham, The Midwich Cuckoos
\n
From the previous tutorial we have generated two @context files defining the context data entities which will be offered in our simple Smart Farm Management System. This means that we have defined an agreed set of unique IDs (URNs or URLs) for all the\ndata entities and every single attribute within those entities so that other external applications will be able to programmatically understand the data held within our broker.
For example the attribute address is within our smart application is defined as follows:
\"@context\": {\n \"schema\": \"https://schema.org/\",\n \"address\": \"schema:address\"\n}\nWhich means that every address attribute follows the definition as defined by schema.org:
https://schema.org/address :
![\"\"](\"https://fiware.github.io/tutorials.Working-with-At-Context/img/architecture-ld.png\")
A program written by a third party would therefore be able to extract information such the fact an address attribute holds a JSON object with a sub-attribute containing the streetAddress by referring to the full schema.org JSON-LD schema
{\n \"@id\": \"http://schema.org/streetAddress\",\n \"@type\": \"rdf:Property\",\n \"http://schema.org/domainIncludes\": {\n \"@id\": \"http://schema.org/PostalAddress\"\n },\n \"http://schema.org/rangeIncludes\": {\n \"@id\": \"http://schema.org/Text\"\n },\n \"rdfs:comment\": \"The street address. For example, 1600 Amphitheatre Pkwy.\",\n \"rdfs:label\": \"streetAddress\"\n}\nThis is the JSON-LD programmatic syntax allowing computers to extract meaningful data The attribute address.streetAddress is a street directly without the need for human intervention.
Imagine the case where a company is contracting agricultural labourers. The farmer will need to be billed for the work done. If such a system is built on JSON-LD, it does not matter if the farmer's Farm Management Information System assigns different names to the attributes of the billing address provided that the farmer and contractor can agree on the well-defined URNS for each attribute as JSON-LD can easily translate between the two formats using common expansion and compaction algorithms.
\nNGSI-LD is a formally structured extended subset of JSON-LD. Therefore NGSI-LD offers all the interoperability and flexibility of JSON-LD itself. It also defines its own core @context which cannot be overridden for NGSI-LD operations. This means that NGSI-LD users agree to a common well defined set of rules for structuring their data, and then supplement this with the rest of the JSON-LD specification.
Whilst interacting directly with NGSI-LD interface of the context broker the additional NGSI-LD rules must be respected. However after the data has been extracted it is possible to loosen this requirement and pass the results to third parties as JSON-LD.
\nThis tutorial is a simple introduction to the rules and restrictions behind NGSI-LD and will create some NGSI-LD entities and then extract the data in different formats. The two main data formats are normalized and key-value-pairs. Data returned in the normalised format respects the NGSI-LD rules and may be used directly by another context broker (or any other component offering an NGSI-LD interface). Data returned in the key-value-pairs format is by definition not NGSI-LD.
\nContent-Type and Accept HeadersDuring content negociation, NGSI-LD offers data in one of three formats, these effect the structure of the payload body.
\nAccept: application/json - the response is in JSON formatAccept: application/ld+json - the response is in JSON-LD formatAccept: application/geo+json - the response is in GeoJSON or GeoJSON-LD formatThe major difference between JSON format and JSON-LD format, is that if JSON-LD format is chosen, then the @context is found as an additional attribute within the body of the response. if the JSON only format is used the @context is passed as an additional Link Header element and is not found in the response body.
Similarly when sending NGSI-LD data to the context broker, an application may choose to send a payload including an additional @context attribute (in which case Content-Type: application/ld+json) or the application may send NGSI-LD data without an additional @context attribute (in which case Content-Type: application/json and the Link header must also be present).
The GeoJSON format is only used when querying a context broker for existing data and returns the context in a format suitable for GIS systems. It is a recent addition to the NGSI-LD specification and therefore will not be discussed further here.
\nTo keep things simple all components will be run using Docker. Docker is a container\ntechnology which allows to different components isolated into their respective environments.
\nDocker Compose is a tool for defining and running multi-container Docker applications. A\nYAML file is used\nconfigure the required services for the application. This means all container services can be brought up in a single\ncommand. Docker Compose is installed by default as part of Docker for Windows and Docker for Mac, however Linux users\nwill need to follow the instructions found here
\nWe will start up our services using a simple bash script. Windows users should download cygwin\nto provide a command-line functionality similar to a Linux distribution on Windows.
\n","schema":"https://schema.getpostman.com/json/collection/v2.0.0/collection.json","toc":[{"content":"Working with @context files","slug":"working-with-context-files"},{"content":"Prerequisites","slug":"prerequisites"}],"owner":"513743","collectionId":"37a6dc59-bcbb-4813-b1b3-5b088f4e3bdd","publishedId":"T1LV7iXM","public":true,"customColor":{"top-bar":"FFFFFF","right-sidebar":"303030","highlight":"233C68"},"publishDate":"2020-08-21T14:22:49.000Z"},"item":[{"name":"Reading @Context","item":[{"name":"Obtain the NGSI-LD Linked Data context","id":"1871d872-9b69-4147-811f-8a290ddc8e84","protocolProfileBehavior":{"disableBodyPruning":true},"request":{"method":"GET","header":[],"url":"http://localhost:3004/ngsi-context.jsonld","description":"The NGSI-LD @context serves to define all attributes when sending data to the context broker or retrieving data in normalized format. This @context must be used for all NGSI-LD to NGSI-LD interactions
The JSON-LD @context can be used when querying the data and returning key-values data. Responses are JSON or JSON-LD and the data can be easily ingested and processed further by the receiving application.
json-context.jsonld is a richer JSON-LD definition of the attributes of the data models.alternate-context.jsonld is an alternative JSON-LD definition of the attributes of the data models used by a third-party (the German sub-contractor of farm labourers). Internally their billing application used different short names for attributes. Their @context file reflects the agreed mapping between attribute names.As usual, you can check if the Orion Context Broker is running by making an HTTP request to the exposed port.
\nThe response will look similar to the following:
\n{\n \"orion\": {\n \"version\": \"1.15.0-next\",\n \"uptime\": \"0 d, 3 h, 1 m, 51 s\",\n \"git_hash\": \"af440c6e316075266094c2a5f3f4e4f8e3bb0668\",\n \"compile_time\": \"Tue Jul 16 15:46:18 UTC 2019\",\n \"compiled_by\": \"root\",\n \"compiled_in\": \"51b4d802385a\",\n \"release_date\": \"Tue Jul 16 15:46:18 UTC 2019\",\n \"doc\": \"https://fiware-orion.readthedocs.org/en/master/\"\n }\n}\nThe format of the version response has not changed. The release_date must be 16th July 2019 or later to be able to\nwork with the requests defined below.
The demo application will send and receive NGSI-LD calls to a compliant context broker. Since the standardized NGSI-LD interface is available across\nmultiple context brokers, so we only need to pick one - for example the\nOrion Context Broker. The application will therefore only make use of one FIWARE component.
\nCurrently, the Orion Context Broker relies on open source MongoDB technology to keep\npersistence of the context data it holds.
\nTo promote interoperability of data exchange, NGSI-LD context brokers explicitly expose a JSON-LD @context file to define the data held within the context entities. This defines a unique URI for every entity type and every attribute so that other services outside of the NGSI domain are able to pick and choose the names of their data structures. Every @context file must be available on the\nnetwork. In our case the tutorial application will be used to host a series of static files.
Therefore, the architecture will consist of three elements:
\n@context files defining the context entities within the system.Since all interactions between the three elements are initiated by HTTP requests, the elements can be containerized and\nrun from exposed ports.
\n![](\"https://fiware.github.io/tutorials.Working-with-At-Context/img/architecture.png\")
The necessary configuration information can be seen in the services section of the associated orion-ld.yml file:
orion:\n image: fiware/orion-ld\n hostname: orion\n container_name: fiware-orion\n depends_on:\n - mongo-db\n networks:\n - default\n ports:\n - \"1026:1026\"\n command: -dbhost mongo-db -logLevel DEBUG\n healthcheck:\n test: curl --fail -s http://orion:1026/version || exit 1\nmongo-db:\n image: mongo:3.6\n hostname: mongo-db\n container_name: db-mongo\n expose:\n - \"27017\"\n ports:\n - \"27017:27017\"\n networks:\n - default\n command: --nojournal\ntutorial:\n image: fiware/tutorials.ngsi-ld\n hostname: tutorial\n container_name: fiware-tutorial\n networks:\n default:\n aliases:\n - context\n expose:\n - 3000\nAll containers are residing on the same network - the Orion Context Broker is listening on Port 1026 and MongoDB is\nlistening on the default port 27017 and the tutorial app is listening on port 3000. All containers are also exposing the same ports externally - this is purely for\nthe tutorial access - so that cUrl or Postman can access them without being part of the same network. The command-line\ninitialization should be self explanatory.
All services can be initialised from the command-line by running the\nservices Bash script provided within the\nrepository. Please clone the repository and create the necessary images by running the commands as shown:
\ngit clone https://github.com/FIWARE/tutorials.Working-with-At-Context.git\ncd tutorials.Working-with-At-Context\n\n./services orion|scorpio\n\n\n","event":[{"listen":"prerequest","script":{"id":"391fedfe-2adf-4403-a57a-a5ea43bbbe56","type":"text/javascript","exec":[""]}},{"listen":"test","script":{"id":"4114c796-0df5-4d11-9397-de5f68ed1bf2","type":"text/javascript","exec":[""]}}],"_postman_id":"8702e883-0048-43e9-925a-0c10f6629062"},{"name":"Creating your first Data Entity","id":"95499812-dd17-48c5-a894-0bab4f5ecb11","protocolProfileBehavior":{"disableBodyPruning":true},"request":{"method":"POST","header":[{"key":"Content-Type","value":"application/ld+json"}],"body":{"mode":"raw","raw":"{\n \"id\": \"urn:ngsi-ld:Building:farm001\",\n \"type\": \"Building\",\n \"category\": {\n \t\"type\": \"VocabProperty\",\n \"vocab\": [\"farm\"]\n },\n \"address\": {\n \"type\": \"Property\",\n \"value\": {\n \"streetAddress\": \"Großer Stern 1\",\n \"addressRegion\": \"Berlin\",\n \"addressLocality\": \"Tiergarten\",\n \"postalCode\": \"10557\"\n },\n \"verified\": {\n\t\t\t\"type\": \"Property\",\n\t\t\t\"value\": true\n\t\t}\n },\n \"location\": {\n \"type\": \"GeoProperty\",\n \"value\": {\n \"type\": \"Point\",\n \"coordinates\": [13.3505, 52.5144]\n }\n },\n \"name\": {\n \"type\": \"Property\",\n \"value\": \"Victory Farm\"\n },\n \"@context\": \"http://context/ngsi-context.jsonld\"\n}"},"url":"http://localhost:1026/ngsi-ld/v1/entities/","description":"Note: If you want to clean up and start over again you can do so with the following command:
\n./services stop\n
New context data entities can be created by making a POST request to the /ngsi-ld/v1/entities endpoint and supply an @context along with structured NGSI-LD data.
The first request will take some time, as the context broker must navigate and load all of the files mentioned in the\n@context.
Since the Content-Type: application/ld+json the @context is supplied in the body of the request. As with all NGSI-LD interactions, the core NGSI-LD @context (https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld is implicitly included as well.)
This means that the actual @context is:
{\n \"@context\": [\n \"http://context-provider:3000/data-models/ngsi-context.jsonld\",\n \"https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld\"\n ]\n}\nwith the core @context being processed last and therefore overriding any terms previously defined with the same @id.
Each subsequent entity must have a unique id for the given type.
In this second case the Content-Type: application/json so the @context is supplied in the associated Link header of the request and not in the payload body.
@context - defining NGSI-LD entitiesThe core @context supplies the vocabulary for creating NGSI-LD data entities. Attributes such as id and type ( which should be familiar to anyone who has used NGSI v2) are mapped to the standard JSON-LD @id and @type keywords. The type should refer to an included data model, in this case Building is being used as a short name\nfor the included URN https://uri.fiware.org/ns/data-models#Building. Thereafter each property is defined as a JSON\nelement containing two attributes, a type and a value.
The type of a property attribute must be one of the following:
\"GeoProperty\": \"http://uri.etsi.org/ngsi-ld/GeoProperty\" for locations. Locations should be specified as\nLongitude-Latitude pairs in GeoJSON format. The preferred name for the\nprimary location attribute is location\"Property\": \"http://uri.etsi.org/ngsi-ld/Property\" - for everything else.\"Property\" shall be used as well, but the property value should be Date, Time or DateTime\nstrings encoded in the ISO 8601 format - e.g. YYYY-MM-DDThh:mm:ssZ\n\nNote: that for simplicity, this data entity has no relationships defined. Relationships must be given the\n
\ntype=Relationship. Relationships will be discussed in a subsequent tutorial.
Properties-of-Properties is the NGSI-LD equivalent of metadata (i.e. \"data about data\"), it is use to describe\nproperties of the attribute value itself like accuracy, provider, or the units to be used. Some built-in metadata\nattributes already exist and these names are reserved:
\ncreatedAt (type: DateTime): attribute creation date as an ISO 8601 string.modifiedAt (type: DateTime): attribute modification date as an ISO 8601 string.Additionally observedAt, datasetId and instanceId may optionally be added in some cases, and location,\nobservationSpace and operationSpace have special meaning for Geoproperties.
In the examples given above, one element of metadata (i.e. a property-of-a-property) can be found within the address\nattribute. a verified flag indicates whether the address has been confirmed. The commonest property-of-a-property is\nunitCode which should be used to hold the UN/CEFACT\nCommon Codes for Units of Measurement.
When filtering by type, a Link header must be supplied to associate the short form type=\"Building\"\nwith the FQN https://uri.fiware.org/ns/data-models/Building.
If a reference to the supplied data is supplied, it is possible to return short name data and limit responses to a\nspecific type of data. For example, the request below returns the data of all Building entities within the context\ndata. Use of the type parameter limits the response to Building entities only, use of the options=keyValues query\nparameter reduces the response down to standard JSON-LD.
Because of the use of the options=keyValues, the response consists of JSON only without the attribute definitions\ntype=\"Property\" or any properties-of-properties elements. You can see that Link header from the request has been\nused as the @context returned in the response.
keyValues option in order to get a more compact and brief representation, including just attribute valuesvalues option combined with a list of attribute values attrs for an ordered list of attributes onlyThis example returns the data of all Building entities within the context data The type parameter is mandatory for\nNGSI-LD and is used to filter the response. The Accept HTTP header is needed to retrieve JSON-LD content in the response body.
Since no explicit @context was sent in the request, the response returns the Core @context by default (https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld) and\nall attributes are expanded whenever possible.
id, type, location and nameare defined in the core context and are not expanded.address has been mapped to http://schema.org/addresscategory has been mapped to https://uri.fiware.org/ns/data-models#categoryNote that if an attribute has not been not associated to an FQN when the entity was created, the short name will\nalways be displayed.
\n","urlObject":{"protocol":"http","path":["ngsi-ld","v1","entities"],"host":["localhost:1026"],"query":[{"key":"type","value":"https://uri.fiware.org/ns/dataModels%23Building"}],"variable":[]}},"response":[],"_postman_id":"e01b9a55-e577-4b6e-b8d8-2521d0f33f4e"},{"name":"Obtain Entity Data by id","id":"6e0f2c4f-b7e7-44ae-8188-0023ad935095","protocolProfileBehavior":{"disableBodyPruning":true},"request":{"method":"GET","header":[{"key":"Accept","value":"application/ld+json","type":"text"},{"key":"Link","value":"This example returns the data of urn:ngsi-ld:Building:farm001. The NGSI-LD @context is supplied as a Link header to define the entities returned. The ngsi-context.jsonld @context file is just supplying short names for every attribute.
The full link header syntax can be seen below:
\nLink: <https://fiware.github.io/data-models/context.jsonld>; rel=\"http://www.w3.org/ns/json-ld#context\"; type=\"application/ld+json\nThe standard HTTP Link header allows metadata (in this case the @context) to be passed in without actually touching\nthe resource in question. In the case of NGSI-LD, the metadata is a file in application/ld+json format.
Entity type, to avoid ambiguity in case there are several entities with the same entity id
\n","type":"text/plain"},"key":"type","value":"Store"},{"disabled":true,"description":{"content":"Ordered list of attribute names to display
\n","type":"text/plain"},"key":"attrs","value":"name"}],"variable":[]}},"response":[],"_postman_id":"6e0f2c4f-b7e7-44ae-8188-0023ad935095"},{"name":"Filter context data by attribute value","id":"40e61360-a25c-47c4-bec2-d19d75c28199","protocolProfileBehavior":{"disableBodyPruning":true},"request":{"method":"GET","header":[{"key":"Link","type":"text","value":"This example returns all Building entities with the name attribute Big Red Barn. Filtering can be done using\nthe q parameter - if a string has spaces in it, it can be URL encoded and held within double quote characters \" =\n%22. Since options=keyValues is sent, this will effect the structure of the payload and we will need to supply a different @context file - json-context.jsonld
The use of the Link header and the options=keyValues parameter reduces the response to short form key-values JSON-LD
This JSON-LD is no longer NGSI-LD (since the type and value elements have been removed) and the @context used reflects this. The json-context.jsonld file does not merely define the attribute names, it also includes additional JSON-LD information within it such as the following:
{\n \"barn\": \"https://wiki.openstreetmap.org/wiki/Tag:building%3Dbarn\",\n \"category\": {\n \"@id\": \"https://uri.fiware.org/ns/data-models#category\",\n \"@type\": \"@vocab\"\n }\n}\nThis indicates the category in this JSON-LD response holds an enumerated value (@vocab) and that the value barn is defined by a full URL. This differs compared to the ngsi-context.jsonld @context file where all we can say is that there is an attribute with the full URL https://uri.fiware.org/ns/data-models#category, because in a normalized NGSI-LD response the category attribute would hold a JSON object (with a type and value) not a string.
keyValues option in order to get a more compact and brief representation, including just attribute valuesvalues option combined with a list of attribute values attrs for an ordered list of attributes onlyThe simple NGSI-LD @context is merely a mechanism for mapping URNs. It is therefore possible to retrieve the same data using a different set of short names.
The alternate-context.jsonld maps the names of various attributes to their equivalents in German. If it is supplied in the request a query can be made using alternate short names (e.g. type=Building becomes type=Gebäude)
The response is returned in JSON-LD format with short form attribute names (addresse, katagorie) which correspond to the short names provided in the alternate context. Note that core context terms (id, type etc.) cannot be overridden directly but would require an additional JSON-LD expansion/compaction operation.
It should also be noted that the sub-attributes of the addrese have also not been amended, since address = addrese =http://schema.org/address and this\ndefinition defines the sub-attributes.
keyValues option in order to get a more compact and brief representation, including just attribute valuesvalues option combined with a list of attribute values attrs for an ordered list of attributes onlyWithin the standard Building model, the category attribute refers to an array of strings. This example returns all\nBuilding entities with a category attribute which contains either commercial or office strings. Filtering can be\ndone using the q parameter, comma separating the acceptable values.
The response is returned in JSON-LD format with short form attribute names.
\n","urlObject":{"protocol":"http","path":["ngsi-ld","v1","entities",""],"host":["localhost:1026"],"query":[{"key":"type","value":"Building"},{"key":"q","value":"category==\"barn\",\"farm_auxiliary\""},{"description":{"content":"keyValues option in order to get a more compact and brief representation, including just attribute valuesvalues option combined with a list of attribute values attrs for an ordered list of attributes onlyThis example returns all stores found in the Tiergarten District.
\nFiltering can be done using the q parameter - sub-attributes are annotated using the bracket syntax e.g.\nq=address[addressLocality]==\"Tiergarten\".
Use of the Link header and the options=keyValues parameter reduces the response to JSON-LD.
keyValues option in order to get a more compact and brief representation, including just attribute valuesvalues option combined with a list of attribute values attrs for an ordered list of attributes onlyThis example returns the data of all Building entities with a verified address. The verified attribute is an example\nof a Property-of-a-Property
Metadata queries (i.e. Properties of Properties) are annotated using the dot syntax e.g. q=address.verified==true.
Because of the use of the options=keyValues together with the Accept HTTP header (application/json), the response\nconsists of JSON only without the attribute type and metadata elements.
keyValues option in order to get a more compact and brief representation, including just attribute valuesvalues option combined with a list of attribute values attrs for an ordered list of attributes onlyThis example return all buildings within 800m the Brandenburg Gate in Berlin (52.5162N 13.3777W). To make a\ngeo-query request, three parameters must be specified, geometry, coordinates and georel.
The coordinates parameter is represented in\ngeoJSON including the square brackets.
Note that by default the geo-query will be applied to the location attribute, as this is default specified in NGSI-LD.\nIf another attribute is to be used, an additional geoproperty parameter is required.
Because of the use of the options=keyValues together with the Accept HTTP header (application/json), the response\nconsists of JSON only without the attribute type and metadata elements.
Ordered list of attribute names to display
\n","type":"text/plain"},"key":"georel","value":"near%3BmaxDistance==800"},{"key":"options","value":"keyValues"}],"variable":[]}},"response":[],"_postman_id":"6943ebaf-ddf1-4412-abcd-af6197cd8e87"}],"event":[{"listen":"prerequest","script":{"id":"3f85b567-512e-461f-81ce-4fa7aceb9c21","type":"text/javascript","exec":[""]}},{"listen":"test","script":{"id":"3b5ad0d2-9d3f-48b8-b8b6-324544a0a0e1","type":"text/javascript","exec":[""]}}],"variable":[{"key":"json-context.jsonld","value":"http://context/json-context.jsonld"},{"key":"ngsi-context.jsonld","value":"http://context/ngsi-context.jsonld"},{"key":"context-broker","value":"localhost:1026"},{"key":"alternate-context.jsonld","value":"http://context/alternate-context.jsonld"}]}