![\"GitHub\"](\"https://fiware.github.io/tutorials.LD-Subscriptions-Registrations/icon/GitHub-Mark-32px.png\"){kind=icon}
FIWARE 604. Linked Data Subscriptions and RegistrationsThis tutorial discusses the usage of subscriptions and registrations within NGSI-LD and highlights the similarities and\ndifferences between the equivalent NGSI-v2 and NGSI-LD operations. The tutorial is an analogue of the original\ncontext-provider and subscriptions tutorials but uses API calls from the NGSI-LD interface throughout.
\nThe docker-compose files for this tutorial can be found on GitHub:
FIWARE 604. Linked Data Subscriptions and Registrations
\n\n“Do not repeat after me words that you do not understand. Do not merely put on a mask of my ideas, for it will be an\nillusion and you will thereby deceive yourself.”
\n― Jiddu Krishnamurti
\n
NGSI-LD Subscriptions and Registrations provide the basic mechanism to allow the components within a Smart Linked Data\nSolution to interact with each other.
\nAs a brief reminder, within a distributed system, subscriptions inform a third party component that a change in the\ncontext data has occurred (and the component needs to take further actions), whereas registrations tell the context\nbroker that additional context information is available from another source.
\nBoth of these operations require that the receiving component fully understands the requests it receives, and is capable\nof creating and interpreting the resultant payloads. The differences here between NGSI-v2 and NGSI-LD operations is\nsmall, but there has been a minor amendment to facilite the incorporation of linked data concepts, and therefore the\ncontract between the various components has changed to include minor updates.
\nThe relationship between our Linked Data entities is defined as shown, in addition to the existing data, the tweets\nattribute will be supplied by a Context Provider. In all other respects this model remains the same as the\nprevious tutorial :
![\"\"](\"https://fiware.github.io/tutorials.LD-Subscriptions-Registrations/img/entities.png\")
The simple Node.js Express application has updated to use NGSI-LD in the previous\ntutorial. We will use the monitor page to watch the\nstatus of recent requests, and a two store pages to buy products. Once the services are running these pages can be\naccessed from the following URLs:
\nThe event monitor can be found at: http://localhost:3000/app/monitor
![\"FIWARE](\"https://fiware.github.io/tutorials.LD-Subscriptions-Registrations/img/monitor.png\")
Store001 can be found at: http://localhost:3000/app/store/urn:ngsi-ld:Building:store001
![\"Store\"](\"https://fiware.github.io/tutorials.LD-Subscriptions-Registrations/img/store.png\")
Store002 can be found at: http://localhost:3000/app/store/urn:ngsi-ld:Building:store002
![\"Store2\"](\"https://fiware.github.io/tutorials.LD-Subscriptions-Registrations/img/store2.png\")
The demo Supermarket application will send and receive NGSI-LD calls to a compliant context broker. Since the NGSI-LD\ninterface is available on an experimental version of the\nOrion Context Broker, the demo application will only make use of one\nFIWARE component.
\nCurrently, the Orion Context Broker relies on open source MongoDB technology to keep\npersistence of the context data it holds. To request context data from external sources, a simple Context Provider NGSI\nproxy has also been added. To visualize and interact with the Context we will add a simple Express application
\nTherefore, the architecture will consist of four elements:
\nSince all interactions between the elements are initiated by HTTP requests, the entities can be containerized and run\nfrom exposed ports.
\n![\"\"](\"https://fiware.github.io/tutorials.LD-Subscriptions-Registrations/img/architecture.png\")
The necessary configuration information can be seen in the services section of the associated orion-ld.yml file. It\nhas been described in a previous tutorial
All services can be initialised from the command-line by running the\nservices Bash script provided\nwithin the repository. Please clone the repository and create the necessary images by running the commands as shown:
\ngit clone https://github.com/FIWARE/tutorials.LD-Subscriptions-Registrations.git\ncd tutorials.LD-Subscriptions-Registrations\n\n./services orion\n\n\n","schema":"https://schema.getpostman.com/json/collection/v2.0.0/collection.json","toc":[{"content":"Understanding Linked Data Subscriptions and Registrations","slug":"understanding-linked-data-subscriptions-and-registrations"},{"content":"Architecture","slug":"architecture"},{"content":"Start Up","slug":"start-up"}],"owner":"513743","collectionId":"8d924393-5886-4687-95ef-9b2648da537c","publishedId":"SzfDvjN5","public":true,"customColor":{"top-bar":"FFFFFF","right-sidebar":"303030","highlight":"233C68"},"publishDate":"2020-04-28T08:28:23.000Z"},"item":[{"name":"Using Subscriptions with NGSI-LD","item":[{"name":"Create a Subscription ( Store 1) - Low Stock","id":"00dca035-3e58-4d50-a838-ddf33acced2e","protocolProfileBehavior":{"disableBodyPruning":true,"disabledSystemHeaders":{"accept":true}},"request":{"method":"POST","header":[{"key":"Content-Type","value":"application/ld+json","type":"text","name":"Accept"},{"key":"Accept","value":"application/ld+json","type":"text"}],"body":{"mode":"raw","raw":"{\n \"description\": \"Notify me of low stock in Store 001\",\n \"type\": \"Subscription\",\n \"entities\": [\n {\n \"type\": \"Shelf\"\n }\n ],\n \"watchedAttributes\": [\n \"numberOfItems\"\n ],\n \"q\": \"numberOfItems<10;locatedIn==%22urn:ngsi-ld:Building:store001%22\",\n \"notification\": {\n \"attributes\": [\n \"numberOfItems\",\n \"stocks\",\n \"locatedIn\"\n ],\n \"format\": \"keyValues\",\n \"endpoint\": {\n \"uri\": \"http://tutorial:3000/subscription/low-stock-store001\",\n \"accept\": \"application/json\"\n }\n },\n \"@context\": \"http://context/user-context.jsonld\"\n}","options":{"raw":{"language":"json"}}},"url":"http://localhost:1026/ngsi-ld/v1/subscriptions","description":"Note: If you want to clean up and start over again you can do so with the following command:
\n./services stop\n
NGSI-LD subscriptions can be set up using the /ngsi-ld/v1/subscriptions/ endpoint and in a similar manner to the\nNGSI-v2 /v2/subscriptions endpoint. The payload body is slightly different however. Firstly the linked data @context\nmust be present either as an attribute or in the Link header. If the @context is placed in the body the\nContext-Type header must state that the payload is application/ld+json - i.e. Linked Data plus JSON. The supplied\n@context will also be used when making notifications as part of the notification request.
The type of the NGSI-LD subscription request is always type=Subscription. The structure of the subscription has\nchanged. When setting up a subscription, there is no longer a separate subject section to the payload, entities to\nwatch and trigger conditions are now set at the same level as the description of the subscription.
condition.attrs has been moved up a level and renamed to watchedAttributescondition.expression has been moved up a level and renamed to qThe notification section of the body states that once the conditions of the subscription have been met, a POST request\ncontaining all affected Shelf entities will be sent to the URL http://tutorial:3000/subscription/low-stock-store001.\nIt is now possible to amend the notification payload by requesting notification.format=keyValues and remove the\n@context from the notification body by stating notification.endpoint.accept=application/json. The @context is not\nlost, it is merely passed as a Link header. In summary, all of the flags within a subscription work in the same manner\nas a GET request to the context broker itself. If no flags are set, a full NGSI-LD response including the @context is\nreturned by default, and the payload can be reduced and amended by adding in further restriction
This second request fires notifications to a different endpoint (URL\nhttp://tutorial:3000/subscription/low-stock-store002.) The notification.format=normalized and\nnotification.endpoint.accept=application/ld+json will ensure that the @context is passed in the body of the\nnotification request and that the payload will consist of the expanded entities.
Subscription details can be read by making a GET request to the /ngsi-ld/v1/subscriptions/. All subscription CRUD\nactions continue to be mapped to the same HTTP verbs as before. Adding the Accept: application/json will remove the\n@context element from the response body.
The response consists of the details of the subscriptions within the system. The parameters within the q attribute\nhave been expanded to use the full URIs, as internally the broker consistently uses long names. The differences between\nthe payloads offered by the two subscriptions will be discussed below.
Goto http://localhost:3000/app/store/urn:ngsi-ld:Building:store001 to display and interact with the Supermarket data.
Open two tabs on a browser. Go to the event monitor (http://localhost:3000/app/monitor) to see the payloads that are\nreceived when a subscription fires, and then go to store001\n(http://localhost:3000/app/store/urn:ngsi-ld:Building:store001) and buy beer until less than 10 items are in stock.\nThe low stock message should be displayed on screen.
![\"low-stock\"](\"https://fiware.github.io/tutorials.LD-Subscriptions-Registrations/img/low-stock-warehouse.png\")
low-stock-store001 is fired when the Products on the shelves within Store001 are getting low, the subscription payload\ncan be seen below:
![\"low-stock-json\"](\"https://fiware.github.io/tutorials.LD-Subscriptions-Registrations/img/low-stock-monitor.png\")
The data within the payload consists of key-value pairs of the attributes which were specified in the request. This is\nbecause the subscription was created using the format=keyValues attribute. The @context is not present in the\npayload body since endpoint.accept=application/json was set. The effect is to return a data array in a very similar\nformat to the v2/subscription/ payload. In addition to the data array, the subscriptionId is included in the\nresponse, along with a notifiedAt element which describes when the notification was fired.
Now go to store002 (http://localhost:3000/app/store/urn:ngsi-ld:Building:store002) and buy beer until fewer than 10\nitems are in stock. The low stock message is once again displayed on screen, the payload can be seen within the event\nmonitor.
![\"low-stock-ld\"](\"https://fiware.github.io/tutorials.LD-Subscriptions-Registrations/img/low-stock-monitor-ld.png\")
The second subscription has been set up to pass the full normalized NGSI-LD payload along with the @context. This has\nbeen achieved by using the using the format=normalized attribute within the subscription itself, as well as setting\nendpoint.accept=application/ld+json, so that the @context is also passed with each entity.
All NGSI-LD Context Provider Registration actions take place on the /ngsi-ld/v1/csourceRegistrations/ endpoint. The
standard CRUD mappings apply. The @context must be passed either as a Link header or within the main body of the
request.
The body of the request is similar to the NGSI-v2 equivalent with the following modifications:
\ndataProvided object is now an array called information.attrs have been split into separate arrays of propertyNames and relationshipNamesprovider.url has moved up to endpointmode and operations are now required - if they are missing the defaults are federationOps andinclusive which does not match the default NGSI-v2 supportedForwardingModecontextSourceInfo usually defines additional HTTP Headers which are passed to the registrant, but jsonldContext is a
special key which fires a JSON-LD expansion/compaction operation to ensure that the attribute names within the request
match the expected NGSI-v2 attribute names.
\n\n","urlObject":{"protocol":"http","path":["ngsi-ld","v1","csourceRegistrations",""],"host":["localhost:1026"],"query":[],"variable":[]}},"response":[],"_postman_id":"fddd7191-fbde-46c8-9f1e-5b4cd1da6e56"},{"name":"Check the registration","id":"d9b3a660-77d9-4ddf-974a-36061c795703","protocolProfileBehavior":{"disableBodyPruning":true},"request":{"method":"GET","header":[{"key":"Accept","value":"application/ld+json","type":"text"},{"key":"Link","value":"Note that
\npropertyNamesandrelationshipNameshave replaced the olderpropertiesattribute that was is
defined in the 1.1.1 NGSI-LD core context. It was replaced in order to offer full GeoJSON-LD support. Your context
broker may or may not support the updated core context
Retrieving the registration details can be made by sending a GET request to the /ngsi-ld/v1/csourceRegistrations/\nendpoint, along with an appropriate JSON-LD context in the Link header.
The response returns the details of the registration. In this case the short names of the properties have been\nreturned, along with the @context.
Once a registration has been set up, the additional registered properties and relationships are transparently\nreturned when an requested entity is requested. For simple registrations, a request to obtain the whole entity will be\nproxied to the registered endpoint, for partial registrations the properties and relationships are added to the\nexisting entity held within the context broker.
The response now holds an additional tweets Property, which returns the values obtained from\nhttp://context-provider:3000/static/tweets/ngsi-ld/v1/entities/urn:ngsi-ld:Building:store001 - i.e. the forwarding\nendpoint.
The same response data can be seen within the supermarket application itself. In practice this data has been created via\na series of requests - the context broker is responsible for the urn:ngsi-ld:Building:store001 data, however it checks\nto see if any further information can be provided from other sources. In our case the CSourceRegistration indicates\nthat one further attribute may be available. The broker then requests tweets information from the context provider,\nand provided that it responds in a timely manner, the tweets information is added to the resultant payload.
The supermarket application displays the received data on screen within the supermarket application itself:
\n![\"tweets-1\"](\"https://fiware.github.io/tutorials.LD-Subscriptions-Registrations/img/tweets-1.png\")
Every context-provider must stand by a fixed contract. At a minimum must be able to respond to varieties of the\n/ngsi-ld/v1/entities/<entity-id> GET request. If the registration is limited to certain properties, this request will\nalso contain an attrs parameter in the query string.
Dependent upon the use case of the context-provider, it may or may not need to be able to interpret JSON-LD @context -\nin this case a request is merely returning the full tweets attribute.
The same request is made by the context broker itself when querying for registered attributes
\nAs can be seen the @context has been returned in the request (since the Content-Type header was set). The rest of\nthe response resembles any standard NGSI-LD request.
For a read-write interface it is also possible to amend context data by making a PATCH request to the relevant\nngsi-ld/v1/entities/<entity-id>/attrs endpoint.
If the regisitered attribute is requested from the context broker, it returns the updated values obtained from\nhttp://context-provider:3000/static/tweets/ngsi-ld/v1/entities/urn:ngsi-ld:Building:store001 - i.e. the forwarding\nendpoint.
This alters the response to match the values updated in the previous PATCH request.
\nSince the context provider is responsible for supplying tweets information, changes in the context provider will\nalways be reflected in requests to the context-broker itself. The supermarket application is calling the context broker\nfor context regardless of origin, so the updated tweets data are displayed on screen within the supermarket\napplication itself:
![\"tweets-2\"](\"https://fiware.github.io/tutorials.LD-Subscriptions-Registrations/img/tweets-2.png\")
The context broker is therefore able to return a complete holistic picture of the current state of the world.
\n","urlObject":{"protocol":"http","path":["ngsi-ld","v1","entities","urn:ngsi-ld:Building:store001"],"host":["localhost:1026"],"query":[{"disabled":true,"key":"options","value":"keyValues"}],"variable":[]}},"response":[],"_postman_id":"5a27cb10-4b2b-476f-98b2-8cdf78c197b8"},{"name":"Forwarded Update","id":"019c2990-a792-42d3-a592-2d26fc6ffdf7","protocolProfileBehavior":{"disableBodyPruning":true},"request":{"method":"PATCH","header":[{"key":"Link","value":"A PATCH request to the context broker ( either ngsi-ld/v1/entities/<entity-id>/ or\nngsi-ld/v1/entities/<entity-id>/attrs) will be forwarded to the registered context provider if a registration is\nfound. It is therefore possible to alter the state of a context-provider as a side effect. Of course, not all context\nproviders are necessarily read-write, so attempting to change the attributes of forwarded context may not be fully\nrespected.
In this case however a request to PATCH ngsi-ld/v1/entities/<entity-id> will be successfully forwarded as a series of\nngsi-ld/v1/entities/<entity-id>/attrs requests for each regsitered attribute that is found in the registration.
The result of the previous operation can be seen by retrieving the whole entity using a GET request.
\nThis alters the response to match the values updated in the previous PATCH request which was sent to the context broker\nand then forwarded to the context provider endpoint.
\nAs can be seen, the updated tweets data is also displayed within the supermarket application itself:
![\"tweets-3\"](\"https://fiware.github.io/tutorials.LD-Subscriptions-Registrations/img/tweets-3.png\")
Context Registrations allow some (or all) data within an entity to be provided by an external context provider. It could
be another full context-provider a separate micro-service which only responds to a subset of the NGSI-LD endpoints.
However, there needs to be a contract created as to who supplies what.
All NGSI-LD registrations can be subdivided into one of four types:
\nA Context Broker is permitted to hold context data about the Entities and Attributes locally itself, and also obtain
data from (possibly multiple) external sources
A Context Broker is not permitted to hold context data about the Entities and Attributes locally itself. All context
data is obtained from the external registered sources.
NGSI-LD also defines groups of operations that are allowed on the registrant. The default group is calledfederationOps and includes all entity GET operations. Three other common operational groups are also definedupdateOps (for actuators), retrieveOps (for \"lazy\" sensors) and redirectionOps (for hierarchical broker
architectures). The details won't be covered here, but it should be noted that unless specified, the default NGSI-LD
operation is federationOps using inclusive mode, whereas the default NGSI-v2 operation isupdateOps + retrieveOps using exclusive mode.
For simplicity, for NGSI-v2 - NGSI-LD comparison, this tutorial will only deal with exclusive mode, which is
the only mode offered by NGSI-v2 brokers.
With the NGSI-LD exclusive mode, all registrations can be subdivided into one of two types. Simple registrations
where a single context provider is responsible for the maintenance of the whole entity, and partial registrations where
attributes are spread across multiple context providers. For a simple registration, all context requests are forwarded
Effectively every simple registration is saying \"this entity is held elsewhere\", but the entity data can be requested
and modified via requests to this context broker.
For partial registrations the situation is more complex
\nEach partial registration is saying \"additional augmented context for this entity is held elsewhere\". The entity data
can be requested and modified via requests to this context broker.
With normal operation, the NGSI-LD response does not expose whether data collated from multiple sources is held directly
within the context broker or whether the information has been retrieved externally. It is only when an error occurs
(e.g. timeout) that the HTTP status error code reveals that externally held information could not be retrieved or
amended.