![\"GitHub\"](\"https://fiware.github.io/tutorials.Subscriptions/icon/GitHub-Mark-32px.png\"){kind=icon}
FIWARE 106: Subscribing to Changes in ContextThis tutorial teaches NGSI-LD users about how to create and manage context data subscriptions. The tutorial builds on the\nentities and Smart Farm\napplication created in the previous examples to enable users to\nunderstand the NGSI-LD Subscribe/Notify paradigm and how to\nuse NGSI subscriptions within their own code.
\nThe tutorial refers to devices and actions made within the browser combined with cUrl\ncommands. The cUrl commands are also available as\nPostman documentation.
\nThe docker-compose file for this tutorial can be found on GitHub:
FIWARE 106: Subscribing to Changes in Context
\n\n'Another sandwich!' said the King.
\n'There's nothing but hay left now,' the Messenger said, peeping into the bag.
\n'Hay, then,' the King murmured in a faint whisper.
\nAlice was glad to see that it revived him a good deal. 'There's nothing like eating hay when you're faint,' he remarked to her, as he munched away.
\n— Lewis Carroll (Through the Looking-Glass and What Alice Found There)
\n
Within the FIWARE platform, an entity represents the state of a physical or conceptual object which exists in the real\nworld. Every smart solution needs to know the current state of these object at any given moment in time.
\nThe context of each of these entities is constantly changing. For example, within the smart farm example, the\ncontext will change as animals and vehicles move, soil dries out, tasks are allocated on the farm and completed and so on. For a smart solution based on IoT\nsensor data, this issue is even more pressing as the system will constantly be reacting to changes in the real world.
\nUntil now all the operations we have used to change the state of the system have been synchronous - changes have\nbeen made by directly by a user or application and they have been informed of the result. The Orion Context Broker\noffers also an asynchronous notification mechanism - applications can subscribe to changes of context information so\nthat they can be informed when something happens. This means the application does not need to continuously poll or\nrepeat query requests.
\nUse of the subscription mechanism will therefore reduce both the volume of requests and amount of data being passed\nbetween components within the system. This reduction in network traffic will improve the overall responsiveness.
\nThe relationship between our entities is defined as shown:
\n![\"\"](\"https://fiware.github.io/tutorials.Subscriptions/img/ngsi-ld-entities.png\")
In a previous tutorial, a simple Node.js Express application\nwas created. This tutorial will use the monitor page to watch the status of recent requests, and the devices page to alter the machines on the farm. Once the services are running these pages can be accessed from the following URLs:
\nThe event monitor can be found at: http://localhost:3000/app/monitor
![\"FIWARE](\"https://fiware.github.io/tutorials.Subscriptions/img/monitor.png\")
For the purpose of this tutorial, a series of dummy agricultural IoT devices have been created, which will be attached\nto the context broker. Details of the architecture and protocol used can be found in the\nIoT Sensors tutorial The state of each device can be\nseen on the UltraLight device monitor web page found at: http://localhost:3000/device/monitor
![\"FIWARE](\"https://fiware.github.io/tutorials.Subscriptions/img/farm-devices.png\")
This application will make use of two FIWARE components - the\nOrion-LD Context Brokerand the\nIoT Agent for UltraLight 2.0. Usage of any NGSI-LD Context Broker is sufficient\nfor an application to qualify as “Powered by FIWARE”.
\nCurrently, the Orion-LD Context Broker relies on open source MongoDB technology to keep\npersistence of the context data it holds. To request context data from external sources, a simple Context Provider\nNGSI proxy has also been added. To visualize and interact with the Context we will add a simple Express Frontend\napplication
\nTherefore, the architecture will consist of four elements:
\n@context files defining the context entities within the system.Since all interactions between the elements are initiated by HTTP requests, the entities can be containerized and run\nfrom exposed ports.
\n![\"\"](\"https://fiware.github.io/tutorials.Subscriptions/img/architecture-ld.png\")
The necessary configuration information can be seen in the services section of the associated docker-compose.yml file.\nIt has been described in a previous tutorial
To keep things simple both components will be run using Docker. Docker is a container\ntechnology which allows to different components isolated into their respective environments.
\nDocker Compose is a tool for defining and running multi-container Docker applications. A\nYAML file is used\nconfigure the required services for the application. This means all container services can be brought up in a single\ncommand. Docker Compose is installed by default as part of Docker for Windows and Docker for Mac, however Linux users\nwill need to follow the instructions found here
\nYou can check your current Docker and Docker Compose versions using the following commands:
\ndocker-compose -v\ndocker version\nPlease ensure that you are using Docker version 18.03 or higher and Docker Compose 1.21 or higher and upgrade if\nnecessary.
\nWe will start up our services using a simple bash script. Windows users should download cygwin\nto provide a command-line functionality similar to a Linux distribution on Windows.
\nAll services can be initialized from the command-line by running the bash script provided within the repository. Please\nclone the repository and create the necessary images by running the commands as shown:
\ngit clone https://github.com/FIWARE/tutorials.Subscriptions.git\ncd tutorials.Subscriptions\ngit checkout NGSI-LD\n\n./services create;\n./services orion;\nThis command will also import seed data from the previous\nFarm Management Information System example on startup, and provision a series of dummy devices on the farm.
\n\n\n:information_source: Note: If you want to clean up and start over again you can do so with the following command:
\n\n./services stop\n
To follow the tutorial correctly please ensure you have the follow two pages available on separate tabs in your browser before you\nenter any cUrl commands.
\nDetails of various buildings around the farm can be found in the tutorial application. Open http://localhost:3000/app/farm/urn:ngsi-ld:Building:farm001 to display a building with an associated filling sensor and thermostat.
![\"\"](\"https://fiware.github.io/tutorials.Subscriptions/img/fmis.png\")
The event monitor can be found at: http://localhost:3000/app/monitor.
![\"\"](\"https://fiware.github.io/tutorials.Subscriptions/img/low-stock-farm.png\")
A new subscription can be added by making a POST request to the /ngsi-ld/v1/subscriptions/ endpoint as shown below.
The body of the POST request consists of two parts, the first section of the request (consisting of entities, type, watchedAttributes and\nq)states that the subscription will be checked whenever the filling attribute of a FillingSensor entity is altered. This is further refined by the q parameter so that the actual subscription is only fired for any FillingSensor entity linked to the Building urn:ngsi-ld:Building:farm001 and only when the filling attribute drops below 0.8
The notification section of the body states that once the conditions of the subscription have been met, a POST\nrequest containing all affected FillingSensor entities will be sent to the URL\nhttp://tutorial:3000/subscription/low-stock-farm001 which is handled by the contractor's own system.
It should be noted that the subscription is using the NGSILD-Tenant header because the IoT Devices have been provisioned using a separate tenant to the buildings for now. Tenants allow for context data to be distributed across separate databases and allow multiple application clients to access the same context broker but keep their own data sets apart.
Go to the Device Monitor http://localhost:3000/app/farm/urn:ngsi-ld:Building:farm001 and start removing hay from the barn. Nothing happens until the barn is half-empty, then a request is sent to\nsubscription/low-stock-farm001 as shown:
http://localhost:3000/app/monitor
{\n \"id\": \"urn:ngsi-ld:Notification:5fd0f3824eb81930c97005d8\",\n \"type\": \"Notification\",\n \"subscriptionId\": \"urn:ngsi-ld:Subscription:5fd0ee554eb81930c97005c1\",\n \"notifiedAt\": \"2020-12-09T15:55:46.520Z\",\n \"data\": [\n {\n \"id\": \"urn:ngsi-ld:Device:filling001\",\n \"type\": \"FillingSensor\",\n \"controllingAsset\": \"urn:ngsi-ld:Building:farm001\",\n \"filling\": 0.59\n }\n ]\n}\nCode within the Farm Management Information System handles received the POST request as shown:
\nconst NOTIFY_ATTRIBUTES = ['controllingAsset', 'type', 'filling', 'humidity', 'temperature'];\n\nrouter.post('/subscription/:type', (req, res) => {\n monitor('notify', req.params.type + ' received', req.body);\n _.forEach(req.body.data, (item) => {\n broadcastEvents(req, item, NOTIFY_ATTRIBUTES);\n });\n res.status(204).send();\n});\n\nfunction broadcastEvents(req, item, types) {\n const message = req.params.type + \" received\";\n _.forEach(types, (type) => {\n if (item[type]) {\n req.app.get(\"io\").emit(item[type], message);\n }\n });\n}\nThis business logic emits socket I/O events to any registered parties (such as the contractor who will then refill the barn.)
\n","urlObject":{"protocol":"http","path":["ngsi-ld","v1","subscriptions",""],"host":["localhost:1026"],"query":[],"variable":[]}},"response":[],"_postman_id":"b38b758c-0b3c-4bbf-ac05-22fbfbac9ce8"},{"name":"NGSI-LD normalized Payload","id":"def367e0-b846-45ad-adc4-7d3536baf72e","protocolProfileBehavior":{"disableBodyPruning":true},"request":{"method":"POST","header":[{"key":"Content-Type","value":"application/ld+json","type":"text"},{"key":"NGSILD-Tenant","value":"openiot","type":"text"}],"body":{"mode":"raw","raw":"{\n \"description\": \"Notify me of low feedstock on Farm:001\",\n \"type\": \"Subscription\",\n \"entities\": [\n {\n \"type\": \"FillingLevelSensor\"\n }\n ],\n \"watchedAttributes\": [\n \"filling\"\n ],\n \"q\": \"filling>0.4;filling<0.6;controlledAsset==%22urn:ngsi-ld:Building:farm001%22\",\n \"notification\": {\n \"attributes\": [\n \"filling\",\n \"controlledAsset\"\n ],\n \"format\": \"normalized\",\n \"endpoint\": {\n \"uri\": \"http://tutorial:3000/subscription/low-stock-farm001-ngsild\",\n \"accept\": \"application/json\"\n }\n },\n \"@context\": \"http://context/user-context.jsonld\"\n}"},"url":"http://localhost:1026/ngsi-ld/v1/subscriptions/","description":"This second subscription will fire when the filling level is between 0.6 and 0.4. The format attribute has been altered to inform the subscriber using NGSI-LD normalized format.
When a low-stock-farm001-ngsild event is fired, the response is as shown:
{\n \"id\": \"urn:ngsi-ld:Notification:5fd0fa684eb81930c97005f3\",\n \"type\": \"Notification\",\n \"subscriptionId\": \"urn:ngsi-ld:Subscription:5fd0f69b4eb81930c97005db\",\n \"notifiedAt\": \"2020-12-09T16:25:12.193Z\",\n \"data\": [\n {\n \"id\": \"urn:ngsi-ld:Device:filling001\",\n \"type\": \"FillingSensor\",\n \"filling\": {\n \"type\": \"Property\",\n \"value\": 0.25,\n \"unitCode\": \"C62\",\n \"observedAt\": \"2020-12-09T16:25:12.000Z\"\n },\n \"controllingAsset\": {\n \"type\": \"Relationship\",\n \"object\": \"urn:ngsi-ld:Building:farm001\",\n \"observedAt\": \"2020-12-09T16:25:12.000Z\"\n }\n }\n ]\n}\nBecause the accept attribute has been set to application/json, the @context is sent as a Link header rather than an attribute within the payload body.
Context brokers may offer additional custom payload formats (typically prefixed with an x-). The Orion-LD broker offers a backwards compatible NGSI-v2 payload option for legacy systems.
This third subscription will fire when the filling level is below 0.4. The format attribute has been altered to inform the subscriber using NGSI-v2 normalized format.
When a low-stock-farm001-ngsiv2 event is fired, the response is a normalzed NGSI-v2 payload as shown:
{\n \"subscriptionId\": \"urn:ngsi-ld:Subscription:5fd1f31e8b9b83697b855a5d\",\n \"data\": [\n {\n \"id\": \"urn:ngsi-ld:Device:filling001\",\n \"type\": \"https://uri.etsi.org/ngsi-ld/default-context/FillingSensor\",\n \"https://w3id.org/saref#fillingLevel\": {\n \"type\": \"Property\",\n \"value\": 0.33,\n \"metadata\": {\n \"unitCode\": \"C62\",\n \"accuracy\": {\n \"type\": \"Property\",\n \"value\": 0.05\n },\n \"observedAt\": \"2020-12-10T10:11:57.000Z\"\n }\n },\n \"https://uri.etsi.org/ngsi-ld/default-context/controllingAsset\": {\n \"type\": \"Relationship\",\n \"value\": \"urn:ngsi-ld:Building:farm001\",\n \"metadata\": {\n \"observedAt\": \"2020-12-10T10:11:57.000Z\"\n }\n }\n }\n ]\n}\nAs can be seen, by default the attributes are returned using URN long names. It is also possible to request that the Orion-LD context broker pre-applies a compaction operation to the payload.
\nx-nsgiv2-keyValues - Key Value pairs with URN attribute namesx-nsgiv2-keyValues-compacted - Key Value pairs with short name attribute aliasesx-ngsiv2-normalized - NGSI-v2 normalized payload with URN attribute namesx-ngsiv2-normalized-compacted- NGSI-v2 normalized payload pairs with short name attribute aliasesThe set of available custom formats will vary between Context Brokers.
\n","urlObject":{"protocol":"http","path":["ngsi-ld","v1","subscriptions",""],"host":["localhost:1026"],"query":[],"variable":[]}},"response":[],"_postman_id":"71288f8c-ab23-4ed2-98e5-11cd0a6ef70d"}],"id":"c6c1b6d9-dd40-4daf-8e47-a019091ce8b0","description":"Within the Farm Management Information System, imagine that the farmer wants a contractor to refill his barn with hay when the level has reduced below a set level. It would be possible\nto set up the system so that the contractor was constantly polling for new information, however hay is not removed very\nfrequently so this would be a waste of resources and create a lot of unnecessary data traffic.
\nThe alternative is to create a subscription which will POST a payload to a \"well-known\" URL whenever a value has\nchanged.
\n","_postman_id":"c6c1b6d9-dd40-4daf-8e47-a019091ce8b0"},{"name":"Subscription CRUD Actions","item":[{"name":"Create a Subscription (Product)","id":"4743f5ce-d7d8-4b0d-aa1c-b1868e05d4b4","protocolProfileBehavior":{"disableBodyPruning":true},"request":{"method":"POST","header":[{"key":"content-type","value":"application/ld+json"}],"body":{"mode":"raw","raw":"{\n \"description\": \"Notify me of all product price changes\",\n \"type\": \"Subscription\",\n \"entities\": [{\"type\": \"Product\"}],\n \"watchedAttributes\": [\"price\"],\n \"notification\": {\n \"format\": \"keyValues\",\n \"endpoint\": {\n \"uri\": \"http://tutorial:3000/subscription/price-change\",\n \"accept\": \"application/json\"\n }\n },\n \"@context\": \"http://context-provider:3000/data-models/ngsi-context.jsonld\"\n}"},"url":"http://localhost:1026/ngsi-ld/v1/subscriptions/","description":"This example creates a new subscription. The subscription will fire an asynchronous notification to a URL whenever the\ncontext is changed and the conditions of the subscription - Any Changes to Product prices - are met.
\nNew subscriptions can be added by making a POST request to the /ngsi-ld/v1/subscriptions/ endpoint.
The subject section of the request states that the subscription will be fired whenever the price attribute of any\nProduct entity is altered.
\nThe notification section of the body states that a POST request containing all affected entities will be sent to the\nhttp://tutorial:3000/subscription/price-change endpoint.
This example deletes the Subscription with id=urn:ngsi-ld:Subscription:5fd8a6ff9d82fd9d6ab1ce75 from the context.
Subscriptions can be deleted by making a DELETE request to the /ngsi-ld/v1/subscriptions/<subscription-id> endpoint.
This example amends an existing subscription with the ID urn:ngsi-ld:Subscription:5fd8a6ff9d82fd9d6ab1ce75 and updates the notification URL.
Subscriptions can be updated making a PATCH request to the /ngsi-ld/v1/subscriptions/<subscription-id> endpoint.
This example lists all subscriptions by making a GET request to the /ngsi-ld/v1/subscriptions/ endpoint. The list of subscriptions is limited to the tenant defined by the NGSILD-Tenant header (or the default tenant if the NGSILD-Tenant header is not sent )
The notification section of each subscription will also include the last time the conditions of the subscription were\nmet, and whether associated the POST action was successful.
\n","urlObject":{"protocol":"http","path":["ngsi-ld","v1","subscriptions",""],"host":["localhost:1026"],"query":[],"variable":[]}},"response":[],"_postman_id":"5ce03a61-042a-474d-b2b0-e7ccfd62e12f"},{"name":"Read the detail of a Subscription","id":"7131ded4-51d1-47f5-82a2-a5c0b2702a31","protocolProfileBehavior":{"disableBodyPruning":true},"request":{"method":"GET","header":[],"url":"http://localhost:1026/ngsi-ld/v1/subscriptions/urn:ngsi-ld:Subscription:5fd8a7259d82fd9d6ab1ce76","description":"This example obtains the full details of a subscription with a given ID.
\nThe response includes additional details in the notification section showing the last time the conditions of the\nsubscription were met, and whether associated the POST action was successful.
\nSubscription details can be read by making a GET request to the /ngsi-ld/v1/subscriptions/<subscription-id> endpoint.
The CRUD operations for subscriptions map on to the expected HTTP verbs under the /ngsi-ld/v1/subscriptions/ endpoint.
The <subscription-id> is auto generated when the subscription is created and returned in Header of the POST response\nto be used by the other operation thereafter.