![\"GitHub\"](\"https://fiware.github.io/tutorials.Historic-Context/icon/GitHub-Mark-32px.png\"){kind=icon}
FIWARE 202: Provisioning an IoT AgentThis tutorial introduces the concept of an IoT Agent and wires up the dummy UltraLight 2.0 IoT devices created in the\nprevious tutorial so that measurements can be read \nand commands can be sent using NGSI requests sent to the Orion Context Broker.
\nThe docker-compose files for this tutorial can be found on GitHub:
FIWARE 202: Provisioning an IoT Agent
\n\n\"In every operation there is an above the line and a below the line.\nAbove the line is what you do by the book. Below the line is how you\ndo the job.\"
\n— John le Carré (A Perfect Spy)
\n
An IoT Agent is a component that lets a group of devices send their data to and be managed from a Context Broker\nusing their own native protocols. IoT Agents should also be able to deal with security aspects of the Fiware \nplatform (authentication and authorization of the channel) and provide other common services to the device programmer.
\nThe Orion Context Broker exclusively uses NGSI requests for all\nof its interactions. Each IoT Agent provides a North Port NGSI\ninterface which is used for context broker interactions and all interactions beneath this port occur using the native protocol\nof the attached devices.
\nIn effect, this brings a standard interface to all IoT interactions at the context information management level. \nEach group of IoT devices are able to use their own propriatory protocols and disparate transport mechanisms under\nthe hood whilst the associated IoT Agent offers a facade pattern to handle this complexity.
\nIoT Agents already exist or are in development for many common transports and protocols. Examples include the following:
\nHTTP requests generated by the from the Context Broker and passed downwards towards an IoT device (via\nan IoT agent) are known as southbound traffic. Southbound traffic consists of commands made to \nactuator devices which alter the state of the real world by their actions.
\nFor example to switch on a real-life UltraLight 2.0 Smart Lamp the following interactions would occur:
\non command of the Smart Lamp via NGSI![\"\"](\"https://fiware.github.io/tutorials.IoT-Agent/img/command-swimlane.png\")
Requests generated from an IoT device and passed back upwards towards the Context Broker (via an \nIoT agent) are known as northbound traffic. Northbound traffic consists of measurements made\nby sensor devices and relays the state of the real world into the context data of the system.
\nFor example for a real-life Motion Sensor to send a count measurement the following interactions would occur:
\n![\"\"](\"https://fiware.github.io/tutorials.IoT-Agent/img/measurement-swimlane.png\")
\n\nNote Other more complex interactions are also possible, but this overview is sufficient to understand the basic\nprinciples of an IoT Agent.
\n
As can be seen from the previous sections, although each IoT Agent will be unique since they interpret different\nprotocols, there will a large degree of similarity between IoT agents.
\nThis base functionality has been abstracted out into a common IoT Agent framework library
\nFor the purpose of this tutorial, a series of dummy IoT devices have been created, which will be attached to the context broker.\nThe state of each device can be seen on the UltraLight device monitor web-page found at: http://localhost:3000/device/monitor
![\"FIWARE](\"https://fiware.github.io/tutorials.IoT-Agent/img/device-monitor.png\")
This application builds on the components created in previous tutorials. It\nwill make use of two FIWARE components - the Orion Context Broker and the IoT Agent for UltraLight 2.0. Usage of the Orion Context Broker is sufficient for an application to qualify as “Powered by FIWARE”.\nBoth the Orion Context Broker and the IoT Agent rely on open source MongoDB technology to keep persistence of the information they hold. We will also be using the dummy IoT devices created in the previous tutorial
\nTherefore the overall architecture will consist of the following elements:
\nSince all interactions between the elements are initiated by HTTP requests, the entities can be containerized and run from exposed ports.
\n![\"\"](\"https://fiware.github.io/tutorials.IoT-Agent/img/architecture.png\")
The necessary configuration information for wiring up the IoT devices and the IoT Agent can be seen in the services section of the associated docker-compose.yml file:
context-provider:\n image: fiware/cp-web-app:latest\n hostname: context-provider\n container_name: context-provider\n networks:\n - default\n expose:\n - \"3000\"\n - \"3001\"\n ports:\n - \"3000:3000\"\n - \"3001:3001\"\n environment:\n - \"DEBUG=proxy:*\"\n - \"PORT=3000\"\n - \"IOTA_HTTP_HOST=iot-agent\"\n - \"IOTA_HTTP_PORT=7896\"\n - \"DUMMY_DEVICES_PORT=3001\"\n - \"DUMMY_DEVICES_API_KEY=4jggokgpepnvsb2uv4s40d59ov\"\nThe context-provider container is listening on two ports:
3000 is exposed so we can see the web-page displaying the Dummy IoT devices.3001 is exposed purely for tutorial access - so that cUrl or Postman can make UltraLight commands\nwithout being part of the same network.The context-provider container is driven by environment variables as shown:
The other context-provider container configuration values described in the YAML file are not used in this tutorial.
The IoT Agent for UltraLight 2.0 can be instantiated within a Docker container. An offical Docker image is available from Docker Hub tagged fiware/iotagent-ul. The \nnecessary configuration can be seen below:
iot-agent:\n image: fiware/iotagent-ul:latest\n hostname: iot-agent\n container_name: iot-agent\n depends_on:\n - context-db\n - orion\n networks:\n - default\n expose:\n - \"4041\"\n - \"7896\"\n ports:\n - \"4041:4041\"\n - \"7896:7896\"\n environment:\n - \"IOTA_CB_HOST=orion\"\n - \"IOTA_CB_PORT=1026\"\n - \"IOTA_NORTH_PORT=4041\"\n - \"IOTA_REGISTRY_TYPE=mongodb\"\n - \"IOTA_LOG_LEVEL=DEBUG\"\n - \"IOTA_TIMESTAMP=true\"\n - \"IOTA_MONGO_HOST=context-db\"\n - \"IOTA_MONGO_PORT=27017\"\n - \"IOTA_MONGO_DB=iotagentul\"\n - \"IOTA_HTTP_PORT=7896\"\n - \"IOTA_PROVIDER_URL=http://iot-agent:4041\"\nThe iot-agent container relies on the precence of the Orion Context Broker and uses a MongoDB database to hold device information such as device URLs and Keys. The container is listening on two ports:
7896 is exposed to receive Ultralight measurements over HTTP from the Dummy IoT devices4041 is exposed purely for tutorial access - so that cUrl or Postman can make provisioning commands\nwithout being part of the same network.The iot-agent container is driven by environment variables as shown:
Once the IoT Agent is running, You can check the status by making an HTTP request to the exposed port. \nIf the response is blank, this is usually because the MongoDB database holding the context information is not running or not connected.
\n\n\n","urlObject":{"protocol":"http","path":["iot","about"],"host":["localhost:4041"],"query":[],"variable":[]}},"response":[],"_postman_id":"ebbafc67-6d8c-4a98-8dd4-0e86f3909da3"},{"name":"IoT Agent - Create a Service Group","id":"ce19cbaf-b6e4-40f9-bf1a-a7d12c86f5a7","protocolProfileBehavior":{"disableBodyPruning":true},"request":{"method":"POST","header":[{"key":"fiware-service","value":"openiot"},{"key":"fiware-servicepath","value":"/"},{"key":"Content-Type","value":"application/json"}],"body":{"mode":"raw","raw":"{\n \"services\": [\n {\n \"apikey\": \"4jggokgpepnvsb2uv4s40d59ov\",\n \"cbroker\": \"http://orion:1026\",\n \"entity_type\": \"Motion\",\n \"resource\": \"/iot/d\"\n }\n ]\n}"},"url":"http://localhost:4041/iot/services","description":"Troubleshooting: What if the response is blank ?
\n\n
\n- To check that a docker container is running try
\n\ndocker ps\nYou should see several containers running. If
\niot-agentis not running, you can restart the containers as necessary.
Invoking group provision is always the the first step in connecting devices since it is always necessary to\nsupply an authentication key with each measurement and the IoT Agent will not initially know which URL \nthe context broker is responding on.
\nIt is possible to set up default commands and attributes for all devices as well, but this\nis not done within this tutorial as we will be provisioning each device separately.
\nThis example provisions an anonymous group of devices. It tells the IoT Agent that a series of devices\nwill be sending messages to the IOTA_HTTP_PORT (where the IoT Agent is listening for Northbound communications)
In the example the IoT Agent is informed that the /iot/d endpoint will be used and that devices will authenticate\nthemselves by including the token 4jggokgpepnvsb2uv4s40d59ov. For an UltraLight IoT Agent this means devices will\nbe sending GET or POST requests to:
http://iot-agent:7896/iot/d?i=<device_id>&k=4jggokgpepnvsb2uv4s40d59ov\nWhich should be familiar UltraLight 2.0 syntax from the previous tutorial.
\nWhen a measurement from an IoT device is received on the resource url it needs to be \ninterpreted and passed to the context broker. The entity_type attribute provides a\ndefault type for each device which has made a request (in this case anonymous devices\nwill be known as Thing entities. Furthermore the location of the\ncontext broker (cbroker) is needed, so that the IoT Agent can pass on any measurements \nreceived to the correct URL. cbroker is an optional attribute - if it is not provided, \nthe IoT Agent uses the context broker URL as defined in the configuration file, however\nit has been included here for completeness.
It is common good practice to use URNs following the NGSI-LD draft recommendation when creating entities. Furthermore it is easier to understand\nmeaningful names when defining data attributes. These mappings can be defined by provisioning a device individually.
\nThree types of meaasurement attributes can be provisioned:
\nattributes are active readings from the devicelazy attributes are only sent on request - The IoT Agent will inform the device to return the measurementstatic_attributes are as the name suggests static data about the device (such as relationships) passed on \nto the context broker.\n\nNote: in the case where individual
\nids are not required, or aggregated data is sufficient \ntheattributescan be defined within the provisioning service rather than individually.
In the request we are assiociating the device motion001 with the URN urn:ngsd-ld:Motion:001\nand mapping the device reading c with the context attribute count (which is defined as an Integer)\nA refStore is defined as a static_attribute, placing the device within Store urn:ngsi-ld:Store:001
This example simulates a request coming from the device motion001
The request to the previously provisioned resource iot/d is in UltraLight 2.0 format\nand identifies the device motion001 and passes a known API key.
A similar request was made in the previous tutorial (before the IoT Agent was connected)\nwhen the door was unlocked, you will have seen the state of each motion sensor changing\nand a Northbound request will be logged in the device monitor.
\n","urlObject":{"protocol":"http","port":"7896","path":["iot","d"],"host":["localhost"],"query":[{"key":"k","value":"4jggokgpepnvsb2uv4s40d59ov"},{"key":"i","value":"motion001"}],"variable":[]}},"response":[],"_postman_id":"904e12d5-6bbb-4aff-b716-ebba93c05b8a"},{"name":"Context Broker - Read the Motion Sensor","id":"b87ee72f-94a1-4b49-aa8c-7fe22cf8dfff","protocolProfileBehavior":{"disableBodyPruning":true},"request":{"method":"GET","header":[{"key":"fiware-service","value":"openiot"},{"key":"fiware-servicepath","value":"/"}],"url":"http://localhost:1026/v2/entities/urn:ngsi-ld:Motion:001","description":"This example returns the data for the Motion entity with the id=urn:ngsd-ld:Motion:001 within the context data.
The response shows that the device with id=motion001 has been successfully identifed by the IoT Agent and mapped to the\nentity id=urn:ngsd-ld:Motion:001. This new entity has been created within the context data. The c \nattribute from the dummy device measurement request has been mapped to the more meaningful count attribute\nwithin the context. As you will notice, a TimeInstant attribute has been added to both the entity and the\nmeta data of the attribute - this represents the last time the entity and attribute have been updated, and is\nautomatically added to each new entity because the IOTA_TIMESTAMP environment variable was set when the\nIoT Agent was started up. The refStore attribute comes from the static_attributes set when the device was provisioned.
Provisioning an actuator is similar to provisioning a sensor. This time an endpoint attribute holds\nthe location where the IoT Agent needs to send the UltraLight command and the commands array includes\na list of each command that can be invoked. The example below provisions a bell with the deviceId=bell001.\nThe endpoint is http://context-provider:3001/iot/bell001 and it can accept the ring command.
An actuator can also be provisioned using a bidirectional attribute. Once again an endpoint attribute holds\nthe location where the IoT Agent needs to send the UltraLight command. The ring attribute is defined using an expression and mapped to itself in the reverse direction. When un update to the ring attribute is received, it is also send to the device itself.
Before we wire-up the context broker, we can test that a command can be send to a device by making a REST request \ndirectly to the IoT Agent's North Port using the /v1/updateContext endpoint.\nIt is this endpoint that will eventually be invoked by the context broker once we have connected it up. \nTo test the configuration you can run the command directly as shown.
If you are viewing the device monitor page, you can also see the state of the bell change.
\n![](\"https://fiware.github.io/tutorials.IoT-Agent/img/bell-ring.gif\")
The result of the command to ring the bell can be read by querying the entity within the Orion Context Broker.
\nThe TimeInstant shows last the time any command associated with the entity has been invoked. the result of ring command can be see in the value of the ring_info attribute
Provisioning a device which offers both commands and measurements is merely a matter of making an HTTP POST request\nwith both attributes and command attributes in the body of the request.
This example provisions a smart door with four commands lock, unlock, open and close and maps a single attribute.
Provisioning a device which offers both commands and measurements is merely a matter of making an HTTP POST request\nwith both attributes and command attributes in the body of the request.
This example provisions a smart door with two commands on and off and maps two attributes.
This example lists all provisioned devices by making a GET request to the /iot/devices endpoint.
The response includes all the commands and attributes mappings associated with all provisioned IoT devices.
\n","urlObject":{"protocol":"http","path":["iot","devices"],"host":["localhost:4041"],"query":[],"variable":[]}},"response":[],"_postman_id":"198f6a0b-bf95-4a26-b3c1-0b97b2f955d5"}],"id":"ab8b5d28-89c4-4a30-a85b-2fbc20d1276f","description":"The IoT Agent acts as a middleware between the IoT devices and the context broker. It therefore\nneeds to be able to create context data entities with unique ids. Once a service has been provisioned\nand an unknown device makes a measurement the IoT Agent add this to the context using the supplied\n<device-id> (unless the device is recognized and can be mapped to a known id.
There is no guarantee that every supplied IoT device <device-id> will always be unique, therefore \nall provisioning requests to the IoT Agent require two mandatory headers:
fiware-service header is defined so that entities for a given service can be held in a separate mongoDB database.fiware-servicepath can be used to differenciate between arrays of devices.For example within a smart city application you would expect different fiware-service headers for different\ndepartments (e.g. parks, transport, refuse collection etc.) and each fiware-servicepath would refer to specific park \nand so on. This would mean that data and devices for each service can be identified and separated as needed, but the\ndata would not be siloed - for example data from a Smart Bin within a park can be combined with the GPS Unit \nof a refuse truck to alter the route of the truck in an efficient manner.
The Smart Bin and GPS Unit are likely to come from different manufacturers and it cannot be \nguaranteed that that there is no overlap within <device-ids>s used. The use of the fiware-service and\nfiware-servicepath headers can ensure that this is always the case, and allows the context broker to identify\nthe original source of the context data.
To invoke the open command, the open attribute must be updated in the context.
To invoke the ring command, the ring attribute must be updated in the context.
If you are viewing the device monitor page, you can also see the state of the bell change.
\n
The result of the command to ring the bell can be read by querying the entity within the Orion Context Broker.
\nThe TimeInstant shows last the time any command associated with the entity has been invoked. the result of ring command can be see in the value of the ring_info attribute
The Bell entity has been mapped to id=\"urn:ngsi-ld:Bell:001\" with an entity type=\"Bell\"
To invoke the open command, the open attribute must be updated in the context.
To invoke the open command, the open attribute must be updated in the context.
The result of the invocation of Smart Door commands can be read by querying the entity within the Orion Context Broker.
\nThe TimeInstant shows last the time any command associated with the entity has been invoked.
open command can be see in the value of the open_info attributeclose command can be see in the value of the close_info attributelock command can be see in the value of the lock_info attributeunlock command can be see in the value of the unlock_info attributeThe Smart Door entity has been mapped to id=\"urn:ngsi-ld:Door:001\" with an entity type=\"Door\"
To invoke the open command, the open attribute must be updated in the context.
To switch on the Smart Lamp, the on attribute must be updated in the context.
The result of the invocation of Smart Lamp commands can be read by querying the entity within the Orion Context Broker.
\nThe TimeInstant shows last the time any command associated with the entity has been invoked.
on command can be see in the value of the on_info attributeoff command can be see in the value of the off_info attributeThe Smart Lamp entity has been mapped to id=\"urn:ngsi-ld:Lamp:001\" with an entity type=\"Lamp\"
This example returns the data of all Store entities within the context data
Having connected up the IoT Agent to the IoT devices, the Orion Context Broker was informed that the commands are\nnow available. In other words the IoT Agent registered itself as a\nContext Provider for the command attributes.
\nOnce the commands have been registered it will be possible to ring the Bell, open and close the Smart Door and\nswitch the Smart Lamp on and off by sending requests to the Orion Context Broker, rather than sending UltraLight 2.0\nrequests directly t the IoT devices as we did in the previous tutorial
\n","event":[{"listen":"prerequest","script":{"id":"e5aa0b75-9618-464b-9547-bccf5360d530","type":"text/javascript","exec":[""]}},{"listen":"test","script":{"id":"862fb9fb-be99-4b36-bb84-8ec7f5638ede","type":"text/javascript","exec":[""]}}],"_postman_id":"f3c4d2b1-d9d8-40df-872c-652ce9b5eae9"},{"name":"Service Group CRUD Actions","item":[{"name":"Create a Service Group","id":"ce39cdf4-b682-47de-a8ef-688cc46d2d8c","request":{"method":"POST","header":[{"key":"fiware-service","value":"openiot"},{"key":"fiware-servicepath","value":"/"},{"key":"Content-Type","value":"application/json"}],"body":{"mode":"raw","raw":"{\n \"services\": [\n {\n \"apikey\": \"4jggokgpepnvsb2uv4s40d59ov\",\n \"cbroker\": \"http://orion:1026\",\n \"entity_type\": \"Thing\",\n \"resource\": \"/iot/d\"\n }\n ]\n}"},"url":"http://localhost:4041/iot/services","description":"This example provisions an anonymous group of devices. It tells the IoT Agent that a series of devices\nwill be sending messages to the IOTA_HTTP_PORT (where the IoT Agent is listening for Northbound communications)
In the example the IoT Agent is informed that the /iot/d endpoint will be used and that devices will authenticate\nthemselves by including the token 4jggokgpepnvsb2uv4s40d59ov.
This example obtains the full details of a provisioned services with a given resource path.
The response includes all the defaults associated with the service group such as the entity_type and any default commands or attribute mappings.
Service group details can be read by making a GET request to the /iot/services endpoint and providing a resource parameter.
This example lists all provisioned services by making a GET request to the /iot/services/ endpoint.
The response includes all the defaults associated with each service group such as the entity_type and any default commands or attribute mappings.
This example updates an existing service group with a given resource path and apikey
Service group details can be updated by making a PUT request to the /iot/services endpoint \nand providing a resource and apikey parameters.
This example removes a provisioned service group by making a DELETE request to the /iot/services/ endpoint.
It means that requests to http://iot-agent:7896/iot/d?i=<device_id>&k=4jggokgpepnvsb2uv4s40d59ov\n(where the IoT Agent is listening for Northbound communications) should no longer be processed by the IoT Agent. \nThe apiKey and resource parameters must be supplied in order to identify the service group to be deleted.
The path that the IoT Agent had been listening on
\n","type":"text/plain"},"key":"resource","value":"/iot/d"},{"key":"apikey","value":"4jggokgpepnvsb2uv4s40d59ov"}],"variable":[]}},"response":[],"_postman_id":"78c8a1c0-eec7-484c-98b3-cd706d85aab6"}],"id":"407d44b0-36af-4097-ba21-4444a7892466","description":"The CRUD operations for a service group map on to the expected HTTP verbs under the /iot/services endpoint
Use the resource and apikey parameters to uniquely identify a service group.
This example provisions an individual device. It maps the device_id=bell002 to the entity URN urn:ngsi-ld:Bell:002 and gives the\nentity a type Bell. The IoT Agent has been informed that the device offers a single ring command and is listening on \nhttp://context-provider:3001/iot/bell002 using HTTP. attributes, lazy attributes and static_attributes can also be provisioned.
This example obtains the full details of a provisioned device with a given <device-id> path.
The response includes all the commands and attributes mappings associated with the device
\nProvisioned Device details can be read by making a GET request to the /iot/devices/<device-id> endpoint.
This example lists all provisioned devices by making a GET request to the /iot/devices endpoint.
The response includes all the commands and attributes mappings associated with all devices
\n","urlObject":{"protocol":"http","path":["iot","devices"],"host":["localhost:4041"],"query":[],"variable":[]}},"response":[],"_postman_id":"f80654a3-ef75-4f3d-aa64-b0abb573a8d6"},{"name":"Update a Provisioned Device","id":"6aead855-aa74-429d-afd7-b8f6ae3943c0","request":{"method":"PUT","header":[{"key":"fiware-service","value":"openiot"},{"key":"fiware-servicepath","value":"/"},{"key":"Content-Type","value":"application/json"}],"body":{"mode":"raw","raw":"{\n \"entity_name\": \"urn:ngsi-ld:Thing:bell002\"\n}"},"url":"http://localhost:4041/iot/devices/bell002","description":"This example updates an existing provisioned device by making a PUT request to the /iot/devices/<device-id> endpoint.
This example removes a provisioned device by making a DELETE request to the /iot/services/<device-id> endpoint.
The device attributes will no longer be mapped and commands can no longer be sent to the device.\nIf the device is making active measurements, they will still be handled with default values\nif the associated service has not been deleted.
\n","urlObject":{"protocol":"http","path":["iot","devices","bell002"],"host":["localhost:4041"],"query":[],"variable":[]}},"response":[],"_postman_id":"ef73bac0-505f-4966-9714-c3a1e9ab7b63"}],"id":"18700aa1-1aa6-42f6-b9d1-5b4e5e8ec8dc","description":"The CRUD operations for an individual device map on to the expected HTTP verbs under the /iot/devices endpoint
Use the <device-id> to uniquely identify a device.