![\"GitHub\"](\"https://fiware.github.io/tutorials.CRUD-Operations/icon/GitHub-Mark-32px.png\"){kind=icon}
FIWARE 103: Manipulating Context Data through CRUD Operations This tutorial builds on the data created in the previous stock management example and introduces the concept of CRUD operations, allowing users to manipulate the data held within the context.
\nThe docker-compose file for this tutorial can be found on GitHub:
FIWARE 103: Manipulating Context Data through CRUD Operations
Within the FIWARE platform, an entity represents the state of a physical or\nconceptual object which exists in the real world.
\nWithin our simple stock management system, we currently have four entity types.\nThe relationships between our entities are defined as shown below:
\n![\"\"](\"https://fiware.github.io/tutorials.Entity-Relationships/img/entities.png\")
As you can see, each of the entities defined above contain some properties which\nare liable to change. For example, product price could change, stock could be sold\nand the number of items on the shelves would drop.
\nThis application will only make use of one FIWARE component - the\nOrion Context Broker. Using\nthe Orion Context Broker is sufficient for an application to qualify as\n“Powered by FIWARE”.
\nCurrently, the Orion Context Broker relies on open source\nMongoDB technology to store the context data it\nmanages. Therefore, the architecture will consist of two components:
\nSince the two components interact by means of HTTP requests, they can be\ncontainerized and run from exposed ports.
\n![\"\"](\"https://fiware.github.io/tutorials.CRUD-Operations/img/architecture.png\")
The necessary configuration information can be seen in the services section of\nthe associated docker-compose.yml file:
orion:\n image: fiware/orion:latest\n hostname: orion\n container_name: orion\n depends_on:\n - mongo-db\n networks:\n - default\n expose:\n - \"1026\"\n ports:\n - \"1026:1026\"\n command: -dbhost mongo-db -logLevel DEBUG\nmongo-db:\n image: mongo:3.6\n hostname: mongo-db\n container_name: db-mongo\n expose:\n - \"27017\"\n ports:\n - \"27017:27017\"\n networks:\n - default\n command: --bind_ip_all --smallfiles\nBoth containers reside on the same network - the Orion Context Broker is\nlistening on port 1026 and MongoDB is listening on the default port 271071.\nFor the sake of this tutorial, we have also made the two ports available from\noutside the network so that cUrl or Postman can access them without having to\nbe run from inside the network. The command-line initialization should be self\nexplanatory.
To keep things simple both components will be run using\nDocker. Docker is a container technology which\nallows to package each component with its environment and run it in isolation.
\nDocker Compose is a tool for defining and running multi-container Docker\napplications. A\nYAML file\nis used configure the required services for the application. This means all\ncontainer services can be brought up with a single command. Docker Compose is\ninstalled by default as part of Docker for Windows and Docker for Mac, however\nLinux users will need to follow the instructions found\nhere
\nYou can check your current Docker and Docker Compose versions using the\nfollowing commands:
\ndocker-compose -v\ndocker version\nPlease ensure that you are using Docker version 18.03 or higher and Docker\nCompose 1.21 or higher and upgrade if necessary.
\nWe will start up our services using a simple bash script. Windows users should\ndownload cygwin to provide a command-line\nfunctionality similar to a Linux distribution on Windows.
\nAll services can be initialised from the command-line by running the bash script\nprovided within the repository. Please clone the repository and create the\nnecessary images by running the commands as shown below:
\ngit clone git@github.com:Fiware/tutorials.CRUD-Operations.git\ncd tutorials.CRUD-Operations\n\n./services start\nThis command will also import seed data from the previous\nStore Finder tutorial\non startup.
\n\n\n:information_source: Note: If you want to clean up and start over again\nyou can do so with the following command:
\n\n./services stop\n
Create, Read, Update and Delete are the four basic functions of\npersistent storage. These operations are usually referred to using the acronym\nCRUD. Within a database each of these operations map directly to a series of\ncommands, however their relationship with a RESTful API is slightly more complex.
\nThe Orion Context Broker uses\nNGSI to manipulate the\ncontext data. As a RESTful API, requests to manipulate the data held within the\ncontext follow the standard conventions found when mapping HTTP verbs to CRUD\noperations.
\nFor operations where the <entity-id> is not yet known within the context, or\nis unspecified, the /v2/entities endpoint is used.
Once an <entity-id> is known within the context, individual data entities can\nbe manipulated using the /v2/entities/<entity-id> endpoint.
It is recommended that entity identifiers should be URNs following the\nNGSI-LD guidelines,\ntherefore each id is a URN which follows a standard format:\nurn:ngsi-ld:<entity-type>:<entity-id>. This helps making every id in the\ncontext data unique.
A complete list of entity endpoints can be found in the\nNGSI v2 Swagger Specification
\nTo perform CRUD operations on attributes, the <entity-id> must be known. Each\nattribute is effectively a key-value pair.
There are three endpoints:
\n/v2/entities/<entity-id>/attrs is only used for a patch operation to\nupdate one or more exisiting attributes./v2/entities/<entity-id>/attrs/<attribute> is used to manipulate an\nattribute as a whole./v2/entities/<entity-id>/attrs/<attribute>/value is used to read or update\nthe value of an attribute, leaving the type untouched.A complete list of attribute endpoints can be found in the\nNGSI v2 Swagger Specification
\nAdditionally the Orion Context Broker has a convenience batch operation endpoint\n/v2/op/update to manipulate multiple entities in a single operation.
Batch operations are always triggered by a POST request where the payload is an\nobject with two properties:
\nactionType specifies the kind of action to invoke (e.g. delete)entities is an array of objects holding the list of entities to update, along\nwith the relevant entity data used to perform the operation.This example adds a new Product entity to the context.
\nNew entities can be added by making a POST request to the /v2/entities/ endpoint.
The request will fail if any of the attributes already exist in the context.
\nAny entity must have a id and type attributes, each additional attributes are optional \nand will depend on the system being described. Each additional attribute should also have a \ndefined type and a value attribute. The product has been assigned a unique id following\nthe NGSI-LD draft recommendation and has been assigned type=Product.
Subsequent requests using the same id will result in an error response.
This example adds a new specialOffer attribute to the existing Product entity with id=urn:ngsi-ld:Product:001
New attributes can be added by making a POST request to the /v2/entities/<entity>/attrs endpoint.
The payload should consist of a JSON object holding the attribute names and values as shown.
\nIf no type is specified a default type (Boolean, Text or Number or StructuredValue) will be assigned.
Subsequent requests using the same id will update the value of the attribute in the context
This example uses the convenience batch processing endpoint to add two new Product entities and one new attribute (offerPrice)\nto the context.
The request will fail if any of the attributes already exist in the context.
\nBatch processing uses the /v2/op/update endpoint with a payload with two attributes
actionType=append_strict means that the request only succeed if all entities / attributes are new.entities attribute holds an array of entities we wish to create.Each product has a unique id following the NGSI-LD draft recommendation and has been assigned type=Product.
Subsequent requests using the same data with the actionType=append_strict batch operation will result in an error response.
This example uses the convenience batch processing endpoint to add or amend two Product entities and one attribute (offerPrice) to the context.
actionType=append means we will overwrite existing entities if they existA subsequent request containing the same data (i.e. same entities and actionType=append)\nwon't change the context state.
Batch processing uses the /v2/op/update endpoint with a payload with two attributes:
actionType=append means we will overwrite existing entities if they existentities attribute holds an array of entities we wish to create/overwrite.Each product has a unique id following the NGSI-LD draft recommendation and has been assigned type=Product.
Create Operations map to HTTP POST.
\n/v2/entities endpoint is used for creating new entities/v2/entities/<entity> endpoint is used for adding new attributesAny newly created entity must have id and type attributes, other attributes\nare optional and will depend on the system being modelled. If additional\nattributes are present though, each should specify both a type and a value.
The response will be 204 - No Content if the operation is successful or\n422 - Unprocessable Entity if the operation fails.
\n","event":[{"listen":"prerequest","script":{"id":"e188a3c9-0964-40d4-bb85-30f8dba187dd","type":"text/javascript","exec":[""]}},{"listen":"test","script":{"id":"50a197ae-0c79-49f9-9f06-850bcee94f15","type":"text/javascript","exec":[""]}}],"_postman_id":"f75eabf1-1a66-45ab-b4c5-47d3dc0696f9"},{"name":"Read Operations","item":[{"name":"Read a Data Entity (verbose)","id":"89b9da60-9976-4e38-8057-6212d20e8705","request":{"method":"GET","header":[],"url":"http://localhost:1026/v2/entities/urn:ngsi-ld:Product:010","description":"This example reads the full context from an existing Product entity with a known id.
\nContext data can be retrieved by making a GET request to the /v2/entities/<entity> endpoint.
This example reads the value of a single attribute (name) from an existing Product entity with a known id.
Context data can be retrieved by making a GET request to the /v2/entities/<entity>/attrs/<attribute>/value endpoint.
This example reads the key-value pairs for two requested attributes (name and price) from the context of existing Product entity with a known id.
Combine the options=keyValues parameter and the attrs parameter to obtain key-values.
keyValues option in order to get a more compact and brief representation, including just attribute valuesvalues option combined with a list of attribute values attrs for an ordered list of attributes onlyOrdered list of attribute names to display
\n","type":"text/plain"},"key":"attrs","value":"name,price"}],"variable":[]}},"response":[],"_postman_id":"c201c35c-e5f7-41c1-b3b4-177c45066ea3"},{"name":"Read Multiple attributes values from a Data Entity","id":"6f8ad5c6-c2c0-4a80-9de6-c172d841bf37","request":{"method":"GET","header":[],"url":"http://localhost:1026/v2/entities/urn:ngsi-ld:Product:001/?options=values&attrs=name,price","description":"This example reads the value of two requested attributes (name and price) from the context of existing Product entity with a known id.
Combine the options=values parameter and the attrs parameter to return a list of values in an array
keyValues option in order to get a more compact and brief representation, including just attribute valuesvalues option combined with a list of attribute values attrs for an ordered list of attributes onlyOrdered list of attribute names to display
\n","type":"text/plain"},"key":"attrs","value":"name,price"}],"variable":[]}},"response":[],"_postman_id":"6f8ad5c6-c2c0-4a80-9de6-c172d841bf37"},{"name":"List all Data Entities (verbose)","id":"f9b62645-5c6e-4a35-a701-e9740f3c0bba","request":{"method":"GET","header":[],"url":"http://localhost:1026/v2/entities/?type=Product","description":"This example lists the full context of all Product entities.
\nFull context data for a specified entity type can be retrieved by making a GET request to the /v2/entities/ endpoint and supplying the type parameter.
This example lists the name and price attributes of all Product entities.
Full context data for a specified entity type can be retrieved by making a GET request to the /v2/entities/ endpoint and supplying the type parameter combine this with the options=keyValues parameter and the attrs parameter to obtain key-values.
Entity type
\n","type":"text/plain"},"key":"type","value":"Product"},{"description":{"content":"keyValues option in order to get a more compact and brief representation, including just attribute valuesvalues option combined with a list of attribute values attrs for an ordered list of attributes onlyOrdered list of attribute names to display
\n","type":"text/plain"},"key":"attrs","value":"name,price"}],"variable":[]}},"response":[],"_postman_id":"e51734e1-f879-4b77-b539-fb6083f43a0a"},{"name":"List Data Entity by id","id":"baa32fe6-fa77-46d8-938e-05eaba831c33","request":{"method":"GET","header":[],"url":"http://localhost:1026/v2/entities/?type=Product&options=count&attrs=id","description":"This example lists the id and type of all Product entities.
Context data for a specified entity type can be retrieved by making a GET request to the /v2/entities/ endpoint and supplying the type parameter. Combine this with options=count and attrs=id to return the id attributes of the given type
Entity type
\n","type":"text/plain"},"key":"type","value":"Product"},{"description":{"content":"keyValues option in order to get a more compact and brief representation, including just attribute valuesvalues option combined with a list of attribute values attrs for an ordered list of attributes onlyOrdered list of attribute names to display
\n","type":"text/plain"},"key":"attrs","value":"id"}],"variable":[]}},"response":[],"_postman_id":"baa32fe6-fa77-46d8-938e-05eaba831c33"}],"id":"8ff1dc7a-3929-4e5b-a6c1-0b4a8074cc03","description":"Read Operations map to HTTP GET.
\n/v2/entities/ endpoint is used for listing operations/v2/entities/<entity> endpoint is used for retrieving the details of a single entityoptions parameter (combined with the attrs parameter) is used to filter the fields returnedq parameter can be used to filter the entities returnedThis example updates the value of the price attribute of the Entity with id=urn:ngsi-ld:Product:001
Exisiting attribute values can be altered by making a PUT request to the /v2/entities/<entity>/attrs/<attribute>/value endpoint.
This example simultaneously updates the values of both the price and name attributes of the Entity with id=urn:ngsi-ld:Product:001
Multiple Existing attribute values can be updated by making a PATCH request to the /v2/entities/<entity>/attrs endpoint.
This example uses the convenience batch processing endpoint to create a series of available products.
\nBatch processing uses the /v2/op/update endpoint with a payload with two attributes - actionType=update means we will overwrite existing entities if they exist whereas the entities attribute holds an array of entities we wish to update.
Each product has a unique id following the NGSI-LD draft recommendation and has been assigned type=Product.
This example uses the convenience batch processing endpoint to create a series of available products.
\nBatch processing uses the /v2/op/update endpoint with a payload with two attributes - actionType=append means we will overwrite existing entities if they exist whereas the entities attribute holds an array of entities we wish to update.
Each product has a unique id following the NGSI-LD draft recommendation and has been assigned type=Product.
This example uses the convenience batch processing endpoint to create a series of available products.
\nBatch processing uses the /v2/op/update endpoint with a payload with two attributes - actionType=replace means we will overwrite existing entities if they exist whereas the entities attribute holds an array of entities we wish to update.
Each product has a unique id following the NGSI-LD draft recommendation and has been assigned type=Product.
Overwrite operations are mapped to HTTP PUT.\nHTTP PATCH can be used to update several attributes at once.
\n","event":[{"listen":"prerequest","script":{"id":"061914f4-8b31-4177-a3e2-5a7d0bf58522","type":"text/javascript","exec":[""]}},{"listen":"test","script":{"id":"b098aeb4-22e9-4224-a638-ac10bb75d84f","type":"text/javascript","exec":[""]}}],"_postman_id":"6474be41-c4b5-4c32-8372-f153fd970e3e"},{"name":"Delete Operations","item":[{"name":"Delete a Data Entity","id":"87223590-54a2-4a23-ad8e-b71c2556d314","request":{"method":"DELETE","header":[],"url":"http://localhost:1026/v2/entities/urn:ngsi-ld:Product:010","description":"This example deletes the Entity with id=urn:ngsi-ld:Product:001 from the context
Entities be deleted by making a DELETE request to the /v2/entities/<entity> endpoint.
Subsequent requests using the same id will result in an error response since the entity no longer exists
This example remove the specialOffer attribute from the Entity with id=urn:ngsi-ld:Product:010
Attributes be deleted by making a DELETE request to the /v2/entities/<entity>/attrs/<attribute> endpoint.
If the attribute does not exist in the context, the result in an error response.
\n","urlObject":{"protocol":"http","path":["v2","entities","urn:ngsi-ld:Product:010","attrs","specialOffer"],"host":["localhost:1026"],"query":[],"variable":[]}},"response":[],"_postman_id":"d64545a0-2851-4554-a9e6-93acedf78b72"},{"name":"Batch Delete Multiple Data Entities","id":"edeb9ba3-d2a5-4f46-8e9d-0d1f33ba6e62","request":{"method":"POST","header":[{"key":"Content-Type","value":"application/json"}],"body":{"mode":"raw","raw":"{\n \"actionType\":\"delete\",\n \"entities\":[\n {\n \"id\":\"urn:ngsi-ld:Product:001\", \"type\":\"Product\"\n },\n {\n \"id\":\"urn:ngsi-ld:Product:002\", \"type\":\"Product\"\n }\n ]\n}"},"url":"http://localhost:1026/v2/op/update","description":"This example uses the convenience batch processing endpoint to delete a series of available products.
\nBatch processing uses the /v2/op/update endpoint with a payload with two attributes - actionType=delete means we will\ndelete something from the context and the entities attribute holds the id of the entities we wish to update.
If any entity does not exist in the context, the result in an error response.
\n","urlObject":{"protocol":"http","path":["v2","op","update"],"host":["localhost:1026"],"query":[],"variable":[]}},"response":[],"_postman_id":"edeb9ba3-d2a5-4f46-8e9d-0d1f33ba6e62"},{"name":"Batch Delete Multiple Attributes from a Data Entity","id":"fdf30b5d-4970-4c13-bdcc-b26c1ce21a00","request":{"method":"POST","header":[{"key":"Content-Type","value":"application/json"}],"body":{"mode":"raw","raw":"{\n \"actionType\":\"delete\",\n \"entities\":[\n {\n \"id\":\"urn:ngsi-ld:Product:010\", \"type\":\"Product\",\n \"price\":{},\n \"name\": {}\n }\n ]\n}"},"url":"http://localhost:1026/v2/op/update","description":"This example uses the convenience batch processing endpoint to delete a series of attributes from an available product.
\nBatch processing uses the /v2/op/update endpoint with a payload with two attributes - actionType=delete means we will\ndelete something from the context and the entities attribute holds an array of attributes we wish to update.
If any attribute does not exist in the context, the result in an error response.
\n","urlObject":{"protocol":"http","path":["v2","op","update"],"host":["localhost:1026"],"query":[],"variable":[]}},"response":[],"_postman_id":"fdf30b5d-4970-4c13-bdcc-b26c1ce21a00"},{"name":"Find existing data relationships","id":"d6a3e12c-4541-46e7-8808-f378b666a6fe","request":{"method":"GET","header":[],"body":{"mode":"raw","raw":"{\n \"actionType\": \"APPEND\",\n \"entities\": [\n {\n \"id\": \"7770\",\n \"type\": \"Shelf\",\n \"store\": { \n \"type\": \"Relationship\",\n \"value\": \"urn:ngsi-ld:Store:store1\"\n },\n \"location\": {\n \"type\": \"geo:json\",\n \"value\": {\n \"type\": \"Point\",\n \"coordinates\": [13.3986112, 52.554699]\n }\n },\n \"name\": {\n \"type\": \"Text\",\n \"value\": \"Corner Unit\"\n },\n \"maxCapacity\": {\n \"type\": \"Integer\",\n \"value\": 50\n }\n},\n\n {\n \"id\": \"7771\",\n \"type\": \"Shelf\",\n \"store\": {\n \"type\": \"Relationship\",\n \"value\": \"urn:ngsi-ld:Store:store1\"\n },\n \"location\": {\n \"type\": \"geo:json\",\n \"value\": {\n \"type\": \"Point\",\n \"coordinates\": [\n 13.3987221,\n 52.5546640\n ]\n }\n },\n \"name\": {\n \"type\": \"Text\",\n \"value\": \"Wall Unit 1\"\n },\n \"maxCapacity\": {\n \"type\": \"Integer\",\n \"value\": 100\n }\n },\n \n {\n \"id\": \"7772\",\n \"type\": \"Shelf\",\n \"store\": {\n \"type\": \"Relationship\",\n \"value\": \"urn:ngsi-ld:Store:store1\"\n },\n \"location\": {\n \"type\": \"geo:json\",\n \"value\": {\n \"type\": \"Point\",\n \"coordinates\": [\n 13.3987221,\n 52.5546640\n ]\n }\n },\n \"name\": {\n \"type\": \"Text\",\n \"value\": \"Wall Unit 2\"\n },\n \"maxCapacity\": {\n \"type\": \"Integer\",\n \"value\": 100\n }\n }\n \n \n\n\n\n\n ]\n}"},"url":"http://localhost:1026/v2/entities/?q=refProduct==urn:ngsi-ld:Product:001&options=count&attrs=type","description":"This example returns the key of all entities directly associated with the urn:ngsi-ld:Product:001.
Note that we deleted Product urn:ngsi-ld:Product:001 earlier, so what we\nsee above is actually a dangling reference, i.e. the returned InventoryItem\nreferences a Product that no longer exists.
keyValues option in order to get a more compact and brief representation, including just attribute valuesvalues option combined with a list of attribute values attrs for an ordered list of attributes onlyOrdered list of attribute names to display
\n","type":"text/plain"},"key":"attrs","value":"type"}],"variable":[]}},"response":[],"_postman_id":"d6a3e12c-4541-46e7-8808-f378b666a6fe"}],"id":"25eb20a8-66e6-4b41-924e-1f43337024c7","description":"Delete Operations map to HTTP DELETE.
\nIf there are entities within the context which relate to one another, you must be careful when deleting an entity. You will need to check that no references are left dangling once the entity has been deleted.
\nOrganizing a cascade of deletions is beyond the scope of this tutorial, but it would be possible using a batch delete request.
\n","event":[{"listen":"prerequest","script":{"id":"add1245d-f9c3-4610-ae32-d12708236de5","type":"text/javascript","exec":[""]}},{"listen":"test","script":{"id":"0dd3c8e8-b424-43ae-a176-72bcb7b2ba7c","type":"text/javascript","exec":[""]}}],"_postman_id":"25eb20a8-66e6-4b41-924e-1f43337024c7"}],"event":[{"listen":"prerequest","script":{"id":"dff76437-6762-4f0f-9258-d0e5c77cca33","type":"text/javascript","exec":[""]}},{"listen":"test","script":{"id":"afdd2592-13bb-436d-a728-8af2d8c3d958","type":"text/javascript","exec":[""]}}],"variable":[{"id":"0b237012-4447-4688-bc10-bd8332c946c3","key":"orion","value":"localhost:1026"}]}