API reference in the Swagger UI can be found at: https://api.magistrala.abstractmachines.fr
+To start working with the Magistrala system, you need to create a user account.
+++Identity, which can be email-address (this must be unique as it identifies the user) and secret (password must contain at least 8 characters).
+
curl -sSiX POST http://localhost/users -H "Content-Type: application/json" [-H "Authorization: Bearer <user_token>"] -d @- << EOF
+{
+ "name": "[name]",
+ "tags": ["[tag1]", "[tag2]"],
+ "credentials": {
+ "identity": "<user_identity>",
+ "secret": "<user_secret>"
+ },
+ "metadata": {
+ "[key1]": "[value1]",
+ "[key2]": "[value2]"
+ },
+ "status": "[status]",
+ "role": "[role]"
+}
+EOF
+For example:
+curl -sSiX POST http://localhost/users -H "Content-Type: application/json" -d @- << EOF
+{
+ "name": "John Doe",
+ "credentials": {
+ "identity": "john.doe@email.com",
+ "secret": "12345678"
+ }
+}
+EOF
+
+HTTP/1.1 201 Created
+Server: nginx/1.23.3
+Date: Wed, 14 Jun 2023 13:45:38 GMT
+Content-Type: application/json
+Content-Length: 223
+Connection: keep-alive
+Location: /users/4f22fa45-50ca-491b-a7c9-680a2608dc13
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "4f22fa45-50ca-491b-a7c9-680a2608dc13",
+ "name": "John Doe",
+ "credentials": { "identity": "john.doe@email.com" },
+ "created_at": "2023-06-14T13:45:38.808423Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+}
+You can also use <user_token> so that the owner of the new user is the one identified by the <user_token> for example:
curl -sSiX POST http://localhost/users -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "name": "John Doe",
+ "credentials": {
+ "identity": "jane.doe@email.com",
+ "secret": "12345678"
+ },
+}
+EOF
+
+HTTP/1.1 201 Created
+Server: nginx/1.23.3
+Date: Wed, 14 Jun 2023 13:46:47 GMT
+Content-Type: application/json
+Content-Length: 252
+Connection: keep-alive
+Location: /users/1890c034-7ef9-4cde-83df-d78ea1d4d281
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "1890c034-7ef9-4cde-83df-d78ea1d4d281",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "identity": "jane.doe@email.com" },
+ "created_at": "2023-06-14T13:46:47.322648Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+}
+To log in to the Magistrala system, you need to create a user_token.
curl -sSiX POST http://localhost/users/tokens/issue -H "Content-Type: application/json" -d @- << EOF
+{
+ "identity": "<user_identity>",
+ "secret": "<user_secret>"
+}
+EOF
+For example:
+curl -sSiX POST http://localhost/users/tokens/issue -H "Content-Type: application/json" -d @- << EOF
+{
+ "identity": "john.doe@email.com",
+ "secret": "12345678"
+}
+EOF
+
+HTTP/1.1 201 Created
+Server: nginx/1.23.3
+Date: Wed, 14 Jun 2023 13:47:32 GMT
+Content-Type: application/json
+Content-Length: 709
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "access_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODY3NTEzNTIsImlhdCI6MTY4Njc1MDQ1MiwiaWRlbnRpdHkiOiJqb2huLmRvZUBlbWFpbC5jb20iLCJpc3MiOiJjbGllbnRzLmF1dGgiLCJzdWIiOiI5NDkzOTE1OS1kMTI5LTRmMTctOWU0ZS1jYzJkNjE1NTM5ZDciLCJ0eXBlIjoiYWNjZXNzIn0.AND1sm6mN2wgUxVkDhpipCoNa87KPMghGaS5-4dU0iZaqGIUhWScrEJwOahT9ts1TZSd1qEcANTIffJ_y2Pbsg",
+ "refresh_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODY4MzY4NTIsImlhdCI6MTY4Njc1MDQ1MiwiaWRlbnRpdHkiOiJqb2huLmRvZUBlbWFpbC5jb20iLCJpc3MiOiJjbGllbnRzLmF1dGgiLCJzdWIiOiI5NDkzOTE1OS1kMTI5LTRmMTctOWU0ZS1jYzJkNjE1NTM5ZDciLCJ0eXBlIjoicmVmcmVzaCJ9.z3OWCHhNHNuvkzBqEAoLKWS6vpFLkIYXhH9cZogSCXd109-BbKVlLvYKmja-hkhaj_XDJKySDN3voiazBr_WTA",
+ "access_type": "Bearer"
+}
+To issue another access_token after getting expired, you need to use a refresh_token.
curl -sSiX POST http://localhost/users/tokens/refresh -H "Content-Type: application/json" -H "Authorization: Bearer <refresh_token>"
+For example:
+curl -sSiX POST http://localhost/users/tokens/refresh -H "Content-Type: application/json" -H "Authorization: Bearer <refresh_token>"
+
+HTTP/1.1 201 Created
+Server: nginx/1.23.3
+Date: Wed, 14 Jun 2023 13:49:45 GMT
+Content-Type: application/json
+Content-Length: 709
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "access_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODY3NTE0ODUsImlhdCI6MTY4Njc1MDU4NSwiaWRlbnRpdHkiOiJqb2huLmRvZUBlbWFpbC5jb20iLCJpc3MiOiJjbGllbnRzLmF1dGgiLCJzdWIiOiI5NDkzOTE1OS1kMTI5LTRmMTctOWU0ZS1jYzJkNjE1NTM5ZDciLCJ0eXBlIjoiYWNjZXNzIn0.zZcUH12x7Tlnecrc3AAFnu3xbW4wAOGifWZMnba2EnhosHWDuSN4N7s2S7OxPOrBGAG_daKvkA65mi5n1sxi9A",
+ "refresh_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODY4MzY5ODUsImlhdCI6MTY4Njc1MDU4NSwiaWRlbnRpdHkiOiJqb2huLmRvZUBlbWFpbC5jb20iLCJpc3MiOiJjbGllbnRzLmF1dGgiLCJzdWIiOiI5NDkzOTE1OS1kMTI5LTRmMTctOWU0ZS1jYzJkNjE1NTM5ZDciLCJ0eXBlIjoicmVmcmVzaCJ9.AjxJ5xlUUSjW99ECUAU19ONeCs8WlRl52Ost2qGTADxHGYBjPMqctruyoTYJbdORtL5f2RTxZsnLX_1vLKRY2A",
+ "access_type": "Bearer"
+}
+You can always check the user profile that is logged-in by using the user_token.
curl -sSiX GET http://localhost/users/profile -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX GET http://localhost/users/profile -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Wed, 14 Jun 2023 13:51:59 GMT
+Content-Type: application/json
+Content-Length: 312
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "1890c034-7ef9-4cde-83df-d78ea1d4d281",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": {
+ "identity": "jane.doe@email.com"
+ },
+ "created_at": "2023-06-14T13:46:47.322648Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+}
+You can always check the user entity by entering the user ID and user_token.
curl -sSiX GET http://localhost/users/<user_id> -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX GET http://localhost/users/1890c034-7ef9-4cde-83df-d78ea1d4d281 -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Wed, 14 Jun 2023 13:51:59 GMT
+Content-Type: application/json
+Content-Length: 312
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "1890c034-7ef9-4cde-83df-d78ea1d4d281",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": {
+ "identity": "jane.doe@email.com"
+ },
+ "created_at": "2023-06-14T13:46:47.322648Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+}
+You can get all users in the database by querying /users endpoint.
curl -sSiX GET http://localhost/users -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX GET http://localhost/users -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Wed, 14 Jun 2023 13:52:36 GMT
+Content-Type: application/json
+Content-Length: 285
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "limit": 10,
+ "total": 1,
+ "users": [
+ {
+ "id": "1890c034-7ef9-4cde-83df-d78ea1d4d281",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "identity": "jane.doe@email.com" },
+ "created_at": "2023-06-14T13:46:47.322648Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }
+ ]
+}
+If you want to paginate your results then use offset, limit, metadata, name, identity, tag, status and visbility as query parameters.
curl -sSiX GET http://localhost/users?[offset=<offset>]&[limit=<limit>]&[identity=<identity>]&[name=<name>]&[tag=<tag>]&[status=<status>]&[visibility=<visibility>] -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX GET http://localhost/users?offset=0&limit=5&identity=jane.doe@email.com -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Wed, 14 Jun 2023 13:53:16 GMT
+Content-Type: application/json
+Content-Length: 284
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "limit": 5,
+ "total": 1,
+ "users": [
+ {
+ "id": "1890c034-7ef9-4cde-83df-d78ea1d4d281",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "identity": "jane.doe@email.com" },
+ "created_at": "2023-06-14T13:46:47.322648Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }
+ ]
+}
+Updating user's name and/or metadata
+curl -sSiX PATCH http://localhost/users/<user_id> -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "name": "[new_name]",
+ "metadata": {
+ "[key]": "[value]",
+ }
+}
+EOF
+For example:
+curl -sSiX PATCH http://localhost/users/1890c034-7ef9-4cde-83df-d78ea1d4d281 -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "name": "Jane Doe",
+ "metadata": {
+ "location": "london",
+ }
+}
+EOF
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Wed, 14 Jun 2023 13:54:40 GMT
+Content-Type: application/json
+Content-Length: 354
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "1890c034-7ef9-4cde-83df-d78ea1d4d281",
+ "name": "Jane Doe",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "identity": "jane.doe@email.com" },
+ "metadata": { "location": "london" },
+ "created_at": "2023-06-14T13:46:47.322648Z",
+ "updated_at": "2023-06-14T13:54:40.208005Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "enabled"
+}
+Updating user's tags
+curl -sSiX PATCH http://localhost/users/<user_id>/tags -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "tags": [
+ "[tag_1]",
+ ...
+ "[tag_N]"
+ ]
+}
+EOF
+For example:
+curl -sSiX PATCH http://localhost/users/1890c034-7ef9-4cde-83df-d78ea1d4d281/tags -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "tags": ["male", "developer"]
+}
+EOF
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Wed, 14 Jun 2023 13:55:18 GMT
+Content-Type: application/json
+Content-Length: 375
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "1890c034-7ef9-4cde-83df-d78ea1d4d281",
+ "name": "Jane Doe",
+ "tags": ["male", "developer"],
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "identity": "jane.doe@email.com" },
+ "metadata": { "location": "london" },
+ "created_at": "2023-06-14T13:46:47.322648Z",
+ "updated_at": "2023-06-14T13:55:18.353027Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "enabled"
+}
+Updating user's owner
+curl -sSiX PATCH http://localhost/users/<user_id>/owner -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "owner": "<owner_id>"
+}
+EOF
+For example:
+curl -sSiX PATCH http://localhost/users/1890c034-7ef9-4cde-83df-d78ea1d4d281/owner -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "owner": "532311a4-c13b-4061-b991-98dcae7a934e"
+}
+EOF
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Wed, 14 Jun 2023 13:56:32 GMT
+Content-Type: application/json
+Content-Length: 375
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "1890c034-7ef9-4cde-83df-d78ea1d4d281",
+ "name": "Jane Doe",
+ "tags": ["male", "developer"],
+ "owner": "532311a4-c13b-4061-b991-98dcae7a934e",
+ "credentials": { "identity": "jane.doe@email.com" },
+ "metadata": { "location": "london" },
+ "created_at": "2023-06-14T13:46:47.322648Z",
+ "updated_at": "2023-06-14T13:56:32.059484Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "enabled"
+}
+Updating user's identity
+curl -sSiX PATCH http://localhost/users/<user_id>/identity -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "identity": "<user_identity>"
+}
+EOF
+For example:
+curl -sSiX PATCH http://localhost/users/1890c034-7ef9-4cde-83df-d78ea1d4d281/identity -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "identity": "updated.jane.doe@gmail.com"
+}
+EOF
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Wed, 14 Jun 2023 13:59:53 GMT
+Content-Type: application/json
+Content-Length: 382
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "1890c034-7ef9-4cde-83df-d78ea1d4d281",
+ "name": "Jane Doe",
+ "tags": ["male", "developer"],
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "identity": "updated.jane.doe@gmail.com" },
+ "metadata": { "location": "london" },
+ "created_at": "2023-06-14T13:46:47.322648Z",
+ "updated_at": "2023-06-14T13:59:53.422595Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "enabled"
+}
+Changing the user secret can be done by calling the update secret method
+curl -sSiX PATCH http://localhost/users/secret -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "old_secret": "<old_secret>",
+ "new_secret": "<new_secret>"
+}
+EOF
+For example:
+curl -sSiX PATCH http://localhost/users/secret -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "old_secret": "12345678",
+ "new_secret": "12345678a"
+}
+EOF
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Wed, 14 Jun 2023 14:00:35 GMT
+Content-Type: application/json
+Content-Length: 281
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+Changing the user status to enabled can be done by calling the enable user method
+curl -sSiX POST http://localhost/users/<user_id>/enable -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX POST http://localhost/users/1890c034-7ef9-4cde-83df-d78ea1d4d281/enable -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Wed, 14 Jun 2023 14:01:25 GMT
+Content-Type: application/json
+Content-Length: 382
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "1890c034-7ef9-4cde-83df-d78ea1d4d281",
+ "name": "Jane Doe",
+ "tags": ["male", "developer"],
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "identity": "updated.jane.doe@gmail.com" },
+ "metadata": { "location": "london" },
+ "created_at": "2023-06-14T13:46:47.322648Z",
+ "updated_at": "2023-06-14T13:59:53.422595Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "enabled"
+}
+Changing the user status to disabled can be done by calling the disable user method
+curl -sSiX POST http://localhost/users/<user_id>/disable -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX POST http://localhost/users/1890c034-7ef9-4cde-83df-d78ea1d4d281/disable -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Wed, 14 Jun 2023 14:01:23 GMT
+Content-Type: application/json
+Content-Length: 383
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "1890c034-7ef9-4cde-83df-d78ea1d4d281",
+ "name": "Jane Doe",
+ "tags": ["male", "developer"],
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "identity": "updated.jane.doe@gmail.com" },
+ "metadata": { "location": "london" },
+ "created_at": "2023-06-14T13:46:47.322648Z",
+ "updated_at": "2023-06-14T13:59:53.422595Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "disabled"
+}
+You can get all groups a user is assigned to by calling the get user memberships method.
+If you want to paginate your results then use offset, limit, metadata, name, status, parentID, ownerID, tree and dir as query parameters.
++The user identified by the
+user_tokenmust be assigned to the same group as the user with iduser_idwithc_listaction. Alternatively, the user identified by theuser_tokenmust be the owner of the user with iduser_id.
curl -sSiX GET http://localhost/users/<user_id>/memberships -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX GET http://localhost/users/1890c034-7ef9-4cde-83df-d78ea1d4d281/memberships -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 11:22:18 GMT
+Content-Type: application/json
+Content-Length: 367
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "limit": 0,
+ "offset": 0,
+ "memberships": [
+ {
+ "id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Data analysts",
+ "description": "This group would be responsible for analyzing data collected from sensors.",
+ "metadata": { "location": "london" },
+ "created_at": "2023-06-15T09:41:42.860481Z",
+ "updated_at": "2023-06-15T10:17:56.475241Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "enabled"
+ }
+ ]
+}
+To create a thing, you need the thing and a user_token
curl -sSiX POST http://localhost/things -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "id": "[thing_id]",
+ "name":"[thing_name]",
+ "tags": ["[tag1]", "[tag2]"],
+ "credentials": {
+ "identity": "[thing-identity]",
+ "secret":"[thing-secret]"
+ },
+ "metadata": {
+ "[key1]": "[value1]",
+ "[key2]": "[value2]"
+ },
+ "status": "[enabled|disabled]"
+}
+EOF
+For example:
+curl -sSiX POST http://localhost/things -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "name":"Temperature Sensor"
+}
+EOF
+
+HTTP/1.1 201 Created
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:04:04 GMT
+Content-Type: application/json
+Content-Length: 280
+Connection: keep-alive
+Location: /things/48101ecd-1535-40c6-9ed8-5b1d21e371bb
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "48101ecd-1535-40c6-9ed8-5b1d21e371bb",
+ "name": "Temperature Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "c3f8c096-c60f-4375-8494-bca20a12fca7" },
+ "created_at": "2023-06-15T09:04:04.292602664Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+}
+It is often the case that the user will want to integrate the existing solutions, e.g. an asset management system, with the Magistrala platform. To simplify the integration between the systems and avoid artificial cross-platform reference, such as special fields in Magistrala Things metadata, it is possible to set Magistrala Thing ID with an existing unique ID while create the Thing. This way, the user can set the existing ID as the Thing ID of a newly created Thing to keep reference between Thing and the asset that Thing represents.
+The limitation is that the existing ID has to be unique in the Magistrala domain.
+To create a thing with an external ID, you need to provide the ID together with thing name, and other fields as well as a user_token
For example:
+curl -sSiX POST http://localhost/things -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "name":"Temperature Sensor"
+}
+EOF
+
+HTTP/1.1 201 Created
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:05:06 GMT
+Content-Type: application/json
+Content-Length: 280
+Connection: keep-alive
+Location: /things/2766ae94-9a08-4418-82ce-3b91cf2ccd3e
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "name": "Temperature Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "65ca03bd-eb6b-420b-9d5d-46d459d4f71c" },
+ "created_at": "2023-06-15T09:05:06.538170496Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+}
+It is often the case that the user will want to integrate the existing solutions, e.g. an asset management system, with the Magistrala platform. To simplify the integration between the systems and avoid artificial cross-platform reference, such as special fields in Magistrala Things metadata, it is possible to set Magistrala Thing secret with an existing unique secret when creating the Thing. This way, the user can set the existing secret as the Thing secret of a newly created Thing to keep reference between Thing and the asset that Thing represents. +The limitation is that the existing secret has to be unique in the Magistrala domain.
+To create a thing with an external secret, you need to provide the secret together with thing name, and other fields as well as a user_token
For example:
+curl -sSiX POST http://localhost/things -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "name":"Temperature Sensor"
+ "credentials": {
+ "secret": "94939159-9a08-4f17-9e4e-3b91cf2ccd3e"
+ }
+}
+EOF
+
+HTTP/1.1 201 Created
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:05:06 GMT
+Content-Type: application/json
+Content-Length: 280
+Connection: keep-alive
+Location: /things/2766ae94-9a08-4418-82ce-3b91cf2ccd3e
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "name": "Temperature Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "94939159-9a08-4f17-9e4e-3b91cf2ccd3e" },
+ "created_at": "2023-06-15T09:05:06.538170496Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+}
+You can create multiple things at once by entering a series of things structures and a user_token
curl -sSiX POST http://localhost/things/bulk -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+[
+ {
+ "id": "[thing_id]",
+ "name":"[thing_name]",
+ "tags": ["[tag1]", "[tag2]"],
+ "credentials": {
+ "identity": "[thing-identity]",
+ "secret":"[thing-secret]"
+ },
+ "metadata": {
+ "[key1]": "[value1]",
+ "[key2]": "[value2]"
+ },
+ "status": "[enabled|disabled]"
+ },
+ {
+ "id": "[thing_id]",
+ "name":"[thing_name]",
+ "tags": ["[tag1]", "[tag2]"],
+ "credentials": {
+ "identity": "[thing-identity]",
+ "secret":"[thing-secret]"
+ },
+ "metadata": {
+ "[key1]": "[value1]",
+ "[key2]": "[value2]"
+ },
+ "status": "[enabled|disabled]"
+ }
+]
+EOF
+For example:
+curl -sSiX POST http://localhost/things/bulk -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+[
+ {
+ "name":"Motion Sensor"
+ },
+ {
+ "name":"Light Sensor"
+ }
+]
+EOF
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:05:45 GMT
+Content-Type: application/json
+Content-Length: 583
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "total": 2,
+ "things": [
+ {
+ "id": "19f59b2d-1e9c-43db-bc84-5432bd52a83f",
+ "name": "Motion Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "941c380a-3a41-40e9-8b79-3087daa4f3a6" },
+ "created_at": "2023-06-15T09:05:45.719182307Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "3709f2b0-9c73-413f-992e-7f6f9b396b0d",
+ "name": "Light Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "798ee6be-311b-4640-99e4-0ccb19e0dcb9" },
+ "created_at": "2023-06-15T09:05:45.719186184Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }
+ ]
+}
+The same as creating a Thing with external ID the user can create multiple things at once by providing UUID v4 format unique ID in a series of things together with a user_token
For example:
+curl -sSiX POST http://localhost/things/bulk -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+[
+ {
+ "id": "eb2670ba-a2be-4ea4-83cb-111111111111",
+ "name":"Motion Sensor"
+ },
+ {
+ "id": "eb2670ba-a2be-4ea4-83cb-111111111112",
+ "name":"Light Sensor"
+ }
+]
+EOF
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:06:17 GMT
+Content-Type: application/json
+Content-Length: 583
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "total": 2,
+ "things": [
+ {
+ "id": "eb2670ba-a2be-4ea4-83cb-111111111111",
+ "name": "Motion Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "325cda17-3a52-465d-89a7-2b63c7d0e3a6" },
+ "created_at": "2023-06-15T09:06:17.967825372Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "eb2670ba-a2be-4ea4-83cb-111111111112",
+ "name": "Light Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "67b6cbb8-4a9e-4d32-8b9c-d7cd3352aa2b" },
+ "created_at": "2023-06-15T09:06:17.967828689Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }
+ ]
+}
+You can get thing entity by entering the thing ID and user_token
curl -sSiX GET http://localhost/things/<thing_id> -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX GET http://localhost/things/48101ecd-1535-40c6-9ed8-5b1d21e371bb -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:07:30 GMT
+Content-Type: application/json
+Content-Length: 277
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "48101ecd-1535-40c6-9ed8-5b1d21e371bb",
+ "name": "Temperature Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "c3f8c096-c60f-4375-8494-bca20a12fca7" },
+ "created_at": "2023-06-15T09:04:04.292602Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+}
+You can get all things in the database by querying /things endpoint.
curl -sSiX GET http://localhost/things -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX GET http://localhost/things -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:07:59 GMT
+Content-Type: application/json
+Transfer-Encoding: chunked
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "limit": 10,
+ "total": 8,
+ "things": [
+ {
+ "id": "f3047c10-f2c7-4d53-b3c0-bc56c560c546",
+ "name": "Humidity Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "6d11a91f-0bd8-41aa-8e1b-4c6338329c9c" },
+ "created_at": "2023-06-14T12:04:12.740098Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "04b0b2d1-fdaf-4b66-96a0-740a3151db4c",
+ "name": "UV Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "a1e5d77f-8903-4cef-87b1-d793a3c28de3" },
+ "created_at": "2023-06-14T12:04:56.245743Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "48101ecd-1535-40c6-9ed8-5b1d21e371bb",
+ "name": "Temperature Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "c3f8c096-c60f-4375-8494-bca20a12fca7" },
+ "created_at": "2023-06-15T09:04:04.292602Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "name": "Temperature Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "65ca03bd-eb6b-420b-9d5d-46d459d4f71c" },
+ "created_at": "2023-06-15T09:05:06.53817Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "19f59b2d-1e9c-43db-bc84-5432bd52a83f",
+ "name": "Motion Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "941c380a-3a41-40e9-8b79-3087daa4f3a6" },
+ "created_at": "2023-06-15T09:05:45.719182Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "3709f2b0-9c73-413f-992e-7f6f9b396b0d",
+ "name": "Light Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "798ee6be-311b-4640-99e4-0ccb19e0dcb9" },
+ "created_at": "2023-06-15T09:05:45.719186Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "eb2670ba-a2be-4ea4-83cb-111111111111",
+ "name": "Motion Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "325cda17-3a52-465d-89a7-2b63c7d0e3a6" },
+ "created_at": "2023-06-15T09:06:17.967825Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "eb2670ba-a2be-4ea4-83cb-111111111112",
+ "name": "Light Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "67b6cbb8-4a9e-4d32-8b9c-d7cd3352aa2b" },
+ "created_at": "2023-06-15T09:06:17.967828Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }
+ ]
+}
+If you want to paginate your results then use offset, limit, metadata, name, status, tags and visibility as query parameters.
curl -sSiX GET http://localhost/things?[offset=<offset>]&[limit=<limit>]&name=[name]&[status=<status>] -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX GET http://localhost/things?offset=1&limit=5&name=Light Sensor -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:08:39 GMT
+Content-Type: application/json
+Content-Length: 321
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "limit": 5,
+ "offset": 1,
+ "total": 2,
+ "things": [
+ {
+ "id": "eb2670ba-a2be-4ea4-83cb-111111111112",
+ "name": "Light Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "67b6cbb8-4a9e-4d32-8b9c-d7cd3352aa2b" },
+ "created_at": "2023-06-15T09:06:17.967828Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }
+ ]
+}
+Updating a thing name and/or metadata
+curl -sSiX PATCH http://localhost/things/<thing_id> -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "name":"[thing_name]",
+ "metadata": {
+ "[key1]": "[value1]",
+ "[key2]": "[value2]"
+ }
+}
+EOF
+For example:
+curl -sSiX PATCH http://localhost/things/48101ecd-1535-40c6-9ed8-5b1d21e371bb -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "name":"Pressure Sensor"
+}
+EOF
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:09:12 GMT
+Content-Type: application/json
+Content-Length: 332
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "48101ecd-1535-40c6-9ed8-5b1d21e371bb",
+ "name": "Pressure Sensor",
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "c3f8c096-c60f-4375-8494-bca20a12fca7" },
+ "created_at": "2023-06-15T09:04:04.292602Z",
+ "updated_at": "2023-06-15T09:09:12.267074Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "enabled"
+}
+Updating a thing tags
+curl -sSiX PATCH http://localhost/things/<thing_id>/tags -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "tags": ["tag_1", ..., "tag_N"]
+}
+EOF
+For example:
+curl -sSiX PATCH http://localhost/things/48101ecd-1535-40c6-9ed8-5b1d21e371bb/tags -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "tags": ["sensor", "smart"]
+}
+EOF
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:09:44 GMT
+Content-Type: application/json
+Content-Length: 347
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "48101ecd-1535-40c6-9ed8-5b1d21e371bb",
+ "name": "Pressure Sensor",
+ "tags": ["sensor", "smart"],
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "c3f8c096-c60f-4375-8494-bca20a12fca7" },
+ "created_at": "2023-06-15T09:04:04.292602Z",
+ "updated_at": "2023-06-15T09:09:44.766726Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "enabled"
+}
+Updating a thing entity
+curl -sSiX PATCH http://localhost/things/<thing_id>/owner -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "owner": "[owner_id]"
+}
+EOF
+For example:
+curl -sSiX PATCH http://localhost/things/48101ecd-1535-40c6-9ed8-5b1d21e371bb/owner -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "owner": "f7c55a1f-dde8-4880-9796-b3a0cd05745b"
+}
+EOF
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:09:44 GMT
+Content-Type: application/json
+Content-Length: 347
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "48101ecd-1535-40c6-9ed8-5b1d21e371bb",
+ "name": "Pressure Sensor",
+ "tags": ["sensor", "smart"],
+ "owner": "f7c55a1f-dde8-4880-9796-b3a0cd05745b",
+ "credentials": { "secret": "c3f8c096-c60f-4375-8494-bca20a12fca7" },
+ "created_at": "2023-06-15T09:04:04.292602Z",
+ "updated_at": "2023-06-15T09:09:44.766726Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "enabled"
+}
+Updating a thing secret
+curl -sSiX PATCH http://localhost/things/<thing_id>/secret -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "secret": "<thing_secret>"
+}
+EOF
+For example:
+curl -sSiX PATCH http://localhost/things/48101ecd-1535-40c6-9ed8-5b1d21e371bb/secret -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "secret": "94939159-9a08-4f17-9e4e-3b91cf2ccd3e"
+}
+EOF
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:10:52 GMT
+Content-Type: application/json
+Content-Length: 321
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "48101ecd-1535-40c6-9ed8-5b1d21e371bb",
+ "name": "Pressure Sensor",
+ "tags": ["sensor", "smart"],
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "94939159-9a08-4f17-9e4e-3b91cf2ccd3e" },
+ "created_at": "2023-06-15T09:04:04.292602Z",
+ "updated_at": "2023-06-15T09:10:52.051497Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "enabled"
+}
+To enable a thing you need a thing_id and a user_token
curl -sSiX POST http://localhost/things/<thing_id>/enable -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX POST http://localhost/things/48101ecd-1535-40c6-9ed8-5b1d21e371bb/enable -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:11:43 GMT
+Content-Type: application/json
+Content-Length: 321
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "48101ecd-1535-40c6-9ed8-5b1d21e371bb",
+ "name": "Pressure Sensor",
+ "tags": ["sensor", "smart"],
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "94939159-9a08-4f17-9e4e-3b91cf2ccd3e" },
+ "created_at": "2023-06-15T09:04:04.292602Z",
+ "updated_at": "2023-06-15T09:10:52.051497Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "enabled"
+}
+To disable a thing you need a thing_id and a user_token
curl -sSiX POST http://localhost/things/<thing_id>/disable -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX POST http://localhost/things/48101ecd-1535-40c6-9ed8-5b1d21e371bb/disable -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:11:38 GMT
+Content-Type: application/json
+Content-Length: 322
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "48101ecd-1535-40c6-9ed8-5b1d21e371bb",
+ "name": "Pressure Sensor",
+ "tags": ["sensor", "smart"],
+ "owner": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "credentials": { "secret": "94939159-9a08-4f17-9e4e-3b91cf2ccd3e" },
+ "created_at": "2023-06-15T09:04:04.292602Z",
+ "updated_at": "2023-06-15T09:10:52.051497Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "disabled"
+}
+To create a channel, you need a user_token
curl -sSiX POST http://localhost/channels -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "id": "[channel_id]",
+ "name":"[channel_name]",
+ "description":"[channel_description]",
+ "owner_id": "[owner_id]",
+ "metadata": {
+ "[key1]": "[value1]",
+ "[key2]": "[value2]"
+ },
+ "status": "[enabled|disabled]"
+}
+EOF
+For example:
+curl -sSiX POST http://localhost/channels -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "name": "Temperature Data"
+}
+EOF
+
+HTTP/1.1 201 Created
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:12:51 GMT
+Content-Type: application/json
+Content-Length: 218
+Connection: keep-alive
+Location: /channels/aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Temperature Data",
+ "created_at": "2023-06-15T09:12:51.162431Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+}
+Channel is a group of things that could represent a special category in existing systems, e.g. a building level channel could represent the level of a smarting building system. For helping to keep the reference, it is possible to set an existing ID while creating the Magistrala channel. There are two limitations - the existing ID has to be in UUID V4 format and it has to be unique in the Magistrala domain.
+To create a channel with external ID, the user needs to provide a UUID v4 format unique ID, and a user_token
For example:
+curl -sSiX POST http://localhost/channels -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "id": "48101ecd-1535-40c6-9ed8-5b1d21e371bb",
+ "name": "Humidity Data"
+}
+EOF
+
+HTTP/1.1 201 Created
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:15:11 GMT
+Content-Type: application/json
+Content-Length: 219
+Connection: keep-alive
+Location: /channels/48101ecd-1535-40c6-9ed8-5b1d21e371bb
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "48101ecd-1535-40c6-9ed8-5b1d21e371bb",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Humidity Data",
+ "created_at": "2023-06-15T09:15:11.477695Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+}
+The same as creating a channel with external ID the user can create multiple channels at once by providing UUID v4 format unique ID in a series of channels together with a user_token
curl -sSiX POST http://localhost/channels/bulk -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+[
+ {
+ "id": "[channel_id]",
+ "name":"[channel_name]",
+ "description":"[channel_description]",
+ "owner_id": "[owner_id]",
+ "metadata": {
+ "[key1]": "[value1]",
+ "[key2]": "[value2]"
+ },
+ "status": "[enabled|disabled]"
+ },
+ {
+ "id": "[channel_id]",
+ "name":"[channel_name]",
+ "description":"[channel_description]",
+ "owner_id": "[owner_id]",
+ "metadata": {
+ "[key1]": "[value1]",
+ "[key2]": "[value2]"
+ },
+ "status": "[enabled|disabled]"
+ }
+]
+EOF
+For example:
+curl -sSiX POST http://localhost/channels/bulk -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+[
+ {
+ "name":"Light Data"
+ },
+ {
+ "name":"Pressure Data"
+ }
+]
+EOF
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:15:44 GMT
+Content-Type: application/json
+Content-Length: 450
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "channels": [
+ {
+ "id": "cb81bbff-850d-471f-bd74-c15d6e1a6c4e",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Light Data",
+ "created_at": "2023-06-15T09:15:44.154283Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "fc9bf029-b1d3-4408-8d53-fc576247a4b3",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Pressure Data",
+ "created_at": "2023-06-15T09:15:44.15721Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }
+ ]
+}
+As with things, you can create multiple channels with external ID at once
+For example:
+curl -sSiX POST http://localhost/channels/bulk -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+[
+ {
+ "id": "977bbd33-5b59-4b7a-a9c3-111111111111",
+ "name":"Light Data"
+ },
+ {
+ "id": "977bbd33-5b59-4b7a-a9c3-111111111112",
+ "name":"Pressure Data"
+ }
+]
+EOF
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:16:16 GMT
+Content-Type: application/json
+Content-Length: 453
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "channels": [
+ {
+ "id": "977bbd33-5b59-4b7a-a9c3-111111111111",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Light Data",
+ "created_at": "2023-06-15T09:16:16.931016Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "977bbd33-5b59-4b7a-a9c3-111111111112",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Pressure Data",
+ "created_at": "2023-06-15T09:16:16.934486Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }
+ ]
+}
+Get a channel entity for a logged-in user
+curl -sSiX GET http://localhost/channels/<channel_id> -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX GET http://localhost/channels/aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8 -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:17:17 GMT
+Content-Type: application/json
+Content-Length: 218
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Temperature Data",
+ "created_at": "2023-06-15T09:12:51.162431Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+}
+You can get all channels for a logged-in user.
+If you want to paginate your results then use offset, limit, metadata, name, status, parentID, ownerID, tree and dir as query parameters.
curl -sSiX GET http://localhost/channels -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX GET http://localhost/channels -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:17:46 GMT
+Content-Type: application/json
+Content-Length: 1754
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "total": 8,
+ "channels": [
+ {
+ "id": "17129934-4f48-4163-bffe-0b7b532edc5c",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Tokyo",
+ "created_at": "2023-06-14T12:10:07.950311Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "48101ecd-1535-40c6-9ed8-5b1d21e371bb",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Humidity Data",
+ "created_at": "2023-06-15T09:15:11.477695Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "977bbd33-5b59-4b7a-a9c3-111111111111",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Light Data",
+ "created_at": "2023-06-15T09:16:16.931016Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "977bbd33-5b59-4b7a-a9c3-111111111112",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Pressure Data",
+ "created_at": "2023-06-15T09:16:16.934486Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Temperature Data",
+ "created_at": "2023-06-15T09:12:51.162431Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "b3867a52-675d-4f05-8cd0-df5a08a63ff3",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "London",
+ "created_at": "2023-06-14T12:09:34.205894Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "cb81bbff-850d-471f-bd74-c15d6e1a6c4e",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Light Data",
+ "created_at": "2023-06-15T09:15:44.154283Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "fc9bf029-b1d3-4408-8d53-fc576247a4b3",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Pressure Data",
+ "created_at": "2023-06-15T09:15:44.15721Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }
+ ]
+}
+Update channel name and/or metadata.
+curl -sSiX PUT http://localhost/channels/<channel_id> -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "name":"[channel_name]",
+ "description":"[channel_description]",
+ "metadata": {
+ "[key1]": "[value1]",
+ "[key2]": "[value2]"
+ }
+}
+EOF
+For example:
+curl -sSiX PUT http://localhost/channels/aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8 -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "name":"Jane Doe",
+ "metadata": {
+ "location": "london"
+ }
+}
+EOF
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:18:26 GMT
+Content-Type: application/json
+Content-Length: 296
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Jane Doe",
+ "metadata": { "location": "london" },
+ "created_at": "2023-06-15T09:12:51.162431Z",
+ "updated_at": "2023-06-15T09:18:26.886913Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "enabled"
+}
+To enable a channel you need a channel_id and a user_token
curl -sSiX POST http://localhost/channels/<channel_id>/enable -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX POST http://localhost/channels/aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8/enable -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:19:29 GMT
+Content-Type: application/json
+Content-Length: 296
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Jane Doe",
+ "metadata": { "location": "london" },
+ "created_at": "2023-06-15T09:12:51.162431Z",
+ "updated_at": "2023-06-15T09:18:26.886913Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "enabled"
+}
+To disable a channel you need a channel_id and a user_token
curl -sSiX POST http://localhost/channels/<channel_id>/disable -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX POST http://localhost/channels/aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8/disable -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:19:24 GMT
+Content-Type: application/json
+Content-Length: 297
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Jane Doe",
+ "metadata": { "location": "london" },
+ "created_at": "2023-06-15T09:12:51.162431Z",
+ "updated_at": "2023-06-15T09:18:26.886913Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "disabled"
+}
+Connect things to channels
++++
actionsis optional, if not provided, the default action ism_readandm_write.
curl -sSiX POST http://localhost/connect -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subjects": ["<thing_id>"],
+ "objects": ["<channel_id>"],
+ "actions": ["[action]"]
+}
+EOF
+For example:
+curl -sSiX POST http://localhost/connect -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subjects": ["48101ecd-1535-40c6-9ed8-5b1d21e371bb"],
+ "objects": ["aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8"]
+}
+EOF
+
+HTTP/1.1 201 Created
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:21:37 GMT
+Content-Type: application/json
+Content-Length: 247
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "policies": [
+ {
+ "owner_id": "",
+ "subject": "48101ecd-1535-40c6-9ed8-5b1d21e371bb",
+ "object": "aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8",
+ "actions": ["m_write", "m_read"],
+ "created_at": "0001-01-01T00:00:00Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "updated_by": ""
+ }
+ ]
+}
+Connect thing to channel
++++
actionsis optional, if not provided, the default actions arem_readandm_write.
curl -sSiX POST http://localhost/things/policies -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subject": "<thing_id>",
+ "object": "<channel_id>",
+ "actions": ["<action>", "[action]"]]
+}
+EOF
+For example:
+curl -sSiX POST http://localhost/things/policies -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subject": "48101ecd-1535-40c6-9ed8-5b1d21e371bb",
+ "object": "aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8"
+}
+EOF
+
+HTTP/1.1 201 Created
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:23:28 GMT
+Content-Type: application/json
+Content-Length: 290
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "policies": [
+ {
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "subject": "48101ecd-1535-40c6-9ed8-5b1d21e371bb",
+ "object": "aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8",
+ "actions": ["m_write", "m_read"],
+ "created_at": "2023-06-15T09:23:28.769729Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "updated_by": ""
+ }
+ ]
+}
+Disconnect things from channels specified by lists of IDs.
+curl -sSiX POST http://localhost/disconnect -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subjects": ["<thing_id_1>", "[thing_id_2]"],
+ "objects": ["<channel_id_1>", "[channel_id_2]"]
+}
+EOF
+For example:
+curl -sSiX POST http://localhost/disconnect -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subjects": ["48101ecd-1535-40c6-9ed8-5b1d21e371bb"],
+ "objects": ["aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8"]
+}
+EOF
+
+HTTP/1.1 204 No Content
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:23:07 GMT
+Content-Type: application/json
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+Disconnect thing from the channel
+curl -sSiX DELETE http://localhost/things/policies/<subject_id>/<object_id> -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX DELETE http://localhost/things/policies/48101ecd-1535-40c6-9ed8-5b1d21e371bb/aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8 -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 204 No Content
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:25:23 GMT
+Content-Type: application/json
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+Checks if thing has access to a channel
+curl -sSiX POST http://localhost/channels/<channel_id>/access -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subject": "<thing_secret>",
+ "action": "m_read" | "m_write",
+ "entity_type": "thing"
+}
+EOF
+For example:
+curl -sSiX POST http://localhost/channels/aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8/access -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subject": "48101ecd-1535-40c6-9ed8-5b1d21e371bb",
+ "action": "m_read",
+ "entity_type": "thing"
+}
+EOF
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:39:26 GMT
+Content-Type: application/json
+Content-Length: 0
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+Validates thing's key and returns it's ID if key is valid
+curl -sSiX POST http://localhost/identify -H "Content-Type: application/json" -H "Authorization: Thing <thing_secret>"
+For example:
+curl -sSiX POST http://localhost/identify -H "Content-Type: application/json" -H "Authorization: Thing 6d11a91f-0bd8-41aa-8e1b-4c6338329c9c"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:28:16 GMT
+Content-Type: application/json
+Content-Length: 46
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{ "id": "f3047c10-f2c7-4d53-b3c0-bc56c560c546" }
+Sends message via HTTP protocol
+curl -sSiX POST http://localhost/http/channels/<channel_id>/messages -H "Content-Type: application/senml+json" -H "Authorization: Thing <thing_secret>" -d @- << EOF
+[
+ {
+ "bn": "<base_name>",
+ "bt": "[base_time]",
+ "bu": "[base_unit]",
+ "bver": [base_version],
+ "n": "<measurement_name>",
+ "u": "<measurement_unit>",
+ "v": <measurement_value>,
+ },
+ {
+ "n": "[measurement_name]",
+ "t": <measurement_time>,
+ "v": <measurement_value>,
+ }
+]
+EOF
+For example:
+curl -sSiX POST http://localhost/http/channels/aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8/messages -H "Content-Type: application/senml+json" -H "Authorization: Thing a83b9afb-9022-4f9e-ba3d-4354a08c273a" -d @- << EOF
+[
+ {
+ "bn": "some-base-name:",
+ "bt": 1.276020076001e+09,
+ "bu": "A",
+ "bver": 5,
+ "n": "voltage",
+ "u": "V",
+ "v": 120.1
+ },
+ {
+ "n": "current",
+ "t": -5,
+ "v": 1.2
+ },
+ {
+ "n": "current",
+ "t": -4,
+ "v": 1.3
+ }
+]
+EOF
+HTTP/1.1 202 Accepted
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:40:44 GMT
+Content-Length: 0
+Connection: keep-alive
+Reads messages from database for a given channel
+curl -sSiX GET http://localhost:<service_port>/channels/<channel_id>/messages?[offset=<offset>]&[limit=<limit>] -H "Authorization: Thing <thing_secret>"
+For example:
+curl -sSiX GET http://localhost:9009/channels/aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8/messages -H "Authorization: Thing a83b9afb-9022-4f9e-ba3d-4354a08c273a"
+
+HTTP/1.1 200 OK
+Content-Type: application/json
+Date: Wed, 05 Apr 2023 16:01:49 GMT
+Content-Length: 660
+
+{
+ "offset": 0,
+ "limit": 10,
+ "format": "messages",
+ "total": 3,
+ "messages": [{
+ "channel": "aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8",
+ "publisher": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "protocol": "http",
+ "name": "some-base-name:voltage",
+ "unit": "V",
+ "time": 1276020076.001,
+ "value": 120.1
+ },
+ {
+ "channel": "aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8",
+ "publisher": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "protocol": "http",
+ "name": "some-base-name:current",
+ "unit": "A",
+ "time": 1276020072.001,
+ "value": 1.3
+ },
+ {
+ "channel": "aecf0902-816d-4e38-a5b3-a1ad9a7cf9e8",
+ "publisher": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "protocol": "http",
+ "name": "some-base-name:current",
+ "unit": "A",
+ "time": 1276020071.001,
+ "value": 1.2
+ }
+ ]
+}
+To create a group, you need the group name and a user_token
curl -sSiX POST http://localhost/groups -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "name":"<group_name>",
+ "description":"[group_description]",
+ "parent_id": "[parent_id]",
+ "owner_id": "[owner_id]",
+ "metadata": {
+ "[key1]": "[value1]",
+ "[key2]": "[value2]"
+ },
+ "status": "[enabled|disabled]"
+}
+EOF
+For example:
+curl -sSiX POST http://localhost/groups -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "name": "Security Engineers",
+ "description": "This group would be responsible for securing the platform."
+}
+EOF
+
+HTTP/1.1 201 Created
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:41:42 GMT
+Content-Type: application/json
+Content-Length: 252
+Connection: keep-alive
+Location: /groups/2766ae94-9a08-4418-82ce-3b91cf2ccd3e
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Security Engineers",
+ "description": "This group would be responsible for securing the platform.",
+ "created_at": "2023-06-15T09:41:42.860481Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+}
+When you use parent_id make sure the parent is an already exisiting group
For example:
+curl -sSiX POST http://localhost/groups -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "name": "Customer Support",
+ "description": "This group would be responsible for providing support to users of the platform.",
+ "parent_id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e"
+}
+EOF
+
+HTTP/1.1 201 Created
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 09:42:34 GMT
+Content-Type: application/json
+Content-Length: 306
+Connection: keep-alive
+Location: /groups/dd2dc8d4-f7cf-42f9-832b-81cae9a8e90a
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "dd2dc8d4-f7cf-42f9-832b-81cae9a8e90a",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "parent_id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "name": "Customer Support",
+ "description": "This group would be responsible for providing support to users of the platform.",
+ "created_at": "2023-06-15T09:42:34.063997Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+}
+Get a group entity for a logged-in user
+curl -sSiX GET http://localhost/groups/<group_id> -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX GET http://localhost/groups/2766ae94-9a08-4418-82ce-3b91cf2ccd3e -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 10:00:52 GMT
+Content-Type: application/json
+Content-Length: 252
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Security Engineers",
+ "description": "This group would be responsible for securing the platform.",
+ "created_at": "2023-06-15T09:41:42.860481Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+}
+You can get all groups for a logged-in user.
+If you want to paginate your results then use offset, limit, metadata, name, status, parentID, ownerID, tree and dir as query parameters.
curl -sSiX GET http://localhost/groups -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX GET http://localhost/groups -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 10:13:50 GMT
+Content-Type: application/json
+Content-Length: 807
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "limit": 0,
+ "offset": 0,
+ "total": 3,
+ "groups": [
+ {
+ "id": "0a4a2c33-2d0e-43df-b51c-d905aba99e17",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Sensor Operators",
+ "created_at": "2023-06-14T13:33:52.249784Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Security Engineers",
+ "description": "This group would be responsible for securing the platform.",
+ "created_at": "2023-06-15T09:41:42.860481Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "dd2dc8d4-f7cf-42f9-832b-81cae9a8e90a",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "parent_id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "name": "Customer Support",
+ "description": "This group would be responsible for providing support to users of the platform.",
+ "created_at": "2023-06-15T09:42:34.063997Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }
+ ]
+}
+You can get all groups that are parents of a group for a logged-in user.
+If you want to paginate your results then use offset, limit, metadata, name, status, parentID, ownerID, tree and dir as query parameters.
curl -sSiX GET http://localhost/groups/<group_id>/parents -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX GET http://localhost/groups/dd2dc8d4-f7cf-42f9-832b-81cae9a8e90a/parents?tree=true -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 10:16:03 GMT
+Content-Type: application/json
+Content-Length: 627
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "limit": 10,
+ "offset": 0,
+ "total": 3,
+ "groups": [
+ {
+ "id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Security Engineers",
+ "description": "This group would be responsible for securing the platform.",
+ "level": -1,
+ "children": [
+ {
+ "id": "dd2dc8d4-f7cf-42f9-832b-81cae9a8e90a",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "parent_id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "name": "Customer Support",
+ "description": "This group would be responsible for providing support to users of the platform.",
+ "created_at": "2023-06-15T09:42:34.063997Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }
+ ],
+ "created_at": "2023-06-15T09:41:42.860481Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }
+ ]
+}
+You can get all groups that are children of a group for a logged-in user.
+If you want to paginate your results then use offset, limit, metadata, name, status, parentID, ownerID, tree and dir as query parameters.
curl -sSiX GET http://localhost/groups/<group_id>/children -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX GET http://localhost/groups/2766ae94-9a08-4418-82ce-3b91cf2ccd3e/children?tree=true -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 10:17:13 GMT
+Content-Type: application/json
+Content-Length: 755
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "limit": 10,
+ "offset": 0,
+ "total": 3,
+ "groups": [
+ {
+ "id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Security Engineers",
+ "description": "This group would be responsible for securing the platform.",
+ "path": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "children": [
+ {
+ "id": "dd2dc8d4-f7cf-42f9-832b-81cae9a8e90a",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "parent_id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "name": "Customer Support",
+ "description": "This group would be responsible for providing support to users of the platform.",
+ "level": 1,
+ "path": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e.dd2dc8d4-f7cf-42f9-832b-81cae9a8e90a",
+ "created_at": "2023-06-15T09:42:34.063997Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }
+ ],
+ "created_at": "2023-06-15T09:41:42.860481Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }
+ ]
+}
+Update group entity
+curl -sSiX PUT http://localhost/groups/<group_id> -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "name":"[group_name]",
+ "description":"[group_description]",
+ "metadata": {
+ "[key1]": "[value1]",
+ "[key2]": "[value2]"
+ }
+}
+EOF
+For example:
+curl -sSiX PUT http://localhost/groups/2766ae94-9a08-4418-82ce-3b91cf2ccd3e -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "name":"Data Analysts",
+ "description":"This group would be responsible for analyzing data collected from sensors.",
+ "metadata": {
+ "location": "london"
+ }
+}
+EOF
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 10:17:56 GMT
+Content-Type: application/json
+Content-Length: 328
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Data Analysts",
+ "description": "This group would be responsible for analyzing data collected from sensors.",
+ "metadata": { "location": "london" },
+ "created_at": "2023-06-15T09:41:42.860481Z",
+ "updated_at": "2023-06-15T10:17:56.475241Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "enabled"
+}
+Disable a group entity
+curl -sSiX POST http://localhost/groups/<group_id>/disable -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX POST http://localhost/groups/2766ae94-9a08-4418-82ce-3b91cf2ccd3e/disable -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 10:18:28 GMT
+Content-Type: application/json
+Content-Length: 329
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Data Analysts",
+ "description": "This group would be responsible for analyzing data collected from sensors.",
+ "metadata": { "location": "london" },
+ "created_at": "2023-06-15T09:41:42.860481Z",
+ "updated_at": "2023-06-15T10:17:56.475241Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "disabled"
+}
+Enable a group entity
+curl -sSiX POST http://localhost/groups/<group_id>/enable -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX POST http://localhost/groups/2766ae94-9a08-4418-82ce-3b91cf2ccd3e/enable -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 10:18:55 GMT
+Content-Type: application/json
+Content-Length: 328
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "id": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "owner_id": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "name": "Data Analysts",
+ "description": "This group would be responsible for analyzing data collected from sensors.",
+ "metadata": { "location": "london" },
+ "created_at": "2023-06-15T09:41:42.860481Z",
+ "updated_at": "2023-06-15T10:17:56.475241Z",
+ "updated_by": "94939159-d129-4f17-9e4e-cc2d615539d7",
+ "status": "enabled"
+}
+Assign user to a group
+curl -sSiX POST http://localhost/users/policies -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subject": "<user_id>",
+ "object": "<group_id>",
+ "actions": ["<member_action>"]
+}
+EOF
+For example:
+curl -sSiX POST http://localhost/users/policies -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subject": "1890c034-7ef9-4cde-83df-d78ea1d4d281",
+ "object": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "actions": ["g_list", "c_list"]
+}
+EOF
+
+HTTP/1.1 201 Created
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 10:19:59 GMT
+Content-Type: application/json
+Content-Length: 0
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+You can get all users assigned to a group.
+If you want to paginate your results then use offset, limit, metadata, name, status, identity, and tag as query parameters.
++Must take into consideration the user identified by the
+user_tokenneeds to be assigned to the same group identified bygroup_idwithg_listaction or be the owner of the group identified bygroup_id.
curl -sSiX GET http://localhost/groups/<group_id>/members -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX GET http://localhost/groups/2766ae94-9a08-4418-82ce-3b91cf2ccd3e/members -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 200 OK
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 11:21:29 GMT
+Content-Type: application/json
+Content-Length: 318
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+
+{
+ "limit": 10,
+ "total": 1,
+ "members": [
+ {
+ "id": "1890c034-7ef9-4cde-83df-d78ea1d4d281",
+ "name": "Jane Doe",
+ "tags": ["male", "developer"],
+ "credentials": { "identity": "updated.jane.doe@gmail.com" },
+ "metadata": { "location": "london" },
+ "created_at": "2023-06-14T13:46:47.322648Z",
+ "updated_at": "2023-06-14T13:59:53.422595Z",
+ "status": "enabled"
+ }
+ ]
+}
+Unassign user from group
+curl -sSiX DELETE http://localhost/users/policies/<subject_id>/<object_id> -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX DELETE http://localhost/users/policies/1890c034-7ef9-4cde-83df-d78ea1d4d281/2766ae94-9a08-4418-82ce-3b91cf2ccd3e -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 204 No Content
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 11:25:27 GMT
+Content-Type: application/json
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+Only actions defined on Predefined Policies section are allowed.
+curl -sSiX POST http://localhost/users/policies -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subject": "<user_id>",
+ "object": "<group_id>",
+ "actions": ["<actions>", "[actions]"]
+}
+EOF
+curl -sSiX POST http://localhost/things/policies -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subject": "<thing_id>",
+ "object": "<channel_id>",
+ "actions": ["<actions>", "[actions]"]
+}
+EOF
+curl -sSiX POST http://localhost/things/policies -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subject": "<user_id>",
+ "object": "<channel_id>",
+ "actions": ["<actions>", "[actions]"]
+ "external": true
+}
+EOF
+For example:
+curl -sSiX POST http://localhost/users/policies -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subject": "1890c034-7ef9-4cde-83df-d78ea1d4d281",
+ "object": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "actions": ["g_add", "c_list"]
+}
+EOF
+
+HTTP/1.1 201 Created
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 11:26:50 GMT
+Content-Type: application/json
+Content-Length: 0
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+Only actions defined on Predefined Policies section are allowed.
+curl -sSiX PUT http://localhost/users/policies -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subject": "<user_id>",
+ "object": "<group_id>",
+ "actions": ["<actions>", "[actions]"]
+}
+EOF
+curl -sSiX PUT http://localhost/things/policies -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subject": "<thing_id> | <user_id>",
+ "object": "<channel_id>",
+ "actions": ["<actions>", "[actions]"]
+}
+EOF
+For example:
+curl -sSiX PUT http://localhost/users/policies -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" -d @- << EOF
+{
+ "subject": "1890c034-7ef9-4cde-83df-d78ea1d4d281",
+ "object": "2766ae94-9a08-4418-82ce-3b91cf2ccd3e",
+ "actions": ["g_list", "c_list"]
+}
+EOF
+
+HTTP/1.1 204 No Content
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 11:27:19 GMT
+Content-Type: application/json
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+Only policies defined on Predefined Policies section are allowed.
+curl -sSiX DELETE http://localhost/users/policies/<user_id>/<channel_id> -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+curl -sSiX DELETE http://localhost/things/policies/<thing_id>/<channel_id> -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+For example:
+curl -sSiX DELETE http://localhost/users/policies/1890c034-7ef9-4cde-83df-d78ea1d4d281/2766ae94-9a08-4418-82ce-3b91cf2ccd3e -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>"
+
+HTTP/1.1 204 No Content
+Server: nginx/1.23.3
+Date: Thu, 15 Jun 2023 11:28:31 GMT
+Content-Type: application/json
+Connection: keep-alive
+Access-Control-Expose-Headers: Location
+Magistrala IoT platform is comprised of the following services:
+| Service | +Description | +
|---|---|
| users | +Manages platform's users and auth concerns in regards to users and groups | +
| things | +Manages platform's things, channels and auth concerns in regards to things and channels | +
| http-adapter | +Provides an HTTP interface for sending messages via HTTP | +
| mqtt-adapter | +Provides an MQTT and MQTT over WS interface for sending and receiving messages via MQTT | +
| ws-adapter | +Provides a WebSocket interface for sending and receiving messages via WS | +
| coap-adapter | +Provides a CoAP interface for sending and receiving messages via CoAP | +
| opcua-adapter | +Provides an OPC-UA interface for sending and receiving messages via OPC-UA | +
| lora-adapter | +Provides a LoRa Server forwarder for sending and receiving messages via LoRa | +
| magistrala-cli | +Command line interface | +
The platform is built around 2 main entities: users and things.
+User represents the real (human) user of the system. Users are represented via their email address used as their identity, and password used as their secret, which they use as platform access credentials in order to obtain an access token. Once logged into the system, a user can manage their resources (i.e. groups, things and channels) in CRUD fashion and define access control policies by connecting them.
Group represents a logical groupping of users. It is used to simplify access control management by allowing users to be grouped together. When assigning a user to a group, we create a policy that defines what that user can do with the resources of the group. This way, a user can be assigned to multiple groups, and each group can have multiple users assigned to it. Users in one group have access to other users in the same group as long as they have the required policy. A group can also be assigned to another group, thus creating a group hierarchy. When assigning a user to a group we create a policy that defines what that user can do with the group and other users in the group.
Thing represents devices (or applications) connected to Magistrala that uses the platform for message exchange with other "things".
Channel represents a communication channel. It serves as a message topic that can be consumed by all of the things connected to it. It also servers as grouping mechanism for things. A thing can be connected to multiple channels, and a channel can have multiple things connected to it. A user can be connected to a channel as well, thus allowing them to have an access to the messages published to that channel and also things connected to that channel with the required policy. A channel can also be assigned to another channel, thus creating a channel hierarchy. Both things and users can be assigned to a channel. When assigning a thing to a channel, we create a policy that defines what that thing can do to the channel, for example reading or writing messages to it. When assigning a user to a channel, we create a policy that defines what that user can do with the channel and things connected to it, hereby enabling the sharing of things between users.
Magistrala uses NATS as its default messaging backbone, due to its lightweight and performant nature. You can treat its subjects as physical representation of Magistrala channels, where subject name is constructed using channel unique identifier. Magistrala also provides the ability to change your default message broker to RabbitMQ, VerneMQ or Kafka.
+In general, there is no constrained put on content that is being exchanged through channels. However, in order to be post-processed and normalized, messages should be formatted using SenML.
+Magistrala platform can be run on the edge as well. Deploying Magistrala on a gateway makes it able to collect, store and analyze data, organize and authenticate devices. To connect Magistrala instances running on a gateway with Magistrala in a cloud we can use two gateway services developed for that purpose:
+ +Running Magistrala on gateway moves computation from cloud towards the edge thus decentralizing IoT system. Since we can deploy same Magistrala code on gateway and in the cloud there are many benefits but the biggest one is easy deployment and adoption - once engineers understand how to deploy and maintain the platform, they will be able to apply those same skills to any part of the edge-fog-cloud continuum. This is because the platform is designed to be consistent, making it easy for engineers to move between them. This consistency will save engineers time and effort, and it will also help to improve the reliability and security of the platform. Same set of tools can be used, same patches and bug fixes can be applied. The whole system is much easier to reason about, and the maintenance is much easier and less costly.
+ + + + + + +For user authentication Magistrala uses Authentication keys. There are two types of authentication keys:
+Authentication keys are represented and distributed by the corresponding JWT. User keys are issued when user logs in. Each user request (other than registration and login) contains user key that is used to authenticate the user.
+Recovery key is the password recovery key. It's short-lived token used for password recovery process.
+The following actions are supported:
+Federated authentication is a process of authenticating users using external identity providers. Magistrala supports federated authentication using OpenID Connect protocol. Magistrala is a resource provider and it uses Google Identity Platform as an identity provider. To use federated authentication, you need to create a project in Google Cloud Platform and enable Google Identity Platform API. After that, you need to create OAuth 2.0 credentials and configure the consent screen. This can be done by following Google's documentation. Once you have created OAuth 2.0 credentials, you need to set the following environment variables:
+MF_USERS_GOOGLE_CLIENT_ID=985229335584-m2mft8lqbgfn5gfw9ftrm3r2sgu4tsrw.apps.googleusercontent.com
+MF_USERS_GOOGLE_CLIENT_SECRET=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YUIpw
+MF_UI_GOOGLE_REDIRECT_URL=http://localhost/google-callback
+MF_USERS_GOOGLE_STATE=pGXVNhEeKfycuBzk5InlSfMlEU9UrhlkTUOSqhsgDzXP2Y4RsN
+MF_USERS_UI_REDIRECT_URL=http://localhost:9090
+MF_USERS_GOOGLE_CLIENT_ID - Google OAuth 2.0 client IDMF_USERS_GOOGLE_CLIENT_SECRET - Google OAuth 2.0 client secretMF_UI_GOOGLE_REDIRECT_URL - Google OAuth 2.0 redirect URL to handle callback after successful authentication. This URL must be registered in the Google Cloud Platform.MF_USERS_GOOGLE_STATE - Random string used to protect against cross-site request forgery attacks.MF_USERS_UI_REDIRECT_URL - URL to redirect user after successful authentication. This can be your Magistrala UI URL.Magistrala handles the authentication callback at <MF_BASE_URL>/google-callback endpoint, where <MF_BASE_URL> is the base URL of your Magistrala instance. This endpoint needs to be registered in the Google Cloud Platform and it must match the value of MF_UI_GOOGLE_REDIRECT_URL environment variable. From the UI, google state is prefixed with the signin or signup operation to be able to distinguish between sign-in and sign-up operations. For example, if a user is not signed up, the UI will display an error message and a button to sign-up. The error message is sent from the backend using a cookie with the name error. The UI will read the error message from the cookie and display it to the user. This cookie expires in 1 second. When a user signs up, Magistrala creates a local copy of the user with an ID provided by Google, the name and email address provided by Google and the password is left empty as the user is authenticated using Google, i.e. external user. The user can be created only once, so if the user already exists, the error will be sent to the UI via the error cookie. Finally, the user is redirected to the URL provided in MF_USERS_UI_REDIRECT_URL environment variable upon successful authentication. This should be the base URL of your UI.
The MF_USERS_GOOGLE_CLIENT_ID, MF_USERS_GOOGLE_CLIENT_SECRET, MF_UI_GOOGLE_REDIRECT_URL and MF_USERS_GOOGLE_STATE environment variables should be the same for the UI and users service. The MF_USERS_UI_REDIRECT_URL environment variable should be the URL of your UI which is used to redirect the user after successful authentication.
Magistrala uses the access_token provided by Google only to fetch user information which includes user id, name, given name, family name, picture and locale. The access_token is not stored in the database and it's not used for any other purpose. The id_token is not used as it presents challenges on refreshing it, thus Magistrala issues its own access_token and refresh_token stored in the HTTP-only cookie and it's used to authenticate the user in the subsequent requests.
By default, Magistrala uses Magistrala Thing secret for authentication. The Thing secret is a secret key that's generated at the Thing creation. In order to authenticate, the Thing needs to send its secret with the message. The way the secret is passed depends on the protocol used to send a message and differs from adapter to adapter. For more details on how this secret is passed around, please check out messaging section. This is the default Magistrala authentication mechanism and this method is used if the composition is started using the following command:
+docker-compose -f docker/docker-compose.yml up
+In most of the cases, HTTPS, WSS, MQTTS or secure CoAP are secure enough. However, sometimes you might need an even more secure connection. Magistrala supports mutual TLS authentication (mTLS) based on X.509 certificates. By default, the TLS protocol only proves the identity of the server to the client using the X.509 certificate and the authentication of the client to the server is left to the application layer. TLS also offers client-to-server authentication using client-side X.509 authentication. This is called two-way or mutual authentication. Magistrala currently supports mTLS over HTTP, WS, MQTT and MQTT over WS protocols. In order to run Docker composition with mTLS turned on, you can execute the following command from the project root:
+AUTH=x509 docker-compose -f docker/docker-compose.yml up -d
+Mutual authentication includes client-side certificates. Certificates can be generated using the simple script provided here. In order to create a valid certificate, you need to create Magistrala thing using the process described in the provisioning section. After that, you need to fetch created thing secret. Thing secret will be used to create x.509 certificate for the corresponding thing. To create a certificate, execute the following commands:
+cd docker/ssl
+make ca CN=<common_name> O=<organization> OU=<organizational_unit> emailAddress=<email_address>
+make server_cert CN=<common_name> O=<organization> OU=<organizational_unit> emailAddress=<email_address>
+make thing_cert THING_SECRET=<thing_secret> CRT_FILE_NAME=<cert_name> O=<organization> OU=<organizational_unit> emailAddress=<email_address>
+These commands use OpenSSL tool, so please make sure that you have it installed and set up before running these commands. The default values for Makefile variables are
+CRT_LOCATION = certs
+THING_SECRET = d7cc2964-a48b-4a6e-871a-08da28e7883d
+O = Magistrala
+OU = magistrala
+EA = info@magistrala.com
+CN = localhost
+CRT_FILE_NAME = thing
+Normally, in order to get things running, you will need to specify only THING_SECRET. The other variables are not mandatory and the termination should work with the default values.
make ca will generate a self-signed certificate that will later be used as a CA to sign other generated certificates. CA will expire in 3 years.make server_cert will generate and sign (with previously created CA) server cert, which will expire after 1000 days. This cert is used as a Magistrala server-side certificate in usual TLS flow to establish HTTPS or MQTTS connection.make thing_cert will finally generate and sign a client-side certificate and private key for the thing.In this example <thing_secret> represents secret of the thing and <cert_name> represents the name of the certificate and key file which will be saved in docker/ssl/certs directory. Generated Certificate will expire after 2 years. The key must be stored in the x.509 certificate CN field. This script is created for testing purposes and is not meant to be used in production. We strongly recommend avoiding self-signed certificates and using a certificate management tool such as Vault for the production.
Once you have created CA and server-side cert, you can spin the composition using:
+AUTH=x509 docker-compose -f docker/docker-compose.yml up -d
+Then, you can create user and provision things and channels. Now, in order to send a message from the specific thing to the channel, you need to connect thing to the channel and generate corresponding client certificate using aforementioned commands. To publish a message to the channel, thing should send following request:
+const WebSocket = require("ws");
+// Do not verify self-signed certificates if you are using one.
+process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";
+// Replace <channel_id> and <thing_secret> with real values.
+const ws = new WebSocket(
+ "wss://localhost/ws/channels/<channel_id>/messages?authorization=<thing_secret>",
+ // This is ClientOptions object that contains client cert and client key in the form of string. You can easily load these strings from cert and key files.
+ {
+ cert: `-----BEGIN CERTIFICATE-----....`,
+ key: `-----BEGIN RSA PRIVATE KEY-----.....`,
+ }
+);
+ws.on("open", () => {
+ ws.send("something");
+});
+ws.on("message", (data) => {
+ console.log(data);
+});
+ws.on("error", (e) => {
+ console.log(e);
+});
+As you can see, Authorization header does not have to be present in the HTTP request, since the secret is present in the certificate. However, if you pass Authorization header, it must be the same as the key in the cert. In the case of MQTTS, password filed in CONNECT message must match the key from the certificate. In the case of WSS, Authorization header or authorization query parameter must match cert key.
curl -s -S -i --cacert docker/ssl/certs/ca.crt --cert docker/ssl/certs/<thing_cert_name>.crt --key docker/ssl/certs/<thing_cert_key>.key -X POST -H "Content-Type: application/senml+json" https://localhost/http/channels/<channel_id>/messages -d '[{"bn":"some-base-name:","bt":1.276020076001e+09, "bu":"A","bver":5, "n":"voltage","u":"V","v":120.1}, {"n":"current","t":-5,"v":1.2}, {"n":"current","t":-4,"v":1.3}]'
+mosquitto_pub -u <thing_id> -P <thing_secret> -t channels/<channel_id>/messages -h localhost -p 8883 --cafile docker/ssl/certs/ca.crt --cert docker/ssl/certs/<thing_cert_name>.crt --key docker/ssl/certs/<thing_cert_key>.key -m '[{"bn":"some-base-name:","bt":1.276020076001e+09, "bu":"A","bver":5, "n":"voltage","u":"V","v":120.1}, {"n":"current","t":-5,"v":1.2}, {"n":"current","t":-4,"v":1.3}]'
+mosquitto_sub -u <thing_id> -P <thing_secret> --cafile docker/ssl/certs/ca.crt --cert docker/ssl/certs/<thing_cert_name>.crt --key docker/ssl/certs/<thing_cert_key>.key -t channels/<channel_id>/messages -h localhost -p 8883
+Magistrala allows for fine-grained control over user permissions, taking into account hierarchical relationships between entities domains, groups, channels, and things. The structure and functionality of an authorization system implemented using SpiceDB and its associated schema language. auth service backed by SpiceDB manages permissions for users, domains, groups, channels, and things.
Domain contains Things, Channels, and Groups. A User can be a member of a domain with different types of available relations. This relation provides access control to the entities in the domain.
+In Magistrala, things, channels, and groups are inherently associated with one particular domain. This means that every group, including its sub-groups, every thing, and every channel is owned by and belongs to a specific domain. Domain acts like a kind of namespace.
+graph TD
+ Do[Domain]
+ Gr["Groups + Sub Groups"]
+ Th[Things]
+ Ch[Channels]
+
+ Do --->|owns| Gr
+ Do --->|owns| Ch
+ Do --->|owns| Th
+Entities within domains have relationships with other entities in hierarchical structure.
+graph
+ subgraph Domain
+ direction BT
+ Gr[Group]
+ Th[Thing]
+ SG["Group (Sub Group)"]
+ Ch[Channel]
+
+ Th --->|connects| Ch
+ Ch --->|parent| SG
+ Ch --->|parent| Gr
+ SG --->|parent| Gr
+ end
+style Domain stroke-width:3px,margin-top:10px,margin-bottom:10pxDomain holds entities such as groups, channels, and things.
+The entities created in a domain don't have any hierarchical structure between them.
Example: In domain_1 a user creates the following entities group_1, group_2, thing_1, thing_2, channel_1, channel_2. By default, there is no relation between the entities, until the user assigns a relation between the entities
graph
+ subgraph domain_1
+ direction TB
+ Gr1["group_1"]
+ Gr2["group_2"]
+
+ Th1["thing_1"]
+ Th2["thing_2"]
+
+
+ Ch1["channel_1"]
+ Ch2["channel_2"]
+
+ endThing represents a device (or an application) connected to Magistrala that uses the platform for message exchange with other things.
Channel is a message conduit between things connected to it. It serves as a message topic that can be consumed by all of the things connected to it.
+Things can publish or subscribe to the channel.
Thing and channel can be connected to multiple channels using the following API.
+curl -sSiX POST http://localhost/connect -H "Content-Type: application/json" -H "Authorization: Bearer <domain_user_access_token>" -d @- << EOF
+{
+ "thing_id": "<thing_id>",
+ "channel_id": "<channel_id>"
+}
+EOF
+The below diagram shows thing_1 is connected to channel_1 and channel_2 , then thing_2 is connected to channel_2. This relationship can be established using the provided request
graph
+ subgraph domain_1
+ direction BT
+ Gr1["group_1"]
+ Gr2["group_2"]
+
+ Th1["thing_1"]
+ Th2["thing_2"]
+
+ Ch1["channel_1"]
+ Ch2["channel_2"]
+
+
+ Th1 --->|connect| Ch1
+ Th1 --->|connect| Ch2
+
+ Th2 --->|connect| Ch2
+ endA group serves as a parent entity that can contain both groups and channels as children. Child groups, in turn, can consist of further child groups or channels, forming a nested hierarchy. Notably, channels, which are distinct entities, cannot have child channels but can connect to multiple things. The concept of parentage signifies the relationship between higher-level entities and their subordinate components. Ancestors in this system refer to entities higher up in the hierarchy, and while a child group can have multiple ancestors, a channel can only belong to a single parent group. This hierarchical arrangement provides a structured and organized framework for managing information within the Magistrala.
+Assigning a group as the parent of a channel can be achieved through the following request.
+curl -sSiX POST 'http://localhost/channels/<channel_id>/groups/assign' -H "Content-Type: application/json" -H "Authorization: Bearer <domain_user_access_token>" -d @- << EOF
+{
+ "group_ids" : [ "<group_id_1>", "<group_id_2>" ]
+}
+EOF
+The diagram below illustrates the parent relationship between channel_1 and channel_2 with group_2. This relationship can be established using the provided request.
graph
+ subgraph domain_1
+ direction BT
+ Gr1["group_1"]
+ Gr2["group_2"]
+
+ Th1["thing_1"]
+ Th2["thing_2"]
+
+ Ch1["channel_1"]
+ Ch2["channel_2"]
+
+
+ Th1 --->|connect| Ch1
+ Th1 --->|connect| Ch2
+
+ Th2 --->|connect| Ch2
+
+ Ch1 --->|parent| Gr2
+ Ch2 --->|parent| Gr2
+ endGroups can establish a parent-child relationship with other groups. The children groups are sub-group and they can also have children groups in nested fashion
+Assigning a group as the parent to another group can be achieved through the following request.
+curl -sSiX POST 'http://localhost/groups/<parent_group_id>/groups/assign' -H "Content-Type: application/json" -H "Authorization: Bearer <domain_user_access_token>" -d @- << EOF
+{
+ "group_ids": ["<child_group_id_1>","<child_group_id_2>"]
+}
+EOF
+The diagram below illustrates the parent relationship between group_1 and group_2. This relationship can be established using the provided request.
graph
+ subgraph domain_1
+ direction BT
+ Gr1["group_1"]
+ Gr2["group_2"]
+
+ Th1["thing_1"]
+ Th2["thing_2"]
+
+ Ch1["channel_1"]
+ Ch2["channel_2"]
+
+
+ Th1 --->|connect| Ch1
+ Th1 --->|connect| Ch2
+
+ Th2 --->|connect| Ch2
+
+ Ch1 --->|parent| Gr2
+ Ch2 --->|parent| Gr2
+
+ Gr2 --->|parent| Gr1
+ endAn example group with channels, things, and groups (sub-groups) within the domain.
+Groups have parent-child relationships, forming a hierarchy where top-level groups (group_1 and group_2) have groups (sub-groups - group_11, group_12, group_21, and group_22) or channels (channel_2) beneath them.
graph
+ subgraph domain_1
+ direction BT
+ Gr1["group_1"]
+ Gr2["group_2"]
+
+ Gr11["group_11 (Sub Group)"]
+ Gr12["group_12 (Sub Group)"]
+
+ Gr21["group_21 (Sub Group)"]
+ Gr22["group_22 (Sub Group)"]
+
+ Th1["thing_1"]
+ Th2["thing_2"]
+ Th3["thing_3"]
+ Th4["thing_4"]
+ Th5["thing_5"]
+ Th6["thing_6"]
+
+
+
+ Ch1["channel_1"]
+ Ch2["channel_2"]
+ Ch3["channel_3"]
+ Ch4["channel_4"]
+
+ Gr11 --->|parent| Gr1
+ Gr12 --->|parent| Gr1
+
+ Ch1 --->|parent| Gr11
+ Th1 --->|connects| Ch1
+ Th5 --->|connects| Ch1
+
+ Ch2 --->|parent| Gr1
+ Th2 --->|connects| Ch2
+
+ Gr21 --->|parent| Gr2
+ Ch3 --->|parent| Gr21
+ Th3 --->|connects| Ch3
+
+ Gr22 --->|parent| Gr21
+ Ch4 --->|parent| Gr22
+ Th4 --->|connects| Ch4
+ Th6 --->|connects| Ch4
+ endAnother example
+graph
+ subgraph domain_1
+ direction BT
+
+ Gr1["group_1"]
+
+ Gr11["group_11 (Sub Group)"]
+ Gr12["group_12 (Sub Group)"]
+ Gr13["group_13 (Sub Group)"]
+
+
+ Th1["thing_1"]
+ Th2["thing_2"]
+ Th3["thing_3"]
+ Th4["thing_4"]
+ Th11["thing_11"]
+ Th22["thing_22"]
+ Th33["thing_33"]
+
+ Ch1["channel_1"]
+ Ch2["channel_2"]
+ Ch3["channel_3"]
+ Ch4["channel_4"]
+
+ Gr11 --->|parent| Gr1
+ Ch4 --->|parent| Gr1
+ Gr12 --->|parent| Gr11
+ Gr13 --->|parent| Gr12
+
+ Ch1 --->|parent| Gr11
+ Ch2 --->|parent| Gr12
+ Ch3 --->|parent| Gr13
+
+
+ Th1 --->|connects| Ch1
+ Th11 --->|connects| Ch1
+
+ Th2 --->|connects| Ch2
+ Th22 --->|connects| Ch2
+ Th3 --->|connects| Ch3
+ Th33 --->|connects| Ch3
+ Th4 --->|connects| Ch4
+
+ endIn Magistrala, when a new user registers, they don't automatically have access to domains. +The domain administrator must invite the user to the domain and assign them a role, such as administrator, editor, viewer, or member.
+Domain administrator can invite an existing user in Magistrala or invite new users to the domain by e-mail ID. +After the user registers with Magistrala, the user can accept the invitations to the domain.
+All the users in Magistrala are allowed to create a new domain. +The user who creates a domain automatically becomes the domain administrator.
+Users can have any one of the following relations with a domain
+Let's take the below domain_1 with entities for explaining about user domain relationship.
+ +Users with administrator relations have full control over all entities (things, channels, groups) within the domain. They can perform actions like creating, updating, and deleting entities created by others. Administrators are also allowed to create their own entities and can view and update the ones they have created.
+Example: +user_1 is administrator of domain_1. user_1 can view all entities created by others and have administrator access to all entities in the domain.
+ +Users with editor relations have access to update all entities (things, channels, groups) created by others within the domain. Editor are also allowed to create their own entities and can view and update the ones they have created.
+Example: +user_2 is editor of domain_1. user_2 can view all entities and have edit access to groups and channel entities, view access to thing entities in the domain, and also able to create & manage new things, channels & groups.
+ +Users with viewer relations have access to view all entities (things, channels, groups) created by others within the domain. Viewer are also allowed to create their own entities and can view and update the ones they have created.
+Example: +user_3 is viewer of domain_1. user_3 can only view entities that are created by others in the domain and also able to create & manage new things, channels & groups
+ +Users with member relations could not view and no access to entities (things, channels, groups) created by others within the domain. Members are also allowed to create their own entities and can view and update the ones they have created. +Domain members will not have access by default to any of the entities in the Domain, access shall be granted for specific entities by the domain administrator or individual entity administrator.
+Example: +user_4 , user_5, user_6, user_7, user_8, user_9 is member of domain_1. These member relation users can able to create & manage new things, channels & groups in the domain. They can have access to the entities to which they have a relation in the domain. They could not view and manage other entities to which they don't have any relation in domain.
+Note: All other users having administrator, editor, viewer relation with domain will also have member relation inherit with domain, which allows them to create new things, channels & groups.
+After the user sign-up to Magistrala, the user is allowed to create a new domain or join an existing domain via invitations, without domain user could not create things, channels, groups.
+All operations, including creating, updating, and deleting things, channels, and groups, occur at the domain level. For instance, when a user creates a new thing using an access token, the newly created thing automatically becomes associated with a specific domain. The domain information is extracted from the access token. When the user obtains a token, the user should specify the domain for which they want to operate.
+So to do operations on a domain, an access token for the domain is required. This can be obtained in two ways which is explained in next section.
+JWT token are used in Magistrala for authentication and authorization. The JWT token has domain, exp, iat, iss, sub, type, and user fields.
+Example JWT Token:
+{
+ "domain": "",
+ "exp": 1706544967,
+ "iat": 1706541367,
+ "iss": "magistrala.auth",
+ "sub": "",
+ "type": 0,
+ "user": "266d00f8-2284-4613-b732-3bd16e7cf8f2"
+}
+In JWT token, the domain field has domain ID and the user field has user ID.
+If the domain field is empty, then with that JWT token following actions are permitted
+Actions related to the creation, update, and deletion of things, channels, and groups are not permitted, requests will fail in authorization. Magistrala operations related to things, channels, and groups take place in domain level. So for these kinds of operations, a JWT token with a domain field containing the operating domain ID is required.
+There are two ways to obtain JWT token for a particular domain
+Request:
+curl -sSiX POST 'http://localhost/users/tokens/issue' -H "Content-Type: application/json" -H "Authorization: Bearer <domain_user_access_token>" -d @- << EOF
+{
+ "identity": "user1@example.com",
+ "secret": "12345678",
+ "domain_id": "903f7ede-3308-4206-89c2-e99688b612f7"
+}
+EOF
+In this request, if the domain ID is empty or if the field is not added, then in response JWT token will have an empty domain field.
+Response:
+{
+ "access_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJkb21haW4iOiI5MDNmN2VkZS0zMzA4LTQyMDYtODljMi1lOTk2ODhiNjEyZjciLCJleHAiOjE3MDY2MDM0NDcsImlhdCI6MTcwNjU5OTg0NywiaXNzIjoibWFnaXN0cmFsYS5hdXRoIiwic3ViIjoiOTAzZjdlZGUtMzMwOC00MjA2LTg5YzItZTk5Njg4YjYxMmY3XzU3NDhkZTY5LTJhNjYtNDBkYS1hODI5LTFiNDdmMDJlOWFkYiIsInR5cGUiOjAsInVzZXIiOiI1NzQ4ZGU2OS0yYTY2LTQwZGEtYTgyOS0xYjQ3ZjAyZTlhZGIifQ.c8a8HhVAbkdq_qZnd1DWHtkoNDPQc6XJY6-UcqqCygRR9svjgkwetN3rmIOWPNV5CjPh5lqlzWv1cOLruKBmzw",
+ "refresh_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJkb21haW4iOiI5MDNmN2VkZS0zMzA4LTQyMDYtODljMi1lOTk2ODhiNjEyZjciLCJleHAiOjE3MDY2ODYyNDcsImlhdCI6MTcwNjU5OTg0NywiaXNzIjoibWFnaXN0cmFsYS5hdXRoIiwic3ViIjoiOTAzZjdlZGUtMzMwOC00MjA2LTg5YzItZTk5Njg4YjYxMmY3XzU3NDhkZTY5LTJhNjYtNDBkYS1hODI5LTFiNDdmMDJlOWFkYiIsInR5cGUiOjEsInVzZXIiOiI1NzQ4ZGU2OS0yYTY2LTQwZGEtYTgyOS0xYjQ3ZjAyZTlhZGIifQ.SEMvEw2hchsvPYJWqnHMKlUmgjfqAvFcjeCDXyvS2xSGsscEci3bMrUohaJNkNDWzTBiBinV7nEcPrwbxDfPBQ"
+}
+In most of the cases user login domain is under determinable. This method will be useful for those kind of cases.
+Step 1: Get token without domain ID +Request:
+curl -sSiX POST 'http://localhost/users/tokens/issue' -H "Content-Type: application/json" -H "Authorization: Bearer <domain_user_access_token>" -d @- << EOF
+{
+ "identity": "user1@example.com",
+ "secret": "12345678"
+}
+EOF
+Response:
+{
+ "access_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJkb21haW4iOiIiLCJleHAiOjE3MDY2MDM1MjYsImlhdCI6MTcwNjU5OTkyNiwiaXNzIjoibWFnaXN0cmFsYS5hdXRoIiwic3ViIjoiIiwidHlwZSI6MCwidXNlciI6IjU3NDhkZTY5LTJhNjYtNDBkYS1hODI5LTFiNDdmMDJlOWFkYiJ9.Cc2Qj_z3gcUTjDo7lpcUVx9ymnUfClwt28kayHvMhY27eDu1vWMAZb_twQ85pbSlf12juo8P_YJcWKl3rDEokQ",
+ "refresh_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJkb21haW4iOiIiLCJleHAiOjE3MDY2ODYzMjYsImlhdCI6MTcwNjU5OTkyNiwiaXNzIjoibWFnaXN0cmFsYS5hdXRoIiwic3ViIjoiIiwidHlwZSI6MSwidXNlciI6IjU3NDhkZTY5LTJhNjYtNDBkYS1hODI5LTFiNDdmMDJlOWFkYiJ9.SiVsctItdR0WFhRbg7omZgR_WDPlLfLF2ov2eqkE1EP8c7RruOEv-KST3xVsohY33t2xevrtorwbjMQsl1YV7Q"
+}
+Decoded Access Token:
+{
+ "domain": "",
+ "exp": 1706603526,
+ "iat": 1706599926,
+ "iss": "magistrala.auth",
+ "sub": "",
+ "type": 0,
+ "user": "5748de69-2a66-40da-a829-1b47f02e9adb"
+}
+Decoded Refresh Token:
+{
+ "domain": "",
+ "exp": 1706686326,
+ "iat": 1706599926,
+ "iss": "magistrala.auth",
+ "sub": "",
+ "type": 1,
+ "user": "5748de69-2a66-40da-a829-1b47f02e9adb"
+}
+In these tokens, the domain field will be empty. As said earlier, this token can be to for user profile update, domain creation & listing, accept domain invitations
+Step 2: List domains users have access +Request:
+curl -sSiX GET 'http://localhost/domains' -H "Authorization: Bearer <ACCESS_TOKEN_FROM_STEP_1>"
+Response:
+{
+ "total": 1,
+ "offset": 0,
+ "limit": 10,
+ "status": "all",
+ "domains": [
+ {
+ "id": "903f7ede-3308-4206-89c2-e99688b612f7",
+ "name": "Domain 1",
+ "alias": "domain_1",
+ "status": "enabled",
+ "permission": "administrator",
+ "created_by": "5748de69-2a66-40da-a829-1b47f02e9adb",
+ "created_at": "2024-01-30T07:30:36.89495Z",
+ "updated_at": "0001-01-01T00:00:00Z"
+ }
+ ]
+}
+Step 3: Send Request to Refresh endpoint with domain id +Request:
+curl -sSiX POST 'http://localhost/users/tokens/refresh' -H "Content-Type: application/json" -H "Authorization: Bearer <REFRESH_TOKEN_FROM_STEP_1>" -d @- << EOF
+{
+ "domain_id": "903f7ede-3308-4206-89c2-e99688b612f7"
+}
+EOF
+Note: Same request also used for switching between domains.
+Response:
+{
+ "access_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJkb21haW4iOiI5MDNmN2VkZS0zMzA4LTQyMDYtODljMi1lOTk2ODhiNjEyZjciLCJleHAiOjE3MDY2MDM3MDYsImlhdCI6MTcwNjYwMDEwNiwiaXNzIjoibWFnaXN0cmFsYS5hdXRoIiwic3ViIjoiOTAzZjdlZGUtMzMwOC00MjA2LTg5YzItZTk5Njg4YjYxMmY3XzU3NDhkZTY5LTJhNjYtNDBkYS1hODI5LTFiNDdmMDJlOWFkYiIsInR5cGUiOjAsInVzZXIiOiI1NzQ4ZGU2OS0yYTY2LTQwZGEtYTgyOS0xYjQ3ZjAyZTlhZGIifQ.3_q4F9CWxmBVjItiE8uR01vlm0du_ISl75E-nfEcc-3IMqHEOLbz1WrDvGbaYcPZ-CQufZuP2j-zqR4lShnu2Q",
+ "refresh_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJkb21haW4iOiI5MDNmN2VkZS0zMzA4LTQyMDYtODljMi1lOTk2ODhiNjEyZjciLCJleHAiOjE3MDY2ODY1MDYsImlhdCI6MTcwNjYwMDEwNiwiaXNzIjoibWFnaXN0cmFsYS5hdXRoIiwic3ViIjoiOTAzZjdlZGUtMzMwOC00MjA2LTg5YzItZTk5Njg4YjYxMmY3XzU3NDhkZTY5LTJhNjYtNDBkYS1hODI5LTFiNDdmMDJlOWFkYiIsInR5cGUiOjEsInVzZXIiOiI1NzQ4ZGU2OS0yYTY2LTQwZGEtYTgyOS0xYjQ3ZjAyZTlhZGIifQ.KFUEGEx0LZStpokGnQHoMbpPRA5RUH7AR5RHRC46KcBIUoD4EcuWBiSreFwyRc4v-tcbp-CQQaBNGhqYMW4QZw"
+}
+Decoded Access Token:
+{
+ "domain": "903f7ede-3308-4206-89c2-e99688b612f7",
+ "exp": 1706603706,
+ "iat": 1706600106,
+ "iss": "magistrala.auth",
+ "sub": "903f7ede-3308-4206-89c2-e99688b612f7_5748de69-2a66-40da-a829-1b47f02e9adb",
+ "type": 0,
+ "user": "5748de69-2a66-40da-a829-1b47f02e9adb"
+}
+Decoded Refresh Token:
+{
+ "domain": "903f7ede-3308-4206-89c2-e99688b612f7",
+ "exp": 1706686506,
+ "iat": 1706600106,
+ "iss": "magistrala.auth",
+ "sub": "903f7ede-3308-4206-89c2-e99688b612f7_5748de69-2a66-40da-a829-1b47f02e9adb",
+ "type": 1,
+ "user": "5748de69-2a66-40da-a829-1b47f02e9adb"
+}
+Domain creator becomes administrator of the domain by default. Domain administrator can assign users to a domain with the following relations administrator, editor, viewer, member. The details about these relations are described in this section
+User can be assigned to domain with endpoint /domain/<domain_id>/users/assign with JSON body like below:
{
+ "user_ids" : ["<user_id>"],
+ "relation" : "editor"
+}
+Example Request:
+curl -sSiX POST 'http://localhost/domains/903f7ede-3308-4206-89c2-e99688b612f7/users/assign' -H "Content-Type: application/json" -H "Authorization: Bearer <DOMAIN_ACCESS_TOKEN_>" -d @- << EOF
+{
+ "user_ids" : ["05dbd66a-ce38-4928-ac86-c1b44909be0d"],
+ "relation" : "editor"
+}
+EOF
+User can be unassigned to domain with endpoint /domain/<domain_id>/users/unassign with JSON body like below:
{
+ "user_ids" : ["<user_id>"],
+ "relation" : "editor"
+}
+Example request:
+curl -sSiX POST 'http://localhost/domains/903f7ede-3308-4206-89c2-e99688b612f7/users/unassign' -H "Content-Type: application/json" -H "Authorization: Bearer <DOMAIN_ACCESS_TOKEN_>" -d @- << EOF
+{
+ "user_ids" : ["05dbd66a-ce38-4928-ac86-c1b44909be0d"],
+ "relation" : "administrator"
+}
+EOF
+Users assigned to a domain with any relationship (administrator, editor, viewer, member ) will have access to create entities (things, groups, channels).
+Domain administrator or individual entity administrator shall grant access to domain member for specific entities.
+Like domains, groups also have four types of relations
+Group administrator users have access to update, delete, assign, and unassign to the group and also have access to update, delete, assign, and unassign all of its child entities
+From the previous viewer example, let's take user_3 who has viewer relation with domain_1, which means user_3 will be able to view all the entities created by others but cannot make any edits or updates on them. user_3 will have access to create entities in domain_1
+user_3 creates new thing_101, channel_101, and group_101.
+user_3 request to create thing_101:
+curl -sSiX POST 'http://localhost/things' -H "Content-Type: application/json" -H "Authorization: Bearer <DOMAIN_ACCESS_TOKEN_>" -d @- << EOF
+{
+ "credentials": {
+ "secret": "a1ca33c0-367e-402b-a239-ff1255fdc440"
+ },
+ "name": "thing_101"
+}
+EOF
+user_3 request to create channel_101:
+curl -sSiX POST 'http://localhost/channels' -H "Content-Type: application/json" -H "Authorization: Bearer <DOMAIN_ACCESS_TOKEN_>" -d @- << EOF
+{
+ "name": "chanel_101"
+}
+EOF
+user_3 request to create group_101:
+curl -sSiX POST 'http://localhost/groups' -H "Content-Type: application/json" -H "Authorization: Bearer <DOMAIN_ACCESS_TOKEN_>" -d @- << EOF
+{
+ "name": "group_101"
+}
+EOF
+The user who creates the entity will be the administrator of the entity by default. +So user_3 is administrator of thing_101, channel_101 and group_101.
+ +user_3 will also have domain viewer relation to thing_101, channel_101, and group_101
+user_3 can make these entities (thing_101, channel_101, group_101) in a hierarchical structure by assigning relations between entities +Example: Connect thing_101 & channel_101, assign group_101 as parent of channel_101.
+user_3 request for connect thing_101 & channel_101:
+curl -sSiX POST 'http://localhost/connect' -H "Content-Type: application/json" -H "Authorization: Bearer <DOMAIN_ACCESS_TOKEN_>" -d @- << EOF
+{
+ "thing_id": "<thing_101_id>",
+ "channel_id": "<channel_101_id>"
+}
+EOF
+user_3 request for assign group_101 as parent of channel_101:
+curl -sSiX POST 'http://localhost/channels/<channel_101_id>/groups/assign' -H "Content-Type: application/json" -H "Authorization: Bearer <DOMAIN_ACCESS_TOKEN_>" -d @- << EOF
+{
+ "group_ids" : [ "<group_101_id>" ]
+}
+EOF
+Members of domain 1 will not have access by default to any of the entities in domain 1, access shall be granted for specific entities by domain administrator or individual entity administrator.
+Administrator of group_101 (user_3), assigns user_4 with administrator relation. +When domain member user_4 becomes an administrator of group_101, user_4 can able to update, delete, assign, and unassign to group_101. Since group_101 has channel_101 and thing_101 as children. The user_5 has administrator access on group_101 child entities channel_101 and thing_101.
+user_3 request for assign user_4 as administrator for group_101:
+curl -sSiX POST 'http://localhost/domains/<DOMINA_1_ID>/users/assign' -H "Content-Type: application/json" -H "Authorization: Bearer <DOMAIN_ACCESS_TOKEN_>" -d @- << EOF
+{
+ "user_ids" : ["<user_4 ID>"],
+ "relation" : "administrator"
+}
+Group editor users have access to view, update, assign, and unassign to the group and also have access to view, update, assign, and unassign all of its child channel and group entities, group editor have only view access to child thing entities in group
+Administrator of group_101 (user_3/user_4), assigns user_5 with editor relation. +When domain member user_5 becomes an editor of group_101, user_5 can able to update, assign, and unassign to group_101. Since group_101 has channel_101 and thing_101 as children. The user_5 has editor access to the group child entities channels, things, and groups. In this case, user_5 has editor access to group_101, and also has edit access to its child entities channel_101 and thing_101
+ +Group viewer users have access to view group and also have access to view all of its child entities
+When domain member user_6 becomes a viewer of group_101, user_6 can able to view all the child and nested child entities in group_101. user_6 can assign child entities under group_101 and also assign child entities under any other group and channels that are children of group_101.
+ +user_6 creates new channel and thing with the names channel_201 and thing_201 respectively. Then connects both channel_201 and thing_201.
+ +Now user_5 can able to assign group_101 as a parent for channel_201
+ +When channel_201 was assigned as a child of group_101, all the administrators, editors, and viewers of group_101 got the same access (relation) to channel_201 and thing_201
+ +user_8 creates a new group with the name group_301 +user_9 creates a new thing and channel with the names thing_301 and channel_301 respectively, then connects both thing and channel. +
+user_8 can able to assign channel_301 as a child of group_301 +
+When channel_301 is assigned as a child of group_301, then the administrators, editors, and viewers of group_301 get the same respective access to channel_301. +The administrator, editor, and viewer of channel_301 get the same respective access to thing_301. +So here user_8 becomes the administrator of both channel_301 and thing_301 +user_5 can able to assign group_301 as a child of group_101 +
+When group_301 becomes a child of group_101, then the administrator, editor, and viewer of group_101 get the same respective access to group_301. +The administrator, editor, and viewer of group_301 get the same respective access to channel_301. +The administrator, editor, and viewer of channel_301 get the same respective access to thing_301. +So here user_5 becomes the editor of group_301, channel_301, and thing_301, user_4 becomes administrator of group_301, channel_301, and thing_301. +user_8 has administrator access only to group_301 and its child entities channel_301 and thing_301. +
+There are two ways to user get registered to Magistrala, self-register and register new users by super admin. +User registration is self register default which can be changed by following environment variable:
+MG_USERS_ALLOW_SELF_REGISTER=true
+MZbench is open-source tool for that can generate large traffic and measure performance of the application. MZBench is distributed, cloud-aware benchmarking tool that can seamlessly scale to millions of requests. It's originally developed by satori-com but we will use mzbench fork because it can run with newest Erlang releases and the original MzBench repository is not maintained anymore.
+We will describe installing MZBench server on Ubuntu 18.04 (this can be on your PC or some external cloud server, like droplet on Digital Ocean)
+Install latest OTP/Erlang (it's version 22.3 for me)
+sudo apt update
+sudo apt install erlang
+For running this tool you will also need libz-dev package:
+sudo apt-get update
+sudo apt-get install libz-dev
+and pip:
+sudo apt install python-pip
+Clone mzbench tool and install the requirements:
+git clone https://github.com/mzbench/mzbench
+cd mzbench
+sudo pip install -r requirements.txt
+This should be enough for installing MZBench, and you can now start MZBench server with this CLI command:
+./bin/mzbench start_server
+The MZBench CLI lets you control the server and benchmarks from the command line.
+Another way of using MZBench is over Dashboard. After starting server you should check dashboard on http://localhost:4800.
Note that if you are installing MZBench on external server (i.e. Digital Ocean droplet), that you'll be able to reach MZBench dashboard on your server's IP address:4800, if you previously:
+network_interface from 127.0.0.1 to 0.0.0.0 in configuration file. Default configuration file location is ~/.config/mzbench/server.config, create it from sample configuration file ~/.config/mzbench/server.config.example4800 with ufw allow 4800MZBench can run your test scenarios on many nodes, simultaneously. For now, you are able to run tests locally, so your nodes will be virtual nodes on machine where MZBench server is installed (your PC or DO droplet). You can try one of our MQTT scenarios that uses vmq_mzbench worker. Copy-paste scenario in MZBench dashboard, click button Environmental variables -> Add from script and add appropriate values. Because it's running locally, you should try with smaller values, for example for fan-in scenario use 100 publishers on 2 nodes. Try this before moving forward in setting up Amazon EC2 plugin.
+For larger-scale tests we will set up MZBench to run each node as one of Amazon EC2 instance with built-in plugin mzb_api_ec2_plugin.
+This is basic architecture when running MZBench:
+
Every node that runs your scenarios will be one of Amazon EC2 instance; plus one more additional node — the director node. The director doesn't run scenarios, it collects the metrics from the other nodes and runs post and pre hooks. So, if you want to run jobs on 10 nodes, actually 11 EC2 instances will be created. All instances will be automatically terminated when the test finishes.
+We will use one of ready-to-use Amazon Machine Images (AMI) with all necessary dependencies. We will choose AMI with OTP 22, because that is the version we have on MZBench server. So, we will search for MZBench-erl22 AMI and find one with id ami-03a169923be706764 available in us-west-1b zone. If you have chosen this AMI, everything you do from now must be in us-west-1 zone. We must have IAM user with AmazonEC2FullAccess and IAMFullAccess permissions policies, and his access_key_id and secret_access_key goes to configuration file. In EC2 dashboard, you must create new security group MZbench_cluster where you will add inbound rules to open ssh and TCP ports 4801-4804. Also, in EC2 dashboard go to section key pairs, click Actions -> Import key pair and upload public key you have on your MZBench server in ~/.ssh/id_rsa.pub (if you need to create new, run ssh-keygen and follow instructions). Give it a name on EC2 dashboard, put that name (key_name) and path (keyfile) in configuration file.
[
+{mzbench_api, [
+{network_interface,"0.0.0.0"},
+{keyfile, "~/.ssh/id_rsa"},
+{cloud_plugins, [
+ {local,#{module => mzb_dummycloud_plugin}},
+ {ec2, #{module => mzb_api_ec2_plugin,
+ instance_spec => [
+ {image_id, "ami-03a169923be706764"},
+ {group_set, ["MZbench_cluster"]},
+ {instance_type, "t2.micro"},
+ {availability_zone, "us-west-1b"},
+ {iam_instance_profile_name, "mzbench"},
+ {key_name, "key_pair_name"}
+ ],
+ config => [
+ {ec2_host, "ec2.us-west-1.amazonaws.com"},
+ {access_key_id, "IAM_USER_ACCESS_KEY_ID"},
+ {secret_access_key, "IAM_USER_SECRET_ACCESS_KEY"}
+ ],
+ instance_user => "ec2-user"
+ }}
+ ]
+}
+]}].
+There is both local and ec2 plugin in this configuration file, so you can choose to run tests on either of them. Default path for configuration file is ~/.config/mzbench/server.config, if it's somewhere else, server is starting with:
./bin/mzbench start_server --config <config_file>
+Note that every time you update the configuration you have to restart the server:
+./bin/mzbench restart_server
+Testing environment to be determined.
+In this scenario, large number of requests are sent to HTTP adapter service every second. This test checks how much time HTTP adapter needs to respond to each request.
+TBD
+In this scenario, large number of requests are sent to things service to create things and than to retrieve their data. This test checks how much time things service needs to respond to each request.
+TBD
+ + + + + + +Bootstrapping refers to a self-starting process that is supposed to proceed without external input. Magistrala platform supports bootstrapping process, but some of the preconditions need to be fulfilled in advance. The device can trigger a bootstrap when:s
++Bootstrapping and provisioning are two different procedures. Provisioning refers to entities management while bootstrapping is related to entity configuration.
+
Bootstrapping procedure is the following:
+
+1) Configure device with Bootstrap service URL, an external key and external ID
+++
Optionally create Magistrala channels if they don't exist
++
Optionally create Magistrala thing if it doesn't exist
+
+2) Upload configuration for the Magistrala thing
+3) Bootstrap - send a request for the configuration
+4) Connect/disconnect thing from channels, update or remove configuration
The configuration of Magistrala thing consists of three major parts:
+Also, the configuration contains an external ID and external key, which will be explained later. +In order to enable the thing to start bootstrapping process, the user needs to upload a valid configuration for that specific thing. This can be done using the following HTTP request:
+
curl -s -S -i -X POST -H "Authorization: Bearer <user_token>" -H "Content-Type: application/json" http://localhost:9013/things/configs -d '{
+ "external_id":"09:6:0:sb:sa",
+ "thing_id": "7d63b564-3092-4cda-b441-e65fc1f285f0",
+ "external_key":"key",
+ "name":"some",
+ "channels":[
+ "78c9b88c-b2c4-4d58-a973-725c32194fb3",
+ "c4d6edb2-4e23-49f2-b6ea-df8bc6769591"
+],
+ "content": "config...",
+ "client_cert": "PEM cert",
+ "client_key": "PEM client cert key",
+ "ca_cert": "PEM CA cert"
+}'
+In this example, channels field represents the list of Magistrala channel IDs the thing is connected to. These channels need to be provisioned before the configuration is uploaded. Field content represents custom configuration. This custom configuration contains parameters that can be used to set up the thing. It can also be empty if no additional set up is needed. Field name is human readable name and thing_id is an ID of the Magistrala thing. This field is not required. If thing_id is empty, corresponding Magistrala thing will be created implicitly and its ID will be sent as a part of Location header of the response. Fields client_cert, client_key and ca_cert represent PEM or base64-encoded DER client certificate, client certificate key and trusted CA, respectively.
There are two more fields: external_id and external_key. External ID represents an ID of the device that corresponds to the given thing. For example, this can be a MAC address or the serial number of the device. The external key represents the device key. This is the secret key that's safely stored on the device and it is used to authorize the thing during the bootstrapping process. Please note that external ID and external key and Magistrala ID and Magistrala key are completely different concepts. External id and key are only used to authenticate a device that corresponds to the specific Magistrala thing during the bootstrapping procedure. As Configuration optionally contains client certificate and issuing CA, it's possible that device is not able to establish TLS encrypted communication with Magistrala before bootstrapping. For that purpose, Bootstrap service exposes endpoint used for secure bootstrapping which can be used regardless of protocol (HTTP or HTTPS). Both device and Bootstrap service use a secret key to encrypt the content. Encryption is done as follows:
++Please have on mind that secret key is passed to the Bootstrap service as an environment variable. As security measurement, Bootstrap service removes this variable once it reads it on startup. However, depending on your deployment, this variable can still be visible as a part of your configuration or terminal emulator environment.
+
For more details on which encryption mechanisms are used, please take a look at the implementation.
+Currently, the bootstrapping procedure is executed over the HTTP protocol. Bootstrapping is nothing else but fetching and applying the configuration that corresponds to the given Magistrala thing. In order to fetch the configuration, the thing needs to send a bootstrapping request:
+curl -s -S -i -H "Authorization: Thing <external_key>" http://localhost:9013/things/bootstrap/<external_id>
+The response body should look something like:
+{
+ "thing_id":"7d63b564-3092-4cda-b441-e65fc1f285f0",
+ "thing_key":"d0f6ff22-f521-4674-9065-e265a9376a78",
+ "channels":[
+ {
+ "id":"c4d6edb2-4e23-49f2-b6ea-df8bc6769591",
+ "name":"c1",
+ "metadata":null
+ },
+ {
+ "id":"78c9b88c-b2c4-4d58-a973-725c32194fb3",
+ "name":"c0",
+ "metadata":null
+ }
+ ],
+ "content":"cofig...",
+ "client_cert":"PEM cert",
+ "client_key":"PEM client cert key",
+ "ca_cert":"PEM CA cert"
+}
+The response consists of an ID and key of the Magistrala thing, the list of channels and custom configuration (content field). The list of channels contains not just channel IDs, but the additional Magistrala channel data (name and metadata fields), as well.
Uploading configuration does not automatically connect thing to the given list of channels. In order to connect the thing to the channels, user needs to send the following HTTP request:
+curl -s -S -i -X PUT -H "Authorization: Bearer <user_token>" -H "Content-Type: application/json" http://localhost:9013/things/state/<thing_id> -d '{"state": 1}'
+In order to disconnect, the same request should be sent with the value of state set to 0.
For more information about the Bootstrap service API, please check out the API documentation.
+ + + + + + +Provisioning is a process of configuration of an IoT platform in which system operator creates and sets-up different entities used in the platform - users, groups, channels and things.
+Issues certificates for things. Certs service can create certificates to be used when Magistrala is deployed to support mTLS.
+Certs service will create certificate for valid thing ID if valid user token is passed and user is owner of the provided thing ID.
Certificate service can create certificates in PKI mode - where certificates issued by PKI, when you deploy Vault as PKI certificate management cert service will proxy requests to Vault previously checking access rights and saving info on successfully created certificate.
When MF_CERTS_VAULT_HOST is set, it is presumed that Vault is installed and certs service will issue certificates using Vault API.
First you'll need to set up Vault.
To setup Vault follow steps in Build Your Own Certificate Authority (CA).
To setup certs service with Vault following environment variables must be set:
MF_CERTS_VAULT_HOST=vault-domain.com
+MF_CERTS_VAULT_PKI_PATH=<vault_pki_path>
+MF_CERTS_VAULT_ROLE=<vault_role>
+MF_CERTS_VAULT_TOKEN=<vault_acces_token>
+For lab purposes you can use docker-compose and script for setting up PKI in meodor-vault.
+Make sure you have an already running instance of Magistrala , Vault and Certs service.
To start Magistrala run:
+make run up args="-d"
+To start vault run:
+make run_addons vault up args="-d"
+When vault service is up and running some initializations steps must be done to setup things for Certs service. For more information about this steps please check magistrala-vault
bash docker/addons/vault/vault-init.sh
+bash docker/addons/vault/vault-unseal.sh
+bash docker/addons/vault/vault-set-pki.sh
+vault-init.sh initializes Vault, generates unseal keys and root tokens, and updates corresponding environment variables in the .env file. It's important to securely store these keys as they are required to unseal Vault.
vault-unseal.sh is used to unseal Vault after initialization, but it's typically not needed since Vault can unseal itself when starting the container.
vault-set-pki.sh generates certificates for Vault, including root and intermediate certificates, and copies them to the docker/ssl/certs folder. The CA parameters are sourced from environment variables in the .env file.
To start certs service run:
+make run_addons certs up args="-d"
+Provision a thing:
+magistrala-cli provision test
+To stop certs service run:
+make run_addons certs down
+To stop vault service run:
+make run_addons vault down
+This step can be skipped if you already have a thing ID.
+magistrala-cli certs issue <thing_id> <user_auth_token> [--ttl=8760h]
+For example:
+magistrala-cli certs issue f13f0f30-f923-4504-8a7a-6aa45bcb4866 $USER_TOKEN
+
+{
+ "cert_serial": "6f:35:d5:9d:47:9d:23:50:08:f7:31:13:82:22:e4:c8:e6:cf:2c:c1",
+ "client_cert": "-----BEGIN CERTIFICATE-----\nMIIEATCCAumgAwIBAgIUbzXVnUedI1AI9zETgiLkyObPLMEwDQYJKoZIhvcNAQEL\nBQAwLjEsMCoGA1UEAxMjbWFpbmZsdXguY29tIEludGVybWVkaWF0ZSBBdXRob3Jp\ndHkwHhcNMjMwOTE0MTEwOTI5WhcNMjMxMDE0MTEwOTU4WjAvMS0wKwYDVQQDEyRi\nYTFmMmIxNi01MjA3LTQ2MDgtYTRkZS01ZmFiZmI4NjI3YzIwggEiMA0GCSqGSIb3\nDQEBAQUAA4IBDwAwggEKAoIBAQC9RxcHaTzn18vBdWWZf37K8Grc5dLW/m8vhwOJ\n8oe3iPUiE7xFijIXKw236R1NBh8fLT6/2lia/p4acZtls3yFRphooDwP7S2OiJRI\ngGb/r0SYmSnQKjHbdbixauNECGk1TDNSGvmpNSzvAZvYSJAvd5ZpYf/8Db9IBW0N\nvbI7TfIJHay8vC/0rn1BsmC3x+3nEm0W+Z5udC/UT4+pQn7QWrBsxjVT4r5WY0SQ\nkVhA9Wo+Wpzmy1CMC4X6yLmiIHmfRFlktDxKgPpyy/3zhAE2CkBpT7JEQ723Mv+m\n37oM2EJog+tgIZMExxDbw3Epqgo07B9DWpSZSBHCISeN/TzdAgMBAAGjggEUMIIB\nEDAOBgNVHQ8BAf8EBAMCA6gwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMC\nMB0GA1UdDgQWBBTAoqWVu8ctNmw5CKUBxsUKVDX+PDAfBgNVHSMEGDAWgBS7dmaT\nr5vJJPtV5dReawbYKhxzYzA7BggrBgEFBQcBAQQvMC0wKwYIKwYBBQUHMAKGH2h0\ndHA6Ly92YXVsdDo4MjAwL3YxL3BraV9pbnQvY2EwLwYDVR0RBCgwJoIkYmExZjJi\nMTYtNTIwNy00NjA4LWE0ZGUtNWZhYmZiODYyN2MyMDEGA1UdHwQqMCgwJqAkoCKG\nIGh0dHA6Ly92YXVsdDo4MjAwL3YxL3BraV9pbnQvY3JsMA0GCSqGSIb3DQEBCwUA\nA4IBAQCKMmDzyWWmuSJPh3O9hppRJ6mkX9gut4jP2rwowNv7haj3iu+hR8+GnTix\nu5oy3bZdmRryhhW0XyJsbCKO/z+wsY/RfVgMxF/c1cbmEzki804+AB4a4yNhQD6g\noEEQBD58b6mFi/vPCRiGZmmo5TqMlA37jBRSVnKO/CoH1CAvjqmfWdSoO4IC4uD4\nJev+QNr9wlOimYcA/usmo7rmqz7IB9R/Laxcdkq9iZelKly/jhftEbKgGf2NR/d7\nEKVONjCEp6fL2iBaQSA/899oJJ7QPqE5X821HhBlXKvNmZnYRyUmAS2h1jnxtovp\nsNGcLFRgIAFdaGl1172C7mBZF4C3\n-----END CERTIFICATE-----",
+ "client_key": "-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQEAvUcXB2k859fLwXVlmX9+yvBq3OXS1v5vL4cDifKHt4j1IhO8\nRYoyFysNt+kdTQYfHy0+v9pYmv6eGnGbZbN8hUaYaKA8D+0tjoiUSIBm/69EmJkp\n0Cox23W4sWrjRAhpNUwzUhr5qTUs7wGb2EiQL3eWaWH//A2/SAVtDb2yO03yCR2s\nvLwv9K59QbJgt8ft5xJtFvmebnQv1E+PqUJ+0FqwbMY1U+K+VmNEkJFYQPVqPlqc\n5stQjAuF+si5oiB5n0RZZLQ8SoD6csv984QBNgpAaU+yREO9tzL/pt+6DNhCaIPr\nYCGTBMcQ28NxKaoKNOwfQ1qUmUgRwiEnjf083QIDAQABAoIBADKd7kSnGgiOJwkn\nUfJIrCmtPYaxVz7zb9xv6LxdRXoJgDSKvpCCMn8LnnGOP623c18tBFjeFU/tw24i\n74G1DBnAFUX1g9pmfQZe8/injePWhSuh2hK3FfowcyHPCdPJxAjixd6xJA7iD5Aj\nCABA934aJvkrof9P1dV2zgEct6sv6GPwUgSZxTYVNyU93T/pmvodvpNTYd3uk71A\nLCC5Ojv2gEOkHUWHhMntz7bl6wcH/atk//uYoYxcjZ811tL7/7xwUbyRxFD/b6kP\niptdoXBv27eWWKOtFMgF9iNkhefSKkmHZZWIL1J5CFE8fUdddeLoOa0e7a9vhYS9\n5TMzC2kCgYEA+TJf60QP3rjEgm6bJw1h48ffkPkZTsdp083GoJB77yXUH7m9Wt9g\nlYSALN+67fnkXPEe/C9SInMDRMp9VoswOHeJCFbCNdx5Klv8KKuMZMk0yCZifhx6\nBl7IsVlmlzq3EhK1ZjOVWMxvwS7MlMpPAcsc8DGhwhv9sXW3k2nMevsCgYEAwnHx\nheuaYgE/HrE/GEcPNAwy/uyBb8wxoKavl8OKEyPH+LK8powo9xss8zi+yEYHfSQP\nnJ45Rdz/HGl5QIwD4CjA3Vrm0sTMh094DPp9KhxcOwIhK/IvUJ0deKwHRWek/+c8\nwbD6HfX2Vtu5RU9z2KS7VtazjU5TkIbKP29LoAcCgYAUKAv0JrQ16rISbsnj9cQm\nPYOK4Ws3oQ+hTzKyyB0OMfwfeNGlKQ5R6b7IYmxnVWAwWFyOP3GgUbdA+DP9LRMA\nbkLKRuI8oxG16GzUCVQ4zsGTMu+ijcEdBMus9LNEpj4qmxLLKn75CMg9UwC/REHx\nvjEgCJOx9LungAMSTGt6wwKBgQCXvSGUt6pvhreCNSGeyX1EyaxWIaxU2U11J/7p\neQ/cJdUc8Cal9cTWKV/nokXHtlaLwsNoHlVlfrOasXiM9XbkzAjN9O0iV6+gfFSc\nFDHu1djnt565U7K2vxVLoTu/XsV1ajeQk5JsJRCK8cbgHsOxscP8XWobAJ/XrkhQ\nPoMOqwKBgD8goECBKj+SofUfqKCnGf3E2MWF3kTZMfPaBcuV8TaGMWRRljMmK8YT\npew6IIkAFrsIaXxQsym2JQ+j/L2AoxQkzlf2VF4SaBfUUByT3NijSBpD/d3xRlWA\n7UUO0d72YFnPTqY98Ch/fbKnaCRL/Usv8c9nCt5IdmnihYnuvxYT\n-----END RSA PRIVATE KEY-----",
+ "expiration": "2023-10-14T11:09:58Z",
+ "thing_id": "f13f0f30-f923-4504-8a7a-6aa45bcb4866"
+}
+magistrala-cli certs get [<cert_serial> | thing <thing_id>] <user_auth_token>
+For example:
+magistrala-cli certs get 6f:35:d5:9d:47:9d:23:50:08:f7:31:13:82:22:e4:c8:e6:cf:2c:c1 $USER_TOKEN
+{
+ "cert_serial": "6f:35:d5:9d:47:9d:23:50:08:f7:31:13:82:22:e4:c8:e6:cf:2c:c1",
+ "client_cert": "-----BEGIN CERTIFICATE-----\nMIIEATCCAumgAwIBAgIUbzXVnUedI1AI9zETgiLkyObPLMEwDQYJKoZIhvcNAQEL\nBQAwLjEsMCoGA1UEAxMjbWFpbmZsdXguY29tIEludGVybWVkaWF0ZSBBdXRob3Jp\ndHkwHhcNMjMwOTE0MTEwOTI5WhcNMjMxMDE0MTEwOTU4WjAvMS0wKwYDVQQDEyRi\nYTFmMmIxNi01MjA3LTQ2MDgtYTRkZS01ZmFiZmI4NjI3YzIwggEiMA0GCSqGSIb3\nDQEBAQUAA4IBDwAwggEKAoIBAQC9RxcHaTzn18vBdWWZf37K8Grc5dLW/m8vhwOJ\n8oe3iPUiE7xFijIXKw236R1NBh8fLT6/2lia/p4acZtls3yFRphooDwP7S2OiJRI\ngGb/r0SYmSnQKjHbdbixauNECGk1TDNSGvmpNSzvAZvYSJAvd5ZpYf/8Db9IBW0N\nvbI7TfIJHay8vC/0rn1BsmC3x+3nEm0W+Z5udC/UT4+pQn7QWrBsxjVT4r5WY0SQ\nkVhA9Wo+Wpzmy1CMC4X6yLmiIHmfRFlktDxKgPpyy/3zhAE2CkBpT7JEQ723Mv+m\n37oM2EJog+tgIZMExxDbw3Epqgo07B9DWpSZSBHCISeN/TzdAgMBAAGjggEUMIIB\nEDAOBgNVHQ8BAf8EBAMCA6gwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMC\nMB0GA1UdDgQWBBTAoqWVu8ctNmw5CKUBxsUKVDX+PDAfBgNVHSMEGDAWgBS7dmaT\nr5vJJPtV5dReawbYKhxzYzA7BggrBgEFBQcBAQQvMC0wKwYIKwYBBQUHMAKGH2h0\ndHA6Ly92YXVsdDo4MjAwL3YxL3BraV9pbnQvY2EwLwYDVR0RBCgwJoIkYmExZjJi\nMTYtNTIwNy00NjA4LWE0ZGUtNWZhYmZiODYyN2MyMDEGA1UdHwQqMCgwJqAkoCKG\nIGh0dHA6Ly92YXVsdDo4MjAwL3YxL3BraV9pbnQvY3JsMA0GCSqGSIb3DQEBCwUA\nA4IBAQCKMmDzyWWmuSJPh3O9hppRJ6mkX9gut4jP2rwowNv7haj3iu+hR8+GnTix\nu5oy3bZdmRryhhW0XyJsbCKO/z+wsY/RfVgMxF/c1cbmEzki804+AB4a4yNhQD6g\noEEQBD58b6mFi/vPCRiGZmmo5TqMlA37jBRSVnKO/CoH1CAvjqmfWdSoO4IC4uD4\nJev+QNr9wlOimYcA/usmo7rmqz7IB9R/Laxcdkq9iZelKly/jhftEbKgGf2NR/d7\nEKVONjCEp6fL2iBaQSA/899oJJ7QPqE5X821HhBlXKvNmZnYRyUmAS2h1jnxtovp\nsNGcLFRgIAFdaGl1172C7mBZF4C3\n-----END CERTIFICATE-----",
+ "expiration": "2023-10-14T11:09:58Z",
+ "thing_id": "f13f0f30-f923-4504-8a7a-6aa45bcb4866"
+}
+magistrala-cli certs get thing f13f0f30-f923-4504-8a7a-6aa45bcb4866 $USER_TOKEN
+{
+ "certs": [
+ {
+ "cert_serial": "6f:35:d5:9d:47:9d:23:50:08:f7:31:13:82:22:e4:c8:e6:cf:2c:c1",
+ "expiration": "0001-01-01T00:00:00Z"
+ }
+ ],
+ "limit": 10,
+ "offset": 0,
+ "total": 1
+}
+magistrala-cli certs revoke <thing_id> <user_auth_token>
+For example:
+magistrala-cli certs revoke f13f0f30-f923-4504-8a7a-6aa45bcb4866 $USER_TOKEN
+
+revoked: 2023-09-14 11:21:44 +0000 UTC
+For more information about the Certification service API, please check out the API documentation.
+ + + + + + +Magistrala CLI makes it easy to manage users, things, channels and messages.
+CLI can be downloaded as separate asset from project realeses or it can be built with GNU Make tool:
Get the Magistrala code
+go get github.com/absmach/magistrala
+Build the magistrala-cli
+make cli
+which will build magistrala-cli in <project_root>/build folder.
Executing build/magistrala-cli without any arguments will output help with all available commands and flags:
Usage:
+ magistrala-cli [command]
+
+Available Commands:
+ bootstrap Bootstrap management
+ certs Certificates management
+ channels Channels management
+ completion Generate the autocompletion script for the specified shell
+ config CLI local config
+ domains Domains management
+ groups Groups management
+ health Health Check
+ help Help about any command
+ invitations Invitations management
+ messages Send or read messages
+ provision Provision things and channels from a config file
+ subscription Subscription management
+ things Things management
+ users Users management
+
+Flags:
+ -b, --bootstrap-url string Bootstrap service URL (default "http://localhost:9013")
+ -s, --certs-url string Certs service URL (default "http://localhost:9019")
+ -c, --config string Config path
+ -C, --contact string Subscription contact query parameter
+ -y, --content-type string Message content type (default "application/senml+json")
+ -d, --domains-url string Domains service URL (default "http://localhost:8189")
+ -h, --help help for magistrala-cli
+ -H, --host-url string Host URL (default "http://localhost")
+ -p, --http-url string HTTP adapter URL (default "http://localhost/http")
+ -I, --identity string User identity query parameter
+ -i, --insecure Do not check for TLS cert
+ -v, --invitations-url string Inivitations URL (default "http://localhost:9020")
+ -l, --limit uint Limit query parameter (default 10)
+ -m, --metadata string Metadata query parameter
+ -n, --name string Name query parameter
+ -o, --offset uint Offset query parameter
+ -r, --raw Enables raw output mode for easier parsing of output
+ -R, --reader-url string Reader URL (default "http://localhost")
+ -z, --state string Bootstrap state query parameter
+ -S, --status string User status query parameter
+ -t, --things-url string Things service URL (default "http://localhost:9000")
+ -T, --topic string Subscription topic query parameter
+ -u, --users-url string Users service URL (default "http://localhost:9002")
+
+Use "magistrala-cli [command] --help" for more information about a command.
+It is also possible to use the docker image magistrala/cli to execute CLI command:
docker run -it --rm magistrala/cli -u http://<IP_SERVER> [command]
+For example:
+docker run -it --rm magistrala/cli -u http://192.168.160.1 users token admin@example.com 12345678
+
+{
+ "access_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODA2MjEzMDcsImlhdCI6MTY4MDYyMDQwNywiaWRlbnRpdHkiOiJhZG1pbkBleGFtcGxlLmNvbSIsImlzcyI6ImNsaWVudHMuYXV0aCIsInN1YiI6ImYxZTA5Y2YxLTgzY2UtNDE4ZS1iZDBmLWU3M2I3M2MxNDM2NSIsInR5cGUiOiJhY2Nlc3MifQ.iKdBv3Ko7PKuhjTC6Xs-DvqfKScjKted3ZMorTwpXCd4QrRSsz6NK_lARG6LjpE0JkymaCMVMZlzykyQ6ZgwpA",
+ "access_type": "Bearer",
+ "refresh_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODA3MDY4MDcsImlhdCI6MTY4MDYyMDQwNywiaWRlbnRpdHkiOiJhZG1pbkBleGFtcGxlLmNvbSIsImlzcyI6ImNsaWVudHMuYXV0aCIsInN1YiI6ImYxZTA5Y2YxLTgzY2UtNDE4ZS1iZDBmLWU3M2I3M2MxNDM2NSIsInR5cGUiOiJyZWZyZXNoIn0.-0tOtXFZi48VS-FxkCnVxnW2RUkJvqUmzRz3_EYSSKFyKealoFrv7sZIUvrdvKomnUFzXshP0EygL8vjWP1SFw"
+}
+You can execute each command with -h flag for more information about that command, e.g.
magistrala-cli channels -h
+Response should look like this:
+Channels management: create, get, update or delete Channel and get list of Things connected or not connected to a Channel
+
+Usage:
+ magistrala-cli channels [command]
+
+Available Commands:
+ assign Assign users or groups to a channel
+ connections Connections list
+ create Create channel
+ delete Delete channel
+ disable Change channel status to disabled
+ enable Change channel status to enabled
+ get Get channel
+ groups List groups
+ unassign Unassign users or groups from a channel
+ update Update channel
+ users List users
+
+Flags:
+ -h, --help help for channels
+
+Global Flags:
+ -b, --bootstrap-url string Bootstrap service URL (default "http://localhost:9013")
+ -s, --certs-url string Certs service URL (default "http://localhost:9019")
+ -c, --config string Config path
+ -C, --contact string Subscription contact query parameter
+ -y, --content-type string Message content type (default "application/senml+json")
+ -d, --domains-url string Domains service URL (default "http://localhost:8189")
+ -H, --host-url string Host URL (default "http://localhost")
+ -p, --http-url string HTTP adapter URL (default "http://localhost/http")
+ -I, --identity string User identity query parameter
+ -i, --insecure Do not check for TLS cert
+ -v, --invitations-url string Inivitations URL (default "http://localhost:9020")
+ -l, --limit uint Limit query parameter (default 10)
+ -m, --metadata string Metadata query parameter
+ -n, --name string Name query parameter
+ -o, --offset uint Offset query parameter
+ -r, --raw Enables raw output mode for easier parsing of output
+ -R, --reader-url string Reader URL (default "http://localhost")
+ -z, --state string Bootstrap state query parameter
+ -S, --status string User status query parameter
+ -t, --things-url string Things service URL (default "http://localhost:9000")
+ -T, --topic string Subscription topic query parameter
+ -u, --users-url string Users service URL (default "http://localhost:9002")
+
+Use "magistrala-cli channels [command] --help" for more information about a command.
+magistrala-cli health <service>
+For "things" service, the response should look like this:
+{
+ "build_time": "2024-03-13_16:12:26",
+ "commit": "3bf59689fb74388415d2655eb43b5d736ac82fc2",
+ "description": "things service",
+ "status": "pass",
+ "version": "v0.14.0"
+}
+Magistrala has two options for user creation. Either the <user_token> is provided or not. If the <user_token> is provided then the created user will be owned by the user identified by the <user_token>. Otherwise, when the token is not used, since everybody can create new users, the user will not have an owner. However, the token is still required, in order to be consistent. For more details, please see Authorization page.
magistrala-cli users create <user_name> <user_email> <user_password>
+
+magistrala-cli users create <user_name> <user_email> <user_password> <user_token>
+magistrala-cli users token <user_email> <user_password>
+magistrala-cli users refreshtoken <refresh_token>
+magistrala-cli users get <user_id> <user_token>
+magistrala-cli users get all <user_token>
+magistrala-cli users update <user_id> '{"name":"value1", "metadata":{"value2": "value3"}}' <user_token>
+magistrala-cli users update tags <user_id> '["tag1", "tag2"]' <user_token>
+magistrala-cli users update identity <user_id> <user_email> <user_token>
+magistrala-cli users update owner <user_id> <owner_id> <user_token>
+magistrala-cli users password <old_password> <password> <user_token>
+magistrala-cli users enable <user_id> <user_token>
+magistrala-cli users disable <user_id> <user_token>
+magistrala-cli users profile <user_token>
+magistrala-cli groups create '{"name":"<group_name>","description":"<description>","parentID":"<parent_id>","metadata":"<metadata>"}' <user_token>
+magistrala-cli groups get <group_id> <user_token>
+magistrala-cli groups get all <user_token>
+magistrala-cli groups update '{"id":"<group_id>","name":"<group_name>","description":"<description>","metadata":"<metadata>"}' <user_token>
+magistrala-cli groups members <group_id> <user_token>
+magistrala-cli groups membership <member_id> <user_token>
+magistrala-cli groups assign <member_ids> <member_type> <group_id> <user_token>
+magistrala-cli groups unassign <member_ids> <group_id> <user_token>
+magistrala-cli groups enable <group_id> <user_token>
+magistrala-cli groups disable <group_id> <user_token>
+magistrala-cli things create '{"name":"myThing"}' <user_token>
+magistrala-cli things create '{"name":"myThing", "metadata": {"key1":"value1"}}' <user_token>
+magistrala-cli provision things <file> <user_token>
+file - A CSV or JSON file containing thing names (must have extension .csv or .json)user_token - A valid user auth token for the current systemAn example CSV file might be:
+thing1,
+thing2,
+thing3,
+in which the first column is thing names.
+A comparable JSON file would be
+[
+ {
+ "name": "<thing1_name>",
+ "status": "enabled"
+ },
+ {
+ "name": "<thing2_name>",
+ "status": "disabled"
+ },
+ {
+ "name": "<thing3_name>",
+ "status": "enabled",
+ "credentials": {
+ "identity": "<thing3_identity>",
+ "secret": "<thing3_secret>"
+ }
+ }
+]
+With JSON you can be able to specify more fields of the channels you want to create
+magistrala-cli things update <thing_id> '{"name":"value1", "metadata":{"key1": "value2"}}' <user_token>
+magistrala-cli things update tags <thing_id> '["tag1", "tag2"]' <user_token>
+magistrala-cli things update owner <thing_id> <owner_id> <user_token>
+magistrala-cli things update secret <thing_id> <secet> <user_token>
+magistrala-cli things identify <thing_secret>
+magistrala-cli things enable <thing_id> <user_token>
+magistrala-cli things disable <thing_id> <user_token>
+magistrala-cli things get <thing_id> <user_token>
+magistrala-cli things get all <user_token>
+magistrala-cli things get all --offset=1 --limit=5 <user_token>
+magistrala-cli things share <channel_id> <user_id> <allowed_actions> <user_token>
+magistrala-cli channels create '{"name":"myChannel"}' <user_token>
+magistrala-cli provision channels <file> <user_token>
+file - A CSV or JSON file containing channel names (must have extension .csv or .json)user_token - A valid user auth token for the current systemAn example CSV file might be:
+<channel1_name>,
+<channel2_name>,
+<channel3_name>,
+in which the first column is channel names.
+A comparable JSON file would be
+[
+ {
+ "name": "<channel1_name>",
+ "description": "<channel1_description>",
+ "status": "enabled"
+ },
+ {
+ "name": "<channel2_name>",
+ "description": "<channel2_description>",
+ "status": "disabled"
+ },
+ {
+ "name": "<channel3_name>",
+ "description": "<channel3_description>",
+ "status": "enabled"
+ }
+]
+With JSON you can be able to specify more fields of the channels you want to create
+magistrala-cli channels update '{"id":"<channel_id>","name":"myNewName"}' <user_token>
+magistrala-cli channels enable <channel_id> <user_token>
+magistrala-cli channels disable <channel_id> <user_token>
+magistrala-cli channels get <channel_id> <user_token>
+magistrala-cli channels get all <user_token>
+magistrala-cli channels get all --offset=1 --limit=5 <user_token>
+magistrala-cli things connect <thing_id> <channel_id> <user_token>
+magistrala-cli provision connect <file> <user_token>
+file - A CSV or JSON file containing thing and channel ids (must have extension .csv or .json)user_token - A valid user auth token for the current systemAn example CSV file might be
+<thing_id1>,<channel_id1>
+<thing_id2>,<channel_id2>
+in which the first column is thing IDs and the second column is channel IDs. A connection will be created for each thing to each channel. This example would result in 4 connections being created.
+A comparable JSON file would be
+{
+ "subjects": ["<thing_id1>", "<thing_id2>"],
+ "objects": ["<channel_id1>", "<channel_id2>"]
+}
+magistrala-cli things disconnect <thing_id> <channel_id> <user_token>
+magistrala-cli things connections <thing_id> <user_token>
+magistrala-cli channels connections <channel_id> <user_token>
+magistrala-cli messages send <channel_id> '[{"bn":"Dev1","n":"temp","v":20}, {"n":"hum","v":40}, {"bn":"Dev2", "n":"temp","v":20}, {"n":"hum","v":40}]' <thing_secret>
+magistrala-cli messages read <channel_id> <user_token> -R <reader_url>
+magistrala-cli bootstrap create '{"external_id": "myExtID", "external_key": "myExtKey", "name": "myName", "content": "myContent"}' <user_token> -b <bootstrap-url>
+magistrala-cli bootstrap get <thing_id> <user_token> -b <bootstrap-url>
+magistrala-cli bootstrap update '{"magistrala_id":"<thing_id>", "name": "newName", "content": "newContent"}' <user_token> -b <bootstrap-url>
+magistrala-cli bootstrap remove <thing_id> <user_token> -b <bootstrap-url>
+magistrala-cli bootstrap bootstrap <external_id> <external_key> -b <bootstrap-url>
+Magistrala CLI tool supports configuration files that contain some of the basic settings so you don't have to specify them through flags. Once you set the settings, they remain stored locally.
+magistrala-cli config <parameter> <value>
+Response should look like this:
+ ok
+This command is used to set the flags to be used by CLI in a local TOML file. The default location of the TOML file is in the same directory as the CLI binary. To change the location of the TOML file you can run the command:
+ magistrala-cli config <parameter> <value> -c "cli/file_name.toml"
+The possible parameters that can be set using the config command are:
+| Flag | +Description | +Default | +
|---|---|---|
| bootstrap_url | +Bootstrap service URL | +"http://localhost:9013" | +
| certs_url | +Certs service URL | +"http://localhost:9019" | +
| http_adapter_url | +HTTP adapter URL | +"http://localhost/http" | +
| msg_content_type | +Message content type | +"application/senml+json" | +
| reader_url | +Reader URL | +"http://localhost" | +
| things_url | +Things service URL | +"http://localhost:9000" | +
| tls_verification | +Do not check for TLS cert | ++ |
| users_url | +Users service URL | +"http://localhost:9002" | +
| state | +Bootstrap state query parameter | ++ |
| status | +User status query parameter | ++ |
| topic | +Subscription topic query parameter | ++ |
| contact | +Subscription contact query parameter | ++ |
| User email query parameter | ++ | |
| limit | +Limit query parameter | +10 | +
| metadata | +Metadata query parameter | ++ |
| name | +Name query parameter | ++ |
| offset | +Offset query parameter | ++ |
| raw_output | +Enables raw output mode for easier parsing of output | ++ |
Magistrala source can be found in the official Magistrala GitHub repository. You should fork this repository in order to make changes to the project. The forked version of the repository should be cloned using the following:
+git clone <forked repository> $SOMEPATH/magistrala
+cd $SOMEPATH/magistrala
+Note: If your $SOMEPATH is equal to $GOPATH/src/github.com/absmach/magistrala, make sure that your $GOROOT and $GOPATH do not overlap (otherwise, go modules won't work).
Make sure that you have Protocol Buffers (version 21.12) compiler (protoc) installed.
Go Protobuf installation instructions are here. Go Protobuf uses C bindings, so you will need to install C++ protobuf as a prerequisite. Magistrala uses Protocol Buffers for Go with Gadgets to generate faster marshaling and unmarshaling Go code. Protocol Buffers for Go with Gadgets installation instructions can be found here.
A copy of Go (version 1.19.4) and docker template (version 3.7) will also need to be installed on your system.
+If any of these versions seem outdated, the latest can always be found in our CI script.
+Use the GNU Make tool to build all Magistrala services:
+make
+Build artifacts will be put in the build directory.
++N.B. All Magistrala services are built as a statically linked binaries. This way they can be portable (transferred to any platform just by placing them there and running them) as they contain all needed libraries and do not relay on shared system libraries. This helps creating FROM scratch dockers.
+
Individual microservices can be built with:
+make <microservice_name>
+For example:
+make http
+will build the HTTP Adapter microservice.
+Dockers can be built with:
+make dockers
+or individually with:
+make docker_<microservice_name>
+For example:
+make docker_http
+++N.B. Magistrala creates
+FROM scratchdocker containers which are compact and small in size.N.B. The
+things-dbandusers-dbcontainers are built from a vanilla PostgreSQL docker image downloaded from docker hub which does not persist the data when these containers are rebuilt. Thus, rebuilding of all docker containers withmake dockersor rebuilding thethings-dbandusers-dbcontainers separately withmake docker_things-dbandmake docker_users-dbrespectively, will cause data loss. All your users, things, channels and connections between them will be lost! As we use this setup only for development, we don't guarantee any permanent data persistence. Though, in order to enable data retention, we have configured persistent volumes for each container that stores some data. If you want to update your Magistrala dockerized installation and want to keep your data, usemake cleandockerto clean the containers and images and keep the data (stored in docker persistent volumes) and thenmake runto update the images and the containers. Check the Cleaning up your dockerized Magistrala setup section for details. Please note that this kind of updating might not work if there are database changes.
In order to speed up build process, you can use commands such as:
+make dockers_dev
+or individually with
+make docker_dev_<microservice_name>
+Commands make dockers and make dockers_dev are similar. The main difference is that building images in the development mode is done on the local machine, rather than an intermediate image, which makes building images much faster. Before running this command, corresponding binary needs to be built in order to make changes visible. This can be done using make or make <service_name> command. Commands make dockers_dev and make docker_dev_<service_name> should be used only for development to speed up the process of image building. For deployment images, commands from section above should be used.
When the project is first cloned to your system, you will need to make sure and build all of the Magistrala services.
+make
+make dockers_dev
+As you develop and test changes, only the services related to your changes will need to be rebuilt. This will reduce compile time and create a much more enjoyable development experience.
+make <microservice_name>
+make docker_dev_<microservice_name>
+make run
+Sometimes, depending on the use case and the user's needs it might be useful to override or add some extra parameters to the docker-compose configuration. These configuration changes can be done by specifying multiple compose files with the docker-compose command line option -f as described here.
+The following format of the docker-compose command can be used to extend or override the configuration:
docker-compose -f docker/docker-compose.yml -f docker/docker-compose.custom1.yml -f docker/docker-compose.custom2.yml up [-d]
+In the command above each successive file overrides the previous parameters.
+A practical example in our case would be to enable debugging and tracing in NATS so that we can see better how are the messages moving around.
+docker-compose.nats-debugging.yml
version: "3"
+
+services:
+ nats:
+ command: --debug -DV
+When we have the override files in place, to compose the whole infrastructure including the persistent volumes we can execute:
+docker-compose -f docker/docker-compose.yml -f docker/docker-compose.nats-debugging.yml up -d
+Note: Please store your customizations to some folder outside the Magistrala's source folder and maybe add them to some other git repository. You can always apply your customizations by pointing to the right file using docker-compose -f ....
If you want to clean your whole dockerized Magistrala installation you can use the make pv=true cleandocker command. Please note that by default the make cleandocker command will stop and delete all of the containers and images, but NOT DELETE persistent volumes. If you want to delete the gathered data in the system (the persistent volumes) please use the following command make pv=true cleandocker (pv = persistent volumes). This form of the command will stop and delete the containers, the images and will also delete the persistent volumes.
To build Magistrala MQTT message broker Docker image, use the following commands:
+cd docker/vernemq
+docker build --no-cache . -t magistrala/vernemq
+The Magistrala uses the VerneMQ for implementation of the MQTT messaging. Therefore, for some questions or problems you can also check out the VerneMQ documentation or reach out its contributors.
+If you've made any changes to .proto files, you should call protoc command prior to compiling individual microservices.
To do this by hand, execute:
+protoc -I. --go_out=. --go_opt=paths=source_relative pkg/messaging/*.proto
+protoc -I. --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative users/policies/*.proto
+protoc -I. --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative things/policies/*.proto
+A shorthand to do this via make tool is:
make proto
+++N.B. This must be done once at the beginning in order to generate protobuf Go structures needed for the build. However, if you don't change any of
+.protofiles, this step is not mandatory, since all generated files are included in the repository (those are files with.pb.goextension).
Magistrala can be compiled for ARM platform and run on Raspberry Pi or other similar IoT gateways, by following the instructions here or here as well as information found here. The environment variables GOARCH=arm and GOARM=7 must be set for the compilation.
Cross-compilation for ARM with Magistrala make:
+GOOS=linux GOARCH=arm GOARM=7 make
+To run all of the tests you can execute:
+make test
+Dockertest is used for the tests, so to run them, you will need the Docker daemon/service running.
+Installing Go binaries is simple: just move them from build to $GOBIN (do not fortget to add $GOBIN to your $PATH).
You can execute:
+make install
+which will do this copying of the binaries.
+++N.B. Only Go binaries will be installed this way. The MQTT broker is a written in Erlang and its build scripts can be found in
+docker/vernemqdir.
Magistrala depends on several infrastructural services, notably the default message broker, NATS and PostgreSQL database.
+Magistrala uses NATS as it's default central message bus. For development purposes (when not run via Docker), it expects that NATS is installed on the local system.
+To do this execute:
+go install github.com/nats-io/nats-server/v2@latest
+This will install nats-server binary that can be simply run by executing:
nats-server
+If you want to change the default message broker to RabbitMQ, VerneMQ or Kafka you need to install it on the local system.
+To run using a different broker you need to set the MF_BROKER_TYPE env variable to nats, rabbitmq or vernemq during make and run process.
MF_BROKER_TYPE=<broker-type> make
+MF_BROKER_TYPE=<broker-type> make run
+Magistrala uses PostgreSQL to store metadata (users, things and channels entities alongside with authorization tokens). It expects that PostgreSQL DB is installed, set up and running on the local system.
Information how to set-up (prepare) PostgreSQL database can be found here, and it is done by executing following commands:
+# Create `users` and `things` databases
+sudo -u postgres createdb users
+sudo -u postgres createdb things
+
+# Set-up Postgres roles
+sudo su - postgres
+psql -U postgres
+postgres=# CREATE ROLE magistrala WITH LOGIN ENCRYPTED PASSWORD 'magistrala';
+postgres=# ALTER USER magistrala WITH LOGIN ENCRYPTED PASSWORD 'magistrala';
+Running of the Magistrala microservices can be tricky, as there is a lot of them and each demand configuration in the form of environment variables.
+The whole system (set of microservices) can be run with one command:
+make rundev
+which will properly configure and run all microservices.
+Please assure that MQTT microservice has node_modules installed, as explained in MQTT Microservice chapter.
++ + + + + + +N.B.
+make rundevactually calls helper scriptscripts/run.sh, so you can inspect this script for the details.
Magistrala IoT platform provides services for supporting management of devices on the edge. Typically, IoT solution includes devices (sensors/actuators) deployed in far edge and connected through some proxy gateway. Although most devices could be connected to the Magistrala directly, using gateways decentralizes system, decreases load on the cloud and makes setup less difficult. Also, gateways can provide additional data processing, filtering and storage.
+Services that can be used on gateway to enable data and control plane for edge:
+ |
+
|---|
| Figure 1 - Edge services deployment | +
Figure shows edge gateway that is running Agent, Export and minimal deployment of Magistrala services. Magistrala services enable device management and MQTT protocol, NATS being a central message bus as it is the default message broker in Magistrala becomes also central message bus for other services like Agent and Export as well as for any new custom developed service that can be built to interface with devices with any of hardware supported interfaces on the gateway, those services would publish data to the message broker where Export service can pick them up and send to cloud.
Agent can be used to control deployed services as well as to monitor their liveliness through subcribing to heartbeat Message Broker subject where services should publish their liveliness status, like Export service does.
Agent is service that is used to manage gateways that are connected to Magistrala in cloud. It provides a way to send commands to gateway and receive response via mqtt. There are two types of channels used for Agent data and control. Over the control we are sending commands and receiving response from commands. Data collected from sensors connected to gateway are being sent over data channel. Agent is able to configure itself provided that bootstrap server is running, it will retrieve configuration from bootstrap server provided few arguments - external_id and external_key see bootstraping.
Agent service has following features:
+bash managed by Agentheartbeat.> it can remotely provide info on running services, if services are publishing heartbeat ( like Export)Before running agent we need to provision a thing and DATA and CONTROL channel. Thing that will be used as gateway representation and make bootstrap configuration. If using Magistrala UI this is done automatically when adding gateway through UI. Gateway can be provisioned with provision service.
When you provisioned gateway as described in provision you can check results
+curl -s -S -X GET http://magistrala-domain.com:9013/things/bootstrap/<external_id> -H "Authorization: Thing <external_key>" -H 'Content-Type: application/json' |jq
+{
+ "thing_id": "e22c383a-d2ab-47c1-89cd-903955da993d",
+ "thing_key": "fc987711-1828-461b-aa4b-16d5b2c642fe",
+ "channels": [
+ {
+ "id": "fa5f9ba8-a1fc-4380-9edb-d0c23eaa24ec",
+ "name": "control-channel",
+ "metadata": {
+ "type": "control"
+ }
+ },
+ {
+ "id": "24e5473e-3cbe-43d9-8a8b-a725ff918c0e",
+ "name": "data-channel",
+ "metadata": {
+ "type": "data"
+ }
+ },
+ {
+ "id": "1eac45c2-0f72-4089-b255-ebd2e5732bbb",
+ "name": "export-channel",
+ "metadata": {
+ "type": "export"
+ }
+ }
+ ],
+ "content": "{\"agent\":{\"edgex\":{\"url\":\"http://localhost:48090/api/v1/\"},\"heartbeat\":{\"interval\":\"30s\"},\"log\":{\"level\":\"debug\"},\"mqtt\":{\"mtls\":false,\"qos\":0,\"retain\":false,\"skip_tls_ver\":true,\"url\":\"tcp://magistrala-domain.com:1883\"},\"server\":{\"nats_url\":\"localhost:4222\",\"port\":\"9000\"},\"terminal\":{\"session_timeout\":\"30s\"}},\"export\":{\"exp\":{\"cache_db\":\"0\",\"cache_pass\":\"\",\"cache_url\":\"localhost:6379\",\"log_level\":\"debug\",\"nats\":\"nats://localhost:4222\",\"port\":\"8172\"},\"mqtt\":{\"ca_path\":\"ca.crt\",\"cert_path\":\"thing.crt\",\"channel\":\"\",\"host\":\"tcp://magistrala-domain.com:1883\",\"mtls\":false,\"password\":\"\",\"priv_key_path\":\"thing.key\",\"qos\":0,\"retain\":false,\"skip_tls_ver\":false,\"username\":\"\"},\"routes\":[{\"mqtt_topic\":\"\",\"nats_topic\":\"channels\",\"subtopic\":\"\",\"type\":\"mfx\",\"workers\":10},{\"mqtt_topic\":\"\",\"nats_topic\":\"export\",\"subtopic\":\"\",\"type\":\"default\",\"workers\":10}]}}"
+}
+external_id is usually MAC address, but anything that suits applications requirements can be usedexternal_key is key that will be provided to agent processthing_id is Magistrala thing idchannels is 2-element array where first channel is CONTROL and second is DATA, both channels should be assigned to thingcontent is used for configuring parameters of agent and export service.Then to start the agent service you can do it like this
+git clone https://github.com/absmach/agent.git
+make
+cd build
+
+MF_AGENT_LOG_LEVEL=debug \
+MF_AGENT_BOOTSTRAP_KEY=edged \
+MF_AGENT_BOOTSTRAP_ID=34:e1:2d:e6:cf:03 ./magistrala-agent
+
+{"level":"info","message":"Requesting config for 34:e1:2d:e6:cf:03 from http://localhost:9013/things/bootstrap","ts":"2019-12-05T04:47:24.98411512Z"}
+{"level":"info","message":"Getting config for 34:e1:2d:e6:cf:03 from http://localhost:9013/things/bootstrap succeeded","ts":"2019-12-05T04:47:24.995465239Z"}
+{"level":"info","message":"Connected to MQTT broker","ts":"2019-12-05T04:47:25.009645082Z"}
+{"level":"info","message":"Agent service started, exposed port 9000","ts":"2019-12-05T04:47:25.009755345Z"}
+{"level":"info","message":"Subscribed to MQTT broker","ts":"2019-12-05T04:47:25.012930443Z"}
+MF_AGENT_BOOTSTRAP_KEY - is external_key in bootstrap configuration.MF_AGENT_BOOSTRAP_ID - is external_id in bootstrap configuration.# Set connection parameters as environment variables in shell
+CH=`curl -s -S -X GET http://some-domain-name:9013/things/bootstrap/34:e1:2d:e6:cf:03 -H "Authorization: Thing <BOOTSTRAP_KEY>" -H 'Content-Type: application/json' | jq -r '.magistrala_channels[0].id'`
+TH=`curl -s -S -X GET http://some-domain-name:9013/things/bootstrap/34:e1:2d:e6:cf:03 -H "Authorization: Thing <BOOTSTRAP_KEY>" -H 'Content-Type: application/json' | jq -r .magistrala_id`
+KEY=`curl -s -S -X GET http://some-domain-name:9013/things/bootstrap/34:e1:2d:e6:cf:03 -H "Authorization: Thing <BOOTSTRAP_KEY>" -H 'Content-Type: application/json' | jq -r .magistrala_key`
+
+# Subscribe for response
+mosquitto_sub -d -u $TH -P $KEY -t "channels/${CH}/messages/res/#" -h some-domain-name -p 1883
+
+# Publish command e.g `ls`
+mosquitto_pub -d -u $TH -P $KEY -t channels/$CH/messages/req -h some-domain-name -p 1883 -m '[{"bn":"1:", "n":"exec", "vs":"ls, -l"}]'
+This can be checked from the UI, click on the details for gateway and below the gateway parameters you will se box with prompt, if agent is running and it is properly connected you should be able to execute commands remotely.
If there are services that are running on same gateway as agent and they are publishing heartbeat to the Message Broker subject heartbeat.service_name.service
+You can get the list of services by sending following mqtt message
# View services that are sending heartbeat
+mosquitto_pub -d -u $TH -P $KEY -t channels/$CH/messages/req -h some-domain-name -p 1883 -m '[{"bn":"1:", "n":"service", "vs":"view"}]'
+Response can be observed on channels/$CH/messages/res/#
You can send commands to services running on the same edge gateway as Agent if they are subscribed on same the Message Broker server and correct subject.
+Service commands are being sent via MQTT to topic:
+channels/<control_channel_id>/messages/services/<service_name>/<subtopic>
when messages is received Agent forwards them to the Message Broker on subject:
+commands.<service_name>.<subtopic>
Payload is up to the application and service itself.
+Edgex control messages are sent and received over control channel. MF sends a control SenML of the following form:
+[{"bn":"<uuid>:", "n":"control", "vs":"<cmd>, <param>, edgexsvc1, edgexsvc2, …, edgexsvcN"}}]
+For example,
+[{"bn":"1:", "n":"control", "vs":"operation, stop, edgex-support-notifications, edgex-core-data"}]
+Agent, on the other hand, returns a response SenML of the following form:
+[{"bn":"<uuid>:", "n":"<>", "v":"<RESP>"}]
+EdgeX defines SMA commands in the following RAML file
+Commands are:
+mosquitto_pub -u <thing_id> -P <thing_secret> -t channels/<channel_id>/messages/req -h localhost -m '[{"bn":"1:", "n":"control", "vs":"edgex-operation, start, edgex-support-notifications, edgex-core-data"}]'
+mosquitto_pub -u <thing_id> -P <thing_secret> -t channels/<channel_id>/messages/req -h localhost -m '[{"bn":"1:", "n":"control", "vs":"edgex-config, edgex-support-notifications, edgex-core-data"}]'
+mosquitto_pub -u <thing_id> -P <thing_secret> -t channels/<channel_id>/messages/req -h localhost -m '[{"bn":"1:", "n":"control", "vs":"edgex-metrics, edgex-support-notifications, edgex-core-data"}]'
+If you subscribe to
+mosquitto_sub -u <thing_id> -P <thing_secret> -t channels/<channel_id>/messages/#
+You can observe commands and response from commands executed against edgex
+[{"bn":"1:", "n":"control", "vs":"edgex-metrics, edgex-support-notifications, edgex-core-data"}]
+[{"bn":"1","n":"edgex-metrics","vs":"{\"Metrics\":{\"edgex-core-data\":{\"CpuBusyAvg\":15.568632467698606,\"Memory\":{\"Alloc\":2040136,\"Frees\":876344,\"LiveObjects\":15134,\"Mallocs\":891478,\"Sys\":73332984,\"TotalAlloc\":80657464}},\"edgex-support-notifications\":{\"CpuBusyAvg\":14.65381169745318,\"Memory\":{\"Alloc\":961784,\"Frees\":127430,\"LiveObjects\":6095,\"Mallocs\":133525,\"Sys\":72808696,\"TotalAlloc\":11665416}}}}\n"}]
+Magistrala Export service can send message from one Magistrala cloud to another via MQTT, or it can send messages from edge gateway to Magistrala Cloud. Export service is subscribed to local message bus and connected to MQTT broker in the cloud. Messages collected on local message bus are redirected to the cloud. When connection is lost, if QoS2 is used, messages from the local bus are stored into file or in memory to be resent upon reconnection. Additonaly Export service publishes liveliness status to Agent via the Message Broker subject heartbeat.export.service
Get the code:
+go get github.com/magistrala/export
+cd $GOPATH/github.com/magistrala/export
+Make:
+make
+cd build
+./magistrala-export
+By default Export service looks for config file at ../configs/config.toml if no env vars are specified.
[exp]
+ log_level = "debug"
+ nats = "localhost:4222"
+ port = "8170"
+
+[mqtt]
+ username = "<thing_id>"
+ password = "<thing_password>"
+ ca_path = "ca.crt"
+ client_cert = ""
+ client_cert_key = ""
+ client_cert_path = "thing.crt"
+ client_priv_key_path = "thing.key"
+ mtls = "false"
+ priv_key = "thing.key"
+ retain = "false"
+ skip_tls_ver = "false"
+ url = "tcp://magistrala.com:1883"
+
+[[routes]]
+ mqtt_topic = "channel/<channel_id>/messages"
+ subtopic = "subtopic"
+ nats_topic = "export"
+ type = "default"
+ workers = 10
+
+[[routes]]
+ mqtt_topic = "channel/<channel_id>/messages"
+ subtopic = "subtopic"
+ nats_topic = "channels"
+ type = "mfx"
+ workers = 10
+Service will first look for MF_EXPORT_CONFIG_FILE for configuration and if not found it will be configured with env variables and new config file specified with MF_EXPORT_CONFIG_FILE (default value will be used if none specified) will be saved with values populated from env vars. The service is configured using the environment variables as presented in the table. Note that any unset variables will be replaced with their default values.
For values in environment variables to take effect make sure that there is no MF_EXPORT_CONFIG_FILE file.
If you run with environment variables you can create config file:
+MF_EXPORT_PORT=8178 \
+MF_EXPORT_LOG_LEVEL=debug \
+MF_EXPORT_MQTT_HOST=tcp://localhost:1883 \
+MF_EXPORT_MQTT_USERNAME=<thing_id> \
+MF_EXPORT_MQTT_PASSWORD=<thing_secret> \
+MF_EXPORT_MQTT_CHANNEL=<channel_id> \
+MF_EXPORT_MQTT_SKIP_TLS=true \
+MF_EXPORT_MQTT_MTLS=false \
+MF_EXPORT_MQTT_CA=ca.crt \
+MF_EXPORT_MQTT_CLIENT_CERT=thing.crt \
+MF_EXPORT_MQTT_CLIENT_PK=thing.key \
+MF_EXPORT_CONFIG_FILE=export.toml \
+../build/magistrala-export&
+Values from environment variables will be used to populate export.toml
+port - HTTP port where status of Export service can be fetched.curl -X GET http://localhost:8170/health
+'{"status": "pass", "version":"0.12.1", "commit":"57cca9677721025da055c47957fc3e869e0325aa" , "description":"export service", "build_time": "2022-01-19_10:13:17"}'
+To establish connection to MQTT broker following settings are needed:
+username - Magistrala password - Magistrala url - url of MQTT brokerAdditionally, you will need MQTT client certificates if you enable mTLS. To obtain certificates ca.crt, thing.crt and key thing.key follow instructions here or here.
To setup MTLS connection Export service requires client certificate and mtls in config or MF_EXPORT_MQTT_MTLS must be set to true. Client certificate can be provided in a file, client_cert_path and client_cert_key_path are used for specifying path to certificate files. If MTLS is used and no certificate file paths are specified then Export will look in client_cert and client_cert_key of config file expecting certificate content stored as string.
Routes are being used for specifying which subscriber's topic(subject) goes to which publishing topic. Currently only MQTT is supported for publishing. To match Magistrala requirements mqtt_topic must contain channel/<channel_id>/messages, additional subtopics can be appended.
mqtt_topic - channel/<channel_id>/messages/<custom_subtopic>nats_topic - Export service will be subscribed to the Message Broker subject <nats_topic>.>subtopic - messages will be published to MQTT topic <mqtt_topic>/<subtopic>/<nats_subject>, where dots in nats_subject are replaced with '/'workers - specifies number of workers that will be used for message forwarding.type - specifies message transformation:default is for sending messages as they are received on the Message Broker with no transformation (so they should be in SenML or JSON format if we want to persist them in Magistrala in cloud). If you don't want to persist messages in Magistrala or you are not exporting to Magistrala cloud - message format can be anything that suits your application as message passes untransformed.mfx is for messages that are being picked up on internal Magistrala Message Broker bus. When using Export along with Magistrala deployed on gateway (Fig. 1) messages coming from MQTT broker that are published to the Message Broker bus are Magistrala message. Using mfx type will extract payload and export will publish it to mqtt_topic. Extracted payload is SenML or JSON if we want to persist messages. nats_topic in this case must be channels, or if you want to pick messages from a specific channel in local Magistrala instance to be exported to cloud you can put channels.<local_magistrala_channel_id>.Before running Export service edit configs/config.toml and provide username, password and url
username - matches thing_id in Magistrala cloud instancepassword - matches thing_secretchannel - MQTT part of the topic where to publish MQTT data (channel/<channel_id>/messages is format of magistrala MQTT topic) and plays a part in authorization.If Magistrala and Export service are deployed on same gateway Export can be configured to send messages from Magistrala internal Message Broker bus to Magistrala in a cloud. In order for Export service to listen on Magistrala Message Broker deployed on the same machine Message Broker port must be exposed. Edit Magistrala docker-compose.yml. Default Message Broker, NATS, section must look like below:
nats:
+ image: nats:2.2.4
+ container_name: magistrala-nats
+ restart: on-failure
+ networks:
+ - magistrala-base-net
+ ports:
+ - 4222:4222
+Configuration file for Export service can be sent over MQTT using Agent service.
mosquitto_pub -u <thing_id> -P <thing_secret> -t channels/<control_ch_id>/messages/req -h localhost -p 18831 -m "[{\"bn\":\"1:\", \"n\":\"config\", \"vs\":\"save, export, <config_file_path>, <file_content_base64>\"}]"
+vs="save, export, config_file_path, file_content_base64" - vs determines where to save file and contains file content in base64 encoding payload:
b,_ := toml.Marshal(export.Config)
+payload := base64.StdEncoding.EncodeToString(b)
+There is a configuration.sh script in a scripts directory that can be used for automatic configuration and start up of remotely deployed export. For this to work it is presumed that magistrala-export and scripts/export_start are placed in executable path on remote device. Additionally this script requires that remote device is provisioned following the steps described for provision service.
To run it first edit script to set parameters
+MTLS=false
+EXTERNAL_KEY='raspberry'
+EXTERNAL_ID='pi'
+MAGISTRALA_HOST='magistrala.com'
+MAGISTRALA_USER_EMAIL='edge@email.com'
+MAGISTRALA_USER_PASSWORD='12345678'
+EXTERNAL_KEY and EXTERNAL_ID are parameters posted to /mapping endpoint of provision service, MAGISTRALA_HOST is location of cloud instance of Magistrala that export should connect to and MAGISTRALA_USER_EMAIL and MAGISTRALA_USER_PASSWORD are users credentials in the cloud.
The following are steps that are an example usage of Magistrala components to connect edge with cloud. We will start Magistrala in the cloud with additional services Bootstrap and Provision. Using Bootstrap and Provision we will create a configuration for use in gateway deployment. On the gateway we will start services Agent and Export using previously created configuration.
+Start the Magistrala:
+docker-compose -f docker/docker-compose.yml up
+Start the Bootstrap service:
+docker-compose -f docker/addons/bootstrap/docker-compose.yml up
+Start the Provision service
+docker-compose -f docker/addons/provision/docker-compose.yml up
+Create user:
+magistrala-cli -m http://localhost:9002 users create test test@email.com 12345678
+Obtain user token:
+magistrala-cli -m http://localhost:9002 users token test@email.com 12345678
+
+{
+ "access_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODY3NTEzNTIsImlhdCI6MTY4Njc1MDQ1MiwiaWRlbnRpdHkiOiJqb2huLmRvZUBlbWFpbC5jb20iLCJpc3MiOiJjbGllbnRzLmF1dGgiLCJzdWIiOiI5NDkzOTE1OS1kMTI5LTRmMTctOWU0ZS1jYzJkNjE1NTM5ZDciLCJ0eXBlIjoiYWNjZXNzIn0.AND1sm6mN2wgUxVkDhpipCoNa87KPMghGaS5-4dU0iZaqGIUhWScrEJwOahT9ts1TZSd1qEcANTIffJ_y2Pbsg",
+ "refresh_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODY4MzY4NTIsImlhdCI6MTY4Njc1MDQ1MiwiaWRlbnRpdHkiOiJqb2huLmRvZUBlbWFpbC5jb20iLCJpc3MiOiJjbGllbnRzLmF1dGgiLCJzdWIiOiI5NDkzOTE1OS1kMTI5LTRmMTctOWU0ZS1jYzJkNjE1NTM5ZDciLCJ0eXBlIjoicmVmcmVzaCJ9.z3OWCHhNHNuvkzBqEAoLKWS6vpFLkIYXhH9cZogSCXd109-BbKVlLvYKmja-hkhaj_XDJKySDN3voiazBr_WTA",
+ "access_type": "Bearer"
+}
+
+USER_TOKEN=<access_token>
+Provision a gateway:
+curl -s -S -X POST http://localhost:9016/mapping -H "Authorization: Bearer $USER_TOKEN" -H 'Content-Type: application/json' -d '{"name":"testing", "external_id" : "54:FG:66:DC:43", "external_key":"223334fw2" }' | jq
+{
+ "things": [
+ {
+ "id": "88529fb2-6c1e-4b60-b9ab-73b5d89f7404",
+ "name": "thing",
+ "key": "3529c1bb-7211-4d40-9cd8-b05833196093",
+ "metadata": {
+ "external_id": "54:FG:66:DC:43"
+ }
+ }
+ ],
+ "channels": [
+ {
+ "id": "1aa3f736-0bd3-44b5-a917-a72cc743f633",
+ "name": "control-channel",
+ "metadata": {
+ "type": "control"
+ }
+ },
+ {
+ "id": "e2adcfa6-96b2-425d-8cd4-ff8cb9c056ce",
+ "name": "data-channel",
+ "metadata": {
+ "type": "data"
+ }
+ }
+ ],
+ "whitelisted": {
+ "88529fb2-6c1e-4b60-b9ab-73b5d89f7404": true
+ }
+}
+Parameters Provision will use them to create a bootstrap configuration that will make a relation with Magistrala entities used for connection, authentication and authorization thing and channel. These parameters will be used by Agent service on the gateway to retrieve that information and establish a connection with the cloud.
Start the NATS and Agent service:
+gnatsd
+MF_AGENT_BOOTSTRAP_ID=54:FG:66:DC:43 \
+MF_AGENT_BOOTSTRAP_KEY="223334fw2" \
+MF_AGENT_BOOTSTRAP_URL=http://localhost:9013/things/bootstrap \
+build/magistrala-agent
+{"level":"info","message":"Requesting config for 54:FG:66:DC:43 from http://localhost:9013/things/bootstrap","ts":"2020-05-07T15:50:58.041145096Z"}
+{"level":"info","message":"Getting config for 54:FG:66:DC:43 from http://localhost:9013/things/bootstrap succeeded","ts":"2020-05-07T15:50:58.120779415Z"}
+{"level":"info","message":"Saving export config file /configs/export/config.toml","ts":"2020-05-07T15:50:58.121602229Z"}
+{"level":"warn","message":"Failed to save export config file Error writing config file: open /configs/export/config.toml: no such file or directory","ts":"2020-05-07T15:50:58.121752142Z"}
+{"level":"info","message":"Client agent-88529fb2-6c1e-4b60-b9ab-73b5d89f7404 connected","ts":"2020-05-07T15:50:58.128500603Z"}
+{"level":"info","message":"Agent service started, exposed port 9003","ts":"2020-05-07T15:50:58.128531057Z"}
+git clone https://github.com/absmach/export.git
+make
+Edit the configs/config.toml setting
username - thing from the results of provision request.password - key from the results of provision request.mqtt_topic - in routes set to channels/<channel_data_id>/messages from results of provision.nats_topic - whatever you need, export will subscribe to export.<nats_topic> and forward messages to MQTT.host - url of MQTT broker.[exp]
+ cache_pass = ""
+ cache_url = ""
+ log_level = "debug"
+ nats = "localhost:4222"
+ port = "8170"
+
+[mqtt]
+ ca_path = ""
+ cert_path = ""
+ host = "tcp://localhost:1883"
+ mtls = false
+ password = "3529c1bb-7211-4d40-9cd8-b05833196093"
+ priv_key_path = ""
+ qos = 0
+ retain = false
+ skip_tls_ver = false
+ username = "88529fb2-6c1e-4b60-b9ab-73b5d89f7404"
+
+[[routes]]
+ mqtt_topic = "channels/e2adcfa6-96b2-425d-8cd4-ff8cb9c056ce/messages"
+ nats_topic = ">"
+ workers = 10
+cd build
+./magistrala-export
+2020/05/07 17:36:57 Configuration loaded from file ../configs/config.toml
+{"level":"info","message":"Export service started, exposed port :8170","ts":"2020-05-07T15:36:57.528398548Z"}
+{"level":"debug","message":"Client export-88529fb2-6c1e-4b60-b9ab-73b5d89f7404 connected","ts":"2020-05-07T15:36:57.528405818Z"}
+git clone https://github.com/absmach/agent.git
+go run ./examples/publish/main.go -s http://localhost:4222 export.test "[{\"bn\":\"test\"}]";
+We have configured route for export, nats_topic = ">" means that it will listen to NATS subject export.> and mqtt_topic is configured so that data will be sent to MQTT broker on topic channels/e2adcfa6-96b2-425d-8cd4-ff8cb9c056ce/messages with appended NATS subject. Other brokers can such as rabbitmq can be used, for more detail refer to dev-guide.
In terminal where export is started you should see following message:
+{"level":"debug","message":"Published to: export.test, payload: [{\"bn\":\"test\"}]","ts":"2020-05-08T15:14:15.757298992Z"}
+In Magistrala mqtt service:
magistrala-mqtt | {"level":"info","message":"Publish - client ID export-88529fb2-6c1e-4b60-b9ab-73b5d89f7404 to the topic: channels/e2adcfa6-96b2-425d-8cd4-ff8cb9c056ce/messages/export/test","ts":"2020-05-08T15:16:02.999684791Z"}
+Client is a component that will replace and unify the Magistrala Things and Users services. The purpose is to represent generic client accounts. Each client is identified using its identity and secret. The client will differ from Things service to Users service but we aim to achieve 1:1 implementation between the clients whilst changing how client secret works. This includes client secret generation, usage, modification and storage
+The client entity is represented by the Client struct in Go. The fields of this struct are as follows:
+// Credentials represent client credentials: its
+// "identity" which can be a username, email, generated name;
+// and "secret" which can be a password or access token.
+type Credentials struct {
+ Identity string `json:"identity,omitempty"` // username or generated login ID
+ Secret string `json:"secret"` // password or token
+}
+
+// Client represents generic Client.
+type Client struct {
+ ID string `json:"id"`
+ Name string `json:"name,omitempty"`
+ Tags []string `json:"tags,omitempty"`
+ Owner string `json:"owner,omitempty"` // nullable
+ Credentials Credentials `json:"credentials"`
+ Metadata Metadata `json:"metadata,omitempty"`
+ CreatedAt time.Time `json:"created_at"`
+ UpdatedAt time.Time `json:"updated_at,omitempty"`
+ UpdatedBy string `json:"updated_by,omitempty"`
+ Status Status `json:"status"` // 1 for enabled, 0 for disabled
+ Role Role `json:"role,omitempty"` // 1 for admin, 0 for normal user
+}
+ID is a unique identifier for each client. It is a string value.Name is an optional field that represents the name of the client.Tags is an optional field that represents the tags related to the client. It is a slice of string values.Owner is an optional field that represents the owner of the client.Credentials is a struct that represents the client credentials. It contains two fields:Identity This is the identity of the client, which can be a username, email, or generated name.Secret This is the secret of the client, which can be a password, secret key, or access token.Metadata is an optional field that is used for customized describing of the client.CreatedAt is a field that represents the time when the client was created. It is a time.Time value.UpdatedAt is a field that represents the time when the client was last updated. It is a time.Time value.UpdatedBy is a field that represents the user who last updated the client.Status is a field that represents the status for the client. It can be either 1 for enabled or 0 for disabled.Role is an optional field that represents the role of the client. It can be either 1 for admin or 0 for the user.Currently, we have the things service and the users service as 2 deployments of the client entity. The things service is used to create, read, update, and delete things. The users service is used to create, read, update, and delete users. The client entity will be used to replace the things and users services. The client entity can be serialized to and from JSON format for communication with other services.
+For grouping Magistrala entities there are groups object in the users service. The users groups can be used for grouping users only. Groups are organized like a tree, group can have one parent and children. Group with no parent is root of the tree.
Things Service manages things and channel. Thing represents a device (or an application) connected to Magistrala that uses the platform for message exchange with other things.
+Channel is a message conduit between things connected to it. It serves as a message topic that can be consumed by all of the things connected to it. Things can publish or subscribe to the Channel.
In order to be easily integratable system, Magistrala is using Redis Streams as an event log for event sourcing. Services that are publishing events to Redis Streams are users service, things service, bootstrap service and mqtt adapter.
For every operation users service will generate new event and publish it to Redis Stream called magistrala.users. Every event has its own event ID that is automatically generated and operation field that can have one of the following values:
user.create for user creationuser.update for user updateuser.remove for user change of stateuser.view for user viewuser.view_profile for user profile viewuser.list for listing usersuser.list_by_group for listing users by groupuser.identify for user identificationuser.generate_reset_token for generating reset tokenuser.issue_token for issuing tokenuser.refresh_token for refreshing tokenuser.reset_secret for resetting secretuser.send_password_reset for sending password resetgroup.create for group creationgroup.update for group updategroup.remove for group change of stategroup.view for group viewgroup.list for listing groupsgroup.list_by_user for listing groups by userpolicy.authorize for policy authorizationpolicy.add for policy creationpolicy.update for policy updatepolicy.remove for policy deletionpolicy.list for listing policiesBy fetching and processing these events you can reconstruct users service state. If you store some of your custom data in metadata field, this is the perfect way to fetch it and process it. If you want to integrate through docker-compose.yml you can use magistrala-es-redis service. Just connect to it and consume events from Redis Stream named magistrala.users.
Whenever user is created, users service will generate new create event. This event will have the following format:
In Redis Streams
+1) "1693307171926-0"
+2) 1) "occurred_at"
+ 2) "1693307171925834295"
+ 3) "operation"
+ 4) "user.create"
+ 5) "id"
+ 6) "e1b982d8-a332-4bc2-aaff-4bbaa86880fc"
+ 7) "status"
+ 8) "enabled"
+ 9) "created_at"
+ 10) "2023-08-29T11:06:11.914074Z"
+ 11) "name"
+ 12) "-dry-sun"
+ 13) "metadata"
+ 14) "{}"
+ 15) "identity"
+ 16) "-small-flower@email.com"
+As you can see from this example, every odd field represents field name while every even field represents field value. This is standard event format for Redis Streams. If you want to extract metadata field from this event, you'll have to read it as string first and then you can deserialize it to some structured format.
In Nats JetStream
+Subject: events.magistrala.users Received: 2023-10-05T15:41:04+03:00
+
+{"created_at":"2023-10-05T12:41:04.743529Z","id":"0a5f2e21-1a8b-460e-bfa9-732e570df095","identity":"wizardly_hopper@email.com","metadata":"e30=","name":"wizardly_hopper","occurred_at":1696509664755440542,"operation":"user.create","status":"enabled"}
+In RabbitMQ Streams +
{"created_at":"2023-10-17T08:43:52.329385Z","id":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4","identity":"-Comley-Hipp@email.com","metadata":"e30=","name":"-Skurka-Brule","occurred_at":1697532232341083347,"operation":"user.create","status":"enabled"}
+Whenever user is viewed, users service will generate new view event. This event will have the following format:
In Redis Streams
+1) "1693307172248-0"
+2) 1) "name"
+ 2) "-holy-pond"
+ 3) "owner"
+ 4) "e1b982d8-a332-4bc2-aaff-4bbaa86880fc"
+ 5) "created_at"
+ 6) "2023-08-29T11:06:12.032254Z"
+ 7) "status"
+ 8) "enabled"
+ 9) "operation"
+ 10) "user.view"
+ 11) "id"
+ 12) "56d2a797-dcb9-4fab-baf9-7c75e707b2c0"
+ 13) "identity"
+ 14) "-snowy-wave@email.com"
+ 15) "metadata"
+ 16) "{}"
+ 17) "occurred_at"
+ 18) "1693307172247989798"
+Subject: events.magistrala.users Received: 2023-10-05T18:38:40+03:00
+
+{"created_at":"2023-10-05T15:38:40.219889Z","id":"d4baecb8-adfa-4c7c-8257-deea5d7f9dba","identity":"-Hilling-Karole@email.com","metadata":"e30=","name":"-Doud-Varley","occurred_at":1696520320355145103,"operation":"user.view","owner":"3264e965-3fe5-4d4e-a857-48de43551d2e","status":"enabled"}
+In rabbitmq streams
+{"created_at":"2023-10-17T08:43:52.441962Z","id":"00397996-f7e0-4035-9271-3e48ee66d525","identity":"-Yanish-Lanctot@email.com","metadata":"e30=","name":"-Busbey-Vadala","occurred_at":1697532237472321186,"operation":"user.view","owner":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4","status":"enabled"}
+Whenever user profile is viewed, users service will generate new view_profile event. This event will have the following format:
In Redis
+1) "1693308867001-0"
+2) 1) "id"
+ 2) "64fd20bf-e8fb-46bf-9b64-2a6572eda21b"
+ 3) "name"
+ 4) "admin"
+ 5) "identity"
+ 6) "admin@example.com"
+ 7) "metadata"
+ 8) "{\"role\":\"admin\"}"
+ 9) "created_at"
+ 10) "2023-08-29T10:55:23.048948Z"
+ 11) "status"
+ 12) "enabled"
+ 13) "occurred_at"
+ 14) "1693308867001792403"
+ 15) "operation"
+ 16) "user.view_profile"
+In Nats JetStreams
+Subject: events.magistrala.users Received: 2023-10-05T19:41:01+03:00
+
+{"created_at":"2023-10-05T11:59:02.029606Z","id":"97466511-6317-4c98-8d58-7bd78bcaf587","identity":"admin@example.com","metadata":"eyJyb2xlIjoiYWRtaW4ifQ==","name":"admin","occurred_at":1696524061363472648,"operation":"user.view_profile","status":"enabled"}
+Whenever user list is fetched, users service will generate new list event. This event will have the following format:
In Redis Streams
+1) "1693307172254-0"
+2) 1) "status"
+ 2) "enabled"
+ 3) "occurred_at"
+ 4) "1693307172254687479"
+ 5) "operation"
+ 6) "user.list"
+ 7) "total"
+ 8) "0"
+ 9) "offset"
+ 10) "0"
+ 11) "limit"
+ 12) "10"
+In Nats JetStreams
+Subject: events.magistrala.users Received: 2023-10-05T18:38:40+03:00
+
+{"limit":10,"occurred_at":1696520320382884278,"offset":0,"operation":"user.list","status":"enabled","total":0}
+In rabbitmq streams
+{"limit":10,"occurred_at":1697532237475179227,"offset":0,"operation":"user.list","status":"enabled","total":0}
+Whenever user list by group is fetched, users service will generate new list_by_group event. This event will have the following format:
1) "1693308952544-0"
+2) 1) "operation"
+ 2) "user.list_by_group"
+ 3) "total"
+ 4) "0"
+ 5) "offset"
+ 6) "0"
+ 7) "limit"
+ 8) "10"
+ 9) "group_id"
+ 10) "bc7fb023-70d5-41aa-bf73-3eab1cf001c9"
+ 11) "status"
+ 12) "enabled"
+ 13) "occurred_at"
+ 14) "1693308952544612677"
+Whenever user is identified, users service will generate new identify event. This event will have the following format:
In Redis Streams
+1) "1693307172168-0"
+2) 1) "operation"
+ 2) "user.identify"
+ 3) "user_id"
+ 4) "e1b982d8-a332-4bc2-aaff-4bbaa86880fc"
+ 5) "occurred_at"
+ 6) "1693307172167980303"
+In Nats JetStreams
+Subject: events.magistrala.users Received: 2023-10-05T15:11:22+03:00
+
+{"occurred_at":1696507882455392181,"operation":"user.identify","user_id":"733005f5-7a69-4da3-adac-1ac3bd6fdedf"}
+In rabbitmq streams
+{"occurred_at":1697532232453091076,"operation":"user.identify","user_id":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4"}
+Whenever user reset token is generated, users service will generate new generate_reset_token event. This event will have the following format:
1) "1693310458376-0"
+2) 1) "operation"
+ 2) "user.generate_reset_token"
+ 3) "email"
+ 4) "rodneydav@gmail.com"
+ 5) "host"
+ 6) "http://localhost"
+ 7) "occurred_at"
+ 8) "1693310458376066097"
+Whenever user token is issued, users service will generate new issue_token event. This event will have the following format:
In Redis Streams
+1) "1693307171987-0"
+2) 1) "operation"
+ 2) "user.issue_token"
+ 3) "identity"
+ 4) "-small-flower@email.com"
+ 5) "occurred_at"
+ 6) "1693307171987023095"
+In Nats JetStreams
+Subject: events.magistrala.users Received: 2023-10-05T14:59:02+03:00
+
+{"identity":"admin@example.com","occurred_at":1696507142218064965,"operation":"user.issue_token"}
+In rabbitmq streams
+{"identity":"-Comley-Hipp@email.com","occurred_at":1697532232391996417,"operation":"user.issue_token"}
+Whenever user token is refreshed, users service will generate new refresh_token event. This event will have the following format:
1) "1693309886622-0"
+2) 1) "operation"
+ 2) "user.refresh_token"
+ 3) "occurred_at"
+ 4) "1693309886622414715"
+Whenever user secret is reset, users service will generate new reset_secret event. This event will have the following format:
1) "1693311075789-0"
+2) 1) "operation"
+ 2) "user.update_secret"
+ 3) "updated_by"
+ 4) "34591d29-13eb-49f8-995b-e474911eeb8a"
+ 5) "name"
+ 6) "rodney"
+ 7) "created_at"
+ 8) "2023-08-29T11:59:51.456429Z"
+ 9) "status"
+ 10) "enabled"
+ 11) "occurred_at"
+ 12) "1693311075789446621"
+ 13) "updated_at"
+ 14) "2023-08-29T12:11:15.785039Z"
+ 15) "id"
+ 16) "34591d29-13eb-49f8-995b-e474911eeb8a"
+ 17) "identity"
+ 18) "rodneydav@gmail.com"
+ 19) "metadata"
+ 20) "{}"
+Whenever user instance is updated, users service will generate new update event. This event will have the following format:
In Redis
+1) "1693307172308-0"
+2) 1) "operation"
+ 2) "user.update"
+ 3) "updated_by"
+ 4) "e1b982d8-a332-4bc2-aaff-4bbaa86880fc"
+ 5) "id"
+ 6) "56d2a797-dcb9-4fab-baf9-7c75e707b2c0"
+ 7) "metadata"
+ 8) "{\"Update\":\"rough-leaf\"}"
+ 9) "updated_at"
+ 10) "2023-08-29T11:06:12.294444Z"
+ 11) "name"
+ 12) "fragrant-voice"
+ 13) "identity"
+ 14) "-snowy-wave@email.com"
+ 15) "created_at"
+ 16) "2023-08-29T11:06:12.032254Z"
+ 17) "status"
+ 18) "enabled"
+ 19) "occurred_at"
+ 20) "1693307172308305030"
+In Nats JetStreams
+Subject: events.magistrala.users Received: 2023-10-05T18:38:40+03:00
+
+{"created_at":"2023-10-05T15:38:40.219889Z","id":"d4baecb8-adfa-4c7c-8257-deea5d7f9dba","identity":"-Hilling-Karole@email.com","metadata":"eyJVcGRhdGUiOiJBbGVncmlhLVdvbGwifQ==","name":"Rhude-Parrillo","occurred_at":1696520320510448519,"operation":"user.update","status":"enabled","updated_at":"2023-10-05T15:38:40.500637Z","updated_by":"3264e965-3fe5-4d4e-a857-48de43551d2e"}
+In rabbitmq streams
+{"created_at":"2023-10-17T08:43:52.441962Z","id":"00397996-f7e0-4035-9271-3e48ee66d525","identity":"-Yanish-Lanctot@email.com","metadata":"eyJVcGRhdGUiOiJWZW5hYmxlcy1IeW1hbiJ9","name":"Eicholtz-Stallabrass","occurred_at":1697532242530064639,"operation":"user.update","status":"enabled","updated_at":"2023-10-17T08:44:02.519652Z","updated_by":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4"}
+Whenever user identity is updated, users service will generate new update_identity event. This event will have the following format:
In Redis
+1) "1693307172321-0"
+2) 1) "metadata"
+ 2) "{\"Update\":\"rough-leaf\"}"
+ 3) "created_at"
+ 4) "2023-08-29T11:06:12.032254Z"
+ 5) "status"
+ 6) "enabled"
+ 7) "updated_at"
+ 8) "2023-08-29T11:06:12.310276Z"
+ 9) "updated_by"
+ 10) "e1b982d8-a332-4bc2-aaff-4bbaa86880fc"
+ 11) "id"
+ 12) "56d2a797-dcb9-4fab-baf9-7c75e707b2c0"
+ 13) "name"
+ 14) "fragrant-voice"
+ 15) "operation"
+ 16) "user.update_identity"
+ 17) "identity"
+ 18) "wandering-brook"
+ 19) "occurred_at"
+ 20) "1693307172320906479"
+In Nats JetStream
+Subject: events.magistrala.users Received: 2023-10-05T18:38:40+03:00
+
+{"created_at":"2023-10-05T15:38:40.219889Z","id":"d4baecb8-adfa-4c7c-8257-deea5d7f9dba","identity":"Andes-Bahgat","metadata":"eyJVcGRhdGUiOiJBbGVncmlhLVdvbGwifQ==","name":"Rhude-Parrillo","occurred_at":1696520320527730565,"operation":"user.update_identity","status":"enabled","updated_at":"2023-10-05T15:38:40.518477Z","updated_by":"3264e965-3fe5-4d4e-a857-48de43551d2e"}
+In rabbitmq streams
+{"created_at":"2023-10-17T08:43:52.441962Z","id":"00397996-f7e0-4035-9271-3e48ee66d525","identity":"Cris-Unkles","metadata":"eyJVcGRhdGUiOiJWZW5hYmxlcy1IeW1hbiJ9","name":"Eicholtz-Stallabrass","occurred_at":1697532242534643742,"operation":"user.update_identity","status":"enabled","updated_at":"2023-10-17T08:44:02.532198Z","updated_by":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4"}
+Whenever user tags are updated, users service will generate new update_tags event. This event will have the following format:
In Redis
+1) "1693307172332-0"
+2) 1) "name"
+ 2) "fragrant-voice"
+ 3) "identity"
+ 4) "wandering-brook"
+ 5) "metadata"
+ 6) "{\"Update\":\"rough-leaf\"}"
+ 7) "status"
+ 8) "enabled"
+ 9) "updated_at"
+ 10) "2023-08-29T11:06:12.323039Z"
+ 11) "updated_by"
+ 12) "e1b982d8-a332-4bc2-aaff-4bbaa86880fc"
+ 13) "id"
+ 14) "56d2a797-dcb9-4fab-baf9-7c75e707b2c0"
+ 15) "occurred_at"
+ 16) "1693307172332766275"
+ 17) "operation"
+ 18) "user.update_tags"
+ 19) "tags"
+ 20) "[patient-thunder]"
+ 21) "created_at"
+ 22) "2023-08-29T11:06:12.032254Z"
+In Nats JetStreams
+Subject: events.magistrala.users Received: 2023-10-05T18:38:40+03:00
+
+{"created_at":"2023-10-05T15:38:40.219889Z","id":"d4baecb8-adfa-4c7c-8257-deea5d7f9dba","identity":"Andes-Bahgat","metadata":"eyJVcGRhdGUiOiJBbGVncmlhLVdvbGwifQ==","name":"Rhude-Parrillo","occurred_at":1696520320537588492,"operation":"user.update_tags","status":"enabled","tags":"[Tischler-Persechino]","updated_at":"2023-10-05T15:38:40.533159Z","updated_by":"3264e965-3fe5-4d4e-a857-48de43551d2e"}
+In rabbitmq streams
+{"created_at":"2023-10-17T08:43:52.441962Z","id":"00397996-f7e0-4035-9271-3e48ee66d525","identity":"Cris-Unkles","metadata":"eyJVcGRhdGUiOiJWZW5hYmxlcy1IeW1hbiJ9","name":"Eicholtz-Stallabrass","occurred_at":1697532242539607155,"operation":"user.update_tags","status":"enabled","tags":"[Jagdish-Daneshzadeh]","updated_at":"2023-10-17T08:44:02.536891Z","updated_by":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4"}
+Whenever user instance changes state in the system, users service will generate and publish new remove event. This event will have the following format:
In Redis Streams
+1) "1693307172345-0"
+2) 1) "operation"
+ 2) "user.remove"
+ 3) "id"
+ 4) "56d2a797-dcb9-4fab-baf9-7c75e707b2c0"
+ 5) "status"
+ 6) "disabled"
+ 7) "updated_at"
+ 8) "2023-08-29T11:06:12.323039Z"
+ 9) "updated_by"
+ 10) "e1b982d8-a332-4bc2-aaff-4bbaa86880fc"
+ 11) "occurred_at"
+ 12) "1693307172345419824"
+
+1) "1693307172359-0"
+2) 1) "id"
+ 2) "56d2a797-dcb9-4fab-baf9-7c75e707b2c0"
+ 3) "status"
+ 4) "enabled"
+ 5) "updated_at"
+ 6) "2023-08-29T11:06:12.323039Z"
+ 7) "updated_by"
+ 8) "e1b982d8-a332-4bc2-aaff-4bbaa86880fc"
+ 9) "occurred_at"
+ 10) "1693307172359445655"
+ 11) "operation"
+ 12) "user.remove"
+In Nats JetStream
+Subject: events.magistrala.users Received: 2023-10-05T17:59:29+03:00
+
+{"id":"0a5f2e21-1a8b-460e-bfa9-732e570df095","occurred_at":1696517969187562377,"operation":"user.remove","status":"disabled","updated_at":"0001-01-01T00:00:00Z","updated_by":""}
+In rabbitmq streams
+{"id":"00397996-f7e0-4035-9271-3e48ee66d525","occurred_at":1697532242544658933,"operation":"user.remove","status":"disabled","updated_at":"2023-10-17T08:44:02.536891Z","updated_by":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4"}
+Whenever group is created, users service will generate new create event. This event will have the following format:
In Redis Streams
+1) "1693307172153-0"
+2) 1) "name"
+ 2) "-fragrant-resonance"
+ 3) "metadata"
+ 4) "{}"
+ 5) "occurred_at"
+ 6) "1693307172152850138"
+ 7) "operation"
+ 8) "group.create"
+ 9) "id"
+ 10) "bc7fb023-70d5-41aa-bf73-3eab1cf001c9"
+ 11) "status"
+ 12) "enabled"
+ 13) "created_at"
+ 14) "2023-08-29T11:06:12.129484Z"
+ 15) "owner"
+ 16) "e1b982d8-a332-4bc2-aaff-4bbaa86880fc"
+As you can see from this example, every odd field represents field name while every even field represents field value. This is standard event format for Redis Streams. If you want to extract metadata field from this event, you'll have to read it as string first and then you can deserialize it to some structured format.
In Nats JetStream
+Subject: events.magistrala.users Received: 2023-10-05T16:12:00+03:00
+
+{"created_at":"2023-10-05T13:12:00.88713Z","id":"f565885c-826d-4c4c-9277-a3c8537aadff","metadata":"e30=","name":"cyclopes","occurred_at":1696511520897093737,"operation":"group.create","owner":"97466511-6317-4c98-8d58-7bd78bcaf587","status":"enabled"}
+In rabbitmq streams
+{"created_at":"2023-10-17T08:43:52.447093Z","id":"d9c550f9-e644-453b-af5b-1ab7e9a99d9d","metadata":"e30=","name":"-Zarella-Knobeloch","occurred_at":1697532232450859819,"operation":"group.create","owner":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4","status":"enabled"}
+Whenever group instance is updated, users service will generate new update event. This event will have the following format:
In Redis
+1) "1693307172445-0"
+2) 1) "operation"
+ 2) "group.update"
+ 3) "owner"
+ 4) "e1b982d8-a332-4bc2-aaff-4bbaa86880fc"
+ 5) "name"
+ 6) "young-paper"
+ 7) "occurred_at"
+ 8) "1693307172445370750"
+ 9) "updated_at"
+ 10) "2023-08-29T11:06:12.429555Z"
+ 11) "updated_by"
+ 12) "e1b982d8-a332-4bc2-aaff-4bbaa86880fc"
+ 13) "id"
+ 14) "bc7fb023-70d5-41aa-bf73-3eab1cf001c9"
+ 15) "metadata"
+ 16) "{\"Update\":\"spring-wood\"}"
+ 17) "created_at"
+ 18) "2023-08-29T11:06:12.129484Z"
+ 19) "status"
+ 20) "enabled"
+In Nats JetStreams
+Subject: events.magistrala.users Received: 2023-10-05T18:38:40+03:00
+
+{"created_at":"2023-10-05T15:38:40.22859Z","id":"cabccc8d-937b-4d92-832b-48d7a466e19e","metadata":"eyJVcGRhdGUiOiJSZWdvLVJlZHdheSJ9","name":"Reiger-Cheal","occurred_at":1696520320573615888,"operation":"group.update","owner":"3264e965-3fe5-4d4e-a857-48de43551d2e","status":"enabled","updated_at":"2023-10-05T15:38:40.56848Z","updated_by":"3264e965-3fe5-4d4e-a857-48de43551d2e"}
+In rabbitmq streams
+{"created_at":"2023-10-17T08:43:52.447093Z","id":"d9c550f9-e644-453b-af5b-1ab7e9a99d9d","metadata":"eyJVcGRhdGUiOiJCaW5ueS1TYXZhZGtvdWhpIn0=","name":"Knick-Doskas","occurred_at":1697532242554817001,"operation":"group.update","owner":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4","status":"enabled","updated_at":"2023-10-17T08:44:02.552421Z","updated_by":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4"}
+Whenever group is viewed, users service will generate new view event. This event will have the following format:
In Redis Streams
+1) "1693307172257-0"
+2) 1) "occurred_at"
+ 2) "1693307172257041358"
+ 3) "operation"
+ 4) "group.view"
+ 5) "id"
+ 6) "bc7fb023-70d5-41aa-bf73-3eab1cf001c9"
+ 7) "owner"
+ 8) "e1b982d8-a332-4bc2-aaff-4bbaa86880fc"
+ 9) "name"
+ 10) "-fragrant-resonance"
+ 11) "metadata"
+ 12) "{}"
+ 13) "created_at"
+ 14) "2023-08-29T11:06:12.129484Z"
+ 15) "status"
+ 16) "enabled"
+In Nats JetStream
+Subject: events.magistrala.users Received: 2023-10-05T18:38:40+03:00
+
+{"created_at":"2023-10-05T15:38:40.22859Z","id":"cabccc8d-937b-4d92-832b-48d7a466e19e","metadata":"e30=","name":"-Conneely-Chiang","occurred_at":1696520320392002702,"operation":"group.view","owner":"3264e965-3fe5-4d4e-a857-48de43551d2e","status":"enabled"}
+In rabbitmq streams
+{"created_at":"2023-10-17T08:43:52.447093Z","id":"d9c550f9-e644-453b-af5b-1ab7e9a99d9d","metadata":"e30=","name":"-Zarella-Knobeloch","occurred_at":1697532237477518526,"operation":"group.view","owner":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4","status":"enabled"}
+Whenever group list is fetched, users service will generate new list event. This event will have the following format:
In Redis
+1) "1693307172264-0"
+2) 1) "occurred_at"
+ 2) "1693307172264183217"
+ 3) "operation"
+ 4) "group.list"
+ 5) "total"
+ 6) "0"
+ 7) "offset"
+ 8) "0"
+ 9) "limit"
+ 10) "10"
+ 11) "status"
+ 12) "enabled"
+In Nats JetStreams
+Subject: events.magistrala.users Received: 2023-10-05T19:41:01+03:00
+
+{"limit":100,"occurred_at":1696524061330756963,"offset":0,"operation":"group.list","status":"all","total":0}
+In rabbitmq streams
+{"limit":10,"occurred_at":1697532237481188226,"offset":0,"operation":"group.list","status":"enabled","total":0}
+Whenever group list by user is fetched, users service will generate new list_by_user event. This event will have the following format:
In Redis
+1) "1693308937283-0"
+2) 1) "limit"
+ 2) "10"
+ 3) "channel_id"
+ 4) "bb1a7b38-cd79-410d-aca7-e744decd7b8e"
+ 5) "status"
+ 6) "enabled"
+ 7) "occurred_at"
+ 8) "1693308937282933017"
+ 9) "operation"
+ 10) "group.list_by_user"
+ 11) "total"
+ 12) "0"
+ 13) "offset"
+ 14) "0"
+In Nats JetStreams
+Subject: events.magistrala.users Received: 2023-10-05T19:41:01+03:00
+
+{"limit":100,"occurred_at":1696524061330756963,"offset":0,"operation":"group.list","status":"all","total":0}
+Whenever group instance changes state in the system, users service will generate and publish new remove event. This event will have the following format:
In Redis
+1) "1693307172460-0"
+2) 1) "updated_by"
+ 2) "e1b982d8-a332-4bc2-aaff-4bbaa86880fc"
+ 3) "occurred_at"
+ 4) "1693307172459828786"
+ 5) "operation"
+ 6) "group.remove"
+ 7) "id"
+ 8) "bc7fb023-70d5-41aa-bf73-3eab1cf001c9"
+ 9) "status"
+ 10) "disabled"
+ 11) "updated_at"
+ 12) "2023-08-29T11:06:12.429555Z"
+
+1) "1693307172473-0"
+2) 1) "id"
+ 2) "bc7fb023-70d5-41aa-bf73-3eab1cf001c9"
+ 3) "status"
+ 4) "enabled"
+ 5) "updated_at"
+ 6) "2023-08-29T11:06:12.429555Z"
+ 7) "updated_by"
+ 8) "e1b982d8-a332-4bc2-aaff-4bbaa86880fc"
+ 9) "occurred_at"
+ 10) "1693307172473661564"
+ 11) "operation"
+ 12) "group.remove"
+In Nats JetStreams
+Subject: events.magistrala.users Received: 2023-10-05T18:38:40+03:00
+
+{"id":"cabccc8d-937b-4d92-832b-48d7a466e19e","occurred_at":1696520320583695115,"operation":"group.remove","status":"disabled","updated_at":"2023-10-05T15:38:40.56848Z","updated_by":"3264e965-3fe5-4d4e-a857-48de43551d2e"}
+
+Subject: events.magistrala.users Received: 2023-10-05T18:38:40+03:00
+
+{"id":"cabccc8d-937b-4d92-832b-48d7a466e19e","occurred_at":1696520320592509139,"operation":"group.remove","status":"enabled","updated_at":"2023-10-05T15:38:40.56848Z","updated_by":"3264e965-3fe5-4d4e-a857-48de43551d2e"}
+In rabbitmq streams
+{"id":"d9c550f9-e644-453b-af5b-1ab7e9a99d9d","occurred_at":1697532242559729288,"operation":"group.remove","status":"disabled","updated_at":"2023-10-17T08:44:02.552421Z","updated_by":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4"}
+Whenever policy is authorized, users service will generate new authorize event. This event will have the following format:
In Redis Streams
+1) "1693311470724-0"
+2) 1) "entity_type"
+ 2) "thing"
+ 3) "object"
+ 4) "8a85e2d5-e783-43ee-8bea-d6d0f8039e78"
+ 5) "actions"
+ 6) "c_list"
+ 7) "occurred_at"
+ 8) "1693311470724174126"
+ 9) "operation"
+ 10) "policies.authorize"
+In Nats JetStreams
+Subject: events.magistrala.users Received: 2023-10-05T15:12:07+03:00
+
+{"action":"c_list","entity_type":"client","object":"things","occurred_at":1696507927648459930,"operation":"policies.authorize"}
+In rabbitmq streams
+{"action":"g_list","entity_type":"group","object":"things","occurred_at":1697536686571995884,"operation":"policies.authorize"}
+Whenever policy is added, users service will generate new add event. This event will have the following format:
In Redis Streams
+1) "1693311470721-0"
+2) 1) "operation"
+ 2) "policies.add"
+ 3) "owner_id"
+ 4) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 5) "subject"
+ 6) "12510af8-b6a7-410d-944c-9feded199d6d"
+ 7) "object"
+ 8) "8a85e2d5-e783-43ee-8bea-d6d0f8039e78"
+ 9) "actions"
+ 10) "[g_add,c_list]"
+ 11) "created_at"
+ 12) "2023-08-29T12:17:50.715541Z"
+ 13) "occurred_at"
+ 14) "1693311470721394773"
+In Nats JetStreams
+Subject: events.magistrala.users Received: 2023-10-05T16:13:21+03:00
+
+{"actions":"[m_read]","object":"f565885c-826d-4c4c-9277-a3c8537aadff","occurred_at":1696511601827118557,"operation":"policies.add","subject":"0a5f2e21-1a8b-460e-bfa9-732e570df095"}
+In rabbitmq streams
+{"actions":"[c_list,c_update,c_delete]","object":"d9c550f9-e644-453b-af5b-1ab7e9a99d9d","occurred_at":1697538038965204061,"operation":"policies.add","subject":"df0a29f6-df00-46e8-bde5-cbdf418510da"}
+Whenever policy is updated, users service will generate new update event. This event will have the following format:
In Redis Streams
+1) "1693312500101-0"
+2) 1) "updated_at"
+ 2) "2023-08-29T12:35:00.095028Z"
+ 3) "occurred_at"
+ 4) "1693312500101367995"
+ 5) "operation"
+ 6) "policies.update"
+ 7) "owner_id"
+ 8) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 9) "subject"
+ 10) "12510af8-b6a7-410d-944c-9feded199d6d"
+ 11) "object"
+ 12) "8a85e2d5-e783-43ee-8bea-d6d0f8039e78"
+ 13) "actions"
+ 14) "[g_add,c_list]"
+ 15) "created_at"
+ 16) "2023-08-29T12:17:50.715541Z"
+In rabbitmq streams
+{"actions":"[g_list]","object":"0f551d14-0efe-4b5e-bc96-b9cd834e91d1","occurred_at":1697538056924321702,"operation":"policies.update","subject":"bd3f51f6-d84e-41d9-8205-f725c6b5e774"}
+Whenever policy is removed, users service will generate new remove event. This event will have the following format:
In Redis Streams
+1) "1693312590631-0"
+2) 1) "occurred_at"
+ 2) "1693312590631691388"
+ 3) "operation"
+ 4) "policies.delete"
+ 5) "subject"
+ 6) "12510af8-b6a7-410d-944c-9feded199d6d"
+ 7) "object"
+ 8) "8a85e2d5-e783-43ee-8bea-d6d0f8039e78"
+ 9) "actions"
+ 10) "[g_add,c_list]"
+In rabbitmq streams
+{"object":"0f551d14-0efe-4b5e-bc96-b9cd834e91d1","occurred_at":1697538064359063507,"operation":"policies.delete","subject":"bd3f51f6-d84e-41d9-8205-f725c6b5e774"}
+Whenever policy list is fetched, things service will generate new list event. This event will have the following format:
In Redis Streams
+1) "1693312633649-0"
+2) 1) "operation"
+ 2) "policies.list"
+ 3) "total"
+ 4) "0"
+ 5) "limit"
+ 6) "10"
+ 7) "offset"
+ 8) "0"
+ 9) "occurred_at"
+ 10) "1693312633649171129"
+In rabbitmq streams
+{"limit":10,"occurred_at":1697536690236286573,"offset":0,"operation":"policies.list","total":0}
+For every operation that has side effects (that is changing service state) things service will generate new event and publish it to Redis Stream called magistrala.things. Every event has its own event ID that is automatically generated and operation field that can have one of the following values:
thing.create for thing creationthing.update for thing updatething.remove for thing change of statething.view for thing viewthing.list for listing thingsthing.list_by_channel for listing things by channelthing.identify for thing identificationchannel.create for channel creationchannel.update for channel updatechannel.remove for channel change of statechannel.view for channel viewchannel.list for listing channelschannel.list_by_thing for listing channels by thingpolicy.authorize for policy authorizationpolicy.add for policy creationpolicy.update for policy updatepolicy.remove for policy deletionpolicy.list for listing policiesBy fetching and processing these events you can reconstruct things service state. If you store some of your custom data in metadata field, this is the perfect way to fetch it and process it. If you want to integrate through docker-compose.yml you can use magistrala-es-redis service. Just connect to it and consume events from Redis Stream named magistrala.things.
Whenever thing is created, things service will generate new create event. This event will have the following format:
In Redis Streams
+1) 1) "1693311470576-0"
+2) 1) "operation"
+ 2) "thing.create"
+ 3) "id"
+ 4) "12510af8-b6a7-410d-944c-9feded199d6d"
+ 5) "status"
+ 6) "enabled"
+ 7) "created_at"
+ 8) "2023-08-29T12:17:50.566453Z"
+ 9) "name"
+ 10) "-broken-cloud"
+ 11) "owner"
+ 12) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 13) "metadata"
+ 14) "{}"
+ 15) "occurred_at"
+ 16) "1693311470576589894"
+As you can see from this example, every odd field represents field name while every even field represents field value. This is standard event format for Redis Streams. If you want to extract metadata field from this event, you'll have to read it as string first and then you can deserialize it to some structured format.
In Nats JetStreams
+Subject: events.magistrala.things Received: 2023-10-05T15:41:04+03:00
+
+{"created_at":"2023-10-05T12:41:04.833207Z","id":"9745f2ea-f776-46b1-9b44-1cfd1ad4c6f1","metadata":"e30=","name":"d0","occurred_at":1696509664860397827,"operation":"thing.create","owner":"0a5f2e21-1a8b-460e-bfa9-732e570df095","status":"enabled"}
+In rabbitmq streams
+{"created_at":"2023-10-17T08:43:52.453621Z","id":"83c884cc-51b7-40ab-a98f-83ea93f4cdd6","metadata":"e30=","name":"-Maune-Tuttle","occurred_at":1697532232459167722,"operation":"thing.create","owner":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4","status":"enabled"}
+Whenever thing instance is updated, things service will generate new update event. This event will have the following format:
In Redis
+1) "1693311470669-0"
+2) 1) "operation"
+ 2) "thing.update"
+ 3) "updated_at"
+ 4) "2023-08-29T12:17:50.665752Z"
+ 5) "updated_by"
+ 6) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 7) "owner"
+ 8) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 9) "created_at"
+ 10) "2023-08-29T12:17:50.566453Z"
+ 11) "status"
+ 12) "enabled"
+ 13) "id"
+ 14) "12510af8-b6a7-410d-944c-9feded199d6d"
+ 15) "name"
+ 16) "lingering-sea"
+ 17) "metadata"
+ 18) "{\"Update\":\"nameless-glitter\"}"
+ 19) "occurred_at"
+ 20) "1693311470669567023"
+In Nats JetStreams
+Subject: events.magistrala.things Received: 2023-10-05T18:38:40+03:00
+
+{"created_at":"2023-10-05T15:38:40.264564Z","id":"47540f84-029b-436f-89b5-3c10f87e302b","metadata":"eyJVcGRhdGUiOiJCZXJuYXJkLUJyaWNrZXkifQ==","name":"Bence-Jefferson","occurred_at":1696520320614766498,"operation":"thing.update","owner":"3264e965-3fe5-4d4e-a857-48de43551d2e","status":"enabled","updated_at":"2023-10-05T15:38:40.606662Z","updated_by":"3264e965-3fe5-4d4e-a857-48de43551d2e"}
+In rabbitmq streams
+{"created_at":"2023-10-17T08:43:52.453621Z","id":"83c884cc-51b7-40ab-a98f-83ea93f4cdd6","metadata":"eyJVcGRhdGUiOiJTdGV2ZW5zb24tTW9udHNpb24ifQ==","name":"Marner-Shapiro","occurred_at":1697532242575070481,"operation":"thing.update","owner":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4","status":"enabled","updated_at":"2023-10-17T08:44:02.571677Z","updated_by":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4"}
+Whenever thing secret is updated, things service will generate new update_secret event. This event will have the following format:
In Redis
+1) "1693311470676-0"
+2) 1) "id"
+ 2) "12510af8-b6a7-410d-944c-9feded199d6d"
+ 3) "name"
+ 4) "lingering-sea"
+ 5) "metadata"
+ 6) "{\"Update\":\"nameless-glitter\"}"
+ 7) "status"
+ 8) "enabled"
+ 9) "occurred_at"
+ 10) "1693311470676563107"
+ 11) "operation"
+ 12) "thing.update_secret"
+ 13) "updated_at"
+ 14) "2023-08-29T12:17:50.672865Z"
+ 15) "updated_by"
+ 16) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 17) "owner"
+ 18) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 19) "created_at"
+ 20) "2023-08-29T12:17:50.566453Z"
+In Nats JetStreams
+Subject: events.magistrala.things Received: 2023-10-05T18:38:40+03:00
+
+{"created_at":"2023-10-05T15:38:40.264564Z","id":"47540f84-029b-436f-89b5-3c10f87e302b","metadata":"eyJVcGRhdGUiOiJCZXJuYXJkLUJyaWNrZXkifQ==","name":"Bence-Jefferson","occurred_at":1696520320633637049,"operation":"thing.update_secret","owner":"3264e965-3fe5-4d4e-a857-48de43551d2e","status":"enabled","updated_at":"2023-10-05T15:38:40.625663Z","updated_by":"3264e965-3fe5-4d4e-a857-48de43551d2e"}
+In rabbitmq streams
+{"created_at":"2023-10-17T08:43:52.453621Z","id":"83c884cc-51b7-40ab-a98f-83ea93f4cdd6","metadata":"eyJVcGRhdGUiOiJTdGV2ZW5zb24tTW9udHNpb24ifQ==","name":"Marner-Shapiro","occurred_at":1697532242583980252,"operation":"thing.update_secret","owner":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4","status":"enabled","updated_at":"2023-10-17T08:44:02.580896Z","updated_by":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4"}
+Whenever thing tags are updated, things service will generate new update_tags event. This event will have the following format:
In Redis
+1) "1693311470682-0"
+2) 1) "operation"
+ 2) "thing.update_tags"
+ 3) "owner"
+ 4) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 5) "metadata"
+ 6) "{\"Update\":\"nameless-glitter\"}"
+ 7) "status"
+ 8) "enabled"
+ 9) "occurred_at"
+ 10) "1693311470682522926"
+ 11) "updated_at"
+ 12) "2023-08-29T12:17:50.679301Z"
+ 13) "updated_by"
+ 14) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 15) "id"
+ 16) "12510af8-b6a7-410d-944c-9feded199d6d"
+ 17) "name"
+ 18) "lingering-sea"
+ 19) "tags"
+ 20) "[morning-pine]"
+ 21) "created_at"
+ 22) "2023-08-29T12:17:50.566453Z"
+In Nats JetStreams
+Subject: events.magistrala.things Received: 2023-10-05T18:38:40+03:00
+
+{"created_at":"2023-10-05T15:38:40.264564Z","id":"47540f84-029b-436f-89b5-3c10f87e302b","metadata":"eyJVcGRhdGUiOiJCZXJuYXJkLUJyaWNrZXkifQ==","name":"Bence-Jefferson","occurred_at":1696520320651750298,"operation":"thing.update_tags","owner":"3264e965-3fe5-4d4e-a857-48de43551d2e","status":"enabled","tags":"[Kac-Kimma]","updated_at":"2023-10-05T15:38:40.643285Z","updated_by":"3264e965-3fe5-4d4e-a857-48de43551d2e"}
+In rabbitmq streams
+{"created_at":"2023-10-17T08:43:52.453621Z","id":"83c884cc-51b7-40ab-a98f-83ea93f4cdd6","metadata":"eyJVcGRhdGUiOiJTdGV2ZW5zb24tTW9udHNpb24ifQ==","name":"Marner-Shapiro","occurred_at":1697532242593091501,"operation":"thing.update_tags","owner":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4","status":"enabled","tags":"[Donni-Planting]","updated_at":"2023-10-17T08:44:02.589939Z","updated_by":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4"}
+Whenever thing instance is removed from the system, things service will generate and publish new remove event. This event will have the following format:
In Redis Streams
+1) "1693311470689-0"
+2) 1) "updated_by"
+ 2) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 3) "occurred_at"
+ 4) "1693311470688911826"
+ 5) "operation"
+ 6) "thing.remove"
+ 7) "id"
+ 8) "12510af8-b6a7-410d-944c-9feded199d6d"
+ 9) "status"
+ 10) "disabled"
+ 11) "updated_at"
+ 12) "2023-08-29T12:17:50.679301Z"
+
+1) "1693311470695-0"
+2) 1) "operation"
+ 2) "thing.remove"
+ 3) "id"
+ 4) "12510af8-b6a7-410d-944c-9feded199d6d"
+ 5) "status"
+ 6) "enabled"
+ 7) "updated_at"
+ 8) "2023-08-29T12:17:50.679301Z"
+ 9) "updated_by"
+ 10) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 11) "occurred_at"
+ 12) "1693311470695446948"
+In Nats JetsStreams
+Subject: events.magistrala.things Received: 2023-10-05T18:38:40+03:00
+
+{"id":"47540f84-029b-436f-89b5-3c10f87e302b","occurred_at":1696520320674227458,"operation":"thing.remove","status":"disabled","updated_at":"2023-10-05T15:38:40.643285Z","updated_by":"3264e965-3fe5-4d4e-a857-48de43551d2e"}
+
+Subject: events.magistrala.things Received: 2023-10-05T18:38:40+03:00
+
+{"id":"47540f84-029b-436f-89b5-3c10f87e302b","occurred_at":1696520320692289306,"operation":"thing.remove","status":"enabled","updated_at":"2023-10-05T15:38:40.643285Z","updated_by":"3264e965-3fe5-4d4e-a857-48de43551d2e"}
+In rabbitmq streams
+{"id":"83c884cc-51b7-40ab-a98f-83ea93f4cdd6","occurred_at":1697532242603326333,"operation":"thing.remove","status":"disabled","updated_at":"2023-10-17T08:44:02.589939Z","updated_by":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4"}
+Whenever thing is viewed, things service will generate new view event. This event will have the following format:
In Redis Streams
+1) "1693311470608-0"
+2) 1) "operation"
+ 2) "thing.view"
+ 3) "id"
+ 4) "12510af8-b6a7-410d-944c-9feded199d6d"
+ 5) "name"
+ 6) "-broken-cloud"
+ 7) "owner"
+ 8) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 9) "metadata"
+ 10) "{}"
+ 11) "created_at"
+ 12) "2023-08-29T12:17:50.566453Z"
+ 13) "status"
+ 14) "enabled"
+ 15) "occurred_at"
+ 16) "1693311470608701504"
+In Nats JetStreams
+Subject: events.magistrala.things Received: 2023-10-05T15:42:00+03:00
+
+{"created_at":"2023-10-05T12:41:04.833207Z","id":"9745f2ea-f776-46b1-9b44-1cfd1ad4c6f1","metadata":"e30=","name":"d0","occurred_at":1696509720925490970,"operation":"thing.view","owner":"0a5f2e21-1a8b-460e-bfa9-732e570df095","status":"enabled"}
+In rabbitmq streams
+{"created_at":"2023-10-17T08:43:52.453621Z","id":"83c884cc-51b7-40ab-a98f-83ea93f4cdd6","metadata":"e30=","name":"-Maune-Tuttle","occurred_at":1697532237490995357,"operation":"thing.view","owner":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4","status":"enabled"}
+Whenever thing list is fetched, things service will generate new list event. This event will have the following format:
In Redis Streams
+1) "1693311470613-0"
+2) 1) "occurred_at"
+ 2) "1693311470613173088"
+ 3) "operation"
+ 4) "thing.list"
+ 5) "total"
+ 6) "0"
+ 7) "offset"
+ 8) "0"
+ 9) "limit"
+ 10) "10"
+ 11) "status"
+ 12) "enabled"
+In Nats JetStreams
+Subject: events.magistrala.things Received: 2023-10-05T18:38:40+03:00
+
+{"limit":10,"occurred_at":1696520320459999484,"offset":0,"operation":"thing.list","status":"enabled","total":0}
+In rabbitmq streams
+{"limit":10,"occurred_at":1697532237496534968,"offset":0,"operation":"thing.list","status":"enabled","total":0}
+Whenever thing list by channel is fetched, things service will generate new list_by_channel event. This event will have the following format:
1) "1693312322620-0"
+2) 1) "operation"
+ 2) "thing.list_by_channel"
+ 3) "total"
+ 4) "0"
+ 5) "offset"
+ 6) "0"
+ 7) "limit"
+ 8) "10"
+ 9) "channel_id"
+ 10) "8d77099e-4911-4140-8555-7d3be65a1694"
+ 11) "status"
+ 12) "enabled"
+ 13) "occurred_at"
+ 14) "1693312322620481072"
+Whenever thing is identified, things service will generate new identify event. This event will have the following format:
In Redis Streams
+1) "1693312391155-0"
+2) 1) "operation"
+ 2) "thing.identify"
+ 3) "thing_id"
+ 4) "dc82d6bf-973b-4582-9806-0230cee11c20"
+ 5) "occurred_at"
+ 6) "1693312391155123548"
+In Nats JetStreams
+Subject: events.magistrala.things Received: 2023-10-05T18:38:40+03:00
+
+{"occurred_at":1696520320934964056,"operation":"thing.identify","thing_id":"47540f84-029b-436f-89b5-3c10f87e302b"}
+Whenever channel instance is created, things service will generate and publish new create event. This event will have the following format:
In Redis Streams
+1) 1) "1693311470584-0"
+2) 1) "owner"
+ 2) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 3) "name"
+ 4) "-frosty-moon"
+ 5) "metadata"
+ 6) "{}"
+ 7) "occurred_at"
+ 8) "1693311470584416323"
+ 9) "operation"
+ 10) "channel.create"
+ 11) "id"
+ 12) "8a85e2d5-e783-43ee-8bea-d6d0f8039e78"
+ 13) "status"
+ 14) "enabled"
+ 15) "created_at"
+ 16) "2023-08-29T12:17:50.57866Z"
+Subject: events.magistrala.things Received: 2023-10-05T15:55:39+03:00
+
+{"created_at":"2023-10-05T12:55:39.175568Z","id":"45eb9f35-1360-4051-81e2-9582433a6607","metadata":"e30=","name":"hephaestus","occurred_at":1696510539182201160,"operation":"channel.create","owner":"97466511-6317-4c98-8d58-7bd78bcaf587","status":"enabled"}
+In rabbitmq streams
+{"created_at":"2023-10-17T08:43:52.461635Z","id":"b9ae027e-9068-4bff-9ced-45a28351a1ce","metadata":"e30=","name":"-Dortch-Peckett","occurred_at":1697532232464401399,"operation":"channel.create","owner":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4","status":"enabled"}
+Whenever channel instance is updated, things service will generate and publish new update event. This event will have the following format:
In Redis Streams
+1) "1693311470701-0"
+2) 1) "updated_by"
+ 2) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 3) "owner"
+ 4) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 5) "created_at"
+ 6) "2023-08-29T12:17:50.57866Z"
+ 7) "status"
+ 8) "enabled"
+ 9) "operation"
+ 10) "channel.update"
+ 11) "updated_at"
+ 12) "2023-08-29T12:17:50.698278Z"
+ 13) "metadata"
+ 14) "{\"Update\":\"silent-hill\"}"
+ 15) "occurred_at"
+ 16) "1693311470701812291"
+ 17) "id"
+ 18) "8a85e2d5-e783-43ee-8bea-d6d0f8039e78"
+ 19) "name"
+ 20) "morning-forest"
+In Nats JetStreams
+Subject: events.magistrala.things Received: 2023-10-05T16:00:29+03:00
+
+{"created_at":"2023-10-05T12:41:04.87198Z","id":"5f9d4b76-0717-4859-8ef8-6fcfb81f44d5","metadata":"e30=","name":"hestia","occurred_at":1696510829837578155,"operation":"channel.update","owner":"0a5f2e21-1a8b-460e-bfa9-732e570df095","status":"enabled","updated_at":"2023-10-05T13:00:29.828132Z","updated_by":"97466511-6317-4c98-8d58-7bd78bcaf587"}
+In rabbitmq streams
+{"created_at":"2023-10-17T08:43:52.461635Z","id":"b9ae027e-9068-4bff-9ced-45a28351a1ce","metadata":"eyJVcGRhdGUiOiJTaXNuZXktS29ra2F0In0=","name":"Pelley-Staffing","occurred_at":1697532242622549294,"operation":"channel.update","owner":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4","status":"enabled","updated_at":"2023-10-17T08:44:02.619432Z","updated_by":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4"}
+Note that update channel event will contain only those fields that were updated using update channel endpoint.
+Whenever channel instance is removed from the system, things service will generate and publish new remove event. This event will have the following format:
In Redis Streams
+1) "1693311470708-0"
+2) 1) "status"
+ 2) "disabled"
+ 3) "updated_at"
+ 4) "2023-08-29T12:17:50.698278Z"
+ 5) "updated_by"
+ 6) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 7) "occurred_at"
+ 8) "1693311470708219296"
+ 9) "operation"
+ 10) "channel.remove"
+ 11) "id"
+ 12) "8a85e2d5-e783-43ee-8bea-d6d0f8039e78"
+
+1) "1693311470714-0"
+2) 1) "occurred_at"
+ 2) "1693311470714118979"
+ 3) "operation"
+ 4) "channel.remove"
+ 5) "id"
+ 6) "8a85e2d5-e783-43ee-8bea-d6d0f8039e78"
+ 7) "status"
+ 8) "enabled"
+ 9) "updated_at"
+ 10) "2023-08-29T12:17:50.698278Z"
+ 11) "updated_by"
+ 12) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+In Nats JetStreams
+Subject: events.magistrala.things Received: 2023-10-05T18:38:40+03:00
+
+{"id":"e4fa015f-bfad-4f41-bebe-142d3e938d3a","occurred_at":1696520320726205997,"operation":"channel.remove","status":"disabled","updated_at":"2023-10-05T15:38:40.702159Z","updated_by":"3264e965-3fe5-4d4e-a857-48de43551d2e"}
+
+Subject: events.magistrala.things Received: 2023-10-05T18:38:40+03:00
+
+{"id":"e4fa015f-bfad-4f41-bebe-142d3e938d3a","occurred_at":1696520320786154457,"operation":"channel.remove","status":"enabled","updated_at":"2023-10-05T15:38:40.702159Z","updated_by":"3264e965-3fe5-4d4e-a857-48de43551d2e"}
+In rabbitmq streams
+{"id":"b9ae027e-9068-4bff-9ced-45a28351a1ce","occurred_at":1697532242632439430,"operation":"channel.remove","status":"disabled","updated_at":"2023-10-17T08:44:02.619432Z","updated_by":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4"}
+Whenever channel is viewed, things service will generate new view event. This event will have the following format:
In Redis Streams
+1) "1693311470615-0"
+2) 1) "id"
+ 2) "8a85e2d5-e783-43ee-8bea-d6d0f8039e78"
+ 3) "owner"
+ 4) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 5) "name"
+ 6) "-frosty-moon"
+ 7) "metadata"
+ 8) "{}"
+ 9) "created_at"
+ 10) "2023-08-29T12:17:50.57866Z"
+ 11) "status"
+ 12) "enabled"
+ 13) "occurred_at"
+ 14) "1693311470615693019"
+ 15) "operation"
+ 16) "channel.view"
+In Nats JetStream +
Subject: events.magistrala.things Received: 2023-10-05T18:38:40+03:00
+
+{"created_at":"2023-10-05T15:38:40.31444Z","id":"e4fa015f-bfad-4f41-bebe-142d3e938d3a","metadata":"e30=","name":"-Cech-Hargreaves","occurred_at":1696520320475816826,"operation":"channel.view","owner":"3264e965-3fe5-4d4e-a857-48de43551d2e","status":"enabled"}
+In rabbitmq streams
+{"created_at":"2023-10-17T08:43:52.461635Z","id":"b9ae027e-9068-4bff-9ced-45a28351a1ce","metadata":"e30=","name":"-Dortch-Peckett","occurred_at":1697532237504403729,"operation":"channel.view","owner":"b2941bd6-ca98-4c56-8d94-05fbf6a967c4","status":"enabled"}
+Whenever channel list is fetched, things service will generate new list event. This event will have the following format:
In Redis
+1) "1693311470619-0"
+2) 1) "limit"
+ 2) "10"
+ 3) "status"
+ 4) "enabled"
+ 5) "occurred_at"
+ 6) "1693311470619194337"
+ 7) "operation"
+ 8) "channel.list"
+ 9) "total"
+ 10) "0"
+ 11) "offset"
+ 12) "0"
+In Nats JetStreams
+Subject: events.magistrala.things Received: 2023-10-05T18:38:40+03:00
+
+{"limit":10,"occurred_at":1696520320495779280,"offset":0,"operation":"channel.list","status":"enabled","total":0}
+In rabbitmq streams
+{"limit":10,"occurred_at":1697532237512138449,"offset":0,"operation":"channel.list","status":"enabled","total":0}
+Whenever channel list by thing is fetched, things service will generate new list_by_thing event. This event will have the following format:
1) "1693312299484-0"
+2) 1) "occurred_at"
+ 2) "1693312299484000183"
+ 3) "operation"
+ 4) "channel.list_by_thing"
+ 5) "total"
+ 6) "0"
+ 7) "offset"
+ 8) "0"
+ 9) "limit"
+ 10) "10"
+ 11) "thing_id"
+ 12) "dc82d6bf-973b-4582-9806-0230cee11c20"
+ 13) "status"
+ 14) "enabled"
+Whenever policy is authorized, things service will generate new authorize event. This event will have the following format:
In Redis Streams
+1) "1693311470724-0"
+2) 1) "entity_type"
+ 2) "thing"
+ 3) "object"
+ 4) "8a85e2d5-e783-43ee-8bea-d6d0f8039e78"
+ 5) "actions"
+ 6) "m_read"
+ 7) "occurred_at"
+ 8) "1693311470724174126"
+ 9) "operation"
+ 10) "policies.authorize"
+In Nats JetStreams
+Subject: events.magistrala.things Received: 2023-10-05T18:38:40+03:00
+
+{"actions":"m_write","entity_type":"thing","object":"e4fa015f-bfad-4f41-bebe-142d3e938d3a","occurred_at":1696520320938561965,"operation":"policies.authorize"}
+In Rabbitmq streams
+{"action":"c_list","entity_type":"client","object":"things","occurred_at":1697536682155214702,"operation":"policies.authorize"}
+Whenever policy is added, things service will generate new add event. This event will have the following format:
In Redis Streams
+1) "1693311470721-0"
+2) 1) "operation"
+ 2) "policies.add"
+ 3) "owner_id"
+ 4) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 5) "subject"
+ 6) "12510af8-b6a7-410d-944c-9feded199d6d"
+ 7) "object"
+ 8) "8a85e2d5-e783-43ee-8bea-d6d0f8039e78"
+ 9) "actions"
+ 10) "[m_write,m_read]"
+ 11) "created_at"
+ 12) "2023-08-29T12:17:50.715541Z"
+ 13) "occurred_at"
+ 14) "1693311470721394773"
+In Nats JetStreams
+Subject: events.magistrala.things Received: 2023-10-05T15:41:04+03:00
+
+{"actions":"[m_write,m_read]","created_at":"2023-10-05T12:41:04.901355Z","object":"5f9d4b76-0717-4859-8ef8-6fcfb81f44d5","occurred_at":1696509664928590911,"operation":"policies.add","owner_id":"0a5f2e21-1a8b-460e-bfa9-732e570df095","subject":"9745f2ea-f776-46b1-9b44-1cfd1ad4c6f1"}
+In rabbitmq streams
+{"actions":"[m_read]","created_at":"2023-10-17T10:19:44.704766Z","object":"b9ae027e-9068-4bff-9ced-45a28351a1ce","occurred_at":1697537984720752374,"operation":"policies.add","owner_id":"9eef7587-fb81-4db9-bedd-2ccc4f680532","subject":"87b6a3c2-e498-477f-813e-102a97ae0141"}
+Whenever policy is updated, things service will generate new update event. This event will have the following format:
In Redis Streams
+1) "1693312500101-0"
+2) 1) "updated_at"
+ 2) "2023-08-29T12:35:00.095028Z"
+ 3) "occurred_at"
+ 4) "1693312500101367995"
+ 5) "operation"
+ 6) "policies.update"
+ 7) "owner_id"
+ 8) "fe2e5de0-9900-4ac5-b364-eae0c35777fb"
+ 9) "subject"
+ 10) "12510af8-b6a7-410d-944c-9feded199d6d"
+ 11) "object"
+ 12) "8a85e2d5-e783-43ee-8bea-d6d0f8039e78"
+ 13) "actions"
+ 14) "[m_write,m_read]"
+ 15) "created_at"
+ 16) "2023-08-29T12:17:50.715541Z"
+In rabbitmq streams
+{"actions":"[m_read,m_write]","created_at":"2023-10-17T10:19:44.704766Z","object":"b9ae027e-9068-4bff-9ced-45a28351a1ce","occurred_at":1697537993401523330,"operation":"policies.update","owner_id":"9eef7587-fb81-4db9-bedd-2ccc4f680532","subject":"87b6a3c2-e498-477f-813e-102a97ae0141","updated_at":"2023-10-17T10:19:53.388419Z"}
+Whenever policy is removed, things service will generate new remove event. This event will have the following format:
In Redis Streams
+1) "1693312590631-0"
+2) 1) "occurred_at"
+ 2) "1693312590631691388"
+ 3) "operation"
+ 4) "policies.delete"
+ 5) "subject"
+ 6) "12510af8-b6a7-410d-944c-9feded199d6d"
+ 7) "object"
+ 8) "8a85e2d5-e783-43ee-8bea-d6d0f8039e78"
+ 9) "actions"
+ 10) "[m_write,m_read]"
+In Nats JetStreams
+Subject: events.magistrala.things Received: 2023-10-05T16:05:15+03:00
+
+{"actions":"[m_write,m_read]","object":"5f9d4b76-0717-4859-8ef8-6fcfb81f44d5","occurred_at":1696511115507500254,"operation":"policies.delete","subject":"9745f2ea-f776-46b1-9b44-1cfd1ad4c6f1"}
+In rabbitmq streams
+{"actions":"[m_write,m_read]","object":"053a122e-ea3b-42de-9f5f-9eef7e19491c","occurred_at":1697537995704396106,"operation":"policies.delete","subject":"323a1297-5ca3-425e-b08f-b4bdf0925f10"}
+Whenever policy list is fetched, things service will generate new list event. This event will have the following format:
In Redis Streams
+1) "1693312633649-0"
+2) 1) "operation"
+ 2) "policies.list"
+ 3) "total"
+ 4) "0"
+ 5) "limit"
+ 6) "10"
+ 7) "offset"
+ 8) "0"
+ 9) "occurred_at"
+ 10) "1693312633649171129"
+In rabbitmq streams
+{"limit":10,"occurred_at":1697536686568649276,"offset":0,"operation":"policies.list","total":0}
+++Note: Every one of these events will omit fields that were not used or are not +relevant for specific operation. Also, field ordering is not guaranteed, so DO NOT +rely on it.
+
Bootstrap service publishes events to Redis Stream called magistrala.bootstrap. Every event from this service contains operation field which indicates one of the following event types:
config.create for configuration creation,config.update for configuration update,config.remove for configuration removal,thing.bootstrap for device bootstrap,thing.state_change for device state change,thing.update_connections for device connection update.If you want to integrate through docker-compose.yml you can use magistrala-es-redis service. Just connect to it and consume events from Redis Stream named magistrala.bootstrap.
Whenever configuration is created, bootstrap service will generate and publish new create event. This event will have the following format:
In Redis Streams
+1) "1693313286544-0"
+2) 1) "state"
+ 2) "0"
+ 3) "operation"
+ 4) "config.create"
+ 5) "name"
+ 6) "demo"
+ 7) "channels"
+ 8) "[8d77099e-4911-4140-8555-7d3be65a1694]"
+ 9) "client_cert"
+ 10) "-----BEGIN ENCRYPTED PRIVATE KEY-----MIIFHDBOBgkqhkiG9w0BBQ0wQTApBgkqhkiG9w0BBQwwHAQIc+VAU9JPnIkCAggAMAwGCCqGSIb3DQIJBQAwFAYIKoZIhvcNAwcECImSB+9qZ8dmBIIEyBW/rZlECWnEcMuTXhfJFe+3HP4rV+TXEEuigwCbtVPHWXoZj7KqGiOFgFaDL5Ne/GRwVD6geaTeQVl3aoHzo8mY0yuX2L36Ho2yHF/Bw89WT3hgP0lZ1lVO7O7n8DwybOaoJ+1S3akyb6OPbqcxJou1IGzKV1kz77R8V8nOFSd1BOepNbanGxVG8Jkgc37dQnICXwwaYkTx9PQBtSux1j3KgX0p+VAUNoUFi7N6b0MeO8iEuLU1dUiVwlH/jtitg0W3AvSV+5gezTT2VQW3CVlz6IBTPI1Rfl/3ss18Tao0NiPUmXMIgreBCamXvb0aJm8JxVbhoFYqWVNVocBD+n1+NwhCRlZM5Kgaes5S2JuFnjTAqEYytlQqEySbaN57XYCDNVmQz2iViz/+npuR9SCGwnNvV/TNsKRwav+0NC0pbf3LNk/KL9/X5ccmPhB5Rl7IS/v1BBLYX/jYWVN0dJiSA7fVIr9Acr7IbxWEQ2Y2qh1wdhayi4FBUHY3weivYSU3uGZizsSGJP/N6DutBgS1aXd5X/CqfF7VzRaKF4cfLO4XxTYUEjOztUNMN2XmW0o+ULjQmbouRPs/PIFmh6rc+h42m6p4SkjcsIKOy+mPTeJqhOVmYoMzO8+7mmXDOjFwvi/w97sdmbjII8Zn2iR/N8GuY23vv5h6LQ3tQ5kTA4IuPbYCVLeggd4iMM6TgpuJn0aG7yo4tDFqMeadCVhP2Bp3JQa8r3B2IJstTTF1OtZCrInjSus9ViOiM02Iz3ZmyglsMonJDlWeJL5jKBgqPbLR82IDhIY4IO6SqoVsWu4iWuLW5/TM3fdfYG3Wdvu7Suz7/anLAaMQEzKhObwgDdKmv4PkF75frex969CB1pQqSVnXmz4GrtxVUzWtlflaTSdSegpUXWLhG+jUNKTu+ptxDNM/JBxRNLSzdvsGbkI0qycOCliVpKkkvuiBGtiDWNax6KhV4/oRjkEkTRks9Xeko+q3uY4B//AGxsotsVhF5vhUDTOl5IX7a7wCPtbTGiaR79eprRzGnP9yP38djVrvXprJFU8P7GUr/f2qJt2jDYuCkaqAMsfjdu6YHitjj3ty4vrASgxJ0vsroWhjgiCwgASqM7GnweHSHy5/OZK8jCZX+g+B63Mu4ec+/nNnjvuLqBBZN/FSzXU5fVmYznfPaqW+1Xep+Aj1yGk3L3tvnKLc3sZ1HAJQEjud5dbME6e0JGxh5xOCnzWUR+YL/96KJAcgkxDJ1DxxHv0Uu/5kO5InOsPjs4YKuzqD4nUmGsFsJzTxG626wdGXJMO4YCRKkKtnNeWqMaslM3paN19/tTWyEbaDqc5mVzYLIb3Mzju+OV4GniDeVIvSIsXK5aFGj1PEhfCprQCqUzdNhFU8hF4kUVhn9dp0ExveT7btHSMlEZAWHRkDuLqaImpQkjYmwt90cxtdZwQvjTDtsFmQrvcSp8n1K3P5PwZpVtIw2UHpx+NjE8ZYwOozpXl/oOMzVTB8mi1dQGFkpac9cwnzCZof0ub4iutBeKc4WeEOytvY+CY7hc+/ncCprZ08nlkQarQV7jhfJj658GfBMLGzJtYkCrHwi/AoseIXa5W7eX+lz7O92H2M5QnEkPStQ9lsz2VkYA==-----END ENCRYPTED PRIVATE KEY-----"
+ 11) "ca_cert"
+ 12) "-----BEGIN ENCRYPTED PRIVATE KEY-----MIIFHDBOBgkqhkiG9w0BBQ0wQTApBgkqhkiG9w0BBQwwHAQIc+VAU9JPnIkCAggAMAwGCCqGSIb3DQIJBQAwFAYIKoZIhvcNAwcECImSB+9qZ8dmBIIEyBW/rZlECWnEcMuTXhfJFe+3HP4rV+TXEEuigwCbtVPHWXoZj7KqGiOFgFaDL5Ne/GRwVD6geaTeQVl3aoHzo8mY0yuX2L36Ho2yHF/Bw89WT3hgP0lZ1lVO7O7n8DwybOaoJ+1S3akyb6OPbqcxJou1IGzKV1kz77R8V8nOFSd1BOepNbanGxVG8Jkgc37dQnICXwwaYkTx9PQBtSux1j3KgX0p+VAUNoUFi7N6b0MeO8iEuLU1dUiVwlH/jtitg0W3AvSV+5gezTT2VQW3CVlz6IBTPI1Rfl/3ss18Tao0NiPUmXMIgreBCamXvb0aJm8JxVbhoFYqWVNVocBD+n1+NwhCRlZM5Kgaes5S2JuFnjTAqEYytlQqEySbaN57XYCDNVmQz2iViz/+npuR9SCGwnNvV/TNsKRwav+0NC0pbf3LNk/KL9/X5ccmPhB5Rl7IS/v1BBLYX/jYWVN0dJiSA7fVIr9Acr7IbxWEQ2Y2qh1wdhayi4FBUHY3weivYSU3uGZizsSGJP/N6DutBgS1aXd5X/CqfF7VzRaKF4cfLO4XxTYUEjOztUNMN2XmW0o+ULjQmbouRPs/PIFmh6rc+h42m6p4SkjcsIKOy+mPTeJqhOVmYoMzO8+7mmXDOjFwvi/w97sdmbjII8Zn2iR/N8GuY23vv5h6LQ3tQ5kTA4IuPbYCVLeggd4iMM6TgpuJn0aG7yo4tDFqMeadCVhP2Bp3JQa8r3B2IJstTTF1OtZCrInjSus9ViOiM02Iz3ZmyglsMonJDlWeJL5jKBgqPbLR82IDhIY4IO6SqoVsWu4iWuLW5/TM3fdfYG3Wdvu7Suz7/anLAaMQEzKhObwgDdKmv4PkF75frex969CB1pQqSVnXmz4GrtxVUzWtlflaTSdSegpUXWLhG+jUNKTu+ptxDNM/JBxRNLSzdvsGbkI0qycOCliVpKkkvuiBGtiDWNax6KhV4/oRjkEkTRks9Xeko+q3uY4B//AGxsotsVhF5vhUDTOl5IX7a7wCPtbTGiaR79eprRzGnP9yP38djVrvXprJFU8P7GUr/f2qJt2jDYuCkaqAMsfjdu6YHitjj3ty4vrASgxJ0vsroWhjgiCwgASqM7GnweHSHy5/OZK8jCZX+g+B63Mu4ec+/nNnjvuLqBBZN/FSzXU5fVmYznfPaqW+1Xep+Aj1yGk3L3tvnKLc3sZ1HAJQEjud5dbME6e0JGxh5xOCnzWUR+YL/96KJAcgkxDJ1DxxHv0Uu/5kO5InOsPjs4YKuzqD4nUmGsFsJzTxG626wdGXJMO4YCRKkKtnNeWqMaslM3paN19/tTWyEbaDqc5mVzYLIb3Mzju+OV4GniDeVIvSIsXK5aFGj1PEhfCprQCqUzdNhFU8hF4kUVhn9dp0ExveT7btHSMlEZAWHRkDuLqaImpQkjYmwt90cxtdZwQvjTDtsFmQrvcSp8n1K3P5PwZpVtIw2UHpx+NjE8ZYwOozpXl/oOMzVTB8mi1dQGFkpac9cwnzCZof0ub4iutBeKc4WeEOytvY+CY7hc+/ncCprZ08nlkQarQV7jhfJj658GfBMLGzJtYkCrHwi/AoseIXa5W7eX+lz7O92H2M5QnEkPStQ9lsz2VkYA==-----END ENCRYPTED PRIVATE KEY-----"
+ 13) "occurred_at"
+ 14) "1693313286544243035"
+ 15) "thing_id"
+ 16) "dc82d6bf-973b-4582-9806-0230cee11c20"
+ 17) "content"
+ 18) "{ \"server\": { \"address\": \"127.0.0.1\", \"port\": 8080 }, \"database\": { \"host\": \"localhost\", \"port\": 5432, \"username\": \"user\", \"password\": \"password\", \"dbname\": \"mydb\" }, \"logging\": { \"level\": \"info\", \"file\": \"app.log\" } }"
+ 19) "owner"
+ 20) "64fd20bf-e8fb-46bf-9b64-2a6572eda21b"
+ 21) "external_id"
+ 22) "209327A2FA2D47E3B05F118D769DC521"
+ 23) "client_key"
+ 24) "-----BEGIN ENCRYPTED PRIVATE KEY-----MIIFHDBOBgkqhkiG9w0BBQ0wQTApBgkqhkiG9w0BBQwwHAQIc+VAU9JPnIkCAggAMAwGCCqGSIb3DQIJBQAwFAYIKoZIhvcNAwcECImSB+9qZ8dmBIIEyBW/rZlECWnEcMuTXhfJFe+3HP4rV+TXEEuigwCbtVPHWXoZj7KqGiOFgFaDL5Ne/GRwVD6geaTeQVl3aoHzo8mY0yuX2L36Ho2yHF/Bw89WT3hgP0lZ1lVO7O7n8DwybOaoJ+1S3akyb6OPbqcxJou1IGzKV1kz77R8V8nOFSd1BOepNbanGxVG8Jkgc37dQnICXwwaYkTx9PQBtSux1j3KgX0p+VAUNoUFi7N6b0MeO8iEuLU1dUiVwlH/jtitg0W3AvSV+5gezTT2VQW3CVlz6IBTPI1Rfl/3ss18Tao0NiPUmXMIgreBCamXvb0aJm8JxVbhoFYqWVNVocBD+n1+NwhCRlZM5Kgaes5S2JuFnjTAqEYytlQqEySbaN57XYCDNVmQz2iViz/+npuR9SCGwnNvV/TNsKRwav+0NC0pbf3LNk/KL9/X5ccmPhB5Rl7IS/v1BBLYX/jYWVN0dJiSA7fVIr9Acr7IbxWEQ2Y2qh1wdhayi4FBUHY3weivYSU3uGZizsSGJP/N6DutBgS1aXd5X/CqfF7VzRaKF4cfLO4XxTYUEjOztUNMN2XmW0o+ULjQmbouRPs/PIFmh6rc+h42m6p4SkjcsIKOy+mPTeJqhOVmYoMzO8+7mmXDOjFwvi/w97sdmbjII8Zn2iR/N8GuY23vv5h6LQ3tQ5kTA4IuPbYCVLeggd4iMM6TgpuJn0aG7yo4tDFqMeadCVhP2Bp3JQa8r3B2IJstTTF1OtZCrInjSus9ViOiM02Iz3ZmyglsMonJDlWeJL5jKBgqPbLR82IDhIY4IO6SqoVsWu4iWuLW5/TM3fdfYG3Wdvu7Suz7/anLAaMQEzKhObwgDdKmv4PkF75frex969CB1pQqSVnXmz4GrtxVUzWtlflaTSdSegpUXWLhG+jUNKTu+ptxDNM/JBxRNLSzdvsGbkI0qycOCliVpKkkvuiBGtiDWNax6KhV4/oRjkEkTRks9Xeko+q3uY4B//AGxsotsVhF5vhUDTOl5IX7a7wCPtbTGiaR79eprRzGnP9yP38djVrvXprJFU8P7GUr/f2qJt2jDYuCkaqAMsfjdu6YHitjj3ty4vrASgxJ0vsroWhjgiCwgASqM7GnweHSHy5/OZK8jCZX+g+B63Mu4ec+/nNnjvuLqBBZN/FSzXU5fVmYznfPaqW+1Xep+Aj1yGk3L3tvnKLc3sZ1HAJQEjud5dbME6e0JGxh5xOCnzWUR+YL/96KJAcgkxDJ1DxxHv0Uu/5kO5InOsPjs4YKuzqD4nUmGsFsJzTxG626wdGXJMO4YCRKkKtnNeWqMaslM3paN19/tTWyEbaDqc5mVzYLIb3Mzju+OV4GniDeVIvSIsXK5aFGj1PEhfCprQCqUzdNhFU8hF4kUVhn9dp0ExveT7btHSMlEZAWHRkDuLqaImpQkjYmwt90cxtdZwQvjTDtsFmQrvcSp8n1K3P5PwZpVtIw2UHpx+NjE8ZYwOozpXl/oOMzVTB8mi1dQGFkpac9cwnzCZof0ub4iutBeKc4WeEOytvY+CY7hc+/ncCprZ08nlkQarQV7jhfJj658GfBMLGzJtYkCrHwi/AoseIXa5W7eX+lz7O92H2M5QnEkPStQ9lsz2VkYA==-----END ENCRYPTED PRIVATE KEY-----"
+In Nats Jet Streams
+Subject: events.magistrala.bootstrap Received: 2023-10-11T19:17:46+03:00
+
+{"client_cert":"-----BEGIN CERTIFICATE-----\nMIIEATCCAumgAwIBAgIUNl+p3eaPJsZRFvW/u5AGBLgx7YMwDQYJKoZIhvcNAQEL\nBQAwLjEsMCoGA1UEAxMjbWFpbmZsdXguY29tIEludGVybWVkaWF0ZSBBdXRob3Jp\ndHkwHhcNMjMxMDExMTYwMDQ0WhcNMjMxMTEwMTYwMTE0WjAvMS0wKwYDVQQDEyQ1\nNmRhNGU5NC0zZDFjLTRhMGQtYTNmNC1hNTdkZDBhNTVkN2IwggEiMA0GCSqGSIb3\nDQEBAQUAA4IBDwAwggEKAoIBAQCt3dFuSvEtihmHKBNCe8AUI/xTJn+JCUF90+ZY\njoz8hWVwd/UhJQKTblBhWL/osGOrr3PItcCVZ1JDaGQMQzhtaPYnBbwJDwMTbcey\nlb/E6O/DD4UCqWpjQjYhc0/z98oM70E+Szs6yp+68qYIGQDH669P91TaURtuGGMr\n8nLN6fQTb9mVZX1L3ps8zHoxb+i7oVhLmjGeGXjf3HP4U+L/hwYhZ/gqqXYZJli6\nR1j5BxJuk1aVKqwrm2qbuFYhWI7Wl6ZEoEk2Q3FV1onzxcVHB5xBacDcNHhyQmoE\nhIg4lt1DTaUKRxlZqbhJQtLrrCeI8NCvEpC/dh1eBwddIxKbAgMBAAGjggEUMIIB\nEDAOBgNVHQ8BAf8EBAMCA6gwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMC\nMB0GA1UdDgQWBBTcmEIW7knfrwBTyBu6WwycgGqzwTAfBgNVHSMEGDAWgBQMxD1V\nt+J/aShmUQJJHmdmmZ/fXDA7BggrBgEFBQcBAQQvMC0wKwYIKwYBBQUHMAKGH2h0\ndHA6Ly92YXVsdDo4MjAwL3YxL3BraV9pbnQvY2EwLwYDVR0RBCgwJoIkNTZkYTRl\nOTQtM2QxYy00YTBkLWEzZjQtYTU3ZGQwYTU1ZDdiMDEGA1UdHwQqMCgwJqAkoCKG\nIGh0dHA6Ly92YXVsdDo4MjAwL3YxL3BraV9pbnQvY3JsMA0GCSqGSIb3DQEBCwUA\nA4IBAQCiCS+dPhXS/upfMieFbt8+QmYQFZ82Ct2oTsDTBaDczE7MiLxrl5iKbynk\nT47y+hyvWFIb31BhrIrS6mRR+9IoJdmje3KxyvNr/TtSw7T4spbfRo30uk2flnA0\nDhJv3bzcYRwjaLdBcZTUS0GETqztdqMThJIY2EAfhfKDM0ecM4tF2c8/RT14BpPO\nEVCUX69A0k6p80xTfmyhvKIfiMsJIcqsHuOiEsiXwgNNUqc332bIV+fQW+vXK8eS\nstp7N2v6axo016wf3qgEUPdPC0sPMHlwIc3s6ddZOoatk6LKH1nW2gIguUH4Ws7s\nAN5Cs0h2D+AWi+IJpDuUzvgrd4/3\n-----END CERTIFICATE-----","client_key":"-----BEGIN RSA PRIVATE KEY-----\nMIIEpQIBAAKCAQEArd3RbkrxLYoZhygTQnvAFCP8UyZ/iQlBfdPmWI6M/IVlcHf1\nISUCk25QYVi/6LBjq69zyLXAlWdSQ2hkDEM4bWj2JwW8CQ8DE23HspW/xOjvww+F\nAqlqY0I2IXNP8/fKDO9BPks7OsqfuvKmCBkAx+uvT/dU2lEbbhhjK/Jyzen0E2/Z\nlWV9S96bPMx6MW/ou6FYS5oxnhl439xz+FPi/4cGIWf4Kql2GSZYukdY+QcSbpNW\nlSqsK5tqm7hWIViO1pemRKBJNkNxVdaJ88XFRwecQWnA3DR4ckJqBISIOJbdQ02l\nCkcZWam4SULS66wniPDQrxKQv3YdXgcHXSMSmwIDAQABAoIBAGFvWKmdd/EUXl/+\n1mRAo5Dl5cbXYUtzk28nbAQexuXQ/9r6brYHXp0uif8z1EBbcU/KgHFvYaCYiWJb\nQw4YMawm0SNnNExDTG7765iqERERlSPUM68dMBC2D03JqHnJWELNZdu6H1RALyl+\nSAtrr6NZ8iI3MicyotOc+R6svSelUigaTKYGadWKNQJjOpLpFhu0D87iTdQ2dJyU\nJKGuB1YCp3rTNXTLuq/omMDyqJYPCNXbKW78bM9UExz0suYJi6URW/aQIK/A8CUh\nXAm40C2aQ8d5tHONsIGB9BlzOkHrXB2CvHKkQWl/TEfo6qcFI6qhx1Gx72SOK7F1\nAE0dDQkCgYEA5Vqq4/LrY/uzjTN918IYrcSi3gr0NoUY/KuZverBZdSR/fS0vMHX\nQ/X7XGE2gOGrzqWN2dtEAmMrisfo9zUlimU+7SWeetOKa5JTtmSZGFWvCrvsnWkH\narZT59t3yJ7YqwbjRmaSC3Uq7veFBsIEKbszm0eeMCtNcSLuykmBM/8CgYEAwhDd\nS3OLj4J62PlSjyBHSDP2Zv5sG6jXDDRPo8HQWcWoOq2stnwgrblkEXpsSpaRaUP8\nQabEtkobckYx3OK6/0Q4c0A6I4lHQpm2m6K2o+8lsG/OwwNplXA+kpYuqUYEjgBl\nFiGblZCtyDPdd4PnHDBjKLUzOYxgRDxfVLLKcWUCgYEAm4+ypyellswqzZPmQAhL\nOtlLanVdjPkbqI0vmwv2Hv5eAzUNvZVwT40w70iUcjgekuvhWamJ6GChMOFE1x96\nFfN0Cd9hLYf7s9is5OI4oLPFJO+vnliVikCein1mMnHjHaVvU9nQJutSsoC5/opr\nzm5Fo4Wg+qT0Qs9hzVyrwLsCgYEAoTk9f6d4dDskMAnByuI4FgYFWL9ZtQjpz1vO\nJe+oVkxdXJJYgCpTQ8BXICYivTylhVxTv376wa6DasZiOm2qiNN2Slk7c7ZimzP0\nfwwIy9yr5Q6eKWk2WE4tzb4y+bIPqqEtWduF1BWkKkTcYqQUZljUqEcRTWgPueCm\nGkmG4fkCgYEAk1rI1pAdfdxLrKTGu2WRfr0UgNx/eUjVuL5GT3Bu092QLXwFc3Py\nsIu7vHRQY1ASGRDvlyx76Uc+mtmXhowwBL1fTiKcfgD3SbqRTw8rwJbF6MNB7fOH\nSWlA4wuzpwVvg0j2DoY5LbwMN7AbrGS451E14xUwpEWdIDChWw14qag=\n-----END RSA PRIVATE KEY-----","external_id":"889","name":"ariadne","occurred_at":1697041066438195084,"operation":"config.create","owner":"1058552c-3f6a-4bfe-827e-ecbcdc95d344","state":"0","thing_id":"bb569875-a268-4ed4-b382-b88b181c61f3"}
+Whenever configuration is updated, bootstrap service will generate and publish new update event. This event will have the following format:
1) "1693313985263-0"
+2) 1) "state"
+ 2) "0"
+ 3) "operation"
+ 4) "config.update"
+ 5) "thing_id"
+ 6) "dc82d6bf-973b-4582-9806-0230cee11c20"
+ 7) "content"
+ 8) "{ \"server\": { \"address\": \"127.0.0.1\", \"port\": 8080 }, \"database\": { \"host\": \"localhost\", \"port\": 5432, \"username\": \"user\", \"password\": \"password\", \"dbname\": \"mydb\" } }"
+ 9) "name"
+ 10) "demo"
+ 11) "occurred_at"
+ 12) "1693313985263381501"
+Whenever certificate is updated, bootstrap service will generate and publish new update event. This event will have the following format:
1) "1693313759203-0"
+2) 1) "thing_key"
+ 2) "dc82d6bf-973b-4582-9806-0230cee11c20"
+ 3) "client_cert"
+ 4) "-----BEGIN ENCRYPTED PRIVATE KEY-----MIIFHDBOBgkqhkiG9w0BBQ0wQTApBgkqhkiG9w0BBQwwHAQIc+VAU9JPnIkCAggAMAwGCCqGSIb3DQIJBQAwFAYIKoZIhvcNAwcECImSB+9qZ8dmBIIEyBW/rZlECWnEcMuTXhfJFe+3HP4rV+TXEEuigwCbtVPHWXoZj7KqGiOFgFaDL5Ne/GRwVD6geaTeQVl3aoHzo8mY0yuX2L36Ho2yHF/Bw89WT3hgP0lZ1lVO7O7n8DwybOaoJ+1S3akyb6OPbqcxJou1IGzKV1kz77R8V8nOFSd1BOepNbanGxVG8Jkgc37dQnICXwwaYkTx9PQBtSux1j3KgX0p+VAUNoUFi7N6b0MeO8iEuLU1dUiVwlH/jtitg0W3AvSV+5gezTT2VQW3CVlz6IBTPI1Rfl/3ss18Tao0NiPUmXMIgreBCamXvb0aJm8JxVbhoFYqWVNVocBD+n1+NwhCRlZM5Kgaes5S2JuFnjTAqEYytlQqEySbaN57XYCDNVmQz2iViz/+npuR9SCGwnNvV/TNsKRwav+0NC0pbf3LNk/KL9/X5ccmPhB5Rl7IS/v1BBLYX/jYWVN0dJiSA7fVIr9Acr7IbxWEQ2Y2qh1wdhayi4FBUHY3weivYSU3uGZizsSGJP/N6DutBgS1aXd5X/CqfF7VzRaKF4cfLO4XxTYUEjOztUNMN2XmW0o+ULjQmbouRPs/PIFmh6rc+h42m6p4SkjcsIKOy+mPTeJqhOVmYoMzO8+7mmXDOjFwvi/w97sdmbjII8Zn2iR/N8GuY23vv5h6LQ3tQ5kTA4IuPbYCVLeggd4iMM6TgpuJn0aG7yo4tDFqMeadCVhP2Bp3JQa8r3B2IJstTTF1OtZCrInjSus9ViOiM02Iz3ZmyglsMonJDlWeJL5jKBgqPbLR82IDhIY4IO6SqoVsWu4iWuLW5/TM3fdfYG3Wdvu7Suz7/anLAaMQEzKhObwgDdKmv4PkF75frex969CB1pQqSVnXmz4GrtxVUzWtlflaTSdSegpUXWLhG+jUNKTu+ptxDNM/JBxRNLSzdvsGbkI0qycOCliVpKkkvuiBGtiDWNax6KhV4/oRjkEkTRks9Xeko+q3uY4B//AGxsotsVhF5vhUDTOl5IX7a7wCPtbTGiaR79eprRzGnP9yP38djVrvXprJFU8P7GUr/f2qJt2jDYuCkaqAMsfjdu6YHitjj3ty4vrASgxJ0vsroWhjgiCwgASqM7GnweHSHy5/OZK8jCZX+g+B63Mu4ec+/nNnjvuLqBBZN/FSzXU5fVmYznfPaqW+1Xep+Aj1yGk3L3tvnKLc3sZ1HAJQEjud5dbME6e0JGxh5xOCnzWUR+YL/96KJAcgkxDJ1DxxHv0Uu/5kO5InOsPjs4YKuzqD4nUmGsFsJzTxG626wdGXJMO4YCRKkKtnNeWqMaslM3paN19/tTWyEbaDqc5mVzYLIb3Mzju+OV4GniDeVIvSIsXK5aFGj1PEhfCprQCqUzdNhFU8hF4kUVhn9dp0ExveT7btHSMlEZAWHRkDuLqaImpQkjYmwt90cxtdZwQvjTDtsFmQrvcSp8n1K3P5PwZpVtIw2UHpx+NjE8ZYwOozpXl/oOMzVTB8mi1dQGFkpac9cwnzCZof0ub4iutBeKc4WeEOytvY+CY7hc+/ncCprZ08nlkQarQV7jhfJj658GfBMLGzJtYkCrHwi/AoseIXa5W7eX+lz7O92H2M5QnEkPStQ9lsz2VkYA==-----END ENCRYPTED PRIVATE KEY-----"
+ 5) "client_key"
+ 6) "-----BEGIN ENCRYPTED PRIVATE KEY-----MIIFHDBOBgkqhkiG9w0BBQ0wQTApBgkqhkiG9w0BBQwwHAQIc+VAU9JPnIkCAggAMAwGCCqGSIb3DQIJBQAwFAYIKoZIhvcNAwcECImSB+9qZ8dmBIIEyBW/rZlECWnEcMuTXhfJFe+3HP4rV+TXEEuigwCbtVPHWXoZj7KqGiOFgFaDL5Ne/GRwVD6geaTeQVl3aoHzo8mY0yuX2L36Ho2yHF/Bw89WT3hgP0lZ1lVO7O7n8DwybOaoJ+1S3akyb6OPbqcxJou1IGzKV1kz77R8V8nOFSd1BOepNbanGxVG8Jkgc37dQnICXwwaYkTx9PQBtSux1j3KgX0p+VAUNoUFi7N6b0MeO8iEuLU1dUiVwlH/jtitg0W3AvSV+5gezTT2VQW3CVlz6IBTPI1Rfl/3ss18Tao0NiPUmXMIgreBCamXvb0aJm8JxVbhoFYqWVNVocBD+n1+NwhCRlZM5Kgaes5S2JuFnjTAqEYytlQqEySbaN57XYCDNVmQz2iViz/+npuR9SCGwnNvV/TNsKRwav+0NC0pbf3LNk/KL9/X5ccmPhB5Rl7IS/v1BBLYX/jYWVN0dJiSA7fVIr9Acr7IbxWEQ2Y2qh1wdhayi4FBUHY3weivYSU3uGZizsSGJP/N6DutBgS1aXd5X/CqfF7VzRaKF4cfLO4XxTYUEjOztUNMN2XmW0o+ULjQmbouRPs/PIFmh6rc+h42m6p4SkjcsIKOy+mPTeJqhOVmYoMzO8+7mmXDOjFwvi/w97sdmbjII8Zn2iR/N8GuY23vv5h6LQ3tQ5kTA4IuPbYCVLeggd4iMM6TgpuJn0aG7yo4tDFqMeadCVhP2Bp3JQa8r3B2IJstTTF1OtZCrInjSus9ViOiM02Iz3ZmyglsMonJDlWeJL5jKBgqPbLR82IDhIY4IO6SqoVsWu4iWuLW5/TM3fdfYG3Wdvu7Suz7/anLAaMQEzKhObwgDdKmv4PkF75frex969CB1pQqSVnXmz4GrtxVUzWtlflaTSdSegpUXWLhG+jUNKTu+ptxDNM/JBxRNLSzdvsGbkI0qycOCliVpKkkvuiBGtiDWNax6KhV4/oRjkEkTRks9Xeko+q3uY4B//AGxsotsVhF5vhUDTOl5IX7a7wCPtbTGiaR79eprRzGnP9yP38djVrvXprJFU8P7GUr/f2qJt2jDYuCkaqAMsfjdu6YHitjj3ty4vrASgxJ0vsroWhjgiCwgASqM7GnweHSHy5/OZK8jCZX+g+B63Mu4ec+/nNnjvuLqBBZN/FSzXU5fVmYznfPaqW+1Xep+Aj1yGk3L3tvnKLc3sZ1HAJQEjud5dbME6e0JGxh5xOCnzWUR+YL/96KJAcgkxDJ1DxxHv0Uu/5kO5InOsPjs4YKuzqD4nUmGsFsJzTxG626wdGXJMO4YCRKkKtnNeWqMaslM3paN19/tTWyEbaDqc5mVzYLIb3Mzju+OV4GniDeVIvSIsXK5aFGj1PEhfCprQCqUzdNhFU8hF4kUVhn9dp0ExveT7btHSMlEZAWHRkDuLqaImpQkjYmwt90cxtdZwQvjTDtsFmQrvcSp8n1K3P5PwZpVtIw2UHpx+NjE8ZYwOozpXl/oOMzVTB8mi1dQGFkpac9cwnzCZof0ub4iutBeKc4WeEOytvY+CY7hc+/ncCprZ08nlkQarQV7jhfJj658GfBMLGzJtYkCrHwi/AoseIXa5W7eX+lz7O92H2M5QnEkPStQ9lsz2VkYA==-----END ENCRYPTED PRIVATE KEY-----"
+ 7) "ca_cert"
+ 8) "-----BEGIN ENCRYPTED PRIVATE KEY-----MIIFHDBOBgkqhkiG9w0BBQ0wQTApBgkqhkiG9w0BBQwwHAQIc+VAU9JPnIkCAggAMAwGCCqGSIb3DQIJBQAwFAYIKoZIhvcNAwcECImSB+9qZ8dmBIIEyBW/rZlECWnEcMuTXhfJFe+3HP4rV+TXEEuigwCbtVPHWXoZj7KqGiOFgFaDL5Ne/GRwVD6geaTeQVl3aoHzo8mY0yuX2L36Ho2yHF/Bw89WT3hgP0lZ1lVO7O7n8DwybOaoJ+1S3akyb6OPbqcxJou1IGzKV1kz77R8V8nOFSd1BOepNbanGxVG8Jkgc37dQnICXwwaYkTx9PQBtSux1j3KgX0p+VAUNoUFi7N6b0MeO8iEuLU1dUiVwlH/jtitg0W3AvSV+5gezTT2VQW3CVlz6IBTPI1Rfl/3ss18Tao0NiPUmXMIgreBCamXvb0aJm8JxVbhoFYqWVNVocBD+n1+NwhCRlZM5Kgaes5S2JuFnjTAqEYytlQqEySbaN57XYCDNVmQz2iViz/+npuR9SCGwnNvV/TNsKRwav+0NC0pbf3LNk/KL9/X5ccmPhB5Rl7IS/v1BBLYX/jYWVN0dJiSA7fVIr9Acr7IbxWEQ2Y2qh1wdhayi4FBUHY3weivYSU3uGZizsSGJP/N6DutBgS1aXd5X/CqfF7VzRaKF4cfLO4XxTYUEjOztUNMN2XmW0o+ULjQmbouRPs/PIFmh6rc+h42m6p4SkjcsIKOy+mPTeJqhOVmYoMzO8+7mmXDOjFwvi/w97sdmbjII8Zn2iR/N8GuY23vv5h6LQ3tQ5kTA4IuPbYCVLeggd4iMM6TgpuJn0aG7yo4tDFqMeadCVhP2Bp3JQa8r3B2IJstTTF1OtZCrInjSus9ViOiM02Iz3ZmyglsMonJDlWeJL5jKBgqPbLR82IDhIY4IO6SqoVsWu4iWuLW5/TM3fdfYG3Wdvu7Suz7/anLAaMQEzKhObwgDdKmv4PkF75frex969CB1pQqSVnXmz4GrtxVUzWtlflaTSdSegpUXWLhG+jUNKTu+ptxDNM/JBxRNLSzdvsGbkI0qycOCliVpKkkvuiBGtiDWNax6KhV4/oRjkEkTRks9Xeko+q3uY4B//AGxsotsVhF5vhUDTOl5IX7a7wCPtbTGiaR79eprRzGnP9yP38djVrvXprJFU8P7GUr/f2qJt2jDYuCkaqAMsfjdu6YHitjj3ty4vrASgxJ0vsroWhjgiCwgASqM7GnweHSHy5/OZK8jCZX+g+B63Mu4ec+/nNnjvuLqBBZN/FSzXU5fVmYznfPaqW+1Xep+Aj1yGk3L3tvnKLc3sZ1HAJQEjud5dbME6e0JGxh5xOCnzWUR+YL/96KJAcgkxDJ1DxxHv0Uu/5kO5InOsPjs4YKuzqD4nUmGsFsJzTxG626wdGXJMO4YCRKkKtnNeWqMaslM3paN19/tTWyEbaDqc5mVzYLIb3Mzju+OV4GniDeVIvSIsXK5aFGj1PEhfCprQCqUzdNhFU8hF4kUVhn9dp0ExveT7btHSMlEZAWHRkDuLqaImpQkjYmwt90cxtdZwQvjTDtsFmQrvcSp8n1K3P5PwZpVtIw2UHpx+NjE8ZYwOozpXl/oOMzVTB8mi1dQGFkpac9cwnzCZof0ub4iutBeKc4WeEOytvY+CY7hc+/ncCprZ08nlkQarQV7jhfJj658GfBMLGzJtYkCrHwi/AoseIXa5W7eX+lz7O92H2M5QnEkPStQ9lsz2VkYA==-----END ENCRYPTED PRIVATE KEY-----"
+ 9) "operation"
+ 10) "cert.update"
+ 11) "occurred_at"
+ 12) "1693313759203076553"
+Whenever configuration list is fetched, bootstrap service will generate new list event. This event will have the following format:
In Redis Streams
+1) "1693339274766-0"
+2) 1) "occurred_at"
+ 2) "1693339274766130265"
+ 3) "offset"
+ 4) "0"
+ 5) "limit"
+ 6) "10"
+ 7) "operation"
+ 8) "config.list"
+In Nats JetStreams
+Subject: events.magistrala.bootstrap Received: 2023-10-11T19:23:05+03:00
+
+{"external_id":"879","name":"aphrodite","occurred_at":1697042445469239430,"operation":"config.list","owner":"1058552c-3f6a-4bfe-827e-ecbcdc95d344","state":"0","thing_id":"1f9c0d1d-0d6b-4a83-83b9-50845e557a85"}
+Whenever configuration is viewed, bootstrap service will generate new view event. This event will have the following format:
1) 1) "1693339152105-0"
+2) 1) "thing_id"
+ 2) "74f00d13-d370-42c0-b528-04fff995275c"
+ 3) "name"
+ 4) "demo"
+ 5) "external_id"
+ 6) "FF-41-EF-BC-90-BC"
+ 7) "channels"
+ 8) "[90aae157-d47f-4d71-9a68-b000c0025ae8]"
+ 9) "client_cert"
+ 10) "-----BEGIN PRIVATE KEY-----MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDVYaZsyd76aSWZexY/OyX8hVdE+ruT3OZrE6gFSjDiaAA2Uf5/eHT1BJdR4LviooXix8vfc/g5CAN/z98zmUmAzx9lk5T4sRhJfqYQ2yDEt1tVDwD3RzL9RHXRWiZu4thk253jOpT15VFvOf5wE6mhVozFl9OetVJb4eqKbHx9RY0rMXwiBiCC2LcUtcp6rVjp4pK6VGjehA8siVX9bnRsIY776jDb/pm2n+y5G+bd1CifSdgTrr7QLKFlx0//5lyslmfUbf76kg9bZ8Qe2NdFKvcpEZ4ENxtwMrqW2i1pTExVHNpka8rhA5936qpDKu1ce+kccIbFsPRAHU5PyXfNAgMBAAECggEAAtBt4c4WcGuwlkHxp4B/3hZix0Md9DOb9QTmWLjYxN5QRRHMbyFHPEVaOuHhjc9M6r0YgD2cTsw/QjvwmqfxOI9YFP6JnsS0faD7pF9EzbNes1QmVByOnJkpi0r1aiL4baQZL0+sz+1n/IqMQ4LO4D+ETcV/LKmsM2VbCDD+wfwsVkTmgaqKtXIFQ3bOU5LjRcxCZFs81z3mYDyP4hfnlmTWOOXcf8yLqx5LGH8erCGXgrhZiN5/mhkzUpkF75Eo4qt3jVZEt+d48RnPsk0TO0rqs4j5F3d/6Dboi3UpRlHZ4vEM7MeDGoMuXTh59MzbV1e/03sY2jTtB2NVQ51pFQKBgQD0kjYorDqu5e82Orp5rRkS58nUDgq3vaxNKJq+32LuuTuNjRrM57XoyBAVnBlfTP5IOQaxjYPNxHkZhYdYREyZKx96g6FZUWLQxKO+vP+E25MXSsnP8FMkQNfgSvMCxfIyFO3aVbDUme6bIScPNCTzKVWSWTj5Zyyig9VQpoRJ5wKBgQDfWlF7krUefQEvdJFxd9IGBvlkWkGi942Hh0H6vJCzhMQO8DeHZjO4oiiCEpRmBdkLDlZs81mykmyFEpjcmv4JD23HQ9IPi0/4Bsuu3SDXF4HC5/QYldaG0behBmMmDYuaQ0NHY5rpCnpZBteYT6V6lcBm/AIKwvz+N8fY2fDCKwKBgQDfBCjQw+SrMc8FI16Br7+KhsR7UuahEBt7LIiXfvom98//TuleafhuMWjBW9ujFIFXeHDLHWFQFFXdWO7HJVi33yPQQxGxcc5q0rUCLDPQga1Kcw8+R0Z5a4uu4olgQQKOepk+HB+obkmvOfb1HTaIaWu3jRawDk4cT50H8x/0hwKBgB63eB9LhNclj+Ur3djCBsNHcELp2r8D1pX99wf5qNjXeHMpfCmF17UbsAB7d6c0RK4tkZs4OGzDkGMYtKcaNbefRJSz8g6rNRtCK/7ncF3EYNciOUKsUK2H5/4gN8CC+mEDwRvvSd2k0ECwHTRYN8TNFYHURJ+gQ1Te7QAYsPCzAoGBAMZnbAY1Q/gK11JaPE2orFb1IltDRKB2IXh5Ton0ZCqhmOhMLQ+4t7DLPUKdXlsBZa/IIm5XehBg6VajbG0zulKLzO4YHuWEduwYON+4DNQxLWhBCBauOZ7+dcGUvYkeKoySYs6hznV9mlMHe1TuhCO8zHjpvBXOrlAR8VX5BXKz-----END PRIVATE KEY-----"
+ 11) "state"
+ 12) "0"
+ 13) "operation"
+ 14) "config.view"
+ 15) "content"
+ 16) "{\"device_id\": \"12345\",\"secure_connection\": true,\"sensor_config\": {\"temperature\": true,\"humidity\": true,\"pressure\": false}}"
+ 17) "owner"
+ 18) "b2972472-c93c-408f-9b77-0f8a81ee47af"
+ 19) "occurred_at"
+ 20) "1693339152105496336"
+Whenever configuration is removed, bootstrap service will generate and publish new remove event. This event will have the following format:
In Redis Streams
+1) "1693339203771-0"
+2) 1) "occurred_at"
+ 2) "1693339203771705590"
+ 3) "thing_id"
+ 4) "853f37b9-513a-41a2-a575-bbaa746961a6"
+ 5) "operation"
+ 6) "config.remove"
+In Nats JetStreams
+Subject: events.magistrala.bootstrap Received: 2023-10-11T19:24:50+03:00
+
+{"occurred_at":1697041490458439515,"operation":"config.remove","thing_id":"6a08150a-cd19-460f-99c9-a760ee50aed3"}
+Whenever a thing is removed, bootstrap service will generate and publish new config.remove_handler event. This event will have the following format:
1) 1) "1693337955655-0"
+2) 1) "config_id"
+ 2) "0198b458-573e-415a-aa05-052ddab9709d"
+ 3) "operation"
+ 4) "config.remove_handler"
+ 5) "occurred_at"
+ 6) "1693337955654969489"
+Whenever thing is bootstrapped, bootstrap service will generate and publish new bootstrap event. This event will have the following format:
1) 1) "1693339161600-0"
+2) 1) "occurred_at"
+ 2) "1693339161600369325"
+ 3) "external_id"
+ 4) "FF-41-EF-BC-90-BC"
+ 5) "success"
+ 6) "1"
+ 7) "operation"
+ 8) "thing.bootstrap"
+ 9) "thing_id"
+ 10) "74f00d13-d370-42c0-b528-04fff995275c"
+ 11) "content"
+ 12) "{\"device_id\": \"12345\",\"secure_connection\": true,\"sensor_config\": {\"temperature\": true,\"humidity\": true,\"pressure\": false}}"
+ 13) "owner"
+ 14) "b2972472-c93c-408f-9b77-0f8a81ee47af"
+ 15) "name"
+ 16) "demo"
+ 17) "channels"
+ 18) "[90aae157-d47f-4d71-9a68-b000c0025ae8]"
+ 19) "ca_cert"
+ 20) "-----BEGIN PRIVATE KEY-----MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDVYaZsyd76aSWZexY/OyX8hVdE+ruT3OZrE6gFSjDiaAA2Uf5/eHT1BJdR4LviooXix8vfc/g5CAN/z98zmUmAzx9lk5T4sRhJfqYQ2yDEt1tVDwD3RzL9RHXRWiZu4thk253jOpT15VFvOf5wE6mhVozFl9OetVJb4eqKbHx9RY0rMXwiBiCC2LcUtcp6rVjp4pK6VGjehA8siVX9bnRsIY776jDb/pm2n+y5G+bd1CifSdgTrr7QLKFlx0//5lyslmfUbf76kg9bZ8Qe2NdFKvcpEZ4ENxtwMrqW2i1pTExVHNpka8rhA5936qpDKu1ce+kccIbFsPRAHU5PyXfNAgMBAAECggEAAtBt4c4WcGuwlkHxp4B/3hZix0Md9DOb9QTmWLjYxN5QRRHMbyFHPEVaOuHhjc9M6r0YgD2cTsw/QjvwmqfxOI9YFP6JnsS0faD7pF9EzbNes1QmVByOnJkpi0r1aiL4baQZL0+sz+1n/IqMQ4LO4D+ETcV/LKmsM2VbCDD+wfwsVkTmgaqKtXIFQ3bOU5LjRcxCZFs81z3mYDyP4hfnlmTWOOXcf8yLqx5LGH8erCGXgrhZiN5/mhkzUpkF75Eo4qt3jVZEt+d48RnPsk0TO0rqs4j5F3d/6Dboi3UpRlHZ4vEM7MeDGoMuXTh59MzbV1e/03sY2jTtB2NVQ51pFQKBgQD0kjYorDqu5e82Orp5rRkS58nUDgq3vaxNKJq+32LuuTuNjRrM57XoyBAVnBlfTP5IOQaxjYPNxHkZhYdYREyZKx96g6FZUWLQxKO+vP+E25MXSsnP8FMkQNfgSvMCxfIyFO3aVbDUme6bIScPNCTzKVWSWTj5Zyyig9VQpoRJ5wKBgQDfWlF7krUefQEvdJFxd9IGBvlkWkGi942Hh0H6vJCzhMQO8DeHZjO4oiiCEpRmBdkLDlZs81mykmyFEpjcmv4JD23HQ9IPi0/4Bsuu3SDXF4HC5/QYldaG0behBmMmDYuaQ0NHY5rpCnpZBteYT6V6lcBm/AIKwvz+N8fY2fDCKwKBgQDfBCjQw+SrMc8FI16Br7+KhsR7UuahEBt7LIiXfvom98//TuleafhuMWjBW9ujFIFXeHDLHWFQFFXdWO7HJVi33yPQQxGxcc5q0rUCLDPQga1Kcw8+R0Z5a4uu4olgQQKOepk+HB+obkmvOfb1HTaIaWu3jRawDk4cT50H8x/0hwKBgB63eB9LhNclj+Ur3djCBsNHcELp2r8D1pX99wf5qNjXeHMpfCmF17UbsAB7d6c0RK4tkZs4OGzDkGMYtKcaNbefRJSz8g6rNRtCK/7ncF3EYNciOUKsUK2H5/4gN8CC+mEDwRvvSd2k0ECwHTRYN8TNFYHURJ+gQ1Te7QAYsPCzAoGBAMZnbAY1Q/gK11JaPE2orFb1IltDRKB2IXh5Ton0ZCqhmOhMLQ+4t7DLPUKdXlsBZa/IIm5XehBg6VajbG0zulKLzO4YHuWEduwYON+4DNQxLWhBCBauOZ7+dcGUvYkeKoySYs6hznV9mlMHe1TuhCO8zHjpvBXOrlAR8VX5BXKz-----END PRIVATE KEY-----"
+Whenever thing's state changes, bootstrap service will generate and publish new change state event. This event will have the following format:
In Redis Streams
+1) "1555405294806-0"
+2) 1) "thing_id"
+2) "63a110d4-2b77-48d2-aa46-2582681eeb82"
+3) "state"
+4) "0"
+5) "timestamp"
+6) "1555405294"
+7) "operation"
+8) "thing.state_change"
+In Nats JetStreams
+Subject: events.magistrala.bootstrap Received: 2023-10-11T19:26:31+03:00
+
+{"occurred_at":1697041591648042067,"operation":"thing.change_state","state":"0","thing_id":"1f9c0d1d-0d6b-4a83-83b9-50845e557a85"}
+Whenever thing's list of connections is updated, bootstrap service will generate and publish new update connections event. This event will have the following format:
1) "1555405373360-0"
+2) 1) "operation"
+ 2) "thing.update_connections"
+ 3) "thing_id"
+ 4) "63a110d4-2b77-48d2-aa46-2582681eeb82"
+ 5) "channels"
+ 6) "ff13ca9c-7322-4c28-a25c-4fe5c7b753fc, 925461e6-edfb-4755-9242-8a57199b90a5, c3642289-501d-4974-82f2-ecccc71b2d82"
+ 7) "timestamp"
+ 8) "1555405373"
+Whenever channel is updated, bootstrap service will generate and publish new update handler event. This event will have the following format:
1) "1693339403536-0"
+2) 1) "operation"
+ 2) "channel.update_handler"
+ 3) "channel_id"
+ 4) "0e602731-36ba-4a29-adba-e5761f356158"
+ 5) "name"
+ 6) "dry-sky"
+ 7) "metadata"
+ 8) "{\"log\":\"info\"}"
+ 9) "occurred_at"
+ 10) "1693339403536636387"
+Whenever channel is removed, bootstrap service will generate and publish new remove handler event. This event will have the following format:
1) "1693339468719-0"
+2) 1) "config_id"
+ 2) "0198b458-573e-415a-aa05-052ddab9709d"
+ 3) "operation"
+ 4) "config.remove_handler"
+ 5) "occurred_at"
+ 6) "1693339468719177463"
+Instead of using heartbeat to know when client is connected through MQTT adapter one can fetch events from Redis Streams that MQTT adapter publishes. MQTT adapter publishes events every time client connects and disconnects to stream named magistrala.mqtt.
Events that are coming from MQTT adapter have following fields:
+thing_id ID of a thing that has connected to MQTT adapter,event_type can have two possible values, connect and disconnect,instance represents MQTT adapter instance.occurred_at is in Epoch UNIX Time Stamp format.If you want to integrate through docker-compose.yml you can use magistrala-es-redis service. Just connect to it and consume events from Redis Stream named magistrala.mqtt.
Examples of connect event:
+In Redis Streams
+1) 1) "1693312937469-0"
+2) 1) "thing_id"
+ 1) "76a58221-e319-492a-be3e-b3d15631e92a"
+ 2) "event_type"
+ 3) "connect"
+ 4) "instance"
+ 5) ""
+ 6) "occurred_at"
+ 7) "1693312937469719069"
+In Nats JetStreams
+Subject: events.magistrala.mqtt Received: 2023-10-09T14:57:36+03:00
+
+{"event_type":"connect","instance":"","occurred_at":1696852656381976408,"thing_id":"9b23fec0-41a2-44ed-af13-7c54706b3291"}
+Example of disconnect event:
+In Redis Streams
+1) 1) "1693312937471-0"
+2) 1) "thing_id"
+ 2) "76a58221-e319-492a-be3e-b3d15631e92a"
+ 3) "event_type"
+ 4) "disconnect"
+ 5) "instance"
+ 6) ""
+ 7) "occurred_at"
+ 8) "1693312937471064150"
+In Nats JetStreams
+Subject: events.magistrala.mqtt Received: 2023-10-09T14:57:36+03:00
+
+{"event_type":"disconnect","instance":"","occurred_at":1696852656435238414,"thing_id":"9b23fec0-41a2-44ed-af13-7c54706b3291"}
+Before proceeding, install the following prerequisites:
+Once everything is installed, execute the following command from project root:
+make run
+This will start Magistrala docker composition, which will output the logs from the containers.
+Open a new terminal from which you can interact with the running Magistrala system. The easiest way to do this is by using the Magistrala CLI, which can be downloaded as a tarball from GitHub (here we use release 0.14.0 but be sure to use the latest CLI release):
wget -O- https://github.com/absmach/magistrala/releases/download/0.14.0/magistrala-cli_0.14.0_linux-amd64.tar.gz | tar xvz -C $GOBIN
+++Make sure that
+$GOBINis added to your$PATHso thatmagistrala-clicommand can be accessible system-wide
Build magistrala-cli if the pre-built CLI is not compatible with your OS, i.e MacOS. Please see the CLI for further details.
Once installed, you can use the CLI to quick-provision the system for testing:
+magistrala-cli provision test
+This command actually creates a temporary testing user, logs it in, then creates two things and two channels on behalf of this user. +This quickly provisions a Magistrala system with one simple testing scenario.
+You can read more about system provisioning in the dedicated Provisioning chapter
+Output of the command follows this pattern:
+{
+ "created_at": "2023-04-04T08:02:47.686337Z",
+ "credentials": {
+ "identity": "crazy_feistel@email.com",
+ "secret": "12345678"
+ },
+ "id": "0216df07-8f08-40ef-ba91-ff0e700f387a",
+ "name": "crazy_feistel",
+ "status": "enabled",
+ "updated_at": "2023-04-04T08:02:47.686337Z"
+}
+
+
+{
+ "access_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODA1OTYyNjcsImlhdCI6MTY4MDU5NTM2NywiaWRlbnRpdHkiOiJjcmF6eV9mZWlzdGVsQGVtYWlsLmNvbSIsImlzcyI6ImNsaWVudHMuYXV0aCIsInN1YiI6IjAyMTZkZjA3LThmMDgtNDBlZi1iYTkxLWZmMGU3MDBmMzg3YSIsInR5cGUiOiJhY2Nlc3MifQ.EpaFDcRjYAHwqhejLfay5ju8L1a7VdhXKohUlwTv7YTeOK-ClfNNx6KznV05Swdj6lgvbmVAfe0wz2JMpfMjdw",
+ "access_type": "Bearer",
+ "refresh_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODA2ODE3NjcsImlhdCI6MTY4MDU5NTM2NywiaWRlbnRpdHkiOiJjcmF6eV9mZWlzdGVsQGVtYWlsLmNvbSIsImlzcyI6ImNsaWVudHMuYXV0aCIsInN1YiI6IjAyMTZkZjA3LThmMDgtNDBlZi1iYTkxLWZmMGU3MDBmMzg3YSIsInR5cGUiOiJyZWZyZXNoIn0.3xcrkIBbi2a8firNHtnK6I8sBBOgrQ6XBa3x7cybKc6omOuqrkkNjXGjKU9tgShvjpfCWT48AR1VqO_VxJxL8g"
+}
+
+
+[
+ {
+ "created_at": "2023-04-04T08:02:47.81865461Z",
+ "credentials": {
+ "secret": "fc9473d8-6756-4fcc-968f-ea43cd0b803b"
+ },
+ "id": "5d5e593b-7629-4cc3-bebc-b20d8ab9dbef",
+ "name": "d0",
+ "owner": "0216df07-8f08-40ef-ba91-ff0e700f387a",
+ "status": "enabled",
+ "updated_at": "2023-04-04T08:02:47.81865461Z"
+ },
+ {
+ "created_at": "2023-04-04T08:02:47.818661382Z",
+ "credentials": {
+ "secret": "56a4b1bd-9750-42b3-a3cb-cf5ee2b86fe4"
+ },
+ "id": "45048a8e-c602-4e91-9556-a9d3af6617fb",
+ "name": "d1",
+ "owner": "0216df07-8f08-40ef-ba91-ff0e700f387a",
+ "status": "enabled",
+ "updated_at": "2023-04-04T08:02:47.818661382Z"
+ }
+]
+
+
+[
+ {
+ "created_at": "2023-04-04T08:02:47.857619Z",
+ "id": "a31e16f8-343c-4366-8b4f-c95e190937f4",
+ "name": "c0",
+ "owner_id": "0216df07-8f08-40ef-ba91-ff0e700f387a",
+ "status": "enabled",
+ "updated_at": "2023-04-04T08:02:47.857619Z"
+ },
+ {
+ "created_at": "2023-04-04T08:02:47.867336Z",
+ "id": "e20ad0bb-c490-47dd-9366-fb8ffd56c5dc",
+ "name": "c1",
+ "owner_id": "0216df07-8f08-40ef-ba91-ff0e700f387a",
+ "status": "enabled",
+ "updated_at": "2023-04-04T08:02:47.867336Z"
+ }
+]
+In the Magistrala system terminal (where docker compose is running) you should see following logs:
+...
+magistrala-users | {"level":"info","message":"Method register_client with id 0216df07-8f08-40ef-ba91-ff0e700f387a using token took 87.335902ms to complete without errors.","ts":"2023-04-04T08:02:47.722776862Z"}
+magistrala-users | {"level":"info","message":"Method issue_token of type Bearer for client crazy_feistel@email.com took 55.342161ms to complete without errors.","ts":"2023-04-04T08:02:47.783884818Z"}
+magistrala-users | {"level":"info","message":"Method identify for token eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODA1OTYyNjcsImlhdCI6MTY4MDU5NTM2NywiaWRlbnRpdHkiOiJjcmF6eV9mZWlzdGVsQGVtYWlsLmNvbSIsImlzcyI6ImNsaWVudHMuYXV0aCIsInN1YiI6IjAyMTZkZjA3LThmMDgtNDBlZi1iYTkxLWZmMGU3MDBmMzg3YSIsInR5cGUiOiJhY2Nlc3MifQ.EpaFDcRjYAHwqhejLfay5ju8L1a7VdhXKohUlwTv7YTeOK-ClfNNx6KznV05Swdj6lgvbmVAfe0wz2JMpfMjdw with id 0216df07-8f08-40ef-ba91-ff0e700f387a took 1.389463ms to complete without errors.","ts":"2023-04-04T08:02:47.817018631Z"}
+magistrala-things | {"level":"info","message":"Method create_things 2 things using token eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODA1OTYyNjcsImlhdCI6MTY4MDU5NTM2NywiaWRlbnRpdHkiOiJjcmF6eV9mZWlzdGVsQGVtYWlsLmNvbSIsImlzcyI6ImNsaWVudHMuYXV0aCIsInN1YiI6IjAyMTZkZjA3LThmMDgtNDBlZi1iYTkxLWZmMGU3MDBmMzg3YSIsInR5cGUiOiJhY2Nlc3MifQ.EpaFDcRjYAHwqhejLfay5ju8L1a7VdhXKohUlwTv7YTeOK-ClfNNx6KznV05Swdj6lgvbmVAfe0wz2JMpfMjdw took 48.137759ms to complete without errors.","ts":"2023-04-04T08:02:47.853310066Z"}
+magistrala-users | {"level":"info","message":"Method identify for token eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODA1OTYyNjcsImlhdCI6MTY4MDU5NTM2NywiaWRlbnRpdHkiOiJjcmF6eV9mZWlzdGVsQGVtYWlsLmNvbSIsImlzcyI6ImNsaWVudHMuYXV0aCIsInN1YiI6IjAyMTZkZjA3LThmMDgtNDBlZi1iYTkxLWZmMGU3MDBmMzg3YSIsInR5cGUiOiJhY2Nlc3MifQ.EpaFDcRjYAHwqhejLfay5ju8L1a7VdhXKohUlwTv7YTeOK-ClfNNx6KznV05Swdj6lgvbmVAfe0wz2JMpfMjdw with id 0216df07-8f08-40ef-ba91-ff0e700f387a took 302.571µs to complete without errors.","ts":"2023-04-04T08:02:47.856820523Z"}
+magistrala-things | {"level":"info","message":"Method create_channel for 2 channels using token eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODA1OTYyNjcsImlhdCI6MTY4MDU5NTM2NywiaWRlbnRpdHkiOiJjcmF6eV9mZWlzdGVsQGVtYWlsLmNvbSIsImlzcyI6ImNsaWVudHMuYXV0aCIsInN1YiI6IjAyMTZkZjA3LThmMDgtNDBlZi1iYTkxLWZmMGU3MDBmMzg3YSIsInR5cGUiOiJhY2Nlc3MifQ.EpaFDcRjYAHwqhejLfay5ju8L1a7VdhXKohUlwTv7YTeOK-ClfNNx6KznV05Swdj6lgvbmVAfe0wz2JMpfMjdw took 15.340692ms to complete without errors.","ts":"2023-04-04T08:02:47.872089509Z"}
+magistrala-users | {"level":"info","message":"Method identify for token eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODA1OTYyNjcsImlhdCI6MTY4MDU5NTM2NywiaWRlbnRpdHkiOiJjcmF6eV9mZWlzdGVsQGVtYWlsLmNvbSIsImlzcyI6ImNsaWVudHMuYXV0aCIsInN1YiI6IjAyMTZkZjA3LThmMDgtNDBlZi1iYTkxLWZmMGU3MDBmMzg3YSIsInR5cGUiOiJhY2Nlc3MifQ.EpaFDcRjYAHwqhejLfay5ju8L1a7VdhXKohUlwTv7YTeOK-ClfNNx6KznV05Swdj6lgvbmVAfe0wz2JMpfMjdw with id 0216df07-8f08-40ef-ba91-ff0e700f387a took 271.162µs to complete without errors.","ts":"2023-04-04T08:02:47.875812318Z"}
+magistrala-things | {"level":"info","message":"Method add_policy for client with id 5d5e593b-7629-4cc3-bebc-b20d8ab9dbef using token eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODA1OTYyNjcsImlhdCI6MTY4MDU5NTM2NywiaWRlbnRpdHkiOiJjcmF6eV9mZWlzdGVsQGVtYWlsLmNvbSIsImlzcyI6ImNsaWVudHMuYXV0aCIsInN1YiI6IjAyMTZkZjA3LThmMDgtNDBlZi1iYTkxLWZmMGU3MDBmMzg3YSIsInR5cGUiOiJhY2Nlc3MifQ.EpaFDcRjYAHwqhejLfay5ju8L1a7VdhXKohUlwTv7YTeOK-ClfNNx6KznV05Swdj6lgvbmVAfe0wz2JMpfMjdw took 28.632906ms to complete without errors.","ts":"2023-04-04T08:02:47.904041832Z"}
+magistrala-users | {"level":"info","message":"Method identify for token eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODA1OTYyNjcsImlhdCI6MTY4MDU5NTM2NywiaWRlbnRpdHkiOiJjcmF6eV9mZWlzdGVsQGVtYWlsLmNvbSIsImlzcyI6ImNsaWVudHMuYXV0aCIsInN1YiI6IjAyMTZkZjA3LThmMDgtNDBlZi1iYTkxLWZmMGU3MDBmMzg3YSIsInR5cGUiOiJhY2Nlc3MifQ.EpaFDcRjYAHwqhejLfay5ju8L1a7VdhXKohUlwTv7YTeOK-ClfNNx6KznV05Swdj6lgvbmVAfe0wz2JMpfMjdw with id 0216df07-8f08-40ef-ba91-ff0e700f387a took 269.959µs to complete without errors.","ts":"2023-04-04T08:02:47.906989497Z"}
+magistrala-things | {"level":"info","message":"Method add_policy for client with id 5d5e593b-7629-4cc3-bebc-b20d8ab9dbef using token eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODA1OTYyNjcsImlhdCI6MTY4MDU5NTM2NywiaWRlbnRpdHkiOiJjcmF6eV9mZWlzdGVsQGVtYWlsLmNvbSIsImlzcyI6ImNsaWVudHMuYXV0aCIsInN1YiI6IjAyMTZkZjA3LThmMDgtNDBlZi1iYTkxLWZmMGU3MDBmMzg3YSIsInR5cGUiOiJhY2Nlc3MifQ.EpaFDcRjYAHwqhejLfay5ju8L1a7VdhXKohUlwTv7YTeOK-ClfNNx6KznV05Swdj6lgvbmVAfe0wz2JMpfMjdw took 6.303771ms to complete without errors.","ts":"2023-04-04T08:02:47.910594262Z"}
+magistrala-users | {"level":"info","message":"Method identify for token eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODA1OTYyNjcsImlhdCI6MTY4MDU5NTM2NywiaWRlbnRpdHkiOiJjcmF6eV9mZWlzdGVsQGVtYWlsLmNvbSIsImlzcyI6ImNsaWVudHMuYXV0aCIsInN1YiI6IjAyMTZkZjA3LThmMDgtNDBlZi1iYTkxLWZmMGU3MDBmMzg3YSIsInR5cGUiOiJhY2Nlc3MifQ.EpaFDcRjYAHwqhejLfay5ju8L1a7VdhXKohUlwTv7YTeOK-ClfNNx6KznV05Swdj6lgvbmVAfe0wz2JMpfMjdw with id 0216df07-8f08-40ef-ba91-ff0e700f387a took 364.448µs to complete without errors.","ts":"2023-04-04T08:02:47.912905436Z"}
+magistrala-things | {"level":"info","message":"Method add_policy for client with id 45048a8e-c602-4e91-9556-a9d3af6617fb using token eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODA1OTYyNjcsImlhdCI6MTY4MDU5NTM2NywiaWRlbnRpdHkiOiJjcmF6eV9mZWlzdGVsQGVtYWlsLmNvbSIsImlzcyI6ImNsaWVudHMuYXV0aCIsInN1YiI6IjAyMTZkZjA3LThmMDgtNDBlZi1iYTkxLWZmMGU3MDBmMzg3YSIsInR5cGUiOiJhY2Nlc3MifQ.EpaFDcRjYAHwqhejLfay5ju8L1a7VdhXKohUlwTv7YTeOK-ClfNNx6KznV05Swdj6lgvbmVAfe0wz2JMpfMjdw took 7.73352ms to complete without errors.","ts":"2023-04-04T08:02:47.920205467Z"}
+...
+This proves that these provisioning commands were sent from the CLI to the Magistrala system.
+Once system is provisioned, a thing can start sending messages on a channel:
magistrala-cli messages send <channel_id> '[{"bn":"some-base-name:","bt":1.276020076001e+09, "bu":"A","bver":5, "n":"voltage","u":"V","v":120.1}, {"n":"current","t":-5,"v":1.2}, {"n":"current","t":-4,"v":1.3}]' <thing_secret>
+For example:
+magistrala-cli messages send a31e16f8-343c-4366-8b4f-c95e190937f4 '[{"bn":"some-base-name:","bt":1.276020076001e+09, "bu":"A","bver":5, "n":"voltage","u":"V","v":120.1}, {"n":"current","t":-5,"v":1.2}, {"n":"current","t":-4,"v":1.3}]' fc9473d8-6756-4fcc-968f-ea43cd0b803b
+In the Magistrala system terminal you should see following logs:
+...
+magistrala-things | {"level":"info","message":"Method authorize_by_key for channel with id a31e16f8-343c-4366-8b4f-c95e190937f4 by client with secret fc9473d8-6756-4fcc-968f-ea43cd0b803b took 7.048706ms to complete without errors.","ts":"2023-04-04T08:06:09.750992633Z"}
+magistrala-broker | [1] 2023/04/04 08:06:09.753072 [TRC] 192.168.144.11:60616 - cid:10 - "v1.18.0:go" - <<- [PUB channels.a31e16f8-343c-4366-8b4f-c95e190937f4 261]
+magistrala-broker | [1] 2023/04/04 08:06:09.754037 [TRC] 192.168.144.11:60616 - cid:10 - "v1.18.0:go" - <<- MSG_PAYLOAD: ["\n$a31e16f8-343c-4366-8b4f-c95e190937f4\x1a$5d5e593b-7629-4cc3-bebc-b20d8ab9dbef\"\x04http*\xa6\x01[{\"bn\":\"some-base-name:\",\"bt\":1.276020076001e+09, \"bu\":\"A\",\"bver\":5, \"n\":\"voltage\",\"u\":\"V\",\"v\":120.1}, {\"n\":\"current\",\"t\":-5,\"v\":1.2}, {\"n\":\"current\",\"t\":-4,\"v\":1.3}]0\xd9\xe6\x8b\xc9Ø\xab\xa9\x17"]
+magistrala-broker | [1] 2023/04/04 08:06:09.755550 [TRC] 192.168.144.13:58572 - cid:8 - "v1.18.0:go" - ->> [MSG channels.a31e16f8-343c-4366-8b4f-c95e190937f4 1 261]
+magistrala-http | {"level":"info","message":"Method publish to channel a31e16f8-343c-4366-8b4f-c95e190937f4 took 15.979094ms to complete without errors.","ts":"2023-04-04T08:06:09.75232571Z"}
+...
+This proves that messages have been correctly sent through the system via the protocol adapter (magistrala-http).
Magistrala is modern, scalable, secure open source and patent-free IoT cloud platform written in Go.
+It accepts user and thing connections over various network protocols (i.e. HTTP, MQTT, WebSocket, CoAP), thus making a seamless bridge between them. It is used as the IoT middleware for building complex IoT solutions.
+
Thank you for your interest in Magistrala and the desire to contribute!
+Take a look at our open issues. The good-first-issue label is specifically for issues that are great for getting started. Checkout the contribution guide to learn more about our style and conventions. Make your changes compatible to our workflow.
+Magistrala can be easily deployed on Kubernetes platform by using Helm Chart from official Magistrala DevOps GitHub repository.
+Kubernetes is an open source container orchestration engine for automating deployment, scaling, and management of containerised applications. Install it locally or have access to a cluster. Follow these instructions if you need more information.
+Kubectl is official Kubernetes command line client. Follow these instructions to install it.
+Regarding the cluster control with kubectl, default config .yaml file should be ~/.kube/config.
Helm is the package manager for Kubernetes. Follow these instructions to install it.
+Add a stable chart repository:
+helm repo add stable https://charts.helm.sh/stable
+Add a bitnami chart repository:
+helm repo add bitnami https://charts.bitnami.com/bitnami
+Follow these instructions to install it or:
+helm install ingress-nginx ingress-nginx/ingress-nginx --version 3.26.0 --create-namespace -n ingress-nginx
+Get Helm charts from Magistrala DevOps GitHub repository:
+git clone https://github.com/absmach/devops.git
+cd devops/charts/mainflux
+Update the on-disk dependencies to mirror Chart.yaml:
+helm dependency update
+If you didn't already have namespace created you should do it with:
+kubectl create namespace mf
+Deploying release named magistrala in namespace named mf is done with just:
helm install magistrala . -n mf
+Magistrala is now deployed on your Kubernetes.
+You can override default values while installing with --set option. For example, if you want to specify ingress hostname and pull latest tag of users image:
helm install magistrala -n mf --set ingress.hostname='example.com' --set users.image.tag='latest'
+Or if release is already installed, you can update it:
+helm upgrade magistrala -n mf --set ingress.hostname='example.com' --set users.image.tag='latest'
+The following table lists the configurable parameters and their default values.
+| Parameter | +Description | +Default | +
|---|---|---|
| defaults.logLevel | +Log level | +debug | +
| defaults.image.pullPolicy | +Docker Image Pull Policy | +IfNotPresent | +
| defaults.image.repository | +Docker Image Repository | +magistrala | +
| defaults.image.tag | +Docker Image Tag | +0.13.0 | +
| defaults.replicaCount | +Replicas of MQTT adapter, Things, Envoy and Authn | +3 | +
| defaults.messageBrokerUrl | +Message broker URL, the default is NATS Url | +nats://nats:4222 | +
| defaults.jaegerPort | +Jaeger port | +6831 | +
| nginxInternal.mtls.tls | +TLS secret which contains the server cert/key | ++ |
| nginxInternal.mtls.intermediateCrt | +Generic secret which contains the intermediate cert used to verify clients | ++ |
| ingress.enabled | +Should the Nginx Ingress be created | +true | +
| ingress.hostname | +Hostname for the Nginx Ingress | ++ |
| ingress.tls.hostname | +Hostname of the Nginx Ingress certificate | ++ |
| ingress.tls.secret | +TLS secret for the Nginx Ingress | ++ |
| messageBroker.maxPayload | +Maximum payload size in bytes that the Message Broker server, if it is NATS, server will accept | +268435456 | +
| messageBroker.replicaCount | +Message Broker replicas | +3 | +
| users.dbPort | +Users service DB port | +5432 | +
| users.httpPort | +Users service HTTP port | +9000 | +
| things.dbPort | +Things service DB port | +5432 | +
| things.httpPort | +Things service HTTP port | +9001 | +
| things.authGrpcPort | +Things service Auth gRPC port | +7000 | +
| things.authHttpPort | +Things service Auth HTTP port | +9002 | +
| things.redisESPort | +Things service Redis Event Store port | +6379 | +
| things.redisCachePort | +Things service Redis Auth Cache port | +6379 | +
| adapter_http.httpPort | +HTTP adapter port | +8185 | +
| mqtt.proxy.mqttPort | +MQTT adapter proxy port | +1884 | +
| mqtt.proxy.wsPort | +MQTT adapter proxy WS port | +8081 | +
| mqtt.broker.mqttPort | +MQTT adapter broker port | +1883 | +
| mqtt.broker.wsPort | +MQTT adapter broker WS port | +8080 | +
| mqtt.broker.persistentVolume.size | +MQTT adapter broker data Persistent Volume size | +5Gi | +
| mqtt.redisESPort | +MQTT adapter Event Store port | +6379 | +
| mqtt.redisCachePort | +MQTT adapter Redis Auth Cache port | +6379 | +
| adapter_coap.udpPort | +CoAP adapter UDP port | +5683 | +
| ui.port | +UI port | +3000 | +
| bootstrap.enabled | +Enable bootstrap service | +false | +
| bootstrap.dbPort | +Bootstrap service DB port | +5432 | +
| bootstrap.httpPort | +Bootstrap service HTTP port | +9013 | +
| bootstrap.redisESPort | +Bootstrap service Redis Event Store port | +6379 | +
| influxdb.enabled | +Enable InfluxDB reader & writer | +false | +
| influxdb.dbPort | +InfluxDB port | +8086 | +
| influxdb.writer.httpPort | +InfluxDB writer HTTP port | +9006 | +
| influxdb.reader.httpPort | +InfluxDB reader HTTP port | +9005 | +
| adapter_opcua.enabled | +Enable OPC-UA adapter | +false | +
| adapter_opcua.httpPort | +OPC-UA adapter HTTP port | +8188 | +
| adapter_opcua.redisRouteMapPort | +OPC-UA adapter Redis Auth Cache port | +6379 | +
| adapter_lora.enabled | +Enable LoRa adapter | +false | +
| adapter_lora.httpPort | +LoRa adapter HTTP port | +8187 | +
| adapter_lora.redisRouteMapPort | +LoRa adapter Redis Auth Cache port | +6379 | +
| twins.enabled | +Enable twins service | +false | +
| twins.dbPort | +Twins service DB port | +27017 | +
| twins.httpPort | +Twins service HTTP port | +9021 | +
| twins.redisCachePort | +Twins service Redis Cache port | +6379 | +
All Magistrala services (both core and add-ons) can have their logLevel, image.pullPolicy, image.repository and image.tag overridden.
Magistrala Core is a minimalistic set of required Magistrala services. They are all installed by default:
+Magistrala Add-ons are optional services that are disabled by default. Find in Configuration table parameters for enabling them, i.e. to enable influxdb reader & writer you should run helm install with --set influxdb=true.
+List of add-ons services in charts:
By default scale of MQTT adapter, Things, Envoy, Authn and the Message Broker will be set to 3. It's recommended that you set this values to number of your nodes in Kubernetes cluster, i.e. --set defaults.replicaCount=3 --set messageBroker.replicaCount=3
To send MQTT messages to your host on ports 1883 and 8883 some additional steps are required in configuring NGINX Ingress Controller.
NGINX Ingress Controller uses ConfigMap to expose TCP and UDP services. That ConfigMaps are included in helm chart in ingress.yaml file assuming that location of ConfigMaps should be ingress-nginx/tcp-services and ingress-nginx/udp-services. These locations was set with --tcp-services-configmap and --udp-services-configmap flags and you can check it in deployment of Ingress Controller or add it there in args section for nginx-ingress-controller if it's not already specified. This is explained in NGINX Ingress documentation
Also, these three ports need to be exposed in the Service defined for the Ingress. You can do that with command that edit your service:
+kubectl edit svc -n ingress-nginx nginx-ingress-ingress-nginx-controller
and add in spec->ports:
+- name: mqtt
+ port: 1883
+ protocol: TCP
+ targetPort: 1883
+- name: mqtts
+ port: 8883
+ protocol: TCP
+ targetPort: 8883
+- name: coap
+ port: 5683
+ protocol: UDP
+ targetPort: 5683
+For testing purposes you can generate certificates as explained in detail in authentication chapter of this document. So, you can use this script and after replacing all localhost with your hostname, run:
make ca
+make server_cert
+make thing_cert KEY=<thing_secret>
+you should get in certs folder these certificates that we will use for setting up TLS and mTLS:
ca.crt
+ca.key
+ca.srl
+magistrala-server.crt
+magistrala-server.key
+thing.crt
+thing.key
+Create kubernetes secrets using those certificates with running commands from secrets script. In this example secrets are created in mf namespace:
kubectl -n mf create secret tls magistrala-server --key magistrala-server.key --cert magistrala-server.crt
+
+kubectl -n mf create secret generic ca --from-file=ca.crt
+You can check if they are succesfully created:
+kubectl get secrets -n mf
+And now set ingress.hostname, ingress.tls.hostname to your hostname and ingress.tls.secret to magistrala-server and after helm update you have secured ingress with TLS certificate.
For mTLS you need to set nginx_internal.mtls.tls="magistrala-server" and nginx_internal.mtls.intermediate_crt="ca".
Now you can test sending mqtt message with this parameters:
+mosquitto_pub -d -L mqtts://<thing_id>:<thing_secret>@example.com:8883/channels/<channel_id>/messages --cert thing.crt --key thing.key --cafile ca.crt -m "test-message"
+Bridging with LoRaWAN Networks can be done over the lora-adapter. This service sits between Magistrala and LoRa Server and just forwards the messages from one system to another via MQTT protocol, using the adequate MQTT topics and in the good message format (JSON and SenML), i.e. respecting the APIs of both systems.
+LoRa Server is used for connectivity layer. Specially for the LoRa Gateway Bridge service, which abstracts the SemTech packet-forwarder UDP protocol into JSON over MQTT. But also for the LoRa Server service, responsible of the de-duplication and handling of uplink frames received by the gateway(s), handling of the LoRaWAN mac-layer and scheduling of downlink data transmissions. Finally the Lora App Server services is used to interact with the system.
+Before to run the lora-adapter you must install and run LoRa Server. First, execute the following command:
go get github.com/brocaar/loraserver-docker
+Once everything is installed, execute the following command from the LoRa Server project root:
+docker-compose up
+Troubleshouting: Magistrala and LoRa Server use their own MQTT brokers which by default occupy MQTT port 1883. If both are ran on the same machine different ports must be used. You can fix this on Magistrala side by configuring the environment variable MF_MQTT_ADAPTER_MQTT_PORT.
Now that both systems are running you must provision LoRa Server, which offers for integration with external services, a RESTful and gRPC API. You can do it as well over the LoRa App Server, which is good example of integration.
+network session key and application session key of your Device. You can generate and copy them on your device configuration or you can use your own pre generated keys and set them using the LoRa App Server UI.
+ Device connect through OTAA. Make sure that loraserver device-profile is using same release as device. If MAC version is 1.0.X, application key = app_key and app_eui = deviceEUI. If MAC version is 1.1 or ABP both parameters will be needed, APP_key and Network key.Once everything is running and the LoRa Server is provisioned, execute the following command from Magistrala project root to run the lora-adapter:
+docker-compose -f docker/addons/lora-adapter/docker-compose.yml up -d
+Troubleshouting: The lora-adapter subscribes to the LoRa Server MQTT broker and will fail if the connection is not established. You must ensure that the environment variable MF_LORA_ADAPTER_MESSAGES_URL is propertly configured.
Remark: By defaut, MF_LORA_ADAPTER_MESSAGES_URL is set as tcp://lora.mqtt.magistrala.io:1883 in the docker-compose.yml file of the adapter. If you run the composition without configure this variable you will start to receive messages from our demo server.
The lora-adapter use Redis database to create a route map between both systems. As in Magistrala we use Channels to connect Things, LoRa Server uses Applications to connect Devices.
+The lora-adapter uses the matadata of provision events emitted by Magistrala system to update his route map. For that, you must provision Magistrala Channels and Things with an extra metadata key in the JSON Body of the HTTP request. It must be a JSON object with key lora which value is another JSON object. This nested JSON object should contain app_id or dev_eui field. In this case app_id or dev_eui must be an existent Lora application ID or device EUI:
Channel structure:
+{
+ "name": "<channel name>",
+ "metadata:": {
+ "lora": {
+ "app_id": "<application ID>"
+ }
+ }
+}
+Thing structure:
+{
+ "type": "device",
+ "name": "<thing name>",
+ "metadata:": {
+ "lora": {
+ "dev_eui": "<device EUI>"
+ }
+ }
+}
+To forward LoRa messages the lora-adapter subscribes to topics applications/+/devices/+ of the LoRa Server MQTT broker. It verifies the app_id and the dev_eui of received messages. If the mapping exists it uses corresponding Channel ID and Thing ID to sign and forwards the content of the LoRa message to the Magistrala message broker.
Once a channel is provisioned and thing is connected to it, it can start to publish messages on the channel. The following sections will provide an example of message publishing for each of the supported protocols.
+To publish message over channel, thing should send following request:
+curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Content-Type: application/senml+json" -H "Authorization: Thing <thing_secret>" https://localhost/http/channels/<channel_id>/messages -d '[{"bn":"some-base-name:","bt":1.276020076001e+09, "bu":"A","bver":5, "n":"voltage","u":"V","v":120.1}, {"n":"current","t":-5,"v":1.2}, {"n":"current","t":-4,"v":1.3}]'
+Note that if you're going to use senml message format, you should always send messages as an array.
+For more information about the HTTP messaging service API, please check out the API documentation.
+To send and receive messages over MQTT you could use Mosquitto tools, or Paho if you want to use MQTT over WebSocket.
+To publish message over channel, thing should call following command:
+mosquitto_pub -u <thing_id> -P <thing_secret> -t channels/<channel_id>/messages -h localhost -m '[{"bn":"some-base-name:","bt":1.276020076001e+09, "bu":"A","bver":5, "n":"voltage","u":"V","v":120.1}, {"n":"current","t":-5,"v":1.2}, {"n":"current","t":-4,"v":1.3}]'
+To subscribe to channel, thing should call following command:
+mosquitto_sub -u <thing_id> -P <thing_secret> -t channels/<channel_id>/messages -h localhost
+If you want to use standard topic such as channels/<channel_id>/messages with SenML content type (JSON or CBOR), you should use following topic channels/<channel_id>/messages.
If you are using TLS to secure MQTT connection, add --cafile docker/ssl/certs/ca.crt
+to every command.
CoAP adapter implements CoAP protocol using underlying UDP and according to RFC 7252. To send and receive messages over CoAP, you can use CoAP CLI. To set the add-on, please follow the installation instructions provided here.
+Examples:
+coap-cli get channels/<channel_id>/messages/subtopic -auth <thing_secret> -o
+coap-cli post channels/<channel_id>/messages/subtopic -auth <thing_secret> -d "hello world"
+coap-cli post channels/<channel_id>/messages/subtopic -auth <thing_secret> -d "hello world" -h 0.0.0.0 -p 1234
+To send a message, use POST request. To subscribe, send GET request with Observe option (flag o) set to false. There are two ways to unsubscribe:
GET request with Observe option set to true.RST message as a response to CONF message received by the server.The most of the notifications received from the Adapter are non-confirmable. By RFC 7641:
+++Server must send a notification in a confirmable message instead of a non-confirmable message at least every 24 hours. This prevents a client that went away or is no longer interested from remaining in the list of observers indefinitely.
+
CoAP Adapter sends these notifications every 12 hours. To configure this period, please check adapter documentation If the client is no longer interested in receiving notifications, the second scenario described above can be used to unsubscribe.
+To publish and receive messages over channel using web socket, you should first send handshake request to /channels/<channel_id>/messages path. Don't forget to send Authorization header with thing authorization token. In order to pass message content type to WS adapter you can use Content-Type header.
If you are not able to send custom headers in your handshake request, send them as query parameter authorization and content-type. Then your path should look like this /channels/<channel_id>/messages?authorization=<thing_secret>&content-type=<content-type>.
If you are using the docker environment prepend the url with ws. So for example /ws/channels/<channel_id>/messages?authorization=<thing_secret>&content-type=<content-type>.
const WebSocket = require("ws");
+// do not verify self-signed certificates if you are using one
+process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";
+// c02ff576-ccd5-40f6-ba5f-c85377aad529 is an example of a thing_auth_key
+const ws = new WebSocket(
+ "ws://localhost:8186/ws/channels/1/messages?authorization=c02ff576-ccd5-40f6-ba5f-c85377aad529"
+);
+ws.on("open", () => {
+ ws.send("something");
+});
+ws.on("message", (data) => {
+ console.log(data);
+});
+ws.on("error", (e) => {
+ console.log(e);
+});
+package main
+
+import (
+ "log"
+ "os"
+ "os/signal"
+ "time"
+
+ "github.com/gorilla/websocket"
+)
+
+var done chan interface{}
+var interrupt chan os.Signal
+
+func receiveHandler(connection *websocket.Conn) {
+ defer close(done)
+
+ for {
+ _, msg, err := connection.ReadMessage()
+ if err != nil {
+ log.Fatal("Error in receive: ", err)
+ return
+ }
+
+ log.Printf("Received: %s\n", msg)
+ }
+}
+
+func main() {
+ done = make(chan interface{})
+ interrupt = make(chan os.Signal)
+
+ signal.Notify(interrupt, os.Interrupt)
+
+ channelId := "30315311-56ba-484d-b500-c1e08305511f"
+ thingSecret := "c02ff576-ccd5-40f6-ba5f-c85377aad529"
+
+ socketUrl := "ws://localhost:8186/channels/" + channelId + "/messages/?authorization=" + thingSecret
+
+ conn, _, err := websocket.DefaultDialer.Dial(socketUrl, nil)
+ if err != nil {
+ log.Fatal("Error connecting to Websocket Server: ", err)
+ } else {
+ log.Println("Connected to the ws adapter")
+ }
+ defer conn.Close()
+
+ go receiveHandler(conn)
+
+ for {
+ select {
+
+ case <-interrupt:
+ log.Println("Interrupt occured, closing the connection...")
+ conn.Close()
+ err := conn.WriteMessage(websocket.TextMessage, []byte("closed this ws client just now"))
+ if err != nil {
+ log.Println("Error during closing websocket: ", err)
+ return
+ }
+
+ select {
+ case <-done:
+ log.Println("Receiver Channel Closed! Exiting...")
+
+ case <-time.After(time.Duration(1) * time.Second):
+ log.Println("Timeout in closing receiving channel. Exiting...")
+ }
+ return
+ }
+ }
+}
+Magistrala also supports MQTT-over-WS, along with pure WS protocol. this bring numerous benefits for IoT applications that are derived from the properties of MQTT - like QoS and PUB/SUB features.
+There are 2 reccomended Javascript libraries for implementing browser support for Magistrala MQTT-over-WS connectivity:
+ +As WS is an extension of HTTP protocol, Magistrala exposes it on port 8008, so it's usage is practically transparent.
+Additionally, please notice that since same port as for HTTP is used (8008), and extension URL /mqtt should be used -
+i.e. connection URL should be ws://<host_addr>/mqtt.
For quick testing you can use HiveMQ UI tool.
+Here is an example of a browser application connecting to Magistrala server and sending and receiving messages over WebSocket using MQTT.js library:
+<script src="https://unpkg.com/mqtt/dist/mqtt.min.js"></script>
+<script>
+ // Initialize a mqtt variable globally
+ console.log(mqtt)
+
+ // connection option
+ const options = {
+ clean: true, // retain session
+ connectTimeout: 4000, // Timeout period
+ // Authentication information
+ clientId: '14d6c682-fb5a-4d28-b670-ee565ab5866c',
+ username: '14d6c682-fb5a-4d28-b670-ee565ab5866c',
+ password: 'ec82f341-d4b5-4c77-ae05-34877a62428f',
+ }
+
+ var channelId = '08676a76-101d-439c-b62e-d4bb3b014337'
+ var topic = 'channels/' + channelId + '/messages'
+
+ // Connect string, and specify the connection method by the protocol
+ // ws Unencrypted WebSocket connection
+ // wss Encrypted WebSocket connection
+ const connectUrl = 'ws://localhost/mqtt'
+ const client = mqtt.connect(connectUrl, options)
+
+ client.on('reconnect', (error) => {
+ console.log('reconnecting:', error)
+ })
+
+ client.on('error', (error) => {
+ console.log('Connection failed:', error)
+ })
+
+ client.on('connect', function () {
+ console.log('client connected:' + options.clientId)
+ client.subscribe(topic, { qos: 0 })
+ client.publish(topic, 'WS connection demo!', { qos: 0, retain: false })
+ })
+
+ client.on('message', function (topic, message, packet) {
+ console.log('Received Message:= ' + message.toString() + '\nOn topic:= ' + topic)
+ })
+
+ client.on('close', function () {
+ console.log(options.clientId + ' disconnected')
+ })
+</script>
+N.B. Eclipse Paho lib adds sub-URL /mqtt automaticlly, so procedure for connecting to the server can be something like this:
var loc = { hostname: "localhost", port: 8008 };
+// Create a client instance
+client = new Paho.MQTT.Client(loc.hostname, Number(loc.port), "clientId");
+// Connect the client
+client.connect({ onSuccess: onConnect });
+In order to use subtopics and give more meaning to your pub/sub channel, you can simply add any suffix to base /channels/<channel_id>/messages topic.
Example subtopic publish/subscribe for bedroom temperature would be channels/<channel_id>/messages/bedroom/temperature.
Subtopics are generic and multilevel. You can use almost any suffix with any depth.
+Topics with subtopics are propagated to Message broker in the following format channels.<channel_id>.<optional_subtopic>.
Our example topic channels/<channel_id>/messages/bedroom/temperature will be translated to appropriate Message Broker topic channels.<channel_id>.bedroom.temperature.
You can use multilevel subtopics, that have multiple parts. These parts are separated by . or / separators.
+When you use combination of these two, have in mind that behind the scene, / separator will be replaced with ..
+Every empty part of subtopic will be removed. What this means is that subtopic a///b is equivalent to a/b.
+When you want to subscribe, you can use the default Message Broker, NATS, wildcards * and >. Every subtopic part can have * or > as it's value, but if there is any other character beside these wildcards, subtopic will be invalid. What this means is that subtopics such as a.b*c.d will be invalid, while a.b.*.c.d will be valid.
Authorization is done on channel level, so you only have to have access to channel in order to have access to +it's subtopics.
+Note: When using MQTT, it's recommended that you use standard MQTT wildcards + and #.
Magistrala supports the MQTT protocol for message exchange. MQTT is a lightweight Publish/Subscribe messaging protocol used to connect restricted devices in low bandwidth, high-latency or unreliable networks. The publish-subscribe messaging pattern requires a message broker. The broker is responsible for distributing messages to and from clients connected to the MQTT adapter.
+Magistrala supports MQTT version 3.1.1. The MQTT adapter is based on Eclipse Paho MQTT client library. The adapter is configured to use nats as the default MQTT broker, but you can use vernemq too.
+In the dev environment, docker profiles are preferred when handling different MQTT and message brokers supported by Magistrala.
+Magistrala uses two types of brokers:
+MQTT_BROKER: Handles MQTT communication between MQTT adapters and message broker.MESSAGE_BROKER: Manages communication between adapters and Magistrala writer services.MQTT_BROKER can be either vernemq or nats.
+MESSAGE_BROKER can be either nats or rabbitmq.
Each broker has a unique profile for configuration. The available profiles are:
+vernemq_nats: Uses vernemq as MQTT_BROKER and nats as MESSAGE_BROKER.vernemq_rabbitmq: Uses vernemq as MQTT_BROKER and rabbitmq as MESSAGE_BROKER.nats_nats: Uses nats as both MQTT_BROKER and MESSAGE_BROKER.nats_rabbitmq: Uses nats as MQTT_BROKER and rabbitmq as MESSAGE_BROKER.The following command will run VerneMQ as an MQTT broker and Nats as a message broker:
+MF_MQTT_BROKER_TYPE=vernemq MF_BROKER_TYPE=nats make run
+The following command will run NATS as an MQTT broker and RabbitMQ as a message broker:
+MF_MQTT_BROKER_TYPE=nats MF_BROKER_TYPE=rabbitmq make run
+By default, NATS is used as an MQTT broker and RabbitMQ as a message broker.
+NATS support for MQTT and it is designed to empower users to leverage their existing IoT deployments. NATS offers significant advantages in terms of security and observability when used end-to-end. NATS server as a drop-in replacement for MQTT is compelling. This approach allows you to retain your existing IoT investments while benefiting from NATS' secure, resilient, and scalable access to your streams and services.
+To enable MQTT support on NATS, JetStream needs to be enabled. This is done by default in Magistrala. This is because persistence is necessary for sessions and retained messages, even for QoS 0 retained messages. Communication between MQTT and NATS involves creating similar NATS subscriptions when MQTT clients subscribe to topics. This ensures that the interest is registered in the NATS cluster, and messages are delivered accordingly. When MQTT publishers send messages, they are converted to NATS subjects, and matching NATS subscriptions receive the MQTT messages.
+NATS supports up to QoS 1 subscriptions, where the server retains messages until it receives the PUBACK for the corresponding packet identifier. If PUBACK is not received within the "ack_wait" interval, the message is resent. The maximum value for "max_ack_pending" is 65535.
+NATS Server persists all sessions, even if they are created with the "clean session" flag. Sessions are identified by client identifiers. If two connections attempt to use the same client identifier, the server will close the existing connection and accept the new one, reducing the flapping rate.
+NATS supports MQTT in a NATS cluster, with the replication factor automatically set based on cluster size.
+VerneMQ is a powerful MQTT publish/subscribe message broker designed to implement the OASIS industry standard MQTT protocol. It is built to take messaging and IoT applications to the next level by providing a unique set of features related to scalability, reliability, high-performance, and operational simplicity.
+Key features of VerneMQ include:
+VerneMQ is designed from the ground up to work as a distributed message broker, ensuring continued operation even in the event of node or network failures. It can easily scale both horizontally and vertically to handle large numbers of concurrent clients.
+VerneMQ uses a master-less clustering technology, which means there are no special nodes like masters or slaves to consider when adding or removing nodes, making cluster operation safe and simple. This allows MQTT clients to connect to any cluster node and receive messages from any other node. However, it acknowledges the challenges of fulfilling MQTT specification guarantees in a distributed environment, particularly during network partitions.
+Magistrala supports multiple message brokers for message exchange. Message brokers are used to distribute messages to and from clients connected to the different protocols adapters and writers. Writers, which are responsible for storing messages in the database, are connected to the message broker using wildcard subscriptions. This means that writers will receive all messages published to the message broker. Clients can subscribe to the message broker using topic and subtopic combinations. The message broker will then forward all messages published to the topic and subtopic combination to the client.
+Magistrala supports NATS, RabbitMQ and Kafka as message brokers.
+Since Magistrala supports configurable message brokers, you can use Nats with JetStream enabled as a message broker. To do so, you need to set MF_BROKER_TYPE to nats and set MF_NATS_URL to the url of your nats instance. When using make command to start Magistrala MF_BROKER_URL is automatically set to MF_NATS_URL.
Since Magistrala is using nats:2.9.21-alpine docker image with the following configuration:
max_payload: 1MB
+max_connections: 1M
+port: $MF_NATS_PORT
+http_port: $MF_NATS_HTTP_PORT
+trace: true
+
+jetstream {
+ store_dir: "/data"
+ cipher: "aes"
+ key: $MF_NATS_JETSTREAM_KEY
+ max_mem: 1G
+}
+These are the default values but you can change them by editing the configuration file. For more information about nats configuration checkout official nats documentation. The health check endpoint is exposed on MF_NATS_HTTP_PORT and its /healthz path.
The main reason for using Nats with JetStream enabled is to have a distributed system with high availability and minimal dependencies. Nats is configure to run as the default message broker, but you can use any other message broker supported by Magistrala. Nats is configured to use JetStream, which is a distributed streaming platform built on top of nats. JetStream is used to store messages and to provide high availability. This makes nats to be used as the default event store, but you can use any other event store supported by Magistrala. Nats with JetStream enabled is also used as a key-value store for caching purposes. This makes nats to be used as the default cache store, but you can use any other cache store supported by Magistrala.
+This versatile architecture allows you to use nats alone for the MQTT broker, message broker, event store and cache store. This is the default configuration, but you can use any other MQTT broker, message broker, event store and cache store supported by Magistrala.
+Since Magistrala uses a configurable message broker, you can use RabbitMQ as a message broker. To do so, you need to set MF_BROKER_TYPE to rabbitmq and set MF_RABBITMQ_URL to the url of your RabbitMQ instance. When using make command to start Magistrala MF_BROKER_URL is automatically set to MF_RABBITMQ_URL.
Since Magistrala is using rabbitmq:3.9.20-management-alpine docker image, the management console is available at port MF_RABBITMQ_HTTP_PORT
Magistrala has one exchange for the entire platform called messages. This exchange is of type topic. The exchange is durable i.e. it will survive broker restarts and remain declared when there are no remaining bindings. The exchange does not auto-delete when all queues have finished using it. When declaring the exchange no_wait is set to false which means that the broker will wait for a confirmation from the server that the exchange was successfully declared. The exchange is not internal i.e. other exchanges can publish messages to it.
Magistrala uses topic-based routing to route messages to the appropriate queues. The routing key is in the format channels.<channel_id>.<optional_subtopic>. A few valid routing key examples: channels.318BC587-A68B-40D3-9026-3356FA4E702C, channels.318BC587-A68B-40D3-9026-3356FA4E702C.bedroom.temperature.
The AMQP published message doesn't contain any headers. The message body is the payload of the message.
+When subscribing to messages from a channel, a queue is created with the name channels.<channel_id>.<optional_subtopic>. The queue is durable i.e. it will survive broker restarts and remain declared when there are no remaining consumers or bindings. The queue does not auto-delete when all consumers have finished using it. The queue is not exclusive i.e. it can be accessed in other connections. When declaring the queue we set no_wait to false which means that the broker waits for a confirmation from the server that the queue was successfully declared. The queue is not passive i.e. the server creates the queue if it does not exist.
The queue is then bound to the exchange with the routing key channels.<channel_id>.<optional_subtopic>. The binding is not no-wait i.e. the broker waits for a confirmation from the server that the binding was successfully created.
Once this is done, the consumer can start consuming messages from the queue with a specific client ID. The consumer is not no-local i.e. the server will not send messages to the connection that published them. The consumer is not exclusive i.e. the queue can be accessed in other connections. The consumer is no-ack i.e. the server acknowledges
+deliveries to this consumer prior to writing the delivery to the network.
When Unsubscribing from a channel, the queue is unbound from the exchange and deleted.
+For more information and examples checkout official nats.io documentation, official rabbitmq documentation, official vernemq documentation and official kafka documentation.
+Since Magistrala uses a configurable message broker, you can use Kafka as a message broker. To do so, you need to set MF_BROKER_TYPE to kafka and set MF_KAFKA_URL to the url of your Kafka instance. When using make command to start Magistrala MF_BROKER_URL is automatically set to MF_KAFKA_URL.
Magistrala utilizes spotify/kafka:latest docker image. The image also exposes kafka:9092 and zookeeper:2181 ports. This is used for development purposes only. For production, it is assumed that you have your own Kafka cluster.
The publisher implementation is based on the segmentio/kafka-go library. Publishing messages is well supported by the library, but subscribing to topics is not. The library does not provide a way to subscribe to all topics, but only to a specific topic. This is a problem because the Magistrala platform uses a topic per channel, and the number of channels is not known in advance. The solution is to use the Zookeeper library to get a list of all topics and then subscribe to each of them. The list of topics is obtained by connecting to the Zookeeper server and reading the list of topics from the /brokers/topics node. The first message published from the topic can be lost if subscription happens closely followed by publishing. After the subscription, we guarantee that all messages will be received.
Magistrala publishes messages to Kafka using the channels.<channel_id>.<optional_subtopic> topic. A few valid topic examples: channels.318BC587-A68B-40D3-9026-3356FA4E702C, channels.318BC587-A68B-40D3-9026-3356FA4E702C.bedroom.temperature. All topics have a single partition and replication factor of 1. On publishing messages a topic is created and a writer is created for that topic. The writer is not closed after publishing a message. The writer is closed when the application is closed. The batchSize is set to 1, which means that messages are written to Kafka as soon as they are published. The requiredAcks is set to RequireAll which means that the server will wait for the leader to receive the message and wait for the full set of in-sync replicas to receive the message.
On subscribing to messages from a channel, a reader is configured with the topic channels.<channel_id>.<optional_subtopic>. The reader is not closed after reading a message. The reader is closed when the application is closed. The maxWait is set to 1 second, which means that the reader will wait for 1 second for new data to come when fetching batches of messages from kafka. The heartbeatInterval is set to 1 second, which means that the reader will send the consumer group heartbeat update every 1 second. The partitionWatchInterval is set to 1 second, which means that the reader will check for partition changes every 1 second. The sessionTimeout is set to 1 second, which means that the coordinator will consider the consumer dead and initiate a rebalance if no heartbeat is received for 1 second. For wildcard topics since kafka does not support wildcards, Magistrala subscribes to all topics and filter the messages received.
When Unsubscribing from a channel, the reader is closed.
+For more information and examples checkout official nats.io documentation, official rabbitmq documentation, official vernemq documentation and official kafka documentation.
+ + + + + + +Bridging with an OPC-UA Server can be done over the opcua-adapter. This service sits between Magistrala and an OPC-UA Server and just forwards the messages from one system to another.
+The OPC-UA Server is used for connectivity layer. It allows various methods to read information from the OPC-UA server and its nodes. The current version of the opcua-adapter still experimental and only Browse and Subscribe methods are implemented. Public OPC-UA test servers are available for testing of OPC-UA clients and can be used for development and test purposes.
Execute the following command from Magistrala project root to run the opcua-adapter:
+docker-compose -f docker/addons/opcua-adapter/docker-compose.yml up -d
+The opcua-adapter use Redis database to create a route-map between Magistrala and an OPC-UA Server. As Magistrala use Things and Channels IDs to sign messages, OPC-UA use node ID (node namespace and node identifier combination) and server URI. The adapter route-map associate a Thing ID with a Node ID and a Channel ID with a Server URI.
The opcua-adapter uses the matadata of provision events emitted by Magistrala system to update its route map. For that, you must provision Magistrala Channels and Things with an extra metadata key in the JSON Body of the HTTP request. It must be a JSON object with key opcua which value is another JSON object. This nested JSON object should contain node_id or server_uri that correspond to an existent OPC-UA Node ID or Server URI:
Channel structure:
+{
+ "name": "<channel name>",
+ "metadata:": {
+ "opcua": {
+ "server_uri": "<Server URI>"
+ }
+ }
+}
+Thing structure:
+{
+ "name": "<thing name>",
+ "metadata:": {
+ "opcua": {
+ "node_id": "<Node ID>"
+ }
+ }
+}
+The opcua-adapter exposes a /browse HTTP endpoint accessible with method GET and configurable throw HTTP query parameters server, namespace and identifier. The server URI, the node namespace and the node identifier represent the parent node and are used to fetch the list of available children nodes starting from the given one. By default the root node ID (node namespace and node identifier combination) of an OPC-UA server is ns=0;i=84. It's also the default value used by the opcua-adapter to do the browsing if only the server URI is specified in the HTTP query.
To create an OPC-UA subscription, user should connect the Thing to the Channel. This will automatically create the connection, enable the redis route-map and run a subscription to the server_uri and node_id defined in the Thing and Channel metadata.
To forward OPC-UA messages the opcua-adapter subscribes to the Node ID of an OPC-UA Server URI. It verifies the server_uri and the node_id of received messages. If the mapping exists it uses corresponding Channel ID and Thing ID to sign and forwards the content of the OPC-UA message to the Magistrala message broker. If the mapping or the connection between the Thing and the Channel don't exist the subscription stops.
Provisioning is a process of configuration of an IoT platform in which system operator creates and sets-up different entities used in the platform - users, groups, channels and things.
+Use the Magistrala API to create user account:
+curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Content-Type: application/json" https://localhost/users -d '{"name": "John Doe", "credentials": {"identity": "john.doe@email.com", "secret": "12345678"}, "status": "enabled"}'
+Response should look like this:
+HTTP/2 201
+server: nginx/1.23.3
+date: Tue, 04 Apr 2023 08:40:39 GMT
+content-type: application/json
+content-length: 229
+location: /users/71db4bb0-591e-4f76-b766-b39ced9fc6b8
+strict-transport-security: max-age=63072000; includeSubdomains
+x-frame-options: DENY
+x-content-type-options: nosniff
+access-control-allow-origin: *
+access-control-allow-methods: *
+access-control-allow-headers: *
+
+{
+ "id": "71db4bb0-591e-4f76-b766-b39ced9fc6b8",
+ "name": "John Doe",
+ "credentials": { "identity": "john.doe@email.com" },
+ "created_at": "2023-04-04T08:40:39.319602Z",
+ "updated_at": "2023-04-04T08:40:39.319602Z",
+ "status": "enabled"
+}
+Note that when using official docker-compose, all services are behind nginx proxy and all traffic is TLS encrypted.
In order for this user to be able to authenticate to the system, you will have to create an authorization token for them:
+curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Content-Type: application/json" https://localhost/users/tokens/issue -d '{"identity":"john.doe@email.com", "secret":"12345678"}'
+Response should look like this:
+HTTP/2 201
+server: nginx/1.23.3
+date: Tue, 04 Apr 2023 08:40:58 GMT
+content-type: application/json
+content-length: 709
+strict-transport-security: max-age=63072000; includeSubdomains
+x-frame-options: DENY
+x-content-type-options: nosniff
+access-control-allow-origin: *
+access-control-allow-methods: *
+access-control-allow-headers: *
+
+{
+ "access_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODA2NTE2NTgsImlhdCI6MTY4MDU5NzY1OCwiaWRlbnRpdHkiOiJqb2huLmRvZUBlbWFpbC5jb20iLCJpc3MiOiJjbGllbnRzLmF1dGgiLCJzdWIiOiI3MWRiNGJiMC01OTFlLTRmNzYtYjc2Ni1iMzljZWQ5ZmM2YjgiLCJ0eXBlIjoiYWNjZXNzIn0.E4v79FvikIVs-eYOJAgepBX67G2Pzd9YnC-k3xkVrRQcAjHSdMx685jttr9-uuZtF1q3yIpvV-NdQJ2CG5eDtw",
+ "refresh_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2ODA2ODQwNTgsImlhdCI6MTY4MDU5NzY1OCwiaWRlbnRpdHkiOiJqb2huLmRvZUBlbWFpbC5jb20iLCJpc3MiOiJjbGllbnRzLmF1dGgiLCJzdWIiOiI3MWRiNGJiMC01OTFlLTRmNzYtYjc2Ni1iMzljZWQ5ZmM2YjgiLCJ0eXBlIjoicmVmcmVzaCJ9.K236Hz9nsm3dnvW6i7myu5xWcBaNFEMAIeekWkiS_X9y0sQ1LZwl997hkkj4IHFFrbn8KLfmkOfTOqVWgUREFg",
+ "access_type": "Bearer"
+}
+For more information about the Users service API, please check out the API documentation.
+Before proceeding, make sure that you have created a new account and obtained an authorization token. You can set your access_token in the USER_TOKEN environment variable:
USER_TOKEN=<access_token>
+++This endpoint will be depreciated in 1.0.0. It will be replaced with the bulk endpoint currently found at /things/bulk.
+
Things are created by executing request POST /things with a JSON payload. Note that you will need user_token in order to create things that belong to this particular user.
curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $USER_TOKEN" https://localhost/things -d '{"name":"weio"}'
+Response will contain Location header whose value represents path to newly created thing:
HTTP/2 201
+server: nginx/1.23.3
+date: Tue, 04 Apr 2023 09:06:50 GMT
+content-type: application/json
+content-length: 282
+location: /things/9dd12d93-21c9-4147-92fe-769386efb6cc
+access-control-expose-headers: Location
+
+{
+ "id": "9dd12d93-21c9-4147-92fe-769386efb6cc",
+ "name": "weio",
+ "owner": "71db4bb0-591e-4f76-b766-b39ced9fc6b8",
+ "credentials": { "secret": "551e9869-d10f-4682-8319-5a4b18073313" },
+ "created_at": "2023-04-04T09:06:50.460258649Z",
+ "updated_at": "2023-04-04T09:06:50.460258649Z",
+ "status": "enabled"
+}
+Multiple things can be created by executing a POST /things/bulk request with a JSON payload. The payload should contain a JSON array of the things to be created. If there is an error any of the things, none of the things will be created.
curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $USER_TOKEN" https://localhost/things/bulk -d '[{"name":"weio"},{"name":"bob"}]'
+The response's body will contain a list of the created things.
+HTTP/2 200
+server: nginx/1.23.3
+date: Tue, 04 Apr 2023 08:42:04 GMT
+content-type: application/json
+content-length: 586
+access-control-expose-headers: Location
+
+{
+ "total": 2,
+ "things": [{
+ "id": "1b1cd38f-62cd-4f17-b47e-5ff4e97881e8",
+ "name": "weio",
+ "owner": "71db4bb0-591e-4f76-b766-b39ced9fc6b8",
+ "credentials": { "secret": "43bd950e-0b3f-46f6-a92c-296a6a0bfe66" },
+ "created_at": "2023-04-04T08:42:04.168388927Z",
+ "updated_at": "2023-04-04T08:42:04.168388927Z",
+ "status": "enabled"
+ },
+ {
+ "id": "b594af97-9550-4b11-86e1-2b6db7e329b9",
+ "name": "bob",
+ "owner": "71db4bb0-591e-4f76-b766-b39ced9fc6b8",
+ "credentials": { "secret": "9f89f52e-1b06-4416-8294-ae753b0c4bea" },
+ "created_at": "2023-04-04T08:42:04.168390109Z",
+ "updated_at": "2023-04-04T08:42:04.168390109Z",
+ "status": "enabled"
+ }
+ ]
+}
+In order to retrieve data of provisioned things that are written in database, you can send following request:
+curl -s -S -i --cacert docker/ssl/certs/ca.crt -H "Authorization: Bearer $USER_TOKEN" https://localhost/things
+Notice that you will receive only those things that were provisioned by user_token owner.
HTTP/2 200
+server: nginx/1.23.3
+date: Tue, 04 Apr 2023 08:42:27 GMT
+content-type: application/json
+content-length: 570
+access-control-expose-headers: Location
+
+{
+ "limit": 10,
+ "total": 2,
+ "things": [{
+ "id": "1b1cd38f-62cd-4f17-b47e-5ff4e97881e8",
+ "name": "weio",
+ "owner": "71db4bb0-591e-4f76-b766-b39ced9fc6b8",
+ "credentials": { "secret": "43bd950e-0b3f-46f6-a92c-296a6a0bfe66" },
+ "created_at": "2023-04-04T08:42:04.168388Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ },
+ {
+ "id": "b594af97-9550-4b11-86e1-2b6db7e329b9",
+ "name": "bob",
+ "owner": "71db4bb0-591e-4f76-b766-b39ced9fc6b8",
+ "credentials": { "secret": "9f89f52e-1b06-4416-8294-ae753b0c4bea" },
+ "created_at": "2023-04-04T08:42:04.16839Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }
+ ]
+}
+You can specify offset and limit parameters in order to fetch a specific subset of things. In that case, your request should look like:
curl -s -S -i --cacert docker/ssl/certs/ca.crt -H "Authorization: Bearer $USER_TOKEN" https://localhost/things?offset=0&limit=5
+You can specify name and/or metadata parameters in order to fetch specific subset of things. When specifying metadata you can specify just a part of the metadata JSON you want to match.
curl -s -S -i --cacert docker/ssl/certs/ca.crt -H "Authorization: Bearer $USER_TOKEN" https://localhost/things?offset=0&limit=5&name="weio"
+HTTP/2 200
+server: nginx/1.23.3
+date: Tue, 04 Apr 2023 08:43:09 GMT
+content-type: application/json
+content-length: 302
+access-control-expose-headers: Location
+
+{
+ "limit": 5,
+ "total": 1,
+ "things": [{
+ "id": "1b1cd38f-62cd-4f17-b47e-5ff4e97881e8",
+ "name": "weio",
+ "owner": "71db4bb0-591e-4f76-b766-b39ced9fc6b8",
+ "credentials": { "secret": "43bd950e-0b3f-46f6-a92c-296a6a0bfe66" },
+ "created_at": "2023-04-04T08:42:04.168388Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }]
+}
+If you don't provide them, default values will be used instead: 0 for offset and 10 for limit. Note that limit cannot be set to values greater than 100. Providing invalid values will be considered malformed request.
This is a special endpoint that allows you to disable a thing, soft deleting it from the database. In order to disable you own thing you can send following request:
+curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Authorization: Bearer $USER_TOKEN" https://localhost/things/1b1cd38f-62cd-4f17-b47e-5ff4e97881e8/disable
+HTTP/2 200
+server: nginx/1.23.3
+date: Tue, 04 Apr 2023 09:00:40 GMT
+content-type: application/json
+content-length: 277
+access-control-expose-headers: Location
+
+{
+ "id": "1b1cd38f-62cd-4f17-b47e-5ff4e97881e8",
+ "name": "weio",
+ "owner": "71db4bb0-591e-4f76-b766-b39ced9fc6b8",
+ "credentials": { "secret": "43bd950e-0b3f-46f6-a92c-296a6a0bfe66" },
+ "created_at": "2023-04-04T08:42:04.168388Z",
+ "updated_at": "2023-04-04T08:42:04.168388Z",
+ "status": "disabled"
+}
+++This endpoint will be depreciated in 1.0.0. It will be replaced with the bulk endpoint currently found at /channels/bulk.
+
Channels are created by executing request POST /channels:
curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $USER_TOKEN" https://localhost/channels -d '{"name":"mychan"}'
+After sending request you should receive response with Location header that contains path to newly created channel:
HTTP/2 201
+server: nginx/1.23.3
+date: Tue, 04 Apr 2023 09:18:10 GMT
+content-type: application/json
+content-length: 235
+location: /channels/0a67a8ee-eda9-408e-af83-f895096b7359
+access-control-expose-headers: Location
+
+{
+ "id": "0a67a8ee-eda9-408e-af83-f895096b7359",
+ "owner_id": "71db4bb0-591e-4f76-b766-b39ced9fc6b8",
+ "name": "mychan",
+ "created_at": "2023-04-04T09:18:10.26603Z",
+ "updated_at": "2023-04-04T09:18:10.26603Z",
+ "status": "enabled"
+}
+Multiple channels can be created by executing a POST /things/bulk request with a JSON payload. The payload should contain a JSON array of the channels to be created. If there is an error any of the channels, none of the channels will be created.
curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $USER_TOKEN" https://localhost/channels/bulk -d '[{"name":"joe"},{"name":"betty"}]'
+The response's body will contain a list of the created channels.
+HTTP/2 200
+server: nginx/1.23.3
+date: Tue, 04 Apr 2023 09:11:16 GMT
+content-type: application/json
+content-length: 487
+access-control-expose-headers: Location
+
+{
+ "channels": [{
+ "id": "5ec1beb9-1b76-47e6-a9ef-baf9e4ae5820",
+ "owner_id": "71db4bb0-591e-4f76-b766-b39ced9fc6b8",
+ "name": "joe",
+ "created_at": "2023-04-04T09:11:16.131972Z",
+ "updated_at": "2023-04-04T09:11:16.131972Z",
+ "status": "disabled"
+ },
+ {
+ "id": "ff1316f1-d3c6-4590-8bf3-33774d79eab2",
+ "owner_id": "71db4bb0-591e-4f76-b766-b39ced9fc6b8",
+ "name": "betty",
+ "created_at": "2023-04-04T09:11:16.138881Z",
+ "updated_at": "2023-04-04T09:11:16.138881Z",
+ "status": "disabled"
+ }
+ ]
+}
+In order to retrieve data of provisioned channels that are written in database, you can send following request:
+curl -s -S -i --cacert docker/ssl/certs/ca.crt -H "Authorization: Bearer $USER_TOKEN" https://localhost/channels
+Notice that you will receive only those things that were provisioned by user_token owner.
HTTP/2 200
+server: nginx/1.23.3
+date: Tue, 04 Apr 2023 09:13:48 GMT
+content-type: application/json
+content-length: 495
+access-control-expose-headers: Location
+
+{
+ "total": 2,
+ "channels": [{
+ "id": "5ec1beb9-1b76-47e6-a9ef-baf9e4ae5820",
+ "owner_id": "71db4bb0-591e-4f76-b766-b39ced9fc6b8",
+ "name": "joe",
+ "created_at": "2023-04-04T09:11:16.131972Z",
+ "updated_at": "2023-04-04T09:11:16.131972Z",
+ "status": "enabled"
+ },
+ {
+ "id": "ff1316f1-d3c6-4590-8bf3-33774d79eab2",
+ "owner_id": "71db4bb0-591e-4f76-b766-b39ced9fc6b8",
+ "name": "betty",
+ "created_at": "2023-04-04T09:11:16.138881Z",
+ "updated_at": "2023-04-04T09:11:16.138881Z",
+ "status": "enabled"
+ }
+ ]
+}
+You can specify offset and limit parameters in order to fetch specific subset of channels. In that case, your request should look like:
curl -s -S -i --cacert docker/ssl/certs/ca.crt -H "Authorization: Bearer $USER_TOKEN" https://localhost/channels?offset=0&limit=5
+If you don't provide them, default values will be used instead: 0 for offset and 10 for limit. Note that limit cannot be set to values greater than 100. Providing invalid values will be considered malformed request.
This is a special endpoint that allows you to disable a channel, soft deleting it from the database. In order to disable you own channel you can send following request:
+curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Authorization: Bearer $USER_TOKEN" https://localhost/channels/5ec1beb9-1b76-47e6-a9ef-baf9e4ae5820/disable
+HTTP/2 200
+server: nginx/1.23.3
+date: Tue, 04 Apr 2023 09:16:31 GMT
+content-type: application/json
+content-length: 235
+access-control-expose-headers: Location
+
+{
+ "id": "5ec1beb9-1b76-47e6-a9ef-baf9e4ae5820",
+ "owner_id": "71db4bb0-591e-4f76-b766-b39ced9fc6b8",
+ "name": "joe",
+ "created_at": "2023-04-04T09:11:16.131972Z",
+ "updated_at": "2023-04-04T09:11:16.131972Z",
+ "status": "disabled"
+}
+Channel can be observed as a communication group of things. Only things that are connected to the channel can send and receive messages from other things in this channel. Things that are not connected to this channel are not allowed to communicate over it. Users may also be assigned to channels, thus sharing things between users. With the necessary policies in place, users can be granted access to things that are not owned by them.
+A user who is the owner of a channel or a user that has been assigned to the channel with the required policy can connect things to the channel. This is equivalent of giving permissions to these things to communicate over given communication group.
+To connect a thing to the channel you should send following request:
+++This endpoint will be depreciated in 1.0.0. It will be replaced with the bulk endpoint found at /connect.
+
curl -s -S -i --cacert docker/ssl/certs/ca.crt -X PUT -H "Authorization: Bearer $USER_TOKEN" https://localhost/channels/<channel_id>/things/<thing_id>
+HTTP/2 201
+server: nginx/1.23.3
+date: Tue, 04 Apr 2023 09:20:23 GMT
+content-type: application/json
+content-length: 266
+access-control-expose-headers: Location
+
+{
+ "owner_id": "71db4bb0-591e-4f76-b766-b39ced9fc6b8",
+ "subject": "b594af97-9550-4b11-86e1-2b6db7e329b9",
+ "object": "ff1316f1-d3c6-4590-8bf3-33774d79eab2",
+ "actions": ["m_write", "m_read"],
+ "created_at": "2023-04-04T09:20:23.015342Z",
+ "updated_at": "2023-04-04T09:20:23.015342Z"
+}
+To connect multiple things to a channel, you can send the following request:
+curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $USER_TOKEN" https://localhost/connect -d '{"channel_ids":["<channel_id>", "<channel_id>"],"thing_ids":["<thing_id>", "<thing_id>"]}'
+You can observe which things are connected to specific channel:
+curl -s -S -i --cacert docker/ssl/certs/ca.crt -H "Authorization: Bearer $USER_TOKEN" https://localhost/channels/<channel_id>/things
+Response that you'll get should look like this:
+HTTP/2 200
+server: nginx/1.23.3
+date: Tue, 04 Apr 2023 09:53:21 GMT
+content-type: application/json
+content-length: 254
+access-control-expose-headers: Location
+
+{
+ "limit": 10,
+ "total": 1,
+ "things": [{
+ "id": "b594af97-9550-4b11-86e1-2b6db7e329b9",
+ "name": "bob",
+ "credentials": { "secret": "9f89f52e-1b06-4416-8294-ae753b0c4bea" },
+ "created_at": "2023-04-04T08:42:04.16839Z",
+ "updated_at": "0001-01-01T00:00:00Z",
+ "status": "enabled"
+ }]
+}
+You can observe to which channels is specified thing connected:
+curl -s -S -i --cacert docker/ssl/certs/ca.crt -H "Authorization: Bearer $USER_TOKEN" https://localhost/things/<thing_id>/channels
+Response that you'll get should look like this:
+HTTP/2 200
+server: nginx/1.23.3
+date: Tue, 04 Apr 2023 09:57:10 GMT
+content-type: application/json
+content-length: 261
+access-control-expose-headers: Location
+
+{
+ "total": 1,
+ "channels": [{
+ "id": "ff1316f1-d3c6-4590-8bf3-33774d79eab2",
+ "owner_id": "71db4bb0-591e-4f76-b766-b39ced9fc6b8",
+ "name": "betty",
+ "created_at": "2023-04-04T09:11:16.138881Z",
+ "updated_at": "2023-04-04T09:11:16.138881Z",
+ "status": "enabled"
+ }]
+}
+If you want to disconnect your thing from the channel, send following request:
+++This endpoint will be depreciated in 1.0.0. It will be replaced with the bulk endpoint found at /disconnect.
+
curl -s -S -i --cacert docker/ssl/certs/ca.crt -X DELETE -H "Authorization: Bearer $USER_TOKEN" https://localhost/channels/<channel_id>/things/<thing_id>
+Response that you'll get should look like this:
+HTTP/2 204
+server: nginx/1.23.3
+date: Tue, 04 Apr 2023 09:57:53 GMT
+access-control-expose-headers: Location
+For more information about the Things service API, please check out the API documentation.
+Provisioning is a process of configuration of an IoT platform in which system operator creates and sets-up different entities used in the platform - users, channels and things. It is part of process of setting up IoT applications where we connect devices on edge with platform in cloud. For provisioning we can use Magistrala CLI for creating users and for each node in the edge (eg. gateway) required number of things, channels, connecting them and creating certificates if needed. Provision service is used to set up initial application configuration once user is created. Provision service creates things, channels, connections and certificates. Once user is created we can use provision to create a setup for edge node in one HTTP request instead of issuing several CLI commands.
+Provision service provides an HTTP API to interact with Magistrala.
+For gateways to communicate with Magistrala configuration is required (MQTT host, thing, channels, certificates...). Gateway will send a request to Bootstrap service providing <external_id> and <external_key> in HTTP request to get the configuration. To make a request to Bootstrap service you can use Agent service on a gateway.
To create bootstrap configuration you can use Bootstrap or Provision service. [Magistrala UI][mfxui] uses Bootstrap service for creating gateway configurations. Provision service should provide an easy way of provisioning your gateways i.e creating bootstrap configuration and as many things and channels that your setup requires.
Also, you may use provision service to create certificates for each thing. Each service running on gateway may require more than one thing and channel for communication.
+If, for example, you are using services Agent and Export on a gateway you will need two channels for Agent (data and control) and one thing for Export.
+Additionally, if you enabled mTLS each service will need its own thing and certificate for access to Magistrala.
+Your setup could require any number of things and channels, this kind of setup we can call provision layout.
Provision service provides a way of specifying this provision layout and creating a setup according to that layout by serving requests on /mapping endpoint. Provision layout is configured in config.toml.
The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.
+By default, call to /mapping endpoint will create one thing and two channels (control and data) and connect it as this is typical setup required by Agent. If there is a requirement for different provision layout we can use config file in addition to environment variables.
For the purposes of running provision as an add-on in docker composition environment variables seems more suitable. Environment variables are set in .env.
+Configuration can be specified in config.toml. Config file can specify all the settings that environment variables can configure and in addition /mapping endpoint provision layout can be configured.
In config.toml we can enlist an array of things and channels that we want to create and make connections between them which we call provision layout.
Things Metadata can be whatever suits your needs. Thing that has metadata with external_id will have bootstrap configuration created, external_id value will be populated with value from request).
+Bootstrap configuration can be fetched with Agent. For channel's metadata type is reserved for control and data which we use with Agent.
Example of provision layout below
+[bootstrap]
+ [bootstrap.content]
+ [bootstrap.content.agent.edgex]
+ url = "http://localhost:48090/api/v1/"
+
+ [bootstrap.content.agent.log]
+ level = "info"
+
+ [bootstrap.content.agent.mqtt]
+ mtls = false
+ qos = 0
+ retain = false
+ skip_tls_ver = true
+ url = "localhost:1883"
+
+ [bootstrap.content.agent.server]
+ nats_url = "localhost:4222"
+ port = "9000"
+
+ [bootstrap.content.agent.heartbeat]
+ interval = "30s"
+
+ [bootstrap.content.agent.terminal]
+ session_timeout = "30s"
+
+ [bootstrap.content.export.exp]
+ log_level = "debug"
+ nats = "nats://localhost:4222"
+ port = "8172"
+ cache_url = "localhost:6379"
+ cache_pass = ""
+ cache_db = "0"
+
+ [bootstrap.content.export.mqtt]
+ ca_path = "ca.crt"
+ cert_path = "thing.crt"
+ channel = ""
+ host = "tcp://localhost:1883"
+ mtls = false
+ password = ""
+ priv_key_path = "thing.key"
+ qos = 0
+ retain = false
+ skip_tls_ver = false
+ username = ""
+
+ [[bootstrap.content.export.routes]]
+ mqtt_topic = ""
+ nats_topic = "channels"
+ subtopic = ""
+ type = "mfx"
+ workers = 10
+
+ [[bootstrap.content.export.routes]]
+ mqtt_topic = ""
+ nats_topic = "export"
+ subtopic = ""
+ type = "default"
+ workers = 10
+
+[[things]]
+ name = "thing"
+
+ [things.metadata]
+ external_id = "xxxxxx"
+
+[[channels]]
+ name = "control-channel"
+
+ [channels.metadata]
+ type = "control"
+
+[[channels]]
+ name = "data-channel"
+
+ [channels.metadata]
+ type = "data"
+
+[[channels]]
+ name = "export-channel"
+
+ [channels.metadata]
+ type = "export"
+[bootstrap.content] will be marshalled and saved into content field in bootstrap configs when request to /mappings is made, content field from bootstrap config is used to create Agent and Export configuration files upon Agent fetching bootstrap configuration.
In order to create necessary entities provision service needs to authenticate against Magistrala.
+To provide authentication credentials to the provision service you can pass it in as an environment variable or in a config file as Magistrala user and password or as API token (that can be issued on /users/tokens/issue endpoint of users service.
Additionally, users or API token can be passed in Authorization header, this authentication takes precedence over others.
+username, password - (MF_PROVISION_USER, MF_PROVISION_PASSWORD in .env, mf_user, mf_pass in config.tomlMF_PROVISION_API_KEY in .env or config.tomlAuthorization: Bearer Token|ApiKey - request authorization header containing users token. Check auth.Provision service can be run as a standalone or in docker composition as addon to the core docker composition.
+Standalone:
+MF_PROVISION_BS_SVC_URL=http://localhost:9013/things \
+MF_PROVISION_THINGS_LOCATION=http://localhost:9000 \
+MF_PROVISION_USERS_LOCATION=http://localhost:9002 \
+MF_PROVISION_CONFIG_FILE=docker/addons/provision/configs/config.toml \
+build/magistrala-provision
+Docker composition:
+docker-compose -f docker/addons/provision/docker-compose.yml up
+For the case that credentials or API token is passed in configuration file or environment variables, call to /mapping endpoint doesn't require Authentication header:
curl -s -S -X POST http://localhost:9016/mapping -H 'Content-Type: application/json' -d '{"external_id": "33:52:77:99:43", "external_key": "223334fw2"}'
+In the case that provision service is not deployed with credentials or API key or you want to use user other than one being set in environment (or config file):
+curl -s -S -X POST http://localhost:9016/mapping -H "Authorization: Bearer <token|api_key>" -H 'Content-Type: application/json' -d '{"external_id": "<external_id>", "external_key": "<external_key>"}'
+Or if you want to specify a name for thing different than in config.toml you can specify post data as:
{
+ "name": "<name>",
+ "external_id": "<external_id>",
+ "external_key": "<external_key>"
+}
+Response contains created things, channels and certificates if any:
+{
+ "things": [
+ {
+ "id": "c22b0c0f-8c03-40da-a06b-37ed3a72c8d1",
+ "name": "thing",
+ "key": "007cce56-e0eb-40d6-b2b9-ed348a97d1eb",
+ "metadata": {
+ "external_id": "33:52:79:C3:43"
+ }
+ }
+ ],
+ "channels": [
+ {
+ "id": "064c680e-181b-4b58-975e-6983313a5170",
+ "name": "control-channel",
+ "metadata": {
+ "type": "control"
+ }
+ },
+ {
+ "id": "579da92d-6078-4801-a18a-dd1cfa2aa44f",
+ "name": "data-channel",
+ "metadata": {
+ "type": "data"
+ }
+ }
+ ],
+ "whitelisted": {
+ "c22b0c0f-8c03-40da-a06b-37ed3a72c8d1": true
+ }
+}
+Deploy Magistrala UI docker composition as it contains all the required services for provisioning to work ( certs, bootstrap and Magistrala core)
git clone https://github.com/absmach/magistrala-ui
+cd magistrala-ui
+docker-compose -f docker/docker-compose.yml up
+Create user and obtain access token
+magistrala-cli users create john.doe@email.com 12345678
+
+# Retrieve token
+magistrala-cli users token john.doe@email.com 12345678
+
+created: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1OTY1ODU3MDUsImlhdCI6MTU5NjU0OTcwNSwiaXNzIjoibWFpbmZsdXguYXV0aG4iLCJzdWIiOiJtaXJrYXNoQGdtYWlsLmNvbSIsInR5cGUiOjB9._vq0zJzFc9tQqc8x74kpn7dXYefUtG9IB0Cb-X2KMK8
+Put a value of token into environment variable
+TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1OTY1ODU3MDUsImlhdCI6MTU5NjU0OTcwNSwiaXNzIjoibWFpbmZsdXguYXV0aG4iLCJzdWIiOiJtaXJrYXNoQGdtYWlsLmNvbSIsInR5cGUiOjB9._vq0zJzFc9tQqc8x74kpn7dXYefUtG9IB0Cb-X2KMK8
+Make a call to provision endpoint
+curl -s -S -X POST http://magistrala.com:9016/mapping -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' -d '{"name":"edge-gw", "external_id" : "gateway", "external_key":"external_key" }'
+To check the results you can make a call to bootstrap endpoint
+curl -s -S -X GET http://magistrala.com:9013/things/bootstrap/gateway -H "Authorization: Thing external_key" -H 'Content-Type: application/json'
+Or you can start Agent with:
git clone https://github.com/absmach/agent.git
+cd agent
+make
+MF_AGENT_BOOTSTRAP_ID=gateway MF_AGENT_BOOTSTRAP_KEY=external_key MF_AGENT_BOOTSTRAP_URL=http://magistrala.ccom:9013/things/bootstrap build/magistrala-agent
+Agent will retrieve connections parameters and connect to Magistrala cloud.
+For more information about the Provision service API, please check out the API documentation.
+ + + + + + +If either the cert or key is not set, the server will use insecure transport.
+MF_USERS_SERVER_CERT the path to server certificate in pem format.
MF_USERS_SERVER_KEY the path to the server key in pem format.
If either the cert or key is not set, the server will use insecure transport.
+MF_THINGS_SERVER_CERT the path to server certificate in pem format.
MF_THINGS_SERVER_KEY the path to the server key in pem format.
Sometimes it makes sense to run Things as a standalone service to reduce network traffic or simplify deployment. This means that Things service operates only using a single user and is able to authorize it without gRPC communication with Auth service. When running Things in the standalone mode, Auth and Users services can be omitted from the deployment.
+To run service in a standalone mode, set MF_THINGS_STANDALONE_EMAIL and MF_THINGS_STANDALONE_TOKEN.
If you wish to secure the gRPC connection to Things and Users services you must define the CAs that you trust. This does not support mutual certificate authentication.
MF_HTTP_ADAPTER_CA_CERTS, MF_MQTT_ADAPTER_CA_CERTS, MF_WS_ADAPTER_CA_CERTS, MF_COAP_ADAPTER_CA_CERTS - the path to a file that contains the CAs in PEM format. If not set, the default connection will be insecure. If it fails to read the file, the adapter will fail to start up.
MF_THINGS_CA_CERTS - the path to a file that contains the CAs in PEM format. If not set, the default connection will be insecure. If it fails to read the file, the service will fail to start up.
By default, Magistrala will connect to Postgres using insecure transport. +If a secured connection is required, you can select the SSL mode and set paths to any extra certificates and keys needed.
+MF_USERS_DB_SSL_MODE the SSL connection mode for Users.
+MF_USERS_DB_SSL_CERT the path to the certificate file for Users.
+MF_USERS_DB_SSL_KEY the path to the key file for Users.
+MF_USERS_DB_SSL_ROOT_CERT the path to the root certificate file for Users.
MF_THINGS_DB_SSL_MODE the SSL connection mode for Things.
+MF_THINGS_DB_SSL_CERT the path to the certificate file for Things.
+MF_THINGS_DB_SSL_KEY the path to the key file for Things.
+MF_THINGS_DB_SSL_ROOT_CERT the path to the root certificate file for Things.
Supported database connection modes are: disabled (default), required, verify-ca and verify-full.
By default gRPC communication is not secure as Magistrala system is most often run in a private network behind the reverse proxy.
+However, TLS can be activated and configured.
+ + + + + + +Magistrala supports various storage databases in which messages are stored:
+These storages are activated via docker-compose add-ons.
+The <project_root>/docker folder contains an addons directory. This directory is used for various services that are not core to the Magistrala platform but could be used for providing additional features.
In order to run these services, core services, as well as the network from the core composition, should be already running.
+Writers provide an implementation of various message writers. Message writers are services that consume Magistrala messages, transform them to desired format and store them in specific data store. The path of the configuration file can be set using the following environment variables: MF_CASSANDRA_WRITER_CONFIG_PATH, MF_POSTGRES_WRITER_CONFIG_PATH, MF_INFLUX_WRITER_CONFIG_PATH, MF_MONGO_WRITER_CONFIG_PATH and MF_TIMESCALE_WRITER_CONFIG_PATH.
Each writer can filter messages based on subjects list that is set in config.toml configuration file. If you want to listen on all subjects, just set the field subjects in the [subscriber] section as ["channels.>"], otherwise pass the list of subjects. Here is an example:
[subscriber]
+subjects = ["channels.*.messages.bedroom.temperature","channels.*.messages.bedroom.humidity"]
+Regarding the Subtopics Section in the messaging page, the example channels/<channel_id>/messages/bedroom/temperature can be filtered as "channels.*.bedroom.temperature". The formatting of this filtering list is determined by the default message broker, NATS, format (Subject-Based Messaging & Wildcards).
There are two types of transformers: SenML and JSON. The transformer type is set in configuration file.
+For SenML transformer, supported message payload formats are SenML+CBOR and SenML+JSON. They are configurable over content_type field in the [transformer] section and expect application/senml+json or application/senml+cbor formats. Here is an example:
[transformer]
+format = "senml"
+content_type = "application/senml+json"
+Usually, the payload of the IoT message contains message time. It can be in different formats (like base time and record time in the case of SenML) and the message field can be under the arbitrary key. Usually, we would want to map that time to the Magistrala Message field Created and for that reason, we need to configure the Transformer to be able to read the field, parse it using proper format and location (if devices time is different than the service time), and map it to Magistrala Message.
+For JSON transformer you can configure time_fields in the [transformer] section to use arbitrary fields from the JSON message payload as timestamp. time_fields is represented by an array of objects with fields field_name, field_format and location that represent respectively the name of the JSON key to use as timestamp, the time format to use for the field value and the time location. Here is an example:
[transformer]
+format = "json"
+time_fields = [{ field_name = "seconds_key", field_format = "unix", location = "UTC"},
+ { field_name = "millis_key", field_format = "unix_ms", location = "UTC"},
+ { field_name = "micros_key", field_format = "unix_us", location = "UTC"},
+ { field_name = "nanos_key", field_format = "unix_ns", location = "UTC"}]
+JSON transformer can be used for any JSON payload. For the messages that contain JSON array as the root element, JSON Transformer does normalization of the data: it creates a separate JSON message for each JSON object in the root. In order to be processed and stored properly, JSON messages need to contain message format information. For the sake of simplicity, nested JSON objects are flatten to a single JSON object in InfluxDB, using composite keys separated by the / separator. This implies that the separator character (/) is not allowed in the JSON object key while using InfluxDB. Apart from InfluxDB, separator character (/) usage in the JSON object key is permitted, since other Writer types do not flat the nested JSON objects. For example, the following JSON object:
{
+ "name": "name",
+ "id": 8659456789564231564,
+ "in": 3.145,
+ "alarm": true,
+ "ts": 1571259850000,
+ "d": {
+ "tmp": 2.564,
+ "hmd": 87,
+ "loc": {
+ "x": 1,
+ "y": 2
+ }
+ }
+}
+for InfluxDB will be transformed to:
+{
+ "name": "name",
+ "id": 8659456789564231564,
+ "in": 3.145,
+ "alarm": true,
+ "ts": 1571259850000,
+ "d/tmp": 2.564,
+ "d/hmd": 87,
+ "d/loc/x": 1,
+ "d/loc/y": 2
+}
+while for other Writers it will preserve its original format.
+The message format is stored in the subtopic. It's the last part of the subtopic. In the example:
+http://localhost:8008/channels/<channelID>/messages/home/temperature/myFormat
+the message format is myFormat. It can be any valid subtopic name, JSON transformer is format-agnostic. The format is used by the JSON message consumers so that they can process the message properly. If the format is not present (i.e. message subtopic is empty), JSON Transformer will report an error. Message writers will store the message(s) in the table/collection/measurement (depending on the underlying database) with the name of the format (which in the example is myFormat). Magistrala writers will try to save any format received (whether it will be successful depends on the writer implementation and the underlying database), but it's recommended that publishers don't send different formats to the same subtopic.
From the project root execute the following command:
+docker-compose -f docker/addons/influxdb-writer/docker-compose.yml up -d
+This will install and start:
+Those new services will take some additional ports:
+To access Influx-UI, navigate to http://localhost:8086 and login with: magistrala, password: magistrala
./docker/addons/cassandra-writer/init.sh
+Please note that Cassandra may not be suitable for your testing environment because of its high system requirements.
+docker-compose -f docker/addons/mongodb-writer/docker-compose.yml up -d
+MongoDB default port (27017) is exposed, so you can use various tools for database inspection and data visualization.
+docker-compose -f docker/addons/postgres-writer/docker-compose.yml up -d
+Postgres default port (5432) is exposed, so you can use various tools for database inspection and data visualization.
+docker-compose -f docker/addons/timescale-writer/docker-compose.yml up -d
+Timescale default port (5432) is exposed, so you can use various tools for database inspection and data visualization.
+Readers provide an implementation of various message readers. Message readers are services that consume normalized (in SenML format) Magistrala messages from data storage and opens HTTP API for message consumption. Installing corresponding writer before reader is implied.
Each of the Reader services exposes the same HTTP API for fetching messages on its default port.
+To read sent messages on channel with id channel_id you should send GET request to /channels/<channel_id>/messages with thing access token in Authorization header. That thing must be connected to channel with channel_id
Response should look like this:
+HTTP/1.1 200 OK
+Content-Type: application/json
+Date: Tue, 18 Sep 2018 18:56:19 GMT
+Content-Length: 228
+
+{
+ "messages": [
+ {
+ "Channel": 1,
+ "Publisher": 2,
+ "Protocol": "mqtt",
+ "Name": "name:voltage",
+ "Unit": "V",
+ "Value": 5.6,
+ "Time": 48.56
+ },
+ {
+ "Channel": 1,
+ "Publisher": 2,
+ "Protocol": "mqtt",
+ "Name": "name:temperature",
+ "Unit": "C",
+ "Value": 24.3,
+ "Time": 48.56
+ }
+ ]
+}
+Note that you will receive only those messages that were sent by authorization token's owner. You can specify offset and limit parameters in order to fetch specific group of messages. An example of HTTP request looks like:
curl -s -S -i -H "Authorization: Thing <thing_secret>" http://localhost:<service_port>/channels/<channel_id>/messages?offset=0&limit=5&format=<subtopic>
+If you don't provide offset and limit parameters, default values will be used instead: 0 for offset and 10 for limit. The format parameter indicates the last subtopic of the message. As indicated under the Writers section, the message format is stored in the subtopic as the last part of the subtopic. In the example:
http://localhost:<service_port>/channels/<channelID>/messages/home/temperature/myFormat
+the message format is myFormat and the value for format=<subtopic> is format=myFormat.
To start InfluxDB reader, execute the following command:
+docker-compose -f docker/addons/influxdb-reader/docker-compose.yml up -d
+To start Cassandra reader, execute the following command:
+docker-compose -f docker/addons/cassandra-reader/docker-compose.yml up -d
+To start MongoDB reader, execute the following command:
+docker-compose -f docker/addons/mongodb-reader/docker-compose.yml up -d
+To start PostgreSQL reader, execute the following command:
+docker-compose -f docker/addons/postgres-reader/docker-compose.yml up -d
+To start Timescale reader, execute the following command:
+docker-compose -f docker/addons/timescale-reader/docker-compose.yml up -d
+Distributed tracing is a method of profiling and monitoring applications. It can provide valuable insight when optimizing and debugging an application. Magistrala includes the Jaeger open tracing framework as a service with its stack by default.
+The Jaeger service will launch with the rest of the Magistrala services. All services can be launched using:
+make run
+The Jaeger UI can then be accessed at http://localhost:16686 from a browser. Details about the UI can be found in Jaeger's official documentation.
The Jaeger service can be disabled by using the scale flag with docker-compose up and setting the jaeger container to 0.
--scale jaeger=0
+Jaeger uses 5 ports within the Magistrala framework. These ports can be edited in the .env file.
| Variable | +Description | +Default | +
|---|---|---|
| MF_JAEGER_PORT | +Agent port for compact jaeger.thrift protocol | +6831 | +
| MF_JAEGER_FRONTEND | +UI port | +16686 | +
| MF_JAEGER_COLLECTOR | +Collector for jaeger.thrift directly from clients | +14268 | +
| MF_JAEGER_CONFIGS | +Configuration server | +5778 | +
| MF_JAEGER_URL | +Jaeger access from within Magistrala | +jaeger:6831 | +
Magistrala provides for tracing of messages ingested into the Magistrala platform. The message metadata such as topic, sub-topic, subscriber and publisher is also included in traces. 
.
The messages are tracked from end to end from the point they are published to the consumers where they are stored. 
As an example for using Jaeger, we can look at the traces generated after provisioning the system. Make sure to have ran the provisioning script that is part of the Getting Started step.
+Before getting started with Jaeger, there are a few terms that are important to define. A trace can be thought of as one transaction within the system. A trace is made up of one or more spans. These are the individual steps that must be taken for a trace to perform its action. A span has tags and logs associated with it. Tags are key-value pairs that provide information such as a database type or http method. Tags are useful when filtering traces in the Jaeger UI. Logs are structured messages used at specific points in the trace's transaction. These are typically used to indicate an error.
When first navigating to the Jaeger UI, it will present a search page with an empty results section. There are multiple fields to search from including service, operation, tags and time frames. Clicking Find Traces will fill the results section with traces containing the selected fields.
The top of the results page includes a scatter plot of the traces and their durations. This can be very useful for finding a trace with a prolonged runtime. Clicking on one of the points will open the trace page of that trace.
+Below the graph is a list of all the traces with a summary of its information. Each trace shows a unique identifier, the overall runtime, the spans it is composed of and when it was ran. Clicking on one of the traces will open the trace page of that trace.
+
The trace page provides a more detailed breakdown of the individual span calls. The top of the page shows a chart breaking down what spans the trace is spending its time in. Below the chart are the individual spans and their details. Expanding the spans shows any tags associated with that span and process information. This is also where any errors or logs seen while running the span will be reported.
+This is just a brief overview of the possibilities of Jaeger and its UI. For more information, check out Jaeger's official documentation.
+ + + + + + +Magistrala twins service is built on top of the Magistrala platform. In order to fully understand what follows, be sure to get acquainted with overall Magistrala architecture.
+Twin refers to a digital representation of a real world data system consisting of possibly multiple data sources/producers and/or destinations/consumers (data agents).
+For example, an industrial machine can use multiple protocols such as MQTT, OPC-UA, a regularly updated machine hosted CSV file etc. to send measurement data - such as flowrate, material temperature, etc. - and state metadata - such as engine and chassis temperature, engine rotations per seconds, identity of the current human operator, etc. - as well as to receive control, i.e. actuation messages - such as, turn on/off light, increment/decrement borer speed, change blades, etc.
+Digital twin is an abstract - and usually less detailed - digital replica of a real world system such as the industrial machine we have just described. It is used to create and store information about system's state at any given moment, to compare system state over a given period of time - so-called diffs or deltas - as well as to control agents composing the system.
+Any data producer or data consumer - which we refer to here collectively as data agent - or an interrelated system of data agents, can be represented by means of possibly multiple Magistrala things, channels and subtopics. For example, an OPC-UA server can be represented as a Magistrala thing and its nodes can be represented as multiple Magistrala channels or multiple subtopics of a single Magistrala channel. What is more, you can invert the representation: you can represent server as a channel and node as things. Magistrala platform is meant to empower you with the freedom of expression so you can make a digital representation of any data agent according to your needs.
+Although this works well, satisfies the requirements of a wide variety of use cases and corresponds to the intended use of Mainlfux IoT platform, this setup can be insufficient in two important ways. Firstly, different things, channels, and their connections - i.e. Magistrala representations of different data agent structures - are unrelated to each other, i.e. they do not form a meaningful whole and, as a consequence, they do not represent a single unified system. Secondly, the semantic aspect, i.e. the meaning of different things and channels is not transparent and defined by the sole use of Magistrala platform entities (channels and things).
+Certainly, we can try to describe things and channels connections and relations as well as their meaning - i.e. their role, position, function in the overall system - by means of their metadata. Although this might work well - with a proviso of a lot of additional effort of writing the relatively complex code to create and parse metadata - it is not a practical approach and we still don't get - at least not out of the box - a readable and useful overview of the system as a whole. Also, this approach does not enable us to answer a simple but very important question, i.e. what was the detailed state of a complete system at a certain moment in time.
+To overcome these problems, Magistrala comes with a digital twin service. The twins service is built on top of the Magistrala platform and relies on its architecture and entities, more precisely, on Magistrala users, things and channels. The primary task of the twin service is to handle Magistrala digital twins. Magistrala digital twin consists of three parts:
+Magistrala Twins service depends on the Magistrala IoT platform. The following diagram shows the place of the twins service in the overall Magistrala architecture:
+
You use an HTTP client to communicate with the twins service. Every request sent to the twins service is authenticated by users service. Twins service handles CRUD requests and creates, retrieves, updates and deletes twins. The CRUD operations depend on the database to persist and fetch already saved twins.
+Twins service listens to the message broker server and intercepts messages passing via the message broker. Every Magistrala message contains information about subchannel and topic used to send a message. Twins service compares this info with attribute definitions of twins persisted in the database, fetches the corresponding twins and updates their respective states.
+Before we dwell into twin's anatomy, it is important to realize that in order to use Magistrala twin service, you have to provision Magistrala things and channels and you have to connect things and channels beforehand. As you go, you can modify your things, channels and connections and you can modify your digital twin to reflect these modifications, but you have to have at least a minimal setup in order to use the twin service.
+Twin's general information stores twin's owner email - owner is represented by Magistrala user -, twin's ID (unique) and name (not necessarily unique), twin's creation and update dates as well as twin's revision number. The latter refers to the sequential number of twin's definition.
+The twin's definition is meant to be a semantic representation of system's data sources and consumers (data agents). Each data data agent is represented by means of attribute. Attribute consists of data agent's name, Magistrala channel and subtopic over which it communicates. Nota bene: each attribute is uniquely defined by the combination of channel and subtopic and we cannot have two or more attributes with the same channel and subtopic in the same definition.
+Attributes have a state persistence flag that determines whether the messages communicated by its corresponding channel and subtopic trigger the creation of a new twin state. Twin states are persisted in the separate collection of the same database. Currently, twins service uses the MongoDB. InfluxDB support for twins and states persistence is on the roadmap.
+When we define our digital twin, its JSON representation might look like this:
+{
+ "owner": "john.doe@email.net",
+ "id": "a838e608-1c1b-4fea-9c34-def877473a89",
+ "name": "grinding machine 2",
+ "revision": 2,
+ "created": "2020-05-05T08:41:39.142Z",
+ "updated": "2020-05-05T08:49:12.638Z",
+ "definitions": [
+ {
+ "id": 0,
+ "created": "2020-05-05T08:41:39.142Z",
+ "attributes": [],
+ "delta": 1000000
+ },
+ {
+ "id": 1,
+ "created": "2020-05-05T08:46:23.207Z",
+ "attributes": [
+ {
+ "name": "engine temperature",
+ "channel": "7ef6c61c-f514-402f-af4b-2401b588bfec",
+ "subtopic": "engine",
+ "persist_state": true
+ },
+ {
+ "name": "chassis temperature",
+ "channel": "7ef6c61c-f514-402f-af4b-2401b588bfec",
+ "subtopic": "chassis",
+ "persist_state": true
+ },
+ {
+ "name": "rotations per sec",
+ "channel": "a254032a-8bb6-4973-a2a1-dbf80f181a86",
+ "subtopic": "",
+ "persist_state": false
+ }
+ ],
+ "delta": 1000000
+ },
+ {
+ "id": 2,
+ "created": "2020-05-05T08:49:12.638Z",
+ "attributes": [
+ {
+ "name": "engine temperature",
+ "channel": "7ef6c61c-f514-402f-af4b-2401b588bfec",
+ "subtopic": "engine",
+ "persist_state": true
+ },
+ {
+ "name": "chassis temperature",
+ "channel": "7ef6c61c-f514-402f-af4b-2401b588bfec",
+ "subtopic": "chassis",
+ "persist_state": true
+ },
+ {
+ "name": "rotations per sec",
+ "channel": "a254032a-8bb6-4973-a2a1-dbf80f181a86",
+ "subtopic": "",
+ "persist_state": false
+ },
+ {
+ "name": "precision",
+ "channel": "aed0fbca-0d1d-4b07-834c-c62f31526569",
+ "subtopic": "",
+ "persist_state": true
+ }
+ ],
+ "delta": 1000000
+ }
+ ]
+}
+In the case of the upper twin, we begin with an empty definition, the one with the id 0 - we could have provided the definition immediately - and over the course of time, we add two more definitions, so the total number of revisions is 2 (revision index is zero-based). We decide not to persist the number of rotation per second in our digital twin state. We define it, though, because the definition and its attributes are used not only to define states of a complex data agent system, but also to define the semantic structure of the system. delta is the number of nanoseconds used to determine whether the received attribute value should trigger the generation of the new state or the same state should be updated. The reason for this is to enable state sampling over the regular intervals of time. Discarded values are written to the database of choice by Magistrala writers, so you can always retrieve intermediate values if need be.
states are created according to the twin's current definition. A state stores twin's ID - every state belongs to a single twin -, its own ID, twin's definition number, creation date and the actual payload. Payload is a set of key-value pairs where a key corresponds to the attribute name and a value is the actual value of the attribute. All SenML value types are supported.
+A JSON representation of a partial list of states might look like this:
+{
+ "total": 28,
+ "offset": 10,
+ "limit": 5,
+ "states": [
+ {
+ "twin_id": "a838e608-1c1b-4fea-9c34-def877473a89",
+ "id": 11,
+ "definition": 1,
+ "created": "2020-05-05T08:49:06.167Z",
+ "payload": {
+ "chassis temperature": 0.3394171011161684,
+ "engine temperature": 0.3814079472715233
+ }
+ },
+ {
+ "twin_id": "a838e608-1c1b-4fea-9c34-def877473a89",
+ "id": 12,
+ "definition": 1,
+ "created": "2020-05-05T08:49:12.168Z",
+ "payload": {
+ "chassis temperature": 1.8116442194724147,
+ "engine temperature": 0.3814079472715233
+ }
+ },
+ {
+ "twin_id": "a838e608-1c1b-4fea-9c34-def877473a89",
+ "id": 13,
+ "definition": 2,
+ "created": "2020-05-05T08:49:18.174Z",
+ "payload": {
+ "chassis temperature": 1.8116442194724147,
+ "engine temperature": 3.2410616702795814
+ }
+ },
+ {
+ "twin_id": "a838e608-1c1b-4fea-9c34-def877473a89",
+ "id": 14,
+ "definition": 2,
+ "created": "2020-05-05T08:49:19.145Z",
+ "payload": {
+ "chassis temperature": 3.2410616702795814,
+ "engine temperature": 3.2410616702795814,
+ "precision": 8.922156489392854
+ }
+ },
+ {
+ "twin_id": "a838e608-1c1b-4fea-9c34-def877473a89",
+ "id": 15,
+ "definition": 2,
+ "created": "2020-05-05T08:49:24.178Z",
+ "payload": {
+ "chassis temperature": 0.8694383878692546,
+ "engine temperature": 3.2410616702795814,
+ "precision": 8.922156489392854
+ }
+ }
+ ]
+}
+As you can see, the first two states correspond to the definition 1 and have only two attributes in the payload. The rest of the states is based on the definition 2, where we persist three attributes and, as a consequence, its payload consists of three entries.
+Twin belongs to a Magistrala user, tenant representing a physical person or an organization. User owns Magistrala things and channels as well as twins. Magistrala user provides authorization and authentication mechanisms to twins service. For more details, please see Authentication with Magistrala keys. In practical terms, we need to create a Magistrala user in order to create a digital twin. Every twin belongs to exactly one user. One user can have unlimited number of digital twins.
+For more information about the Twins service HTTP API please refer to the twins service OpenAPI file.
+Create and update requests use JSON body to initialize and modify, respectively, twin. You can omit every piece of data - every key-value pair - from the JSON. However, you must send at least an empty JSON body.
+{
+ "name": "twin_name",
+ "definition": {
+ "attributes": [
+ {
+ "name": "temperature",
+ "channel": "3b57b952-318e-47b5-b0d7-a14f61ecd03b",
+ "subtopic": "temperature",
+ "persist_state": true
+ },
+ {
+ "name": "humidity",
+ "channel": "3b57b952-318e-47b5-b0d7-a14f61ecd03b",
+ "subtopic": "humidity",
+ "persist_state": false
+ },
+ {
+ "name": "pressure",
+ "channel": "7ef6c61c-f514-402f-af4b-2401b588bfec",
+ "subtopic": "",
+ "persist_state": true
+ }
+ ],
+ "delta": 1
+ }
+}
+Create request uses POST HTTP method to create twin:
+curl -s -S -i -X POST -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" http://localhost:9018/twins -d '<twin_data>'
+If you do not suply the definition, the empty definition of the form
+{
+ "id": 0,
+ "created": "2020-05-05T08:41:39.142Z",
+ "attributes": [],
+ "delta": 1000000
+}
+will be created.
+curl -s -S -i -X PUT -H "Content-Type: application/json" -H "Authorization: Bearer <user_token>" http://localhost:9018/<twin_id> -d '<twin_data>'
+curl -s -S -i -X GET -H "Authorization: Bearer <user_token>" http://localhost:9018/twins/<twin_id>
+curl -s -S -i -X GET -H "Authorization: Bearer <user_token>" http://localhost:9018/twins
+List requests accepts limit and offset query parameters. By default, i.e. without these parameters, list requests fetches only first ten twins (or less, if there are less then ten twins).
You can fetch twins [10-29) like this:
+curl -s -S -i -X GET -H "Authorization: Bearer <user_token>" http://localhost:9018/twins?offset=10&limit=20
+curl -s -S -i -X DELETE -H "Authorization: Bearer <user_token>" http://localhost:9018/twins/<twin_id>
+curl -s -S -i -X GET -H "Authorization: Bearer <user_token>" http://localhost:9018/states/<twin_id>
+List requests accepts limit and offset query parameters. By default, i.e. without these parameters, list requests fetches only first ten states (or less, if there are less then ten states).
You can fetch states [10-29) like this:
+curl -s -S -i -X GET -H "Authorization: Bearer <user_token>" http://localhost:9018/states/<twin_id>?offset=10&limit=20
+Every twin and states related operation publishes notifications via the message broker. To fully understand what follows, please read about Magistrala messaging capabilities and utilities.
+In order to pick up this notifications, you have to create a Magistrala channel before you start the twins service and inform the twins service about the channel by means of the environment variable, like this:
+export MF_TWINS_CHANNEL_ID=f6894dfe-a7c9-4eef-a614-637ebeea5b4c
+The twins service will use this channel to publish notifications related to twins creation, update, retrieval and deletion. It will also publish notifications related to state saving into the database.
+All notifications will be published on the following message broker subject:
+channels.<mf_twins_channel_id>.<optional_subtopic>
+where <optional_subtopic> is one of the following:
create.success - on successful twin creation,create.failure - on twin creation failure,update.success - on successful twin update,update.failure - on twin update failure,get.success - on successful twin retrieval,get.failure - on twin retrieval failure,remove.success - on successful twin deletion,remove.failure - on twin deletion failure,save.success - on successful state savesave.failure - on state save failure.Normally, you can use the default message broker, NATS, wildcards. In order to learn more about Magistrala channel topic composition, please read about subtopics. The point is to be able to subscribe to all subjects or any operation pair subject - e.g. create.success/failure - by means of one connection and read all messages or all operation related messages in the context of the same subscription.
+Since messages published on message broker are republished on any other protocol supported by Magistrala - HTTP, MQTT, CoAP and WS - you can use any supported protocol client to pick up notifications.
+ + + + + + +