{"activeVersionTag":"latest","latestAvailableVersionTag":"latest","collection":{"info":{"_postman_id":"0396aedd-a176-4dc7-9082-262894bb45b7","name":"LetsData Control SDK API","description":"# LetsData Control SDK\n\nThis is the guide for LetsData Control SDK. Control SDK has APIs for creation and management of the APIs to create datasets, update code, list tasks, logs, metrics, errors and usage records etc. You'll use the Control SDK to orchestrate the data pipelines (how to create and manage a datasets / data processing jobs). The Data SDK, on the other hand, is the interface for the user's code that needs to executed for data processing. (Data SDK Docs are at this [link](https://www.letsdata.io/docs/sdk-interface/))\n\nHere is a list of the different resources and actions that are supported by the LetsData Control SDK API:\n\n<u><b>Datasets</b></u>\n\n- dataset\n    - view a dataset\n- dataset/create\n    - create a dataset\n- dataset/list\n    - list all datasets\n- dataset/delete\n    - delete a dataset\n    - view the status of a delete request\n- dataset/code\n    - update (update code jar)\n    - view (view status of existing update code request)\n- dataset/compute\n    - update (update dataset compute config)\n\n<u><b>Tasks</b></u>\n\n- tasks/list\n    - list tasks for a dataset\n- tasks/redrive\n    - create (create a dataset task redrive request)\n    - view (view the status of a dataset task redrive request)\n- tasks/stop\n    - create (create a dataset task stop request)\n\n<u><b>Errors</b></u>\n\n- errors\n    - view an error record file\n- errors/list\n    \n\n<u><b>Logs</b></u>\n\n- logs\n    - view a task's log file\n\n<u><b>Metrics</b></u>\n\n- metrics\n    - view a dataset's metrics\n\n<u><b>VPC</b></u>\n\n- vpc\n    - list vpcs for a dataset\n- vpc/listvpcpeeringconnection\n    - list vpc peering connections for a dataset's vpc\n- vpc/deletevpcpeeringconnection\n    - delete vpc peering connections for a dataset's vpc\n- vpc/acceptvpcpeeringconnection\n    - accept vpc peering connection for a dataset's vpc\n\n<u><b>Users</b></u>\n\n- users\n    - view a user's detail\n- users/create\n    - create a user\n- users/update\n    - update a user's details\n- users/delete\n    - delete a user\n- users/list\n    - list the users\n\n<u><b>Costs</b></u>\n\n- costs\n    - list the costs for a tenant user\n\n<u><b>Usage Records</b></u>\n\n- usagerecords/list\n    - list the usage records for a dataset\n\n# **Prequisites**\n\nTo start using the LetsData Control SDK, you need to:\n\n- Have a valid LetsData account (username and a password). You can sign-up for a LetsData account at: [https://www.letsdata.io/#signup](https://www.letsdata.io/#signup)\n- For any serious calls, you'll need a valid Client Id to use the LetsData Control SDK. You can request one by emailing at [support@letsdata.io](https://mailto:support@letsdata.io) or logging an issue at [https://www.letsdata.io/#support](https://www.letsdata.io/#support) - we'll enable Control API access for your tenant. For any testing and experimentation, you can use the testing and experimentation clientId `6ent0fqtc4v5ud6i8o41ado8rj`. Being a multi tenant system, the clientId helps us differentiate API calls from different clients.\n    \n- Currently there are no API usage rate limits and restrictions.\n    \n\n# Authentication\n\nYou'll need to obtain `AccessToken` and `IdToken` by calling AWS Cognito. Here is a sample request and response:\n\n- Post data saved in file`auth_data.json`\n    \n\n``` json\n{\n    \"AuthParameters\" : {\n        \"USERNAME\" : \"{{LetsData Username}}\",\n        \"PASSWORD\" : \"{{LetsData Password}}\"\n    },\n    \"AuthFlow\" : \"USER_PASSWORD_AUTH\", \n    \"ClientId\" : \"{{LetsData ClientId}}\"\n}\n\n ```\n\n- Post request\n    \n\n``` bash\ncurl -X POST --data @auth_data.json \\\n    -H 'X-Amz-Target: AWSCognitoIdentityProviderService.InitiateAuth' \\\n    -H 'Content-Type: application/x-amz-json-1.1' \\\n    https://cognito-idp.us-east-1.amazonaws.com/ \\\n    --output creds.json\n\n ```\n\n- Example response is saved in the `creds.json` - a quick `cat creds.json` show the following json - copy the `AccessToken` and `IdToken`, you'll need these for API calls. Also note the expiry which is the duration the token is valid for.\n    \n\n``` json\n{\n  \"AuthenticationResult\": {\n    \"AccessToken\": \"eyJraWQiOiJaQk...<redacted>\",\n    \"ExpiresIn\": 3600,\n    \"IdToken\": \"eyJraWQiOiJuSktcL1JN...<redacted>\",\n    \"RefreshToken\": \"eyJjdHkiOiJKV1Qi...<redacted>\",\n    \"TokenType\": \"Bearer\"\n  },\n  \"ChallengeParameters\": {}\n}\n\n ```\n\n- You can call any of the LetsData Control SDK API by adding the `-H \"Authorization: Bearer {{IdToken}}\"` and `-H \"LetsDataAuthorization: Bearer {{AccessToken}}\"` headers. Here is an example API call that does a GET to retrieve a dataset's details.\n    \n\n``` bash\ncurl \"https://www.letsdata.io/api/dataset?tenantId={{tenantId}}&userId={{userId}}&datasetName={{datasetName}}\" \\\n    -H \"Authorization: Bearer {{IdToken}}\" \\\n    -H \"LetsDataAuthorization: Bearer {{AccessToken}}\"\n\n ```\n\n- Almost every Control SDK API requires the `tenantId` and the `userId` for the authenticated user (TenantAdmins can pass a different userId to retrieve data for other users in the organization - see [user roles documentation](https://www.letsdata.io/docs/user-management/#user-roles)). You can find your tenantId, userId via the console, cli ([docs](https://www.letsdata.io/docs/access-grants/#create-access-grants-role)) or decode the `IdToken` to get the tenant and user ids ([jwt.io](https://jwt.io/) has a decoder). Here is a decoded Id token - the `sub` field is the `userId` and the `custom:tenantid` is the `tenantId`\n    \n\n``` json\n{\n  \"sub\": \"accb3567-2b6e-41ae-b00d-6ce1f9a58d94\",\n  \"custom:companyaddress\": \"{\\\"addressLine1\\\":\\\"1234 Some Street\\\",\\\"addressLine2\\\":\\\"Apt F8\\\",\\\"city\\\":\\\"Bellevue\\\",\\\"state\\\":\\\"WA\\\",\\\"country\\\":\\\"US\\\",\\\"postalCode\\\":\\\"98006\\\"}\",\n  \"cognito:groups\": [\n    \"Tenant-d5feaf90-71a9-41ee-b1b9-35e4242c3155-Users\"\n  ],\n  \"custom:userrole\": \"TenantAdmin\",\n  \"iss\": \"https://cognito-idp.us-east-1.amazonaws.com/us-east-1_asdjery68Ts\",\n  \"cognito:username\": \"user@letsdata.io\",\n  \"custom:companyname\": \"LetsData IO\",\n  \"origin_jti\": \"1a0038a2-f8dd-4a71-996e-e64cde31003c\",\n  \"custom:tenantid\": \"d5feaf90-71a9-41ee-b1b9-35e4242c3155\",\n  \"aud\": \"11bbm85f3niuukca8su98dqc2t\",\n  \"event_id\": \"978d2a81-fa94-4fdf-a5ce-52240c02aaaa\",\n  \"token_use\": \"id\",\n  \"auth_time\": 1708220818,\n  \"exp\": 1708224418,\n  \"iat\": 1708220818,\n  \"jti\": \"194095f3-f350-425b-ad84-c5f8e1dd67fc\",\n  \"email\": \"user@letsdata.io\"\n}\n\n ```\n\n# API Common Errors\n\nHere are common errors for the API that you may see from time to time and an explanation of what they mean:\n\n## HTTP 401 Unauthorized\n\nIf the request could not be authorized, you will receive an HTTP 401 Unauthorized response code. The response body will have the following JSON:\n\n``` json\n{\n    \"statusCode\": \"{{LETSDATA_STATUS_CODE}}\",\n    \"message\": \"{{LETSDATA_ERROR_MESSAGE}}\"\n}\n\n ```\n\nThe following status codes / messages are expected\n\n- **LETSDATA_ACCESS_TOKEN_UNAUTHORIZED:** AWS Cognito was unable to verify the access token, so some parameter is possibly wrong in the auth request, expired token etc.\n    \n\n``` json\n{\n    \"statusCode\": \"LETSDATA_ACCESS_TOKEN_UNAUTHORIZED\",\n    \"message\": \"The access token in the request is unauthorized to view this resource.\"\n}\n\n ```\n\n- **UNAUTHORIZED:** LetsData could not authenticate / authorize the request, some validation check failed. These could be different tenantId in request data and id token, different userIds, incorrect clientId, expired token or incorrect user role ([user roles docs](https://www.letsdata.io/docs/user-management/#user-roles)) similar.\n    \n\n``` json\n{\n    \"statusCode\": \"UNAUTHORIZED\",\n    \"message\": \"The user is unauthorized to view this resource.\"\n}\n\n ```\n\n## HTTP 404 NotFound\n\nHTTP 404 can be returned when the resource could not be found. Also, if the Http verb is different (for example GET was issued but the code requires a POST), you'll also see Http 404 NotFound response with the following body `\"Unsupported Http Method. Expected HttpMethod: POST, Actual HttpMethod: GET\"`","schema":"https://schema.getpostman.com/json/collection/v2.0.0/collection.json","isPublicCollection":false,"owner":"33034393","collectionId":"0396aedd-a176-4dc7-9082-262894bb45b7","publishedId":"2sA2xh3D7y","public":true,"publicUrl":"https://documenter-api.postman.tech/view/33034393/2sA2xh3D7y","privateUrl":"https://go.postman.co/documentation/33034393-0396aedd-a176-4dc7-9082-262894bb45b7","customColor":{"top-bar":"FFFFFF","right-sidebar":"303030","highlight":"FF6C37"},"documentationLayout":"classic-single-column","customisation":{"metaTags":[{"name":"description","value":""},{"name":"title","value":"LetsData Control SDK"}],"appearance":{"default":"dark","themes":[{"name":"dark","logo":"https://content.pstmn.io/00cdffd1-5707-4d1f-853c-539b75b8bc2e/bG9nby1jb2xvci5wbmc=","colors":{"top-bar":"212121","right-sidebar":"303030","highlight":"FF6C37"}},{"name":"light","logo":"https://content.pstmn.io/00cdffd1-5707-4d1f-853c-539b75b8bc2e/bG9nby1jb2xvci5wbmc=","colors":{"top-bar":"FFFFFF","right-sidebar":"303030","highlight":"FF6C37"}}]}},"version":"8.10.0","publishDate":"2024-03-09T21:59:27.000Z","activeVersionTag":"latest","documentationTheme":"light","metaTags":{"title":"LetsData Control SDK","description":""},"logos":{"logoLight":"https://content.pstmn.io/00cdffd1-5707-4d1f-853c-539b75b8bc2e/bG9nby1jb2xvci5wbmc=","logoDark":"https://content.pstmn.io/00cdffd1-5707-4d1f-853c-539b75b8bc2e/bG9nby1jb2xvci5wbmc="}},"statusCode":200},"environments":[],"user":{"authenticated":false,"permissions":{"publish":false}},"run":{"button":{"js":"https://run.pstmn.io/button.js","css":"https://run.pstmn.io/button.css"}},"web":"https://www.getpostman.com/","team":{"logo":"https://res.cloudinary.com/postman/image/upload/t_team_logo_pubdoc/v1/team/768118b36f06c94b0306958b980558e6915839447e859fe16906e29d683976f0","favicon":""},"isEnvFetchError":false,"languages":"[{\"key\":\"csharp\",\"label\":\"C#\",\"variant\":\"HttpClient\"},{\"key\":\"csharp\",\"label\":\"C#\",\"variant\":\"RestSharp\"},{\"key\":\"curl\",\"label\":\"cURL\",\"variant\":\"cURL\"},{\"key\":\"dart\",\"label\":\"Dart\",\"variant\":\"http\"},{\"key\":\"go\",\"label\":\"Go\",\"variant\":\"Native\"},{\"key\":\"http\",\"label\":\"HTTP\",\"variant\":\"HTTP\"},{\"key\":\"java\",\"label\":\"Java\",\"variant\":\"OkHttp\"},{\"key\":\"java\",\"label\":\"Java\",\"variant\":\"Unirest\"},{\"key\":\"javascript\",\"label\":\"JavaScript\",\"variant\":\"Fetch\"},{\"key\":\"javascript\",\"label\":\"JavaScript\",\"variant\":\"jQuery\"},{\"key\":\"javascript\",\"label\":\"JavaScript\",\"variant\":\"XHR\"},{\"key\":\"c\",\"label\":\"C\",\"variant\":\"libcurl\"},{\"key\":\"nodejs\",\"label\":\"NodeJs\",\"variant\":\"Axios\"},{\"key\":\"nodejs\",\"label\":\"NodeJs\",\"variant\":\"Native\"},{\"key\":\"nodejs\",\"label\":\"NodeJs\",\"variant\":\"Request\"},{\"key\":\"nodejs\",\"label\":\"NodeJs\",\"variant\":\"Unirest\"},{\"key\":\"objective-c\",\"label\":\"Objective-C\",\"variant\":\"NSURLSession\"},{\"key\":\"ocaml\",\"label\":\"OCaml\",\"variant\":\"Cohttp\"},{\"key\":\"php\",\"label\":\"PHP\",\"variant\":\"cURL\"},{\"key\":\"php\",\"label\":\"PHP\",\"variant\":\"Guzzle\"},{\"key\":\"php\",\"label\":\"PHP\",\"variant\":\"HTTP_Request2\"},{\"key\":\"php\",\"label\":\"PHP\",\"variant\":\"pecl_http\"},{\"key\":\"powershell\",\"label\":\"PowerShell\",\"variant\":\"RestMethod\"},{\"key\":\"python\",\"label\":\"Python\",\"variant\":\"http.client\"},{\"key\":\"python\",\"label\":\"Python\",\"variant\":\"Requests\"},{\"key\":\"r\",\"label\":\"R\",\"variant\":\"httr\"},{\"key\":\"r\",\"label\":\"R\",\"variant\":\"RCurl\"},{\"key\":\"ruby\",\"label\":\"Ruby\",\"variant\":\"Net::HTTP\"},{\"key\":\"shell\",\"label\":\"Shell\",\"variant\":\"Httpie\"},{\"key\":\"shell\",\"label\":\"Shell\",\"variant\":\"wget\"},{\"key\":\"swift\",\"label\":\"Swift\",\"variant\":\"URLSession\"}]","languageSettings":[{"key":"csharp","label":"C#","variant":"HttpClient"},{"key":"csharp","label":"C#","variant":"RestSharp"},{"key":"curl","label":"cURL","variant":"cURL"},{"key":"dart","label":"Dart","variant":"http"},{"key":"go","label":"Go","variant":"Native"},{"key":"http","label":"HTTP","variant":"HTTP"},{"key":"java","label":"Java","variant":"OkHttp"},{"key":"java","label":"Java","variant":"Unirest"},{"key":"javascript","label":"JavaScript","variant":"Fetch"},{"key":"javascript","label":"JavaScript","variant":"jQuery"},{"key":"javascript","label":"JavaScript","variant":"XHR"},{"key":"c","label":"C","variant":"libcurl"},{"key":"nodejs","label":"NodeJs","variant":"Axios"},{"key":"nodejs","label":"NodeJs","variant":"Native"},{"key":"nodejs","label":"NodeJs","variant":"Request"},{"key":"nodejs","label":"NodeJs","variant":"Unirest"},{"key":"objective-c","label":"Objective-C","variant":"NSURLSession"},{"key":"ocaml","label":"OCaml","variant":"Cohttp"},{"key":"php","label":"PHP","variant":"cURL"},{"key":"php","label":"PHP","variant":"Guzzle"},{"key":"php","label":"PHP","variant":"HTTP_Request2"},{"key":"php","label":"PHP","variant":"pecl_http"},{"key":"powershell","label":"PowerShell","variant":"RestMethod"},{"key":"python","label":"Python","variant":"http.client"},{"key":"python","label":"Python","variant":"Requests"},{"key":"r","label":"R","variant":"httr"},{"key":"r","label":"R","variant":"RCurl"},{"key":"ruby","label":"Ruby","variant":"Net::HTTP"},{"key":"shell","label":"Shell","variant":"Httpie"},{"key":"shell","label":"Shell","variant":"wget"},{"key":"swift","label":"Swift","variant":"URLSession"}],"languageOptions":[{"label":"C# - HttpClient","value":"csharp - HttpClient - C#"},{"label":"C# - RestSharp","value":"csharp - RestSharp - C#"},{"label":"cURL - cURL","value":"curl - cURL - cURL"},{"label":"Dart - http","value":"dart - http - Dart"},{"label":"Go - Native","value":"go - Native - Go"},{"label":"HTTP - HTTP","value":"http - HTTP - HTTP"},{"label":"Java - OkHttp","value":"java - OkHttp - Java"},{"label":"Java - Unirest","value":"java - Unirest - Java"},{"label":"JavaScript - Fetch","value":"javascript - Fetch - JavaScript"},{"label":"JavaScript - jQuery","value":"javascript - jQuery - JavaScript"},{"label":"JavaScript - XHR","value":"javascript - XHR - JavaScript"},{"label":"C - libcurl","value":"c - libcurl - C"},{"label":"NodeJs - Axios","value":"nodejs - Axios - NodeJs"},{"label":"NodeJs - Native","value":"nodejs - Native - NodeJs"},{"label":"NodeJs - Request","value":"nodejs - Request - NodeJs"},{"label":"NodeJs - Unirest","value":"nodejs - Unirest - NodeJs"},{"label":"Objective-C - NSURLSession","value":"objective-c - NSURLSession - Objective-C"},{"label":"OCaml - Cohttp","value":"ocaml - Cohttp - OCaml"},{"label":"PHP - cURL","value":"php - cURL - PHP"},{"label":"PHP - Guzzle","value":"php - Guzzle - PHP"},{"label":"PHP - HTTP_Request2","value":"php - HTTP_Request2 - PHP"},{"label":"PHP - pecl_http","value":"php - pecl_http - PHP"},{"label":"PowerShell - RestMethod","value":"powershell - RestMethod - PowerShell"},{"label":"Python - http.client","value":"python - http.client - Python"},{"label":"Python - Requests","value":"python - Requests - Python"},{"label":"R - httr","value":"r - httr - R"},{"label":"R - RCurl","value":"r - RCurl - R"},{"label":"Ruby - Net::HTTP","value":"ruby - Net::HTTP - Ruby"},{"label":"Shell - Httpie","value":"shell - Httpie - Shell"},{"label":"Shell - wget","value":"shell - wget - Shell"},{"label":"Swift - URLSession","value":"swift - URLSession - Swift"}],"layoutOptions":[{"value":"classic-single-column","label":"Single Column"},{"value":"classic-double-column","label":"Double Column"}],"versionOptions":[],"environmentOptions":[{"value":"0","label":"No Environment"}],"canonicalUrl":"https://documenter.gw.postman.com/view/metadata/2sA2xh3D7y"}