Export job reference
Job creation
Request
To submit an export job, send a POST request to https://api.botify.com/v1/jobs with the appropriate payload.
Payload:
{
"job_type": "export",
"payload": {
"username": USERNAME,
"project": PROJECT_SLUG,
"export_size": EXPORT_SIZE,
"formatter": FORMATTER,
"formatter_config": FORMATTER_CONFIG,
"connector": CONNECTOR,
"extra_config": EXTRA_CONFIG,
"query": BQL_QUERY
}
}
Let's dive into the details:
- job_type: will always have the value export for retrieve data. Other types of jobs exist, but they don't interest us here.
- username: the username to which belongs the project
- project: the project slug that can be found in Botify. See Getting started for how to retrieve your username+project slug combination
- export_size: the number of data rows that will be exported. A positive integer that will have an upper limit depending on the connector. See Backends and connectors
- formatter: the format type, see available values at Formatters
- formatter_config: the configuration linked to the formatter. See Formatters
- connector: the backend to which we will want to export the data. See Backends and connectors
- extra_config: the configuration linked to the connector/backend. See Backends and connectors
- query: the BQL query expressing what data we want to retrieve. See BQL Introduction
Headers:
Authentication: Token TOKEN
, authentication header, see Getting startedContent-Type: application/json
With all this information, you can create an export job.
Changing the Compression File Format
The default compression file format for exports is .GZ. To export to a .ZIP file instead, use the extra_config
key:
{
"extra_config": {
"filetype": "zip",
}
See the Backends and Connectors page for more information.
Response
Once created, you will get a response that looks like this:
{
"job_id": JOB_ID,
"job_type": "export",
"job_url": "/v1/jobs/JOB_ID",
"job_status": "CREATED",
"payload": PAYLOAD,
"results": null,
"date_created": DATE_CREATED,
"user": USERNAME,
"metadata": null
}
About this payload:
- job_id: the unique job identifier that will allow you to retrieve the state of the job
- job_type: is just reminding you of the job type you just created: export.
- job_url: the URL on which you will be able to query your job's status.
- job_status: Can be:
-
- CREATED: we didn't start scheduling the job yet.
-
- PENDING: we are waiting for a worker to be available to start working on your export.
-
- PROCESSING: we are computing the export
-
- DONE: the job was completed successfully, the results key should be filled with the relevant information
-
- FAILED: oops, something went wrong. The results key should contain information. If not, don't hesitate to contact our support.
Job status
Once you create a job, it will be queued in our computation system. Depending on the volume and complexity of the export you requested, it might take some time, up to multiple hours.
Do a GET request on https://api.botify.com/v1/jobs/JOB_ID in order to know if the job is done or is still processing.
The only needed header is authentication, see Getting started to fetch it.
The response payload is the same as the job creation response.
Updated about 1 year ago
Check out what backends and formatters are available, and how to write a BQL query: