Job creation

Request

To submit an export job, send a POST request to https://api.botify.com/v1/jobs with the appropriate payload.

Payload:

{
  "job_type": "export",
  "payload": {
    "username": USERNAME,
    "project": PROJECT_SLUG,
    "export_size": EXPORT_SIZE,
    "formatter": FORMATTER,
    "formatter_config": FORMATTER_CONFIG,
    "connector": CONNECTOR,
    "extra_config": EXTRA_CONFIG,
    "query": BQL_QUERY
  }
}

Let's dive into the details:

• job_type: will always have the value export for retrieve data. Other types of jobs exist, but they don't interest us here.
• username: the username to which belongs the project
• project: the project slug that can be found in Botify. See Getting started for how to retrieve your username+project slug combination
• export_size: the number of data rows that will be exported. A positive integer that will have an upper limit depending on the connector. See Backends and connectors
• formatter: the format type, see available values at Formatters
• formatter_config: the configuration linked to the formatter. See Formatters
• connector: the backend to which we will want to export the data. See Backends and connectors
• extra_config: the configuration linked to the connector/backend. See Backends and connectors
• query: the BQL query expressing what data we want to retrieve. See BQL Introduction

Headers:

• Authentication: Token TOKEN, authentication header, see Getting started
• Content-Type: application/json

With all this information, you can create an export job.

Changing the Compression File Format

The default compression file format for exports is .GZ. To export to a .ZIP file instead, use the extra_config key:

{
  "extra_config": {
    "filetype": "zip",
  }

See the Backends and Connectors page for more information.

Response

Once created, you will get a response that looks like this:

{
    "job_id": JOB_ID,
    "job_type": "export",
    "job_url": "/v1/jobs/JOB_ID",
    "job_status": "CREATED",
    "payload": PAYLOAD,
    "results": null,
    "date_created": DATE_CREATED,
    "user": USERNAME,
    "metadata": null
}

About this payload:

• job_id: the unique job identifier that will allow you to retrieve the state of the job
• job_type: is just reminding you of the job type you just created: export.
• job_url: the URL on which you will be able to query your job's status.
• job_status: Can be:
• • CREATED: we didn't start scheduling the job yet.
• • PENDING: we are waiting for a worker to be available to start working on your export.
• • PROCESSING: we are computing the export
• • DONE: the job was completed successfully, the results key should be filled with the relevant information
• • FAILED: oops, something went wrong. The results key should contain information. If not, don't hesitate to contact our support.

Job status

Once you create a job, it will be queued in our computation system. Depending on the volume and complexity of the export you requested, it might take some time, up to multiple hours.

Do a GET request on https://api.botify.com/v1/jobs/JOB_ID in order to know if the job is done or is still processing.
The only needed header is authentication, see Getting started to fetch it.

The response payload is the same as the job creation response.