Bulk exports and incremental syncs

Recommended patterns for backfills and ongoing sync jobs using updated_at

View as Markdown

Use the top-level list endpoints with filter[updated_at][...], order_by, and cursor pagination to build both one-time exports and recurring sync jobs:

  • GET /v2/contacts
  • GET /v2/companies
  • GET /v2/users
  • GET /v2/tags
  • GET /v2/engagements

The shared contract is:

  • limit defaults to 25 and is capped at 100
  • order_by supports updated_at:asc and updated_at:desc
  • filter[updated_at][gt], filter[updated_at][gte], filter[updated_at][lt], and filter[updated_at][lte] accept ISO-8601 UTC timestamps
  • next_cursor continues the current traversal until has_more becomes false

Which pattern to use

GoalRecommended request shapeWhy
Initial bulk exportorder_by=updated_at:asc&limit=100Walks oldest to newest so your checkpoint moves forward predictably.
Incremental syncfilter[updated_at][gte]=<checkpoint>&order_by=updated_at:asc&limit=100Re-reads the boundary timestamp so your sync can deduplicate instead of risking a gap.
Latest activity vieworder_by=updated_at:desc&limit=25Returns newest updates first for dashboards or ad hoc inspection.

Initial bulk export

For a first-time export, start from the beginning and walk forward in ascending updated_at order.

  1. Request the resource without an updated_at filter.
  2. Set order_by=updated_at:asc.
  3. Set limit=100 unless you need a smaller page size.
  4. Follow next_cursor until has_more is false.
  5. Upsert each record into your destination system.
  6. Record the largest updated_at value you processed, but only promote it to your saved checkpoint after the full run finishes successfully.

Example:

curl --request GET \
--url 'https://app.askelephant.ai/api/v2/contacts?limit=100&order_by=updated_at:asc' \
--header 'Authorization: sk-apik_<id>.<secret>' \
--header 'Accept: application/json'

When the response contains a next_cursor, continue the same traversal:

curl --request GET \
--url 'https://app.askelephant.ai/api/v2/contacts?limit=100&order_by=updated_at:asc&cursor=<next_cursor>' \
--header 'Authorization: sk-apik_<id>.<secret>' \
--header 'Accept: application/json'

Incremental sync

For recurring syncs, use your saved checkpoint as the lower bound and keep walking forward.

  1. Load the last completed checkpoint for the resource.
  2. Request filter[updated_at][gte]=<checkpoint> with order_by=updated_at:asc.
  3. Follow next_cursor until the run is complete.
  4. Upsert records by id.
  5. Deduplicate records you have already processed at the checkpoint boundary.
  6. Before advancing the checkpoint, run an overlap/lookback or reconciliation pass below the prior checkpoint to recover rows whose updated_at moved backward during the cursor walk.
  7. After the final page and recovery pass succeed, save the highest updated_at seen during the run as the next checkpoint.

Example:

curl --request GET \
--url 'https://app.askelephant.ai/api/v2/contacts?limit=100&filter[updated_at][gte]=2026-03-01T00:00:00.000Z&order_by=updated_at:asc' \
--header 'Authorization: sk-apik_<id>.<secret>' \
--header 'Accept: application/json'

Why gte is the safer default for syncs

gt is available and can reduce duplicate work:

filter[updated_at][gt]=2026-03-01T00:00:00.000Z

For most data pipelines, gte is the safer default because it intentionally replays the checkpoint boundary. That lets your consumer deduplicate by id and updated_at instead of depending on a strict handoff at exactly one timestamp value.

If duplicate reads are significantly more expensive than deduplication, you can switch to gt. The tradeoff is that your checkpoint handling must be tighter because you are no longer replaying the boundary.

Persist checkpoint state per resource type. A minimal shape is:

{
"resource": "contacts",
"updated_at": "2026-03-04T18:25:00Z"
}

If your destination supports idempotent upserts, use that. It makes retries and boundary replays much simpler.

Structured transcript timeline backfills

To export structured transcript timelines (speakers, timed entries, and speaker identity) for many engagements, use list pagination with expand=transcript_timeline instead of one GET /v2/engagements/{engagement_id}/transcript_timeline per engagement.

  1. Request GET /v2/engagements with expand=transcript_timeline, limit=25, and your preferred order_by and filters.
  2. Follow next_cursor until has_more is false.
  3. Read transcript_timeline from each engagement in data. The object matches the per-engagement timeline endpoint. When no primary timeline exists, transcript_timeline is null.

This expand requires transcript_timelines:read in addition to engagements:read. Plaintext expand=transcript is a separate field and does not return structured timelines. Pages that include expand=transcript_timeline are capped at 25 engagements to bound synchronous database, serialization, and response-size work. Supplying a larger limit still returns at most 25 engagements.

Each list request counts as one read against workspace rate limits, even when up to 25 structured timelines are embedded. A 10,000-engagement backfill therefore needs on the order of 400 list requests (plus pagination overhead), not 10,000 timeline GETs.

Example:

curl --request GET \
--globoff \
--url 'https://app.askelephant.ai/api/v2/engagements?limit=25&order_by=updated_at:asc&filter[processing_status][eq]=COMPLETED&expand=transcript_timeline' \
--header 'Authorization: sk-apik_<id>.<secret>' \
--header 'Accept: application/json'

The same expand=transcript_timeline parameter is supported on GET /v2/companies/{company_id}/engagements and GET /v2/contacts/{contact_id}/engagements.

See also: Searching engagements for transcript expand patterns and scope requirements.

Cursor walks are live, not snapshot-isolated

List cursors use keyset pagination on the active sort key plus id (see Pagination and filtering). While you walk next_cursor pages, the underlying dataset can change:

  • A row whose updated_at increases during an order_by=updated_at:asc export can appear on a later page. Upsert by id so these duplicates are safe.
  • A row whose updated_at decreases during the same walk may be skipped relative to your in-progress cursor.

Checkpoint filters (filter[updated_at][gte]=<checkpoint>) narrow what you read, but they do not freeze the collection for the duration of a multi-page walk. Boundary replay with gte handles equal timestamps and upserts handle duplicates, but neither recovers a row that moved below the checkpoint. Before promoting the checkpoint, run an overlap/lookback window and reconcile by id; for loss-intolerant exports, also schedule a periodic full reconciliation because updated_at is not guaranteed to move only forward.

Practical notes

  • Keep separate checkpoints for each resource type because each collection paginates independently.
  • Do not mix cursor values between different endpoints or different query shapes.
  • Keep the same order_by and filter values for every page in a single traversal.
  • Save checkpoints only after a successful run, not after each page.
  • Use updated_at:desc for operational views and debugging, not for forward-moving export jobs.
  • Design sync consumers to tolerate duplicate rows from live cursor pagination; see How cursors work.

See also: Pagination and filtering