Bulk exports and incremental syncs
Recommended patterns for backfills and ongoing sync jobs using updated_at
Use the top-level list endpoints with filter[updated_at][...], order_by, and cursor pagination to build both one-time exports and recurring sync jobs:
GET /v2/contactsGET /v2/companiesGET /v2/usersGET /v2/tagsGET /v2/engagements
The shared contract is:
limitdefaults to25and is capped at100order_bysupportsupdated_at:ascandupdated_at:descfilter[updated_at][gt],filter[updated_at][gte],filter[updated_at][lt], andfilter[updated_at][lte]accept ISO-8601 UTC timestampsnext_cursorcontinues the current traversal untilhas_morebecomesfalse
Which pattern to use
Initial bulk export
For a first-time export, start from the beginning and walk forward in ascending updated_at order.
- Request the resource without an
updated_atfilter. - Set
order_by=updated_at:asc. - Set
limit=100unless you need a smaller page size. - Follow
next_cursoruntilhas_moreisfalse. - Upsert each record into your destination system.
- Record the largest
updated_atvalue you processed, but only promote it to your saved checkpoint after the full run finishes successfully.
Example:
When the response contains a next_cursor, continue the same traversal:
Incremental sync
For recurring syncs, use your saved checkpoint as the lower bound and keep walking forward.
- Load the last completed checkpoint for the resource.
- Request
filter[updated_at][gte]=<checkpoint>withorder_by=updated_at:asc. - Follow
next_cursoruntil the run is complete. - Upsert records by
id. - Deduplicate records you have already processed at the checkpoint boundary.
- Before advancing the checkpoint, run an overlap/lookback or reconciliation pass below the prior checkpoint to recover rows whose
updated_atmoved backward during the cursor walk. - After the final page and recovery pass succeed, save the highest
updated_atseen during the run as the next checkpoint.
Example:
Why gte is the safer default for syncs
gt is available and can reduce duplicate work:
For most data pipelines, gte is the safer default because it intentionally replays the checkpoint boundary. That lets your consumer deduplicate by id and updated_at instead of depending on a strict handoff at exactly one timestamp value.
If duplicate reads are significantly more expensive than deduplication, you can switch to gt. The tradeoff is that your checkpoint handling must be tighter because you are no longer replaying the boundary.
Recommended checkpoint shape
Persist checkpoint state per resource type. A minimal shape is:
If your destination supports idempotent upserts, use that. It makes retries and boundary replays much simpler.
Structured transcript timeline backfills
To export structured transcript timelines (speakers, timed entries, and speaker identity) for many engagements, use list pagination with expand=transcript_timeline instead of one GET /v2/engagements/{engagement_id}/transcript_timeline per engagement.
- Request
GET /v2/engagementswithexpand=transcript_timeline,limit=25, and your preferredorder_byand filters. - Follow
next_cursoruntilhas_moreisfalse. - Read
transcript_timelinefrom each engagement indata. The object matches the per-engagement timeline endpoint. When no primary timeline exists,transcript_timelineisnull.
This expand requires transcript_timelines:read in addition to engagements:read. Plaintext expand=transcript is a separate field and does not return structured timelines.
Pages that include expand=transcript_timeline are capped at 25 engagements to bound synchronous database, serialization, and response-size work. Supplying a larger limit still returns at most 25 engagements.
Each list request counts as one read against workspace rate limits, even when up to 25 structured timelines are embedded. A 10,000-engagement backfill therefore needs on the order of 400 list requests (plus pagination overhead), not 10,000 timeline GETs.
Example:
The same expand=transcript_timeline parameter is supported on GET /v2/companies/{company_id}/engagements and GET /v2/contacts/{contact_id}/engagements.
See also: Searching engagements for transcript expand patterns and scope requirements.
Cursor walks are live, not snapshot-isolated
List cursors use keyset pagination on the active sort key plus id (see Pagination and filtering). While you walk next_cursor pages, the underlying dataset can change:
- A row whose
updated_atincreases during anorder_by=updated_at:ascexport can appear on a later page. Upsert byidso these duplicates are safe. - A row whose
updated_atdecreases during the same walk may be skipped relative to your in-progress cursor.
Checkpoint filters (filter[updated_at][gte]=<checkpoint>) narrow what you read, but they do not freeze the collection for the duration of a multi-page walk. Boundary replay with gte handles equal timestamps and upserts handle duplicates, but neither recovers a row that moved below the checkpoint. Before promoting the checkpoint, run an overlap/lookback window and reconcile by id; for loss-intolerant exports, also schedule a periodic full reconciliation because updated_at is not guaranteed to move only forward.
Practical notes
- Keep separate checkpoints for each resource type because each collection paginates independently.
- Do not mix
cursorvalues between different endpoints or different query shapes. - Keep the same
order_byand filter values for every page in a single traversal. - Save checkpoints only after a successful run, not after each page.
- Use
updated_at:descfor operational views and debugging, not for forward-moving export jobs. - Design sync consumers to tolerate duplicate rows from live cursor pagination; see How cursors work.
See also: Pagination and filtering