Bulk exports and incremental syncs
Recommended patterns for backfills and ongoing sync jobs using updated_at
Use the top-level list endpoints with filter[updated_at][...], order_by, and cursor pagination to build both one-time exports and recurring sync jobs:
GET /v2/contactsGET /v2/companiesGET /v2/usersGET /v2/tagsGET /v2/engagements
The shared contract is:
limitdefaults to25and is capped at100order_bysupportsupdated_at:ascandupdated_at:descfilter[updated_at][gt],filter[updated_at][gte],filter[updated_at][lt], andfilter[updated_at][lte]accept ISO-8601 UTC timestampsnext_cursorcontinues the current traversal untilhas_morebecomesfalse
Which pattern to use
Initial bulk export
For a first-time export, start from the beginning and walk forward in ascending updated_at order.
- Request the resource without an
updated_atfilter. - Set
order_by=updated_at:asc. - Set
limit=100unless you need a smaller page size. - Follow
next_cursoruntilhas_moreisfalse. - Upsert each record into your destination system.
- Record the largest
updated_atvalue you processed, but only promote it to your saved checkpoint after the full run finishes successfully.
Example:
When the response contains a next_cursor, continue the same traversal:
Incremental sync
For recurring syncs, use your saved checkpoint as the lower bound and keep walking forward.
- Load the last completed checkpoint for the resource.
- Request
filter[updated_at][gte]=<checkpoint>withorder_by=updated_at:asc. - Follow
next_cursoruntil the run is complete. - Upsert records by
id. - Deduplicate records you have already processed at the checkpoint boundary.
- After the final page succeeds, save the highest
updated_atseen during the run as the next checkpoint.
Example:
Why gte is the safer default for syncs
gt is available and can reduce duplicate work:
For most data pipelines, gte is the safer default because it intentionally replays the checkpoint boundary. That lets your consumer deduplicate by id and updated_at instead of depending on a strict handoff at exactly one timestamp value.
If duplicate reads are significantly more expensive than deduplication, you can switch to gt. The tradeoff is that your checkpoint handling must be tighter because you are no longer replaying the boundary.
Recommended checkpoint shape
Persist checkpoint state per resource type. A minimal shape is:
If your destination supports idempotent upserts, use that. It makes retries and boundary replays much simpler.
Practical notes
- Keep separate checkpoints for each resource type because each collection paginates independently.
- Do not mix
cursorvalues between different endpoints or different query shapes. - Keep the same
order_byand filter values for every page in a single traversal. - Save checkpoints only after a successful run, not after each page.
- Use
updated_at:descfor operational views and debugging, not for forward-moving export jobs.
See also: Pagination and filtering