1
General Board / Entities skipped during sync due to max updated time filter – possible delay or
« on: January 14, 2026, 06:20:16 pm »
We are facing an issue where one entity is being skipped during synchronization when using timestamp-based filtering. Below are the details and our questions regarding the observed behavior.
Scenario Overview
Our automation works in the following manner:
Fetch the maximum updated time from previously synced entities.
Fetch all entities created or updated after that time up to “now”.
Sync those entities to the target project.
Timeline of Operations (Source Project)
Log Time (IST) Entity ID Operation
12:38:43 C6259946-7847-4488-B583-3C4BFD264388 Create
12:38:50 2BC19F08-DFC9-4b17-8F33-C4E12A49E216 Create
12:40:01 887C860A-DE37-460e-86D7-CB15CE044392 Create
12:40:04 9ED1417A-D878-4dc5-ACA8-E8DB9B79AA82 Create
12:40:09 C6259946-7847-4488-B583-3C4BFD264388 Update
12:40:14 2BC19F08-DFC9-4b17-8F33-C4E12A49E216 Update
After these operations, the system calculated:
Max Update Time (per project): Tue Jan 13 12:40:08 IST 2026
Global Max Update Time: Tue Jan 13 12:40:08 IST 2026
Based on this, the next fetch was executed with the following filter:
Fetching entities between:
Tue Jan 06 12:40:27 IST 2026 and Tue Jan 13 12:40:08 IST 2026
Result of Fetch
The following entities were picked:
9ED1417A-D878-4dc5-ACA8-E8DB9B79AA82 (Create – 12:40:04)
C6259946-7847-4488-B583-3C4BFD264388 (Update – 12:40:09)
2BC19F08-DFC9-4b17-8F33-C4E12A49E216 (Update – 12:40:14)
However, the entity below was not picked:
887C860A-DE37-460e-86D7-CB15CE044392 (Create – 12:40:01)
Questions / Clarifications Required
Created vs Updated Time
When an entity is created and no updates occur afterward, is the updatedTime always equal to the createdTime?
If yes, should such entities be reliably returned when filtering using updatedTime-based queries?
Search API Data Source
Do the search APIs (fetch entities created/updated before or after a given time) rely on a separate indexed table rather than the primary entity table?
If so, is there a known eventual consistency or indexing delay between entity creation/update and its availability in the search table?
Observed Behavior
We are observing a delay where a newly created entity appears in logs but is not immediately returned by the search API using time filters.
This resulted in entity 887C860A-DE37-460e-86D7-CB15CE044392 being skipped during sync.
Recommended Fix / Best Practice
Is this behavior expected due to indexing or reconciliation lag?
What is the recommended buffer/delay (if any) before querying for newly created or updated entities to avoid missing data?
Alternatively, is there a more reliable watermark strategy we should use instead of strict timestamp boundaries?
This issue can lead to permanent data gaps in the target project, so guidance on the correct and safe approach would be very helpful.
Looking forward to your clarification and recommendations.
Scenario Overview
Our automation works in the following manner:
Fetch the maximum updated time from previously synced entities.
Fetch all entities created or updated after that time up to “now”.
Sync those entities to the target project.
Timeline of Operations (Source Project)
Log Time (IST) Entity ID Operation
12:38:43 C6259946-7847-4488-B583-3C4BFD264388 Create
12:38:50 2BC19F08-DFC9-4b17-8F33-C4E12A49E216 Create
12:40:01 887C860A-DE37-460e-86D7-CB15CE044392 Create
12:40:04 9ED1417A-D878-4dc5-ACA8-E8DB9B79AA82 Create
12:40:09 C6259946-7847-4488-B583-3C4BFD264388 Update
12:40:14 2BC19F08-DFC9-4b17-8F33-C4E12A49E216 Update
After these operations, the system calculated:
Max Update Time (per project): Tue Jan 13 12:40:08 IST 2026
Global Max Update Time: Tue Jan 13 12:40:08 IST 2026
Based on this, the next fetch was executed with the following filter:
Fetching entities between:
Tue Jan 06 12:40:27 IST 2026 and Tue Jan 13 12:40:08 IST 2026
Result of Fetch
The following entities were picked:
9ED1417A-D878-4dc5-ACA8-E8DB9B79AA82 (Create – 12:40:04)
C6259946-7847-4488-B583-3C4BFD264388 (Update – 12:40:09)
2BC19F08-DFC9-4b17-8F33-C4E12A49E216 (Update – 12:40:14)
However, the entity below was not picked:
887C860A-DE37-460e-86D7-CB15CE044392 (Create – 12:40:01)
Questions / Clarifications Required
Created vs Updated Time
When an entity is created and no updates occur afterward, is the updatedTime always equal to the createdTime?
If yes, should such entities be reliably returned when filtering using updatedTime-based queries?
Search API Data Source
Do the search APIs (fetch entities created/updated before or after a given time) rely on a separate indexed table rather than the primary entity table?
If so, is there a known eventual consistency or indexing delay between entity creation/update and its availability in the search table?
Observed Behavior
We are observing a delay where a newly created entity appears in logs but is not immediately returned by the search API using time filters.
This resulted in entity 887C860A-DE37-460e-86D7-CB15CE044392 being skipped during sync.
Recommended Fix / Best Practice
Is this behavior expected due to indexing or reconciliation lag?
What is the recommended buffer/delay (if any) before querying for newly created or updated entities to avoid missing data?
Alternatively, is there a more reliable watermark strategy we should use instead of strict timestamp boundaries?
This issue can lead to permanent data gaps in the target project, so guidance on the correct and safe approach would be very helpful.
Looking forward to your clarification and recommendations.