bypassing the 5000 records read limit
We have different users doing different read process on objects that should result in more then 5000 records being read. It is impossible to start creating and extra column for every businessprocess on every object to filter out the records already processed. Is there a way get the next 5000 records, without having to update a dedicated flag for that business proces? Some of those business processes should run simultaneously, so in that case a dedicated flag to trace the already processed records doesn't work. Is there another way to bypass that?
Like
Hello.
In Creatio, there is no built-in way to keep track of already processed records across multiple business processes without using an additional attribute, such as a flag, status, or timestamp. The platform’s business process engine does not store pagination state between runs, so without a dedicated field to mark processed records, each process run will read the same data repeatedly.
A dedicated column or flag ensures that processes can consistently filter out records that have already been processed, especially when multiple processes run in parallel.
We can recommend the following solutions:
- Use pagination logic (LIMIT + OFFSET) and store the last offset value in a system setting or a separate table.
- Implement a queue mechanism outside the standard process engine, where each process pulls its portion of data from a central queue or via a custom API.
- Use temporary tables or a staging area to store the IDs of records being processed, but this also requires custom logic.
In summary, Creatio does not currently provide an out-of-the-box feature for this scenario.
We recommend implementing a simple status or flag to reliably trace which records have been processed.