How can we process more than 500 records in a single Pabbly Connect workflow?
Status: Open · Asked by WellnessVibe LLP on · 0 views
Hi Pabbly Team,
We have a Pabbly Connect workflow that processes workshop/customer records from Google Sheets and sends WhatsApp and Gmail notifications for each applicable record.
Our workflow is structured as follows:
- 1.Schedule (Pabbly) – Schedule Workflow
- 2.DateTime Formatter (Pabbly) – Current Date
- 3.DateTime Formatter – Workshop Date
- 4.Google Sheets – Get Row(s)
- 5.Code (Pabbly) – Run JavaScript
- 6.Iterator (Pabbly) – Process Arrays
- 7.JSON Extractor (Pabbly) – Extract JSON Data
- 8.Iterator (Pabbly) – Process Arrays
- 9.WhatsApp API – Send Notification
- 10.Gmail – Send Email
- 11.Code (Pabbly) – Run JavaScript
- 12.Google Sheets – Add New Row(logs)
Our requirement
The Google Sheets step can return a large number of records, and we need to process more than 500 records automatically.
For example, if there are:
- -500 records → workflow processes them
- -1,000 records → we need all 1,000 processed
- -2,000+ records → we need all applicable records processed
At the moment, we are facing a limitation when trying to process more than approximately 500 records.
What we want to know
Could you please clarify:
- 1.Is there a 500-record/iteration limit in Pabbly Connect for Google Sheets → Get Row(s), Iterator, or workflow execution?
- 2.If there is a limit, which specific step is responsible for it?
3.Is there a recommended way to implement pagination or batch processing so that the workflow can process records in batches of 500 and then continue with the next batch automatically?
4.Can the Google Sheets Get Row(s) action retrieve records using a starting row/offset so that we can process:- -Rows 1–500
- -Rows 501–1,000
- -Rows 1,001–1,500
- etc.
5.Is there a recommended Pabbly-native approach for processing 1,000–5,000+ records in this type of workflow?
6.Since we have two Iterator steps, is there any additional limitation caused by nested/multiple iterations?
7.If this is a task-limit issue rather than a record-limit issue, what would be the recommended architecture to handle this efficiently?
We are looking for a reliable way to process the complete dataset automatically without manually creating separate workflows for every 500 records.
Please let us know the recommended Pabbly Connect architecture for this use case.
Thank you.
