How can we process more than 500 records in this workflow?
Status: Open · Asked by WellnessVibe LLP on · 0 views
Hi Pabbly Team,
We have a Pabbly Connect workflow that processes workshop/customer records from Google Sheets and sends WhatsApp and Gmail notifications for each applicable record.
Our workflow is:
Schedule (Pabbly) – Schedule Workflow
DateTime Formatter (Pabbly) – Current Date
DateTime Formatter – Workshop Date
Google Sheets – Get Row(s)
Code (Pabbly) – Run JavaScript
Iterator (Pabbly) – Process Arrays
JSON Extractor (Pabbly) – Extract JSON Data
Iterator (Pabbly) – Process Arrays
WhatsApp API – Send Notification
Gmail – Send Email
Code (Pabbly) – Run JavaScript
Google Sheets – Add New Row
We are able to process around 500 records, but we need to process more than 500 records automatically.
For example, if our Google Sheet contains 1,000, 2,000, or more applicable records, we want the workflow to process all of them without manually splitting the data.
Could you please clarify:
Is there a 500-record or iteration limit in Pabbly Connect?
Which step in our workflow is responsible for this limitation?
Is there a recommended way to use pagination or batch processing to process records in batches of 500 and then automatically continue with the next batch?
Can Google Sheets – Get Row(s) retrieve records using an offset, starting row, or pagination mechanism?
What is the recommended Pabbly architecture for processing 1,000–5,000+ Google Sheets records?
Could the two Iterator steps in our workflow be causing an additional limitation because of nested/multiple iterations?
Is there any Pabbly-native solution to process all records reliably without creating separate workflows for every 500 records?
We would appreciate your recommended approach for handling large-volume record processing in this workflow.
Thank you.