How to download images with Pabbly Connect?

NextLevelBros · May 27, 2023

Hi,

I would like to download images from a URL that is sent thorugh a webhook and cannot figure out how to do this. Let me explain the setup and use case:

I have a webhook integration setup that sends data from a form to Pabbly Connect. Pabbly connect recieves the data, does some basic formating and then sends it on to a Google Sheet to add as a new row. Part of the data being sent to the Google sheets are some URL's that link to images. The images are freely downloadable and accessible, you do not need to be logged in anywhere.

What I want to do is to download those images and either embed them to the Google Sheet in some cells or send them somewhere else (for example to an email account, or to a google Drive Folder, or what other options do I have?).

The only caveat here is that the images need to be saved or sent on to the final recipient/storage location in a way that can be corresponded to the form that was submitted and thus to the rest of the data from that form. For example if the images are sent to a Google Drive folder where they are mixed with other images from other forms, that will not be convient as it will be confusing which images correspond to which form. If however the images are sent to a subfolder that can be named according to a field from the submitted form data (for example the name, which we also ask for in the form), then that is good for our use case. Of if the images can be sent as an email with the subject being an identifyable field from the submitted form fields, that is also a good functionality for us.

How can this be achieved with Pabbly connect?

Thanks!

ArshilAhmad · May 27, 2023

Hi @NextLevelBros,

The use case you are describing can be easily achieved by simply adding two steps to your workflow:

First, use the 'Google Drive: Create a File in a Specific Folder' action event to generate a new subfolder in your Google Drive. The subfolder's name will correspond to the name you received from your form responses.
Next, proceed to upload the file to the previously created subfolder using the 'Google Drive: Upload a File' action event.

NextLevelBros said:
Of if the images can be sent as an email with the subject being an identifyable field from the submitted form fields, that is also a good functionality for us.

This use case is even simpler. You just need to map the Identifiable Field in the subject line of the email you intend to send.

NextLevelBros · May 31, 2023

Hi @ArshilAhmad,

Thanks a lot for your prompt response, I appreciate that a lot. I am close to getting this to work, but have not managed to get this to work for me 100% yet.

I have setup a workflow to test what you have described. For this test I am just sending an email that only contains a single URL in the body. The URL links to an image. That link is placed in the body of the email (with no other text, just that link), and the folder name will come from the subject line of the email. As I said this is just to test to see that I am correctly understanding and setting up the workflow you have desribed. If it works, I will use it the context I described in my original question with the form responses. So for now I am trying to get my image from the link in the email uploaded to the folder on Google Drive.

I am using this following link for now, this is simply a link to a jpg file on my website, it is publicly available with no restrictions:
https://nextlevelbros.com/wp-content/uploads/09-Tattoo-Marketing-Reporting-Thumbnail-694x379-1.jpg

The first step to create an appropriately named folder in Google Drive has worked, no questions there.

The second part to "Upload a file" to Google drive is not working for me because I am getting an error. This is the exact error:

Code:

Exception: Request failed for https://nextlevelbros.com returned code 404. Truncated server response: <!doctype html>
<html lang="en-US" class="no-js" itemtype="https://schema.org/WebPage" itemscope>
<head>
<title>Page not found - NextLevelBros</tit... (use muteHttpExceptions option to examine full response)

Not sure what is going wrong here, can you advise how to resolve this error? It says something about a 404 error, but my file and site are 100% online and available.

Here is a screenshot of my action step and error:

screenshot-connect.pabbly.com-2023.05.30.jpg

I do also have a follow up question that I would like to ask you, and it has to do with the file name of the file I wish to upload.

It says in the "File Name" field to include the file extension. I will be recieving and want to upload image files, that could be jpg, jpeg, png, gif, psd, pr even a pdf with an image embedded in it. There is no single file type, there are multiple ones that may need to be uploaded in this step. In addition, I do not really wish to rename the files I am uploading. The image files I want to upload already have a filename and a file extension, I jsut want to upload those to google drive and that's all. I must upload the files that I recieve with no further transformations, simple as that. If I need to rename the file for this to work, so be it, I will do that, but the issue with file type exensions remains and there is nothing I can do to change the fact that I will be recieving image files in whatever format is convenient and available for the people who are sending the images to me.

One more question about handling multiple images. Right now I am testing one single url and image file, but in my final use-case I will be processing forms that contain a varying amount of images. It could be zero images, it could be one image, it could be 8 images. It is a form and people will upload how ever many images they want. All images from a single form should go to the same folder though. How should I proceed to set that up?

Look forward to hear from you with advice how to proceed to get this to work, thanks.

ArshilAhmad · May 31, 2023

NextLevelBros said:
The second part to "Upload a file" to Google drive is not working for me because I am getting an error. This is the exact error:

Code:

Exception: Request failed for https://nextlevelbros.com returned code 404. Truncated server response: <!doctype html> <html lang="en-US" class="no-js" itemtype="https://schema.org/WebPage" itemscope> <head> <title>Page not found - NextLevelBros</tit... (use muteHttpExceptions option to examine full response)

Not sure what is going wrong here, can you advise how to resolve this error? It says something about a 404 error, but my file and site are 100% online and available.

It seems that the presence of a "Space" in your URL is causing this error, as I was able to upload the image using the link you provided.

NextLevelBros said:
I do also have a follow up question that I would like to ask you, and it has to do with the file name of the file I wish to upload.

It says in the "File Name" field to include the file extension. I will be recieving and want to upload image files, that could be jpg, jpeg, png, gif, psd, pr even a pdf with an image embedded in it. There is no single file type, there are multiple ones that may need to be uploaded in this step. In addition, I do not really wish to rename the files I am uploading. The image files I want to upload already have a filename and a file extension, I jsut want to upload those to google drive and that's all. I must upload the files that I recieve with no further transformations, simple as that. If I need to rename the file for this to work, so be it, I will do that, but the issue with file type exensions remains and there is nothing I can do to change the fact that I will be recieving image files in whatever format is convenient and available for the people who are sending the images to me.

File Name is not a mandatory field. You do have the option to skip that field, and your file will be saved as it is.

NextLevelBros said:
One more question about handling multiple images. Right now I am testing one single url and image file, but in my final use-case I will be processing forms that contain a varying amount of images. It could be zero images, it could be one image, it could be 8 images. It is a form and people will upload how ever many images they want. All images from a single form should go to the same folder though. How should I proceed to set that up?

Look forward to hear from you with advice how to proceed to get this to work, thanks.

This can work if the URL from your form is captured in an Array format, which would allow us to upload the images individually using the Iterator.

NextLevelBros · May 31, 2023

Hi @ArshilAhmad,

Thanks for your response.

About the space in the URL, that space is not present in the URL I am sending via the email:

That gives rise to two questions:

1: Is the email parser / Pabbly Connect somehow adding a space there? I see the space in the email parser output, see screenshot below.
2. If so, how to prevent that from happening as my intention is to capture the url data as it arrives and not have filepath changes inserted into my url.

I also would like to confirm that the problem is that the space in the URL is causing the URL to simply no longer link to the desired file, and not because Pabbly is unable to process URL's that link to files that may have spaces in the filename, am I understanding corectly? In other words if a user sends me a file with spaces in its name, this will still work, correct?

About the name field, all clear.

About using the iterator, it seems this is a bit more complex as I am recieving a dozen text fields alongside a variable amount of image urls, and I will have to learn how to re-setup all the mappings once I turn off the simple response since Ihave never used that before, and it is not clear to me how this will all work with the iterator. For now I would like to get my test-scenario working with a single image, and then I will move on to seeing how it will work with the iterator in a live environment and get back to you if I have any questions on that.

ArshilAhmad · Jun 1, 2023

Could you please grant me access to the workflow by adding me as a team member?

NextLevelBros said:
That gives rise to two questions:

1: Is the email parser / Pabbly Connect somehow adding a space there? I see the space in the email parser output, see screenshot below.
2. If so, how to prevent that from happening as my intention is to capture the url data as it arrives and not have filepath changes inserted into my url.

We do not make any changes to the data received through Email Parser; your URL will remain intact. I will have to discuss this particular instance with my team.

NextLevelBros said:
I also would like to confirm that the problem is that the space in the URL is causing the URL to simply no longer link to the desired file, and not because Pabbly is unable to process URL's that link to files that may have spaces in the filename, am I understanding corectly? In other words if a user sends me a file with spaces in its name, this will still work, correct?

Indeed, the problem lies with the specific URL where the presence of a space is causing the issue. This issue is unrelated to the File Name.

NextLevelBros · Jun 2, 2023

@ArshilAhmad what is your email for me to add you as a teammember?

ArshilAhmad · Jun 2, 2023

Email: [email protected]

NextLevelBros · Jun 2, 2023

ArshilAhmad said:
Email: [email protected]

Done.

ArshilAhmad · Jun 2, 2023

I have taken note of your concern and will discuss the issue of added space in the URL with my team. In the meantime, I would suggest that you add a Text Formatter to your workflow to remove any blank spaces in your URL.

NextLevelBros · Jun 2, 2023

Thanks for your help so far and for pro-actively thinking of solutions for the problems I am encountering.

The suggestion to use the text formatter has solved the issue with the file upload error, so the test scenario has completed succesfully, that is great. I would be interested to see a resolution to the issue of the extra space being added to the url when it goes through the email parser for the long term, to avoid needless complexity and setps when using this type of setup.

Now that the test scenario has been proven to work, I have moved on to trying to implement this with my booking form in a live environment with actual user data and variable amount of incoming images.

I have made two screen recording videos to explain issues, questions and suggestions that I have aroud this topic. Please view these here:

Video 1 - Workflow with Iterator
https://videoshare.nextlevelbros.com/conversations/c62818de-efca-5177-bb7f-a1d6f62539a1
My main issue here is that it is not so clear how to set this up so that I can get my form fields sent to Google sheets as well as my images uploaded to Google Drive.

Video 2 - Workflow using multiple Google Drive File Upload Tasks
https://videoshare.nextlevelbros.com/conversations/0b02efff-00f2-5667-83be-bd84c7cfae5c
I have considered using multiple Google Drive tasks to upload my images, instead of the iterator, but I don't know it that will throw up errors if there are no images uploaded or less images than the amount of Google Drive tasks, and also think it would not really be efficient in terms of task usage from my plan. although that is not a deal breaker. I show and explain everything better in the video so please check that out.

If the videos dont open when you click the link, please copy paste the URL into the browser address bar.

ArshilAhmad · Jun 3, 2023

I appreciate you sharing those videos; they helped me understand your use case much more comprehensively.

NextLevelBros said:
Video 1 - Workflow with Iterator
https://videoshare.nextlevelbros.com/conversations/c62818de-efca-5177-bb7f-a1d6f62539a1
My main issue here is that it is not so clear how to set this up so that I can get my form fields sent to Google sheets as well as my images uploaded to Google Drive.

The Iterator, as its name suggests, performs multiple iterations of the same task on similar types of data. In your case, the data consists of different file URLs that you wish to upload to Google Drive.

Regarding task consumption, it depends on the number of files you receive from your form. For example, if you receive 4 photos, the Iterator will transfer each file to Google Drive individually, consuming 4 tasks in total (one for each upload).

I observed that you are attempting to add data to your Google Sheets within the same workflow. Please note that each time the Iterator runs, it will add the data to a new row in your Google Sheets. As a result, if you have 4 URLs, the Iterator will add all 4 URLs to separate rows, along with any other details.

To avoid this, I recommend creating a distinct workflow dedicated to adding data to your Google Sheets.
=============================================================================================

NextLevelBros said:
Video 2 - Workflow using multiple Google Drive File Upload Tasks
https://videoshare.nextlevelbros.com/conversations/0b02efff-00f2-5667-83be-bd84c7cfae5c
I have considered using multiple Google Drive tasks to upload my images, instead of the iterator, but I don't know it that will throw up errors if there are no images uploaded or less images than the amount of Google Drive tasks, and also think it would not really be efficient in terms of task usage from my plan. although that is not a deal breaker. I show and explain everything better in the video so please check that out.

I was not aware that you would only receive a maximum of 5 images; otherwise, I would have suggested you take this exact route.

Regarding task consumption, if we solely consider the 'Google Drive: Upload File' action event and disregard other steps in your workflow, using this approach will consistently consume 5 tasks, regardless of whether you receive less than 5 images from your form data.

Although errors may arise, they will not impact your workflow.
=============================================================================================

Uploading multiple files at the same time is not a feature that the Google Drive API currently has. You can check the attached API documentation.

Upload file data | Google Drive | Google for Developers

Learn how to run simple, multipart, or resumable file uploads.

developers.google.com

NextLevelBros · Jun 5, 2023

Thanks a lot for your response and the information you have provided, I have now managed to succesfully complete setting up this workflow that this post was about. You have been an amazing and I am very happy to have spoken with you about this. I wish every support experience I had was as good as this

The explanations about the iterator will help me figure out when and how to use the iterator for future workflows.

For this workflow I decided to use the setup with multiple Google Drive file upload tasks since the total task consumption is ultimately not really more than if I setup two workflows, one for the data and another for the images with the iterator.

Understood about the Google Drive API, thanks for explaining that too.

=============================================================================================

Now that this is all setup and ready, I am thinking it would be usefull to just place the information from the form (aside from the images, I mean the info I am sending to google sheets) also in the same folder as the images on Google Drive, in a text file (or word file or pdf) so that all the information is in one place and easily accessible for the end user, as opposed to having the images in one place and the text info in another place.

I do have some questions about how I might achieve that, should I open a new ticket/post about that or can I ask you here? Specifically I would like to know how to take the data I have recieved in the form and create a single text file that contains all of that. Then I will upload that to Google Drive in the way that you have shown me.

ArshilAhmad · Jun 5, 2023

I am glad I was able to help you with your workflow

=========================================================================================

Based on my limited understanding, I assume you want to store the collected data in a Google Doc or a PDF. I would suggest adding these action events to your workflow to achieve the desired result:

Set up Google Docs: Create Document from Template as the Action App to create a document from your template.
Set up Google Drive: Share a File With Anyone as the Action App to make the document publicly accessible.
Set up Google Drive: Upload a File as the Action App to upload your PDF File

NextLevelBros · Jun 7, 2023

Thansk a lot for that explanation Arshil. It is not exactly what I was looking for, but usefull nonethless to know how this is done. I will create a new post with more info when I need further assistance with my idea of creating text files.

How to download images with Pabbly Connect?

NextLevelBros

Member

ArshilAhmad

Moderator

NextLevelBros

Member

ArshilAhmad

Moderator

NextLevelBros

Member

ArshilAhmad

Moderator

NextLevelBros

Member

ArshilAhmad

Moderator

NextLevelBros

Member

ArshilAhmad

Moderator

NextLevelBros

Member

ArshilAhmad

Moderator

Upload file data | Google Drive | Google for Developers

NextLevelBros

Member

ArshilAhmad

Moderator

NextLevelBros

Member

Similar threads