Build GPT with Custom Action using Pabbly Connect

The following post shows the method of building your own GPT with Custom Actions where the custom actions are run in the backgroud using Pabbly Connect.

Use-case: The following GPT uses Custom Action to work. It transcribes any audio file and returns the transcription within the ChatGPT User Interface.

1. You have to send the Audio File URL to the GPT and it will give you the transcript in the ChatGPT interface itself.
2. The GPT uses Pabbly Connect on the backend to transcribe a audio from the Audio File URL.

Step by Step process for building your own GPT with custom actions are available below:

---

Firstly, you will have to build the automation workflow inside Pabbly Connect. The workflow is 3 steps long.

Step 1: Get the Audio File URL through a webhook. This Audio File URL will sent from ChatGPT User Interface.
Step 2: Transcribe the audio using WhisperAI. You can also use any other third party API for transcribing audio.
Step 3: Return the response back to the caller of the webhook.

---

Step 1: Webhook is the Trigger application. We will be capturing the AudioFileURL in the trigger.

Step 2: Took this AudioFileURL to WhisperAI action to transcribe the audio and get the returned transcription back. This is a private action currently. I built it for my own use.

Step 3: Now we will send the text back to ChatGPT using the Data Forwarder action step which immediately sends the data back to the caller who called the webhook URL.

After this, I went on to create my own GPT using the GPTs Editor.

This was the basic configuration. Need to add the Name, Description, Instructions.

This is the action that I setup. I am adding the schema below.

I didn't know the schema creation myself. So I asked ChatGPT itself to write the schema for me.

I mentioned that it should ask for the Audio File URL from the User and the transcription will be done and sent back to the user in the response.

Prompt that I used to generate schema is mentioned at the bottom of this post.

This was the schema ChatGPT generated for me that I pasted directly in the Schema without any modification. It includes my webhook URL endpoint as well. I am sharing it here because I want others to follow the steps. But it is not a good practice to share your webhook URLs publicly.

{
  "openapi": "3.1.0",
  "info": {
    "title": "Transcribe Audio",
    "description": "API for transcribing audio from a file URL provided by the user.",
    "version": "v1.0.0"
  },
  "servers": [
    {
      "url": "https://connect.pabbly.com/workflow/sendwebhookdata"
    }
  ],
  "paths": {
    "/IjU3NjUwNTZlMDYzMzA0MzM1MjZjNTUzNTUxMzMi_pc": {
      "post": {
        "description": "Transcribes the audio file from a URL provided by the user.",
        "operationId": "TranscribeAudio",
        "requestBody": {
          "description": "Payload containing the URL of the audio file to be transcribed.",
          "required": true,
          "content": {
            "application/json": {
              "schema": {
                "type": "object",
                "properties": {
                  "AudioFileURL": {
                    "type": "string",
                    "format": "uri",
                    "description": "URL of the audio file to be transcribed, provided by the user."
                  }
                },
                "required": [
                  "AudioFileURL"
                ]
              }
            }
          }
        },
        "responses": {
          "200": {
            "description": "Transcription successful."
          },
          "400": {
            "description": "Invalid request or AudioFileURL."
          }
        },
        "deprecated": false
      }
    }
  },
  "components": {
    "schemas": {}
  }
}

After this I simply added the Privacy URL of Pabbly. This is a requirement to make the GPT public.

The GPT with my custom action is ready to rock and roll.

This is the final usage on the ChatGPT screen.

---

Lastly below is the prompt that I used within ChatGPT to generate the OpenAI GPT Schema for my action.

Write a OpenAI schema in JSON for creating my own GPT.

My API takes the Audio File URL as input through POST method from the user and outputs the transcribed text in response.

My API takes a parameter of AudioFileURL where the user has to submit the File URL of his audio file that is uploaded somewhere.

The request has to be made to the following URL.

https://connect.pabbly.com/.../IjU3NjUwNTZlMDYzMzA0MzM1BB...

In response to the request, it will output the Transcribed audio back.

The response returned will be JSON in the following format.

{ "AudioData" : "Transcribed Audio Data will come here"}

This is the sample schema.
{
"openapi": "3.1.0",
"info": {
"title": "Transcribe Audio",
"description": "API for transcribing audio from the file URL.",
"version": "v1.0.0"
},
"servers": [
{
"url": "https://connect.pabbly.com/workflow/sendwebhookdata"
}
],
"paths": {
"/IjU3NjUwNTZlMDYzMzA0MzM1BBBVjZjNTUzNTUxMzMi_pc": {
"post": {
"description": "Transcribes the Audio through a AudioFileURL",
"operationId": "AudioFileURL",
"requestBody": {
"description": "Payload containing the AudioFileURL for transcribing.",
"required": true,
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"email": {
"type": "string",
"format": "email",
"description": "User's email address to send the appointment confirmation."
}
},
"required": [
"email"
]
}
}
}
},
"responses": {
"200": {
"description": "Confirmation email sent successfully."
},
"400": {
"description": "Invalid request or email address."
}
},
"deprecated": false
}
}
},
"components": {
"schemas": {}
}
}

This is the response with the working schema that I used while building my GPT.