Prompt Optimizer Made Easy with Vertex AI in GCP

In generative AI, crafting well-structured prompts is key to achieving accurate and meaningful responses. Google Cloud’s Vertex AI offers a powerful way to optimize prompts, enhancing their efficiency and improving model outputs.

This article walks you through a step-by-step guide on improving your AI prompts' performance within Google Cloud’s Vertex AI. It directs you through a process designed to make your prompts more accurate and efficient, leading to better results from generative AI models.

The purpose of the Prompt Optimizer is to enhance how we structure the primary instruction for the model. To illustrate this, consider the following example:

Prompt:
“Please show me the color of a strawberry.”
Expected result: “Red”

The Prompt Optimizer refines the prompt by breaking it down into three components: System Prompt, Question, and Target.

In this case:

· System Prompt: "Show me the color of …"
· Question: "Strawberry"
· Target: "Red"

After training, the machine learning model will generate a list of optimized system prompts along with their rankings.

For example, it might suggest an improved prompt such as “Show me the main color of …”.

1. Create New Project / Open Project

To set up your GCP project, log in to the Google Cloud Console and sign in with your Google account. Once inside, create a new project by clicking on the project selector dropdown, selecting New Project, entering a project name, choosing a billing account, and clicking Create.

2. Open Notebook in Vertex AI

Scan this QR code to access the provided notebook.

Launch the Notebook by clicking the link on the website when opening the GCP Console.

3. Fix the typo in the notebook.

There are two unclosed bracket issues in different sections of the notebook. Fix them by correctly placing the closing brackets in their respective positions.

Move the closing bracket to the correct position in the notebook to fix the unclosed bracket issue.

Move the closing bracket to the correct position in the notebook to fix the unclosed bracket issue.

4. Fill in the Project ID

You can find your active project ID in the Project List

5. Create the first Storage Bucket

Create the first bucket that meets this criteria.\

1. Provide any name (ex: inputstorage)
2. Location Type: Region (us-central1)
3. Default Storage Class: Standard
4. Uncheck the "Enforce public access prevention on this bucket"
5. Access Control: Fine-Grained
6. Create

6. Upload the JSONL file to the First Bucket

The notebook is set up to accept a JSONL file as the default input dataset. Download the dataset below

https://s.id/gen_ai_day_jsonl

Here are the examples of the dataset.

{"question": "What is the capital of France?", "target": "Paris"}
{"question": "Who wrote Hamlet?", "target": "William Shakespeare"}
{"question": "What is the square root of 144?", "target": "12"}
{"question": "What is the chemical symbol for gold?", "target": "Au"}
{"question": "How many continents are there?", "target": "7"}
{"question": "What year did World War II end?", "target": "1945"}

Based on the dataset mentioned above, the “question” serves as the prompt sent to the selected model, while the “target” represents the expected response from the model.

The target or selected model (gemini-2.0-flash-001) is configured in this section.

7. Grant Public Access to the First Bucket

Grant public access by clicking on the “Permissions” tab, then:

1. Click "Grant Access"
2. New principals: AllUsers
3. Role: Storage Object Viewer
4. Save

8. Grant Public Access to JSONL object

Grant public access by clicking on the JSONL object, then:

1. Click "Edit Access"
2. Add entry
3. Entity: Public
4. Name: allUsers
5. Access: reader
6. Save

9. Copy the GSUTIL URL of the JSONL object

10. Paste the GSUTIL URL of the JSONL object into `input_data_path`.

11. Create the second storage bucket.

Create the second bucket that meets this criteria:

1. Provide any name (ex: outputstorage)
2. Location Type: Region (us-central1)
3. Default Storage Class: Standard
4. Uncheck the "Enforce public access prevention on this bucket"
5. Access Control: Uniform
6. Create

12. Grant public access to the second bucket

Grant public access by clicking on the “Permissions” tab, then:

1. Click "Grant Access"
2. New principals: AllUsers
3. Role: Storage Object Viewer
4. Save

13. Create the `/output` folder in the second bucket

14. Copy the GSUTIL URL of the output bucket from the Configurations tab.

15. Paste the GSUTIL URL into `output_path`

Add “/output” at the end of the GSUTIL URL.

16. Run the notebook cells sequentially until step 5

17. Monitor the progress in the Vertex AI Custom Jobs menu.

18. View the results in Step 6

Fill the result_path the same as output_path :

Check the results

The initial prompt is: “Answer the following question. Let’s think step by step.”

One example output achieved 0.86 accuracy with the following prompt:

“What is the speed of light in kilometers per second? Calculate the answer and clearly state the final result. Show your work, explaining the steps involved in the calculation.”

Following this step-by-step guide, you can efficiently optimize AI prompts using Google Cloud’s Vertex AI, improving accuracy and performance. This structured approach ensures that your generative AI model produces high-quality responses tailored to your needs. Whether you’re refining prompts for general knowledge questions or complex calculations, the Prompt Optimizer streamlines the process, leading to more precise and reliable outputs. As AI evolves, leveraging tools like Vertex AI for prompt optimization will be crucial in maximizing model efficiency and effectiveness. With your optimized prompts in place, you’re ready to harness the full potential of generative AI within Google Cloud. Happy prompting!

Special thanks to Kevin Jonathan Halim for collaborating with me to write this article.