Program of Thoughts Prompting

November 27, 2025 · View on GitHub

Authored by Kalyan KS. You can follow me on Twitter and LinkedIn for latest LLM, RAG and Agent updates.

Overview

Program of Thoughts (PoT) Prompting is a beak down prompting technique in which a Large Language Model (LLM) solves mathematical or symbolic problems by generating executable Python code rather than explaining its reasoning in natural language.

This method separates thinking from calculating:

The LLM performs the reasoning by writing a clear, semantically meaningful program.
The interpreter performs the computation, ensuring perfect numerical accuracy.

PoT is especially effective for tasks involving arithmetic, geometry, algebra, symbolic manipulation, or multi-step calculations because code execution is reliable, deterministic, and precise.

Program of Thoughts prompting

Figure from Program of Thoughts prompting paper.

Prompt Template

Here is the prompt template for program of thoughts promtping

You are an expert numerical reasoning assistant.

You must solve the problem using **Program-of-Thoughts (PoT)** prompting.

Your output MUST be ONLY Python code:

- Use step-by-step reasoning expressed as variable assignments.
- Do NOT include comments.
- Do NOT include print statements.
- Use clear variable names.
- The last line MUST be: ans = <final value>
- The code MUST run in a Python interpreter.

Do NOT output natural language.  
Do NOT add explanations.  
ONLY return Python code.

Problem:
{question}

Stay Updated with Generative AI, LLMs, Agents and RAG

Join 🚀 AIxFunda free newsletter to get latest updates and interesting tutorials related to Generative AI, LLMs, Agents and RAG.

✨ Weekly GenAI updates
📄 Weekly LLM, Agents and RAG research paper updates
📝 1 fresh blog post on an interesting topic every week

Zero-Shot Implementation

Now let's see the implementation of zero-shot program of thoughts promtping technique using LangChain v1.0

# !pip install langchain langchain-google-genai pydantic

import os
from langchain.chat_models import init_chat_model
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import PydanticOutputParser
from pydantic import BaseModel, Field
from langchain_experimental.utilities import PythonREPL

# 1. Set your Gemini API key
os.environ["GOOGLE_API_KEY"] = userdata.get('GOOGLE_API_KEY')

# 2. Define PoT structured output
class PoTResponse(BaseModel):
    program: str = Field(..., description="Python code that computes the answer. Must assign final result to 'ans'.")

# 3. Parser
parser = PydanticOutputParser(pydantic_object=PoTResponse)

# 4. Initialize Gemini model
model = init_chat_model(
    "gemini-2.5-flash",
    model_provider="google_genai",
    temperature=0
)

# 5. Python Interpreter Tool (LangChain)
python_repl = PythonREPL()

# 6. Zero-Shot PoT Prompt Template
prompt_template = ChatPromptTemplate.from_template(
    """
You are an expert numerical reasoning assistant.

You must solve the problem using **Program-of-Thoughts (PoT)** prompting.

Your output MUST be ONLY Python code:

- Use step-by-step reasoning expressed as variable assignments.
- Do NOT include comments.
- Do NOT include print statements.
- Use clear variable names.
- The last line MUST be: ans = <final value>
- The code MUST run in a Python interpreter.

Do NOT output natural language.  
Do NOT add explanations.  
ONLY return Python code.

Problem:
{question}

Provide the solution in this JSON format:
{format_instructions}
"""
)

# 7. Insert parser instructions
prompt = prompt_template.partial(format_instructions=parser.get_format_instructions())

# 8. Build chain
chain = prompt | model | parser

# 9. Problem
question = """
Janet's ducks lay 16 eggs per day. She eats three for breakfast every morning and
bakes muffins for her friends every day with four. She sells the remainder at the 
farmers' market daily for \$2 per fresh duck egg. How much in dollars does she make 
every day at the farmers' market?
"""

# 10. Invoke LLM → get Python program
result = chain.invoke({"question": question})

print("\n--- Program Generated by LLM ---\n")
print(result.program)

# 11. Execute using LangChain Python Interpreter Tool
execution_output = python_repl.run(result.program)

# 12. Retrieve 'ans' from REPL environment
final_answer = python_repl.locals.get("ans", None)

print("\n--- Final Answer (from Python interpreter) ---\n")
print(final_answer)

Here the output is

--- Program Generated by LLM ---

eggs_laid_per_day = 16
eggs_eaten_for_breakfast = 3
eggs_used_for_muffins = 4
price_per_egg = 2

eggs_remaining_for_sale = eggs_laid_per_day - eggs_eaten_for_breakfast - eggs_used_for_muffins
daily_earnings = eggs_remaining_for_sale * price_per_egg

ans = daily_earnings

--- Final Answer (from Python interpreter) ---

18

Few-Shot Implementation

Now let's see the implementation of few-shot program of thoughts promtping technique using LangChain v1.0

!pip install langchain langchain-google-genai pydantic langchain-experimental

import os
from google.colab import userdata
from langchain.chat_models import init_chat_model
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import PydanticOutputParser
from pydantic import BaseModel, Field
from langchain_experimental.utilities import PythonREPL   # UPDATED IMPORT

# 1. Set API key
os.environ["GOOGLE_API_KEY"] = userdata.get('GOOGLE_API_KEY')

# 2. Structured PoT Response
class PoTResponse(BaseModel):
    program: str = Field(..., description="Python code that computes the answer, must assign to 'ans'.")

# 3. Parser
parser = PydanticOutputParser(pydantic_object=PoTResponse)

# 4. Initialize Gemini model
model = init_chat_model(
    "gemini-2.5-flash",
    model_provider="google_genai",
    temperature=0
)

# 5. Python Interpreter Tool (Experimental REPL)
python_repl = PythonREPL()

# 6. FEW-SHOT EXAMPLE (Only the first one kept)
few_shot_examples = """
Question: Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins with four. She sells the remainder at \$2 per egg. How much does she make daily?
# Python code, return ans
total_eggs = 16
eaten = 3
baked = 4
sold = total_eggs - eaten - baked
price = 2
ans = sold * price
"""

# 7. Few-Shot Prompt Template
prompt_template = ChatPromptTemplate.from_template(
    """
You are an expert numerical reasoning assistant.

Below is an example demonstrating **Program of Thoughts (PoT) prompting**.
The solution is expressed entirely as Python code with the final value stored in `ans`.

{few_shot_examples}

Now solve the following problem using the SAME format:

Problem:
{question}

Output Instructions:
- Output ONLY executable Python code.
- Use clear variable names.
- No comments.
- No print statements.
- Last line MUST be: ans = <final value>

Return your output in this JSON format:
{format_instructions}
"""
)

# 8. Insert few-shot example + parser instructions
prompt = prompt_template.partial(
    few_shot_examples=few_shot_examples,
    format_instructions=parser.get_format_instructions()
)

# 9. Build chain
chain = prompt | model | parser

# 10. Current Problem
question = """
A cylindrical water storage tank has a height of 5 meters and a radius of 2 meters.
The cost of water is \$0.50 per cubic meter.. Calculate the total cost to fill the tank
completely. Use 3.14159 for π.
"""

# 11. Generate PoT Program
result = chain.invoke({"question": question})

print("\n--- Program Generated by LLM ---\n")
print(result.program)

# 12. Execute the generated Python program using REPL
execution_output = python_repl.run(result.program)

# 13. Retrieve answer
final_answer = python_repl.locals.get("ans", None)

print("\n--- Final Answer (from Python interpreter) ---\n")
print(final_answer)

Here the output is

--- Program Generated by LLM ---

height = 5
radius = 2
pi = 3.14159
cost_per_cubic_meter = 0.50
volume = pi * (radius ** 2) * height
total_cost = volume * cost_per_cubic_meter
ans = total_cost

--- Final Answer (from Python interpreter) ---

31.4159