ShanNote: This is a documented version of ChatGPT Prompt Engineering for Developers course. You can find the course here.
Note: In this Article, you'll practice one prompting principle and it's related tactics in order to write effective prompts for large language models. You can view my previous article here.
Principle 2: Give the model time to “think”.
Large language models can make reasoning errors by rushing to an incorrect conclusion. This can happen if the model is given a task that is too complex or if it is not given enough time to consider all of the evidence. To prevent this, you can reframe the query to request a chain or series of relevant reasoning. This will help the model to slow down and consider all of the evidence before drawing a conclusion. Just like humans, large language models can make mistakes if they are rushed or given too much to do. Instructing the model to think longer about a problem can help to improve its accuracy.
Here are some tips for reframing a query to request a chain or series of relevant reasoning:
- Use the word "explain" or "justify" in your query.
- Ask the model to provide evidence to support its conclusion.
- Ask the model to consider all of the relevant information.
These are the outputs that were generated for the same question asked in different ways.
Tactic 1: Specify the steps required to complete a task
text = """
In a charming village, siblings Jack and Jill set out on \
a quest to fetch water from a hilltop \
well. As they climbed, singing joyfully, misfortune \
struck—Jack tripped on a stone and tumbled \
down the hill, with Jill following suit. \
Though slightly battered, the pair returned home to \
comforting embraces. Despite the mishap, \
their adventurous spirits remained undimmed, and they \
continued exploring with delight.
"""
prompt_1 = """
Perform the following actions:
1 - Summarise the following text delimited by triple \
hyphen with 1 sentence.
2 - Translate the summary into French.
3 - List each name in the French summary.
4 - Output a json object that contains the following \
keys: french_summary, num_names.
Separate your answers with line breaks.
Text:
---{text}---
"""
In this prompt we are specifying all the steps that should be performed by the model before arriving at a conclusion.
As seen from the screenshot, the model provided a clear and concise answer for our query.
Let us try to give another prompt that will give us a more structured output.
text = """
In a charming village, siblings Jack and Jill set out on \
a quest to fetch water from a hilltop \
well. As they climbed, singing joyfully, misfortune \
struck—Jack tripped on a stone and tumbled \
down the hill, with Jill following suit. \
Though slightly battered, the pair returned home to \
comforting embraces. Despite the mishap, \
their adventurous spirits remained undimmed, and they \
continued exploring with delight.
"""
Your task is to perform the following actions:
1 - Summarise the following text delimited by
<> with 1 sentence.
2 - Translate the summary into French.
3 - List each name in the French summary.
4 - Output a json object that contains the
following keys: french_summary, num_names.
Use the following format:
Summary: summary
Translation: summary translation
Names: list of names in Italian summary
Output JSON: json with summary and num_names
Text: <{text}>
As seen from the screenshot the output is now more structured and follows a format.
Output:
Tactic 2: Instruct the model to work out its own solution before rushing to a conclusion
Determine if the student's solution is correct or not.
Question:
I'm building a solar power installation and I need \
help working out the financials.
- Land costs $100 / square foot
- I can buy solar panels for $250 / square foot
- I negotiated a contract for maintenance that will cost \
me a flat $100k per year, and an additional $10 / square \
foot
What is the total cost for the first year of operations
as a function of the number of square feet.
Student's Solution:
Let x be the size of the installation in square feet.
Costs:
1. Land cost: 100x
2. Solar panel cost: 250x
3. Maintenance cost: 100,000 + 100x
Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000
This prompt will give the output as the student's answer is correct but the answer is wrong because the calculation "100,000 + 100x" is wrong it should be "100,000 + 10x". So the answer should be "360x + 100,000".
Output:
To get the correct output we need to ask the model to first generate it's own answer and then compare the answer with the student's answer. Here we asking the model to do it's own analysis on the question and generate an answer for us.
To solve the problem do the following:
- First, work out your own solution to the problem.
- Then compare your solution to the student's solution
and evaluate if the student's solution is correct or not.
Don't decide if the student's solution is correct until
you have done the problem yourself.
Use the following format:
Question:
----
question here
----
Student's solution:
----
student's solution here
----
Actual solution:
----
steps to work out the solution and your solution here
----
Is the student's solution the same as actual solution \
just calculated:
---
yes or no
---
Student grade:
---
correct or incorrect
----
Question:
""""
I'm building a solar power installation and I need help \
working out the financials.
- Land costs $100 / square foot
- I can buy solar panels for $250 / square foot
- I negotiated a contract for maintenance that will cost \
me a flat $100k per year, and an additional $10 / square \
foot
What is the total cost for the first year of operations \
as a function of the number of square feet.
""""
Student's solution:
""""
Let x be the size of the installation in square feet.
Costs:
1. Land cost: 100x
2. Solar panel cost: 250x
3. Maintenance cost: 100,000 + 100x
Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000
""""
Actual solution:
Output:
This is an example of how asking the model to do a calculation itself and breaking down the task into steps to give the model more time to think can help you
get more accurate responses. Here i am using hyphen and triple double quotes as delimiters. You can use any delimiters as you like.
To improve the accuracy and quality of the responses from large language models, it is important to give them time to "think". This can be done by reframing your query to request a chain or series of relevant reasoning, or by instructing the model to work out its own solution before rushing to a conclusion. Additionally, it is important to be as specific as possible in your queries, use natural language, use keywords, and be patient when waiting for a response.
