Prompt Engineering in LLM Models

Prompt Engineering in LLM Models

Guide with examples for Prompt Engineering

Prompt engineering is the process of designing and optimizing prompts to get the most out of language models like GPT-4. Good prompt engineering can vastly improve the performance and output of a language model, and is often an essential part of deploying such models in real-world scenarios.

Prompts are the input strings that we provide to a language model to instruct it on the kind of response we want. They're our primary means of directing the model's behavior.

Tips to write good prompts

To write better prompts, consider the following tips:

  1. Write clear and specific instructions: Be explicit about what you want the model to do. Clarity in instructions does not necessarily mean brevity; detailed instructions can help the model produce more accurate results.

  2. Give the model time to think: If a model is making reasoning errors by rushing to an incorrect conclusion, you should try reframing the query to request a chain or series of relevant reasoning before the model provides its final answer. Another way to think about this is that if you give a model a task that's too complex for it to do in a short amount of time or in a small number of words, it may make up a guess which is likely to be incorrect. And you know, this would happen for a person too.

  3. Use delimiters properly: Delimiters like "-" or ">" can help structure the prompt and guide the model's output.

var prompt = `Summarize the text delimited by triple backticks 
into a single sentence.
${text}`;
var prompt = `Summarize the text delimited by triple backticks 
into a single sentence.
${text}`;
  1. Ask for a structured output: Asking the model to return the answer in a specific format (for example, a bulleted list or a numbered sequence) can help you get the results you want.
var prompt = `
Your task is to perform the following actions:
1 - Summarize the following text delimited by
  <> with 1 sentence.
2 - Translate the summary into French.
3 - List each name in the French summary.
4 - Output a json object that contains the
  following keys: french_summary, num_names.
 
Use the following format:
Text: <text to summarize>
Summary: <summary>
Translation: <summary translation>
Names: <list of names in Italian summary>
Output JSON: <json with summary and num_names>
 
Text: <${text}>
`;
var prompt = `
Your task is to perform the following actions:
1 - Summarize the following text delimited by
  <> with 1 sentence.
2 - Translate the summary into French.
3 - List each name in the French summary.
4 - Output a json object that contains the
  following keys: french_summary, num_names.
 
Use the following format:
Text: <text to summarize>
Summary: <summary>
Translation: <summary translation>
Names: <list of names in Italian summary>
Output JSON: <json with summary and num_names>
 
Text: <${text}>
`;
  1. Give examples: Examples can be highly effective, especially for more complex tasks. By showing the model examples of the input and output format you want, you can guide its behavior.
var prompt = `
You will be provided with text delimited by triple quotes.
If it contains a sequence of instructions, 
re-write those instructions in the following format:
 
Step 1 - ...
Step 2 - …

Step N - …
 
If the text does not contain a sequence of instructions, 
then simply write "No steps provided."
 
"""${text_2}""";
`;
var prompt = `
You will be provided with text delimited by triple quotes.
If it contains a sequence of instructions, 
re-write those instructions in the following format:
 
Step 1 - ...
Step 2 - …

Step N - …
 
If the text does not contain a sequence of instructions, 
then simply write "No steps provided."
 
"""${text_2}""";
`;
  1. Few-shot prompting: This involves providing a series of examples before your actual prompt, which can improve the model's understanding of the task.
var prompt = `
Your task is to answer in a consistent style.
 
<child>: Teach me about patience.
 
<grandparent>: The river that carves the deepest 
valley flows from a modest spring; the 
grandest symphony originates from a single note;
the most intricate tapestry begins with a solitary thread.
 
<child>: Teach me about resilience.
`;
var prompt = `
Your task is to answer in a consistent style.
 
<child>: Teach me about patience.
 
<grandparent>: The river that carves the deepest 
valley flows from a modest spring; the 
grandest symphony originates from a single note;
the most intricate tapestry begins with a solitary thread.
 
<child>: Teach me about resilience.
`;
  1. Specify the steps to complete a task: Providing a step-by-step breakdown of a task can guide the model and improve its responses.
var prompt_1 = `
Perform the following actions:
1 - Summarize the following text delimited by triple
backticks with 1 sentence.
2 - Translate the summary into French.
3 - List each name in the French summary.
4 - Output a json object that contains the following
keys: french_summary, num_names.
 
Separate your answers with line breaks.
 
Text:
s${text}
`;
var prompt_1 = `
Perform the following actions:
1 - Summarize the following text delimited by triple
backticks with 1 sentence.
2 - Translate the summary into French.
3 - List each name in the French summary.
4 - Output a json object that contains the following
keys: french_summary, num_names.
 
Separate your answers with line breaks.
 
Text:
s${text}
`;
  1. Instruct the model to work out its own solution: Instead of asking for a direct answer, consider instructing the model to 'think through' the problem or task and describe its reasoning.
var prompt = `
Your task is to determine if the student's solution 
is correct or not.
To solve the problem do the following:
- First, work out your own solution to the problem.
- Then compare your solution to the student's solution 
and evaluate if the student's solution is correct or not.
Don't decide if the student's solution is correct until
you have done the problem yourself.
 
Use the following format:
Question:
'''
question here
'''
 
Student's solution:
'''
student's solution here
'''
 
Actual solution:
'''
steps to work out the solution and your solution here
'''
 
Is the student's solution the same as actual solution 
just calculated:
'''
yes or no
'''
 
Student grade:
'''
correct or incorrect
'''
`;
var prompt = `
Your task is to determine if the student's solution 
is correct or not.
To solve the problem do the following:
- First, work out your own solution to the problem.
- Then compare your solution to the student's solution 
and evaluate if the student's solution is correct or not.
Don't decide if the student's solution is correct until
you have done the problem yourself.
 
Use the following format:
Question:
'''
question here
'''
 
Student's solution:
'''
student's solution here
'''
 
Actual solution:
'''
steps to work out the solution and your solution here
'''
 
Is the student's solution the same as actual solution 
just calculated:
'''
yes or no
'''
 
Student grade:
'''
correct or incorrect
'''
`;
  1. Temperature and Top P Setting: These parameters control the randomness of the model's output. Adjusting these can help you get more focused and relevant responses.

temperature: What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. Default value is 1.

top_p: An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. Default value is 1.

It's recommended altering temperature or top_p but not both.

var prompt = `
You are a customer service AI assistant.
Your task is to send an email reply to a valued customer.
Given the customer email delimited by |, 
Generate a reply to thank the customer for their review.
If the sentiment is positive or neutral, thank them for 
their review.
If the sentiment is negative, apologize and suggest that 
they can reach out to customer service.
Make sure to use specific details from the review.
Write in a concise and professional tone.
Sign the email as 'AI customer agent'.
Customer email: |${review}|
Review sentiment: ${sentiment}
`;
 
var response = get_completion(prompt, { temperature: 0.7 });
console.log(response);
var prompt = `
You are a customer service AI assistant.
Your task is to send an email reply to a valued customer.
Given the customer email delimited by |, 
Generate a reply to thank the customer for their review.
If the sentiment is positive or neutral, thank them for 
their review.
If the sentiment is negative, apologize and suggest that 
they can reach out to customer service.
Make sure to use specific details from the review.
Write in a concise and professional tone.
Sign the email as 'AI customer agent'.
Customer email: |${review}|
Review sentiment: ${sentiment}
`;
 
var response = get_completion(prompt, { temperature: 0.7 });
console.log(response);
  1. Revision: Iteratively refining and testing your prompts is a critical part of prompt engineering.

Examples:

1. Expanding:

var prompt = `
You are a customer service AI assistant.
Your task is to send an email reply to a valued customer.
Given the customer email delimited by |,
Generate a reply to thank the customer for their review.
If the sentiment is positive or neutral, thank them for their review.
If the sentiment is negative, apologize and suggest that they can reach out to customer service.
Make sure to use specific details from the review.
Write in a concise and professional tone.
Sign the email as 'AI customer agent'.
Customer review: |${review}|
Review sentiment: ${sentiment}
`;
var prompt = `
You are a customer service AI assistant.
Your task is to send an email reply to a valued customer.
Given the customer email delimited by |,
Generate a reply to thank the customer for their review.
If the sentiment is positive or neutral, thank them for their review.
If the sentiment is negative, apologize and suggest that they can reach out to customer service.
Make sure to use specific details from the review.
Write in a concise and professional tone.
Sign the email as 'AI customer agent'.
Customer review: |${review}|
Review sentiment: ${sentiment}
`;

2. Translation:

var prompt = `Translate the following  text to French and Spanish
and English pirate: 
"""I want to order a basketball"""`;
var prompt = `Translate the following  text to French and Spanish
and English pirate: 
"""I want to order a basketball"""`;

3. Tone Transformation:

var prompt = `
Translate the following from slang to a business letter:
'Dude, This is Joe, check out this spec on this standing lamp.'
`;
var prompt = `
Translate the following from slang to a business letter:
'Dude, This is Joe, check out this spec on this standing lamp.'
`;

4. Format Conversion:

var data_json = {
  "restaurant employees": [
    { name: "Shyam", email: "shyamjaiswal@gmail.com" },
    { name: "Bob", email: "bob32@gmail.com" },
    { name: "Jai", email: "jai87@gmail.com" },
  ],
};
 
var prompt = `
Translate the following python dictionary from JSON to an HTML \
table with column headers and title: ${JSON.stringify(data_json)}
`;
var data_json = {
  "restaurant employees": [
    { name: "Shyam", email: "shyamjaiswal@gmail.com" },
    { name: "Bob", email: "bob32@gmail.com" },
    { name: "Jai", email: "jai87@gmail.com" },
  ],
};
 
var prompt = `
Translate the following python dictionary from JSON to an HTML \
table with column headers and title: ${JSON.stringify(data_json)}
`;

5. Spelling and Grammar Check:

var text = `
Got this for my daughter for her birthday cuz she keeps taking
mine from my room.  Yes, adults also like pandas too.  She takes
it everywhere with her, and it's super soft and cute.  One of
the ears is a bit lower than the other, and I don't think that
was designed to be asymmetrical. It's a bit small for what I
paid for it though. I think there might be other options that
are bigger for the same price.  It arrived a day earlier than
expected, so I got to play with it myself before I gave it to my daughter.
`;
 
var prompt = `proofread and correct this review enclosed in \: 
\${text}\
`;
var text = `
Got this for my daughter for her birthday cuz she keeps taking
mine from my room.  Yes, adults also like pandas too.  She takes
it everywhere with her, and it's super soft and cute.  One of
the ears is a bit lower than the other, and I don't think that
was designed to be asymmetrical. It's a bit small for what I
paid for it though. I think there might be other options that
are bigger for the same price.  It arrived a day earlier than
expected, so I got to play with it myself before I gave it to my daughter.
`;
 
var prompt = `proofread and correct this review enclosed in \: 
\${text}\
`;

6. Finding Sentiment:

var lamp_review = `
Needed a nice lamp for my bedroom, and this one had additional storage
and not too high of a price point. Got it fast. The string to our lamp 
broke during the transit and the company happily sent over a new one. 
Came within a few days as well. It was easy to put together. I had a 
missing part, so I contacted their support and they very quickly got me 
the missing piece! Lumina seems to me to be a great company that cares 
about their customers and products!!
`;
 
var prompt = `
What is the sentiment of the following product review,
which is delimited with triple backticks?
 
Review text: '''${lamp_review}'''
`;
 
var response = get_completion(prompt);
console.log(response);
var lamp_review = `
Needed a nice lamp for my bedroom, and this one had additional storage
and not too high of a price point. Got it fast. The string to our lamp 
broke during the transit and the company happily sent over a new one. 
Came within a few days as well. It was easy to put together. I had a 
missing part, so I contacted their support and they very quickly got me 
the missing piece! Lumina seems to me to be a great company that cares 
about their customers and products!!
`;
 
var prompt = `
What is the sentiment of the following product review,
which is delimited with triple backticks?
 
Review text: '''${lamp_review}'''
`;
 
var response = get_completion(prompt);
console.log(response);

Output: The sentiment of the product review is positive.

7. Identify Emotions:

var prompt = `
Identify a list of emotions that the writer of the following 
review is expressing. Include no more than five items in the list. 
Format your answer as a list of lower-case words separated by commas.
 
Review text: '''${lamp_review}'''
`;
 
var response = get_completion(prompt);
console.log(response);
var prompt = `
Identify a list of emotions that the writer of the following 
review is expressing. Include no more than five items in the list. 
Format your answer as a list of lower-case words separated by commas.
 
Review text: '''${lamp_review}'''
`;
 
var response = get_completion(prompt);
console.log(response);

Output: happy, satisfied, grateful, impressed, content

Remember, prompt engineering is more art than science, and often requires a fair bit of trial and error. The key is to keep refining your prompts until you get the results you want.

References