What is prompt en­gin­eer­ing and how does it work?

Prompt en­gin­eer­ing comprises various tech­niques and methods for op­tim­ising prompts for gen­er­at­ive AI tools. We’ll explain the defin­i­tion of prompt en­gin­eer­ing, why it’s important, and go over examples and best practices.

Properly for­mu­lat­ing prompts for AI tools is im­per­at­ive if you want to get the most out of language models. As ar­ti­fi­cial in­tel­li­gence continues to evolve, so has the need for pro­fes­sion­als who know how to navigate it most ef­fi­ciently, which is how the pro­fes­sion of prompt engineer came about.

What is prompt en­gin­eer­ing?

The term ‘prompt en­gin­eer­ing’ refers to tech­niques and methods used to optimise prompts for natural language pro­cessing (NLP) and large language models (LLMs) such as GPT-3 or GPT-4, which are based on machine learning. The way a question or in­struc­tions are for­mu­lated greatly in­flu­ences the quality and relevance of the answer generated by the ar­ti­fi­cial in­tel­li­gence tool.

Prompt en­gin­eer­ing for AI models requires not only cre­ativ­ity and precision but also a deep un­der­stand­ing of the re­spect­ive language model, as the choice of words and their order can sig­ni­fic­antly influence the output. Prompts can include text in natural language, images, or other types of data inputs. The same prompt can produce different results across various AI platforms. Therefore, prompt en­gin­eer­ing must be tailored in­di­vidu­ally for each AI text generator or AI video generator.

Why is prompt en­gin­eer­ing important for AI?

Prompt en­gin­eer­ing is essential if you want to achieve better results with gen­er­at­ive AI and fully harness the potential of language models. For example, a prompt engineer may ex­per­i­ment by posing a question in many different ways to see how it in­flu­ences the answer. Vari­ations in word order and using a modifier once or several times (e.g., ‘very’ or ‘very, very, very’) can sig­ni­fic­antly affect the results.

For AI image websites, prompt en­gin­eer­ing can help fine-tune various features of generated images. These often provide the ability to create AI images in a par­tic­u­lar style, per­spect­ive, aspect ratio or image res­ol­u­tion. The first prompt is usually just a starting point. The following prompts can be used, for example, to soften or strengthen certain elements and add or remove objects in an image.

Prompt en­gin­eer­ing can also help align LLMs and optimise workflows for specific outcomes when de­vel­op­ing new tools. There are also other reasons why prompt en­gin­eer­ing is important for AI:

• Result op­tim­isa­tion: Carefully designed prompt en­gin­eer­ing can enable language models to deliver higher quality and more relevant results.
• Ef­fi­ciency: Well-for­mu­lated prompts result in the model de­liv­er­ing the desired in­form­a­tion faster, without the need for multiple prompts or it­er­a­tions.
• Control over output: Clever prompt en­gin­eer­ing allows the user to control the way the AI responds, including the length, style and tone of the response.
• Error reduction: Clear and concise prompts help minimise potential biases, mis­un­der­stand­ings or in­ac­cur­ate answers that a model might give.
• Advanced ap­plic­a­tions: With proper prompt en­gin­eer­ing, AI models can be used for specific tasks or in other areas that they were not ori­gin­ally developed for.
• Ex­per­i­ment­al insights: Ex­per­i­ment­ing with different prompts can help gain a deeper un­der­stand­ing of how a par­tic­u­lar gen­er­at­ive AI works and how it responds to different inputs.

Examples of prompt en­gin­eer­ing

Prompts that can be used to create text, images or videos differ sig­ni­fic­antly from one another. However, for all AI websites, targeted prompt en­gin­eer­ing allows users to interact more ef­fect­ively with the re­spect­ive AI tool.

Prompt examples for text gen­er­at­ors

Here is an example of targeted prompt en­gin­eer­ing for text gen­er­at­ors:

1. Spe­cificity

• original prompt: ‘Tell me about trees’.
• improved prompt: ‘Explain the process of pho­to­syn­thes­is in deciduous trees’.

2. Answer format­ting

• original prompt: ‘What are the benefits of solar energy?’
• improved prompt: ‘Name five benefits of solar energy’.

3. Inserting sample answers

• original prompt: ‘Write a sentence about Paris’.
• improved prompt: ‘Write a sentence about Paris in the style of Hemingway’.

4. Length and details

• original prompt: ‘Describe water’.
• improved prompt: ‘Give me a detailed sci­entif­ic ex­plan­a­tion of the molecular structure of water’.

5. Avoiding prejudice

• original prompt: ‘What do you think about crypto­cur­ren­cies?’
• improved prompt: ‘Describe crypto­cur­ren­cies neutrally and ob­ject­ively’.

6. Context

• original prompt: ‘Why do stocks fall?’
• improved prompt: ‘Con­sid­er­ing economic factors, why might tech­no­logy stocks fall in a recession?’

7. Styles or per­spect­ives

• original prompt: ‘Tell me the story of Napoleon’.
• improved prompt: ‘Tell me the story of Napoleon from the per­spect­ive of one of his soldiers’.

Prompt examples for image gen­er­at­ors

Prompt en­gin­eer­ing is not only relevant for language models, but also for Gen­er­at­ive Ad­versari­al Networks that generate images, such as DALL-E. For image gen­er­at­ors, prompts must textually describe what kind of image should be generated:

1. Spe­cificity

• original prompt: ‘Cat’.
• improved prompt: ‘Orange cat sleeping on a blue pillow’.

2. Com­bin­a­tion of elements

• original prompt: ‘Buildings and clouds’.
• improved prompt: ‘An old Victorian house resting on floating clouds’.

3. Style and era

• original prompt: ‘Cars’.
• improved prompt: ‘1950s retro-style fu­tur­ist­ic cars’.

4. Feelings and at­mo­sphere

• original prompt: ‘Forest’.
• improved prompt: ‘A dark, misty forest bathed in moonlight’.

5. Com­bin­a­tion of unusual elements

• original prompt: ‘Table and fruit’.
• improved prompt: ‘A table made of wa­ter­mel­ons with a top made of dried banana slices’.

6. Per­spect­ive and dimension

• original prompt: ‘Mountains’.
• improved prompt: ‘A huge mountain in the shape of an upside-down tea glass’.

7. Ab­strac­tion

• original prompt: ‘Feelings’.
• improved prompt: ‘Joy visu­al­ised as a bright explosion of colour’.

Prompt examples for video gen­er­at­ors

For video gen­er­at­ors, the challenge is to capture not just a single moment or still image, but a dynamic, timed sequence of actions and events. Good prompt en­gin­eer­ing helps to precisely specify the action, en­vir­on­ment and duration of the video as well as how elements in the video should interact:

1. Action sequence

• original prompt: ‘Cat walking’.
• improved prompt: ‘Orange cat walks slowly past a puddle and then jumps into it’.

2. En­vir­on­ment and mood

• original prompt: ‘Beach scene’.
• improved prompt: ‘A deserted beach at sunset, with gently crashing waves and a flock of birds flying on the horizon’.

3. Temporal de­vel­op­ment

• original prompt: ‘A growing flower’.
• improved prompt: ‘A rose growing from a bud to a fully bloomed flower in 30 seconds’.

4. Dynamic actions

• original prompt: ‘Sports game’.
• improved prompt: ‘A bas­ket­ball game in which a player makes a crucial three-point goal in the final seconds of the game’.

5. Com­bin­a­tion of elements and trans­itions

• original prompt: ‘Times of day’.
• improved prompt: ‘A city panorama trans­ition­ing from morning to night, with the lights of the city coming on as darkness falls’.

6. Story and narration

• original prompt: ‘A bird flying’.
• improved prompt: ‘A young bird trying to fly for the first time. After a few failed attempts, the bird finally conquers the skies and returns safely to its nest’.

What are best practices for prompt en­gin­eer­ing?

With targeted prompt en­gin­eer­ing, it’s possible to obtain optimal results from gen­er­at­ive AI tools. There are some proven best practices that should be taken into account when for­mu­lat­ing prompts:

• Be precise: Being clear when wording a prompt helps the AI better un­der­stand what you expect it to generate.
• Be specific: Make sure your prompts are specific enough to obtain the type of response you want.
• Ex­per­i­ment: If you don’t get the answer you want right away, try phrasing the question dif­fer­ently or adding more context.
• Format in­struc­tions: If you want the answer to be in a specific format (e.g., list, short paragraph, formal language), you should specify this in the prompt.
• Sample responses: Providing sample responses can be helpful as it can give the AI an example of the answer you want and steer it in the right direction.
• Context: Some AI tools benefit from being given ad­di­tion­al in­form­a­tion or more context before the actual question is asked.
• Avoid ambiguity: Avoid unclear or ambiguous wording.
• Limit and direct: If you are concerned that the AI tool may answer in a biased way, or if you want a par­tic­u­lar style or per­spect­ive, give clear in­struc­tions.
• Review: It is important to crit­ic­ally review an AI tool’s responses and ensure they are both accurate and free of unwanted bias.
• Iterative approach: It is often useful to take an iterative approach and refine the question based on the answers received.

What qual­i­fic­a­tions should a prompt engineer have?

Prompt en­gin­eer­ing offers promising op­por­tun­it­ies for in­di­vidu­als with a deep un­der­stand­ing of language pro­cessing and a creative mindset. As AI and NLP tech­no­lo­gies become more prevalent across a wide range of in­dus­tries, the demand for skilled prompt engineers will continue to grow.

Although there are no re­quire­ments in terms of specific education, a degree in a related field can be helpful. Although pro­gram­ming skills are not essential, a degree in computer science or lin­guist­ics can make it easier to un­der­stand language models and develop prompts. Prompt en­gin­eer­ing is primarily about un­der­stand­ing how language works and how to structure it to receive the results you want. The following skills can be helpful in this process:

• Un­der­stand­ing AI and machine learning: It’s important to have a basic un­der­stand­ing of how neural networks work, par­tic­u­larly language models, so you can better un­der­stand the mech­an­isms behind the results.
• Ana­lyt­ic­al thinking: Analysing results and adjusting prompts based on them requires ana­lyt­ic­al thinking.
• Com­mu­nic­a­tion skills: The ability to ar­tic­u­late clear and concise in­struc­tions is essential to prompt en­gin­eer­ing.
• Error detection: The ability to detect in­ac­curacies or errors in an AI model’s responses and make ap­pro­pri­ate ad­just­ments.
• Domain-specific knowledge: Depending on which domain you are using it for, spe­cial­ised domain knowledge may be required to ef­fect­ively design and evaluate prompts and responses.
• Con­tinu­ous learning: Ar­ti­fi­cial in­tel­li­gence and machine learning are rapidly evolving. Good prompt en­gin­eer­ing therefore requires a com­mit­ment to con­tinu­ous learning and a will­ing­ness to con­stantly adapt to new tech­no­lo­gies.
• Teamwork: A prompt engineer often has to col­lab­or­ate with other pro­fes­sion­als such as data sci­ent­ists, software engineers and business analysts.