We all have our go-to LLM implementation. It’s easy to use and generally tells us how best to get the most out of it. You can check your work, create content, and quite often people completely substitute web searches for those of an LLM query (this is not always advisable!). But did you know that there are ways in which you can get more out of an LLM? Are you using the power of prompt engineering to make the most of what’s available?
Prompt engineering sounds complicated, but really once we understand a few simple rules about how an LLM is actually working, it’s straightforward to implement.
Initial Context (System Prompt)
Before you even start writing your prompt, there is likely a starter sentence (or twenty) which you can’t see, being fed to the LLM in front of what you write. This is called a “System Prompt”. The idea behind this is to tell the LLM how to behave, things like being helpful, safe, honest, and staying within policies. Within company specific implementations, this can even be tailored to start conversation off on the right foot without the user having to type out the same prompt every time they use it. For instance adding “Please ensure your answers are in the context of an advisor in a financial institution” might help tailor responses for banking employees.
Structure & Role Play
It is often best to set the scene for the chat agent in the LLM for it to understand what you would like. Part of this is helping it understand who you are, and who you would like it to be. If you start each prompt by saying “I am an {insert role here} and I would like you to be a {insert role here}” it provides the initial setting. Further, if you start to describe the task you would like the agent to perform, you should specify as much about the output as you can. A good way of thinking about this is to set the scene, give the goal of what you’re trying to do, set output parameters, and where possible give an indication where you would like the information to come from. As an example “I am an analyst in a bank and I would like you to take the role of a financial advisor. I’m looking to create a report for senior executives which will highlight some good potential investments. I would like you to write the report using financial language, based on the 3 presentations which I have uploaded”
Give Examples
As alluded to in the previous paragraph, the more indication you give about what you would like, the better your answer will be. If you want something written in a certain style, upload a few files which show that style. If you’re creating a picture or a slide for a presentation, similar. The more that the system has to work with, the more likely it will get closer to your answer. You’ll be surprised at how something like “I’ve uploaded a word file and an image of a slide. Please create a new slide with the format of the image and the content of word file” can give you a super-quick image of what you can use in a slide deck.
Tables and comparisons
LLMs are brilliant at outputting tables showing comparisons across many different sources. Try giving multiple formats, including pdfs, word files, or even just using the web. If you specify what the categories are you’re looking to compare across, you’ll find that tables often give a very concise view contrasting different options. For instance “I'm looking to buy new cast iron cookware. Please find a list of the top 10 pieces of cookware I can use on an induction stovetop. I'm looking for the categories cost, user rating, and main dishes that can be cooked. Please give the output in a tabular format”
Experiment with small changes to prompts
Quite often it appears that LLM outputs create chaotic output: i.e. a small change in input leads to a large change in output. Use this to your advantage, and see whether a complete copy/paste of your previous prompt with a few small changes does the trick. If you have the “temperature” setting available, try setting this differently depending on what you’re looking for.
Start Again from Fresh
Bear in mind that your entire conversation fits in the context window of the LLM. When you experiment, quite often the answers you get can be limited by the previous answers given. It could be that various elements of what you have tried will work together, but given the context of the conversation it’s going to be difficult to bring everything in line with what you would like. At this point, copy the most relevant prompt, click “new chat”, and try starting completely from scratch. This will clear the context window and give you a fresh start
Experiment with Different Formats
You may not know that the major LLMs can output many different formats, not just written content. Try outputting images, slides, files. You’ll be surprised at what you can achieve.
Super Prompts
Why not try asking the LLM the best way to prompt it? You can start by trying to prompt the LLM in the way that you normally do. After that you can continue the conversation by saying you’re looking to build a prompt to give a better output. The LLM will give a nice template which you can then use in a new chat (without context window) to get an output which should be much closer to what you’re expecting
In short, if you’re not using prompting properly, you’re leaving value on the table. Don’t limit the output of LLMs to just becoming a web search agent for you. How can an LLM work best for you?
About the authors
Larry is a lifelong technologist with a strong passion for problem-solving. With over a decade of trading experience and another decade of technical expertise within financial institutions, he has built, grown, and managed highly profitable businesses. Having witnessed both successful and unsuccessful projects, particularly in the banking sector, Larry brings a pragmatic and seasoned perspective to his work. Outside of his professional life, he enjoys Brazilian Jiu-Jitsu, climbing and solving cryptic crosswords.
LinkedIn
Ash is a strategy and operations professional with 14 years of experience in financial services, driven by a deep passion for technology. He has led teams and projects spanning full-scale technology builds to client-facing strategic initiatives. His motivation comes from connecting people, processes, data and ideas to create solutions that deliver real-world impact. Beyond work, Ash enjoys exploring different cultures through food and cocktails and practices yoga regularly.
LinkedIn