PyCon Nigeria Annual Conference

Steering LLMs Towards Reliable and Structured Outputs

speaker-foto

David Okpare

David is an accomplished AI engineer at Siennai Analytics, where he leads the development of applications with integrated large language models (LLMs). With over four years of experience in Python development, David has established himself as a skilled programmer in the field. Beyond his professional endeavors, David is a dedicated contributor to the open-source community. His open-source contributions extend to notable projects like Instructor, EleutherAI, GPT4All, JupySQL, and many others, showcasing his commitment to advancing the field of artificial intelligence and promoting open-source software. David's expertise in LLM integration and his passion for open-source development make him a valuable asset in the AI community. His contributions have not only facilitated the adoption of cutting-edge technologies but have also fostered collaboration and knowledge-sharing within the developer community.

Description

As the use for LLMs is moving from chatbots towards integrations within product, the need for reliable and structured outputs is required. However, LLMs can exhibit unpredictable behavior and undesired outputs, necessitating additional guardrails to ensure more predictable outputs and minimize hallucinations. This talk explores methods to achieve desired outcomes.

Abstract

The widespread adoption of large language models (LLMs) in applications and product integrations necessitates reliable and structured outputs. However, LLMs can exhibit unpredictable behavior and generate undesired outputs, requiring additional safeguards to ensure more predictable outputs and minimize hallucinations. While prompt engineering efforts have been made to reduce the likelihood of unexpected outcomes, the success rate leaves room for further exploration. This talk explores techniques and guardrails to obtain structured output, validate LLM results, implement caching to reduce latency and costs, and handle errors. Additionally, it explores the use of popular Python packages such as Pydantic in guiding the outputs of LLMs and achieving structured outputs where prompting fails.

Audience level: Intermediate or Advanced