Retrieval-Augmented Generation Playbook

Overview

The Retrieval-Augmented Generation (RAG) Playbook is a comprehensive guide for developers focused on building, evaluating, and improving RAG systems within the government sector. RAG systems leverage the power of both retrieval and generation processes to provide contextually accurate and relevant information, enhancing the quality and efficiency of government services.

Objective

The objectives of the RAG Playbook are as follows:

• Facilitate the development of high-quality RAG applications by providing systematic guidance on building, evaluating, and improving RAG pipelines.
• Enhance the performance of RAG systems in government-specific use cases by offering best practices and metrics for evaluation.

Scope

The RAG Playbook is applicable to Whole-of-Government (WOG) projects that aim to integrate RAG systems into their digital services.

Target Audience

The RAG Playbook is intended for government software engineers, data scientists, and vendors involved in digital transformation projects. It caters to various levels such as individuals who are just getting started with RAG to those who have advanced their use of RAG.

Standards, Guidelines and Assessment Criteria

For teams interested in adopting RAG systems, consider the following criteria:

• Determine the suitability of RAG systems for the specific use case and the potential impact on service delivery.
• Evaluate the relevance and accuracy of retrieved information and generated responses using established metrics.
• Engage in continuous improvement by iteratively refining RAG pipelines based on user feedback and performance evaluations.

Resources and Templates

RAG Playbook: Access the full playbook for detailed guidance on RAG development here.

Contact Information

For more information on the RAG Playbook and how to get started, reach out to the AI Practice team through this form.

Last updated 17 April 2025