View Proposal

---

Proposer

John See

Title

Self-Retrieval Augmented Generation for Legal Document Analysis

Goal

The aim of this project is to develop a self-RAG system for querying and analysing legal documents. The system is expected to be evaluated on response accuracy, retrieval relevance and faithfulness to source documents, and compared with conventional RAG and state-of-the-art LLMs

Description

This project will investigate the implementation of Self-RAG for legal document analysis, using a collection of legal documents such as contracts, legislation or case law. The system will allow an LLM to dynamically determine when external legal evidence is required, retrieve relevant passages, evaluate their relevance, and generate responses grounded in the retrieved material. Its performance will be compared against a conventional RAG approach to examine whether self-reflection and adaptive retrieval can improve the accuracy, relevance and factual grounding of legal responses.

Resources

There are a few starter code frameworks available on GitHub (for research purposes). LLM APIs might be required (some with paid subscription) but it may be possible to use open-source LLMs. Documents on the legal cases (written judgement, etc.) are to be obtained from publicly available (free and gazetted) sources.

Background

Large Language Models (LLMs) can assist with legal document analysis, but their tendency to generate plausible yet unsupported information presents significant challenges in legal applications. Retrieval-Augmented Generation (RAG) can improve factual grounding by providing relevant legal documents as evidence, but conventional RAG typically retrieves information for every query regardless of whether retrieval is necessary and may still produce responses based on irrelevant or insufficient evidence. Self-RAG introduces a more adaptive approach in which the model can decide when to retrieve information, assess the relevance of retrieved evidence, and critique its own generated response. The benefits of using Self-RAG for legal document analysis would stem from its ability to map every assertion directly to specific source segments or paragraph citations, avoiding hallucinations and fabrications that are not factually grounded. It also provides an avenue for error correction (and automatic retry) if retrieved answers do not completely answer a complex multi-step question.

Url

Difficulty Level

Moderate

Ethical Approval

None

Number Of Students

1

Supervisor

John See

Keywords

self-rag, legal document analysis

Degrees

Bachelor of Science in Computing Science