Is RAG Dead? Lessons from Building AI for Tax Law with Alex Bowcut

Episode 769

0:00-51:32

As context windows grow into the millions of tokens, many AI practitioners are questioning whether retrieval-augmented generation (RAG) is still necessary. If modern models can ingest entire libraries of documents, why bother with retrieval at all? In this episode, Alex Bowcut, Head of Engineering at Sphere, explains why the answer depends on the application. Sphere uses AI to automate global tax compliance—an environment where getting the answer right isn’t enough. Every conclusion must be backed by the correct legal citation, and every decision must withstand expert review. We explore how Sphere built TRAM (Tax Review and Assessment Model), a production AI system that combines retrieval, reasoning models, legal review workflows, reinforcement learning, and deterministic systems to help tax experts move nearly two orders of magnitude faster while maintaining accuracy. Along the way, we discuss why RAG remains critical in high-stakes domains, how Sphere processes legal and regulatory documents from jurisdictions around the world, retrieval architectures, semantic chunking, dense versus sparse retrieval, expert feedback loops, and the challenges of building AI systems that people can actually trust.

Publisher

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

Link

twimlai.com/podcast/twimlai/rag-dead-lessons-building-ai-tax-law

Published

June 9, 2026

Duration

52 min

Format

Podcast

Category

Issues · Tech News

Source

RSS feed

Host

Sam Charrington

TT

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

@twimlai

#ragdead #buildingai #alexbowcut

The Nile