Julie Kallini
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
Dynamic Token Merging for Efficient Byte-level Language Models with Julie Kallini
- Published
- March 24, 2025
- Duration
- 50:32
- Summary source
- description
- Last updated
- Jun 7, 2026
Discusses Today, we're joined by Julie Kallini, PhD student at Stanford University to discuss her recent paper…
Summary
Today, we're joined by Julie Kallini, PhD student at Stanford University to discuss her recent papers, “MrT5: Dynamic Token Merging for Efficient Byte-level Language Models” and “Mission: Impossible Language Models.” For the MrT5 paper, we explore the importance and failings of tokenization in large language models—including inefficient compression rates …
Intelligent report
Sign in to read teasers, or upgrade to Research Pro to commission a new dossier for this episode. Learn more →
Show notes
Today, we're joined by Julie Kallini, PhD student at Stanford University to discuss her recent papers, “MrT5: Dynamic Token Merging for Efficient Byte-level Language Models” and “Mission: Impossible Language Models.” For the MrT5 paper, we explore the importance and failings of tokenization in large language models—including inefficient compression rates for under-resourced languages—and dig into byte-level modeling as an alternative. We discuss the architecture of MrT5, its ability to learn lan