arxiv preprint - Gecko: Versatile Text Embeddings Distilled from Large Language Models
In this episode, we discuss Gecko: Versatile Text Embeddings Distilled from Large Language Models by Jinhyuk Lee, Zhuyun Dai, Xiaoqi Ren, Blair Chen, Daniel Cer, Jeremy R. Cole, Kai Hui, Michael Boratko, Rajvi Kapadia, Wen Ding, Yi Luan, Sai Meher Karthik Duddu, Gustavo Hernandez Abrego, Weiqiang Shi, Nithi Gupta, Aditya Kusupati, Prateek Jain, Siddhartha Reddy Jonnalagadda, Ming-Wei Chang, Iftekhar Naim. Gecko is a new text embedding model designed for efficient retrieval, using a novel two-step knowledge distillation process from large language models. First, it creates varied synthetic query-passage pairs, then it improves the data by selecting and relabeling high-quality candidates. Despite its smaller size, Gecko demonstrates superior retrieval performance, outpacing larger models with higher dimensionality on a benchmark test.
AI Breakdown
The podcast where we use AI to breakdown the recent AI papers and provide simplified explanations of intricate AI topics for educational purposes.
The content presented here is generated automatically by utilizing LLM and text to speech technologies. While every effort is made to ensure accuracy, any potential misrepresentations or inaccuracies are unintentional due to evolving technology. We value your feedback to enhance our podcast and provide you with the best possible learning experience.
- No. of episodes: 400
- Latest episode: 2024-07-22
- Education Technology
Where can you listen?
