At the International Conference on Machine Learning (ICML), researchers from Intel Labs and the Weizmann Institute of Science introduced a major advance in speculative decoding. The new technique, presented at the conference in Vancouver, Canada, enables any small "draft" model to accelerate any large language model (LLM) regardless of vocabulary differences. "We have solved a core inefficiency in generative AI. Our research shows how to turn speculative acceleration into a universal tool. This isn't just a theoretical improvement; these are practical tools

The CodeSteer system could boost large language models’ accuracy when solving complex problems, such as scheduling shipments in a supply chain.