AMD has announced ROCm version 6.3 , which adds many new updates to the ROCm ecosystem. The latest iteration of the open-source driver stack features several additions, including SGLang, FlashAttention-2, and a Fortran Compiler.
SGLang is a new runtime in ROCm 6.3 that purportedly improves latency, throughput, and resource utilization by optimizing "cutting-edge" generative AI models on AMD's homebrewed Instinct GPUs. SGLang purportedly achieves up to 6X higher performance on large
Luma Labs has given its Dream Machine AI video generator a makeover by introducing a new text-to-image foundation model called Photon.
[ Read More ]
While Apple is in the midst of rolling out new Apple Intelligence features as part of iOS 18, it’s also working on yet-to-be-announced features for iOS 19. One of those features is an upgraded version of Siri powered by more advanced large language models, or LLMs. The news was detailed in a new report today .
Mark Gurman reports that this is being referred to as “LLM Siri” inside Apple. The upgraded assistant is currently being tested as part of a separate app
A Raspberry Pi 5 hooked up to an AMD Radeon -powered eGPU has been demonstrated using the graphics hardware to accelerate running a Large Language Model (LLM). Of course, it's Pi wizard Jeff Geerling again, and in the video embedded below, he talks us through his experience of leveraging the Vulkan API support to enjoy GPU-accelerated local AI on the Raspberry Pi 5.
"Slang empowers real-time graphics developers with innovative features that complement existing shading languages, including modular code development, portable deployment to multiple target APIs, and neural computation in graphics shaders. Hosting under multi-company governance at Khronos will enable and foster industry-
Cerebras got Meta’s Llama 3.1 405B large language model to run at 969 tokens per second, 75 times faster than Amazon Web Services' fastest AI service with GPUs could muster.
The LLM was run on Cerebras’s cloud AI service Cerebras Inference, which uses the chip company’s third-generation Wafer Scale Engines rather than GPUs from Nvidia or AMD. Cerebras has always claimed its Inference service is the fastest for generating tokens, the individual