Apple Inc. AAPL and Nvidia Corporation NVDA have announced a collaboration for the improvement in the performance of large language models.
What Happened: The partnership focuses on integrating Apple’s Recurrent Drafter (ReDrafter) technique with Nvidia’s TensorRT-LLM, to boost text generation speeds.
ReDrafter, which Apple open-sourced earlier this year, combines beam search and dynamic tree attention to improve LLM performance.
The collaboration with Nvidia has led to the integration of ReDrafter into TensorRT-LLM, a tool designed to accelerate LLMs on Nvidia GPUs.
This integration involved adding new operators to improve TensorRT-LLM’s ability to handle complex models and decoding methods.
Benchmarking results show a 2.7x increase in token generation speed for greedy decoding on Nvidia GPUs, significantly reducing latency and power consumption.
Apple’s machine learning researchers noted that this advancement could lower computational costs and improve user experience by reducing latency in production applications.
Subscribe to the Benzinga Tech Trends newsletter to get all the latest tech developments delivered to your inbox.
Why It Matters: In October earlier this year, Apple reported fourth-quarter revenue of $94.9 billion, surpassing analyst expectations of $94.56 billion.
Last month, Nvidia reported third-quarter revenue of $35.1 billion, marking a 94% increase compared to the last year, and exceeding the Street consensus estimate of $33.12 billion, as per Benzinga Pro data.
Together these two tech giants are worth about $7 trillion, with Apple being the most valuable company in the world and Nvidia ranking third.
Price Action: Apple shares fell 2.14% to $248.05 on Wednesday, and dipped further to $247.19 in after-hours trading. Meanwhile, Nvidia shares gained 2.01% to $131.50 in after-hours trading, recovering from a 1.14% decline to $128.91 during the regular session.
Photo Courtesy: Koshiro K On Shutterstock.com
Check out more of Benzinga's Consumer Tech coverage by following this link.
Read Next:
Disclaimer: This content was partially produced with the help of Benzinga Neuro and was reviewed and published by Benzinga editors.
© 2024 Benzinga.com. Benzinga does not provide investment advice. All rights reserved.
Trade confidently with insights and alerts from analyst ratings, free reports and breaking news that affects the stocks you care about.
