Technology
Leave a comment

Apple Researchers Reveal New AI System That Can Beat GPT-4

Apple Researchers Reveal New AI System That Can Beat GPT-4


Apple researchers have developed an artificial intelligence system named ReALM (Reference Resolution as Language Modeling) that aims to radically enhance how voice assistants understand and respond to commands.

hey siri banner apple
In a research paper (via VentureBeat), Apple outlines a new system for how large language models tackle reference resolution, which involves deciphering ambiguous references to on-screen entities, as well as understanding conversational and background context. As a result, ReALM could lead to more intuitive and natural interactions with devices.

Reference resolution is an important part of natural language understanding, enabling users to use pronouns and other indirect references in conversation without confusion. For digital assistants, this capability has historically been a significant challenge, limited by the need to interpret a wide range of verbal cues and visual information. Apple’s ReALM system seeks to address this by converting the complex process of reference resolution into a pure language modeling problem. In doing so, it can comprehend references to visual elements displayed on a screen and integrate this understanding into the conversational flow.

ReALM reconstructs the visual layout of a screen using textual representations. This involves parsing on-screen entities and their locations to generate a textual format that captures the screen’s content and structure. Apple researchers found that this strategy, combined with specific fine-tuning of language models for reference resolution tasks, significantly outperforms traditional methods, including the capabilities of OpenAI’s GPT-4.

ReALM could enable users to interact with digital assistants much more efficiently with reference to what is currently displayed on their screen without the need for precise, detailed instructions. This has the potential to make voice assistants much more useful in a variety of settings, such as helping drivers navigate infotainment systems while driving or assisting users with disabilities by providing an easier and more accurate means of indirect interaction.

Apple has now published several AI research papers. Last month, the company revealed a new method for training large language models that seamlessly integrates both text and visual information. Apple is widely expected to unveil an array of AI features at WWDC in June.

Popular Stories

Alleged iOS 18 Design Resource Reveals visionOS-Like Redesign [Updated]

A first look at iOS 18’s rumored visionOS-style redesign may have been revealed by a new image of the Camera app. Alleged iOS 18 design resource. MacRumors received the above iPhone frame template from an anonymous source who claims they obtained it from an iOS engineer. It will allegedly be included as part of the Apple Design Resources for iOS 18, which helps developers visually design apps …

What to Expect From iOS 17.5

Apple has yet to release the first beta of iOS 17.5 for the iPhone, but two changes are already expected with the upcoming software update. iOS 17.5 will likely allow iPhone users in the EU to download apps directly from the websites of eligible developers, and the update might include some changes to how Apple ID recovery contacts work. More details about these potential changes follow. W…

Apple Says iPhone 6 Plus Now ‘Obsolete’ and iPad Mini 4 Now ‘Vintage’

Apple today added a handful of devices to its public-facing vintage and obsolete products list, including some older iPhone and iPad models. Apple now considers the iPhone 6 Plus to be “obsolete” worldwide, meaning that Apple Stores and Apple Authorized Service Providers no longer offer repairs or other hardware service for the device. Apple says it considers a product “obsolete” once seven…

Apple Card Savings Account to Receive First-Ever Interest Rate Decrease

Nearly one year after it launched in the U.S., the Apple Card’s high-yield savings account will be receiving its first-ever interest rate decrease. Starting on April 3, the Apple Card savings account’s annual percentage yield (APY) will be lowered to 4.4%, according to data on Apple’s backend discovered by MacRumors contributor Aaron Perris. The account currently has a 4.5% APY. 4.4% will …

Top Stories: WWDC 2024 Announced, New iPads Delayed, and More

Apple’s WWDC 2024 dates have been announced, giving us timing for the unveiling of the company’s next round of major operating system updates and likely some other announcements. This week also saw some disappointing news on the iPad front, with update timing for the iPad Pro and iPad Air pushed back from previous rumors. We did hear some new tidbits about what might be coming in iOS 18 and…

Best Buy Introduces All-Time Low Prices on Apple’s M3 MacBook Pro for Members

Best Buy is discounting a large collection of M3 MacBook Pro computers today, including both the 14-inch and 16-inch versions of the laptop. Every deal in this sale requires you to have a My Best Buy Plus or Total membership, although non-members can still get solid second-best prices on these MacBook Pro models. Note: MacRumors is an affiliate partner with Best Buy. When you click a link and…

Apple to Launch New iPad Pro and iPad Air Models in May

Apple will introduce new iPad Pro and iPad Air models in early May, according to Bloomberg’s Mark Gurman. Gurman previously suggested the new iPads would come out in March, and then April, but the timeline has been pushed back once again. Subscribe to the MacRumors YouTube channel for more videos. Apple is working on updates to both the iPad Pro and iPad Air models. The iPad Pro models will…



Source link

Leave a Reply