Amazon is using a secret new AI model to bring human-level understanding to robots

The company often encourages staff to imagine future press releases for new technology they are developing. In one of these documents related to Burnham, Amazon describes an Astro product that costs $995. There's an additional "Burnham Plus" $24.99 monthly fee for standard home monitoring features, and Burnham Plus with Amazon's Ring doorbell camera would be $34.99 a month, according to this document.

Burnham is a broader initiative that could support a host of other products. The technology "remembers what it saw and understood," and derives meaning from those interactions, the documents said. It can also engage in a Q&A dialogue on what it saw and understood, using large language model-based technology commonly found in apps like ChatGPT, and take appropriate actions based on that. 

• For instance, if Burnham finds a stove left burning or a water faucet running unattended, it will find the owner and bring it to their attention, the document said.
• If an elderly man slips in the kitchen, Burnham can check to make sure he's ok, and call others to come help. If it's an emergency situation, Burnham will automatically call 911.
• Owners can ask Burnham where they left their keys.
• It can check whether the kitchen window was left open last night.
• The robot can also monitor whether the kids had friends over after school.

"To put it simply: Our robot has a strong body. What we need next is a brain," one of the documents stated. "A robot with Burnham would understand — in the same way a human understands — the thousands of things that happen within a home every day without having to explicitly code for each one because that 'common-sense' knowledge is implicit in the data the language model was trained on."

The move represents the next phase of growth for Astro, a heavily-invested, Alexa-powered home monitoring robot that so far appears to have failed to live up to Amazon's lofty expectations. Despite years of investment and hundreds of people working on it, Astro has faced mediocre reviews and internal scorn, and is still hard to buy, as it remains invite-only, even after 18 months since its launch.

It's also the latest example of Amazon incorporating generative AI and LLM technology into its existing products and services, as competitors like Microsoft and Google race ahead in the AI chatbot market. Amazon is planning to upgrade its Alexa voice-technology with ChatGPT-like features, as Insider previously reported, while it wants to build AI tools to auto-generate photos and videos for advertisers, according to The Information. Amazon's CFO Brian Olsavsky said last month that more investment dollars are shifting from its core logistics business to such AI and LLM technologies.

In an email to Insider, Amazon's spokesperson said Astro is "off to a very promising start" and that invite requests "remain strong," without providing specific numbers.

"We're excited by the potential of generative AI technologies, and are looking forward to using these technologies to invent new experiences for Astro that will delight customers and make their lives even easier in the future," the spokesperson said.

'Convinced it will work'

The idea for Burnham was inspired by a research paper in the field of large language models. As LLMs were growing in scale, they started demonstrating "emergent skills in both inference and problem-solving," one of the Amazon documents stated. Inference happens when AI models make predictions from new data. 

For example, with LLMs, Amazon found that home robots could go beyond simple detection of on-off or open-closed discoveries, such as spotting if a light was not switched off or a door left open. Instead, they could now handle more complex tasks like detecting broken glass on the floor and knowing that glass shards are dangerous and need to be swept up before someone steps on it, the documents show.

Amazon ran a conceptual demo to test its new home robot's ability to demonstrate inference and problem-solving capabilities. The test was a success. "You can imagine our excitement as we understood what this would mean for our robot," one of the documents said. "We know we still have a long way to go before Burnham can be put into a product but we have learned enough to be convinced it will work."

Like other teams at Amazon, the Astro unit faced severe cost pressures over the past year, according to a person familiar with the matter, who requested anonymity because they're not authorized to speak to the press. The team axed an internal plan to release a lower-cost model last year, and instead, announced a new focus on small business security.

The future of the Burnham project

Burnham may not be limited to just one product. In one of the internal documents, Amazon defines Burnham as a "set of core technologies — not a product in itself," and says it expects Burnham to "appear in a series of future robots."

Burnham could test the limits of large language models as Amazon attempts to move the AI technology out of the digital sphere and into real-life physical settings — in this case, into the center of your home.

Amazon appears convinced that Burnham is a significantly improved version of home monitoring that "brings peace of mind to the family that within their home, all is well," according to the documents. 

"Burnham's friendly personality, social awareness, and conversational fluency ensure it is exceptionally useful and would be welcome as a junior family member in any home," it said.