In April 2020, Sony launched Sony AI, an organization with offices in Japan, the U.S., and Europe undertaking AI R&D with a focus on gaming, imaging, sensing, and more in collaboration with Sony’s other business units. Today, Sony AI unveiled its first major research initiative in the Gastronomy Flagship Project, which aims to enhance the “creativity and techniques” of chefs around the world.
Sony began investing in AI with cooking applications in April 2018, when the company’s U.S. division — Sony Corporation of America — signed an agreement with Carnegie Mellon University to work on AI and robotics research. At the time, Sony said its initial research and development would look into optimizing food prep, cooking, and delivery because the technology needed for a robot to handle “the complex and varied task of food preparation” could in turn be applied to a wider range of industries. According to a McKinsey & Company analysis, 73% of the activities food service workers perform have the potential for automation.
The Gastronomy Flagship Project comprises an AI-powered app for creating recipes, a robot that can assist chefs with cooking, and a “community co-creation” plan that’ll serve as the foundation for the first two efforts.
To create the app, Sony AI says it’ll leverage a range of data sources including including recipes and ingredient information like taste, aroma, flavor, molecular structure, and nutrients to train “proprietary AI algorithms” that’ll assist “the world’s top chefs” in pairing, recipe design, and menu creation. The company notes that recipe creation is a challenging research area for AI because there exists an infinite combination of ingredients as well as constraints such as location, climate, season, and health and food preferences that must be taken into account. “Through this app, [we aim] not only to assist in making delicious food, but also to contribute to people’s health and the sustainability of the environment,” Sony.AI wrote.
Above: Sony AI’s gastronomy app concept.
An AI system that can whip up food pairings and recipes might not be as far-fetched as it seems. IBM recently announced that it’s teaming up with McCormick & Company to create new flavors and foods with machine learning. IBM’s Chef Watson, a research project that sought to create new recipes by analyzing the chemical composition of hundreds of different ingredients, produced more than 10,000 novel dishes. And New York startup Analytical Flavor Systems’ platform — Gastrograph — taps sensory data and machine learning algorithms to suss out products’ flavor profiles and identify areas for improvement. There’s also Los Angeles-based Halla’s I/O platform, which uses AI to generate Netflix-like recommendations for grocery, restaurant, and food delivery apps and websites, in part by leveraging a database of restaurant dish, recipe, ingredient, and grocery item taste and flavor attributes. Others like Foodpairing, Plant Jammer, Tastewise, and Dishq offer proprietary recommendation systems that take into account personal preferences.
As for Sony AI’s forthcoming cooking robot, tentatively dubbed Chef Assisting Cooking Robot, Sony says it’ll help through the entire cooking process from prep to plating through the use of sensors and machine learning approaches like imitation learning. The ambitious goal is to to replicate and in some cases exceed the skills and techniques of chefs with “high precision and speed,” and to develop robots that can be teleoperated to serve meals to people in remote locations.
Sony AI says that it’ll support these initiatives with partnerships involving with universities, research institutes, and companies “at the forefront” of machine learning research. The division is also pledging to “drive dialogue” with creators and experts in a “wide range” of food-related fields like Eneko Atxa, chef owner of the three-Michelin-stars restaurant Azurmendi in Bilbao, Spain; sommelier, chef, author, and aromas specialist François Chartier; and restauranteur Joan Roca, and to apply those learnings to the development of AI apps and robots.
Robots in the kitchen are nothing new. Just last year at the Consumer Electronics Show, Samsung unveiled Chef Bot, a collaborative robot with arms designed to help chefs with ingredient prep. Startups like Chowbotics, Creator, Miso Robotics, and Picnic have taken the concept commercial with fast-service robots that can quickly prepare pizzas, french fries, burgers, and more fresh to order. But some robotic chef concepts have struggled to get off the ground. In June, Zume, a robotics startup focused on pizza-making, shut down its business and pivoted to food packaging, production, and delivery systems. Sony’s multifaceted approach is perhaps reflective of the reality that robots designed for the kitchen aren’t a surefire investment.
