DARPA Contracts Xerox to Build Augmented Reality Trainer AI

The U.S Defense Advanced Research Projects Agency (DARPA) has chosen Xerox’s Paolo Alto Research Center (PARC) to create an augmented reality system with an AI to convert text and video instruction guides into an interactive digital teacher.

AMIGOS AR

PARC is working with the University of California at Santa Barbara, the University of Rostock and augmented reality developer Patched Reality to come up with the system. The idea is that trainers could feed a bunch of text manuals and video instructions into an AI that could then parse them well enough to give real-time instruction to a student wearing smart goggles or a similar headset with conversational instructions and visual aides generated from the uploaded instructions. The guidance could then be personalized to the user’s own expertise and enable people to perform duties they have no training in, from mechanics to medical care.

The project is officially titled the Autonomous Multimodal Ingestion for Goal-Oriented Support or AMIGOS. The friendly acronym indicates the kind of support the government hopes PARC and its partners will produce. An offline system will extract information from multiple sources to complete tasks and an online hybrid AI system for more interactive and personalized AR guidance. DARPA awarded a $5.8 million contract to PARC as the lead contractor for the new system, which is likely more to get the concept off the ground and see if it’s worthwhile putting more government research money into it.

“Augmented reality, computer vision, language processing, dialogue processing and reasoning are all AI technologies that have disrupted a variety of industries individually but never in such a coordinated and synergistic fashion,” AMIGOS principal investigator Dr. Charles Ortiz said. “By leveraging existing instructional materials to create new AR guidance, the AMIGOS project stands to accelerate this movement, making real-time task guidance and feedback available on-demand.”

DARPA Guides

The AMIGOS project falls under DARPA’s Perceptually-enabled Task Guidance (PTG) program, which started soliciting ideas like AMIGOS earlier this year.  The goal is an AI assistant for the military that can help with the increasingly complex tasks that make a modern military function. Information and coordination support need AR and other tools to help avoid mistakes and answer questions. DARPA has a long history in the field, helping fund the research that led to Siri, and developing the Personalized Assistant that Learns (PAL) two decades ago.

The military is exploring other uses for AI assistants too. Lockheed Martin is training F-35 Lightning II pilots with a voice assistant synthetically generated by digital voice interface creator ReadSpeaker. The Navy is searching for a virtual assistant to help run sonar scans for submarines. Meanwhile, Microsoft won a U.S. Army contract worth a potential $22 billion to produce a version of its Hololens 2 smart goggles specifically for military use. The Integrated Visual Augmentation System (IVAS) devices provide better vision and more information about the environment around the soldier, with a built-in voice assistant and tactile controls to operate the modified headset.

Follow @voicebotai   Follow @erichschwartz

US Military Unveils Plans for Developing an AI Assistant at DARPA

Microsoft Scores $22B Army Contract for Hololens-Based AR Smart Googles

Lockheed Martin Picks Readspeaker’s Synthetic Voice Tech to Train F-35 Pilots