Jump to content

ATLAS

Sprachsteuerung von generativen künstlichen Intelligenzen in einer Augmented Reality-Umgebung

Fast facts

About the project

As part of their project work 1, students of the Bachelor's study program Digital Technologies have developed a voice control system for artificial intelligence in a...

What is it about?

As part of the first project, two students on the Bachelor's degree course in Digital Technologies developed the ATLAS application. This name stands for an "augmentation technology for linguistic analysis and control" and consists of an application for the HoloLens 2, an augmented reality headset from Microsoft. ATLAS enables its users to interact with the OpenAI interface for Generative Pretrained Transformer - and thus with generative artificial intelligence - by means of voice-based interaction.

In addition to general information output, the application can be used to launch various functions of the HoloLens 2 through spoken conversation. These functions include taking photos, generating images, visually displaying longer outputs and displaying the environment recognized by the HoloLens in the user's area.

How was it realized?

The application is based on the Unity game engine and the Mixed Reality Toolkit 2 (MRTK2) from Microsoft. With the help of MRTK, the output generated by OpenAI was visualized in an interactive augmented reality display. The transcription of the user's spoken input (conversion from speech to text) and the synthesizing of the output (conversion from text to speech) are also carried out by the language libraries within the MRTK. The C# programming language was used to implement the application.

Contact & Team

Students involved in the project

Tim Schlierkamp
Tim Blankenstein

Portrait von Karsten Lehn __ Portrait of Karsten Lehn
Prof. Karsten Lehn, Dr.-Ing.
Fax
  • +49 231 91128183
Office hours

by arrangement

This site uses cookies to ensure the functionality of the website and to collect statistical data. You can object to the statistical collection via the data protection settings (opt-out).

Settings(Opens in a new tab)