SensAR

The project SensAR visualizes sensor data in industrial environments via augmented reality (AR) applications. The research focuses on generalized industrial processes with selected use cases as a basis. Its aim is to support workers using sensor data and digitized assistance systems. The user stories are defined in cooperation with industrial partners.

The goal of the project is to develop sample applications for augmenting the real world with meaningful, context-sensitive information based on retrieved sensor data. The main focus currently lies in supporting three different use cases; facility management, classification and modelling of industrial objects, indoor navigation. The project is a holistic approach that combines research areas such as dynamic object recognition, positioning, sensor technology and standards, UI and data security.

Parts of the project:

• Object recognition, for example of equipment or load carriers in industrial halls, from laser point clouds or cameras (Prof. Eberhard Gülch, Prof. Ursula Voß, Prof. Franz-Josef Schneider)
• Localization of employees via smartphone using indoor positioning (Prof. Stefan Knauth)
• Retrieval of data from sensors with different standards (Prof. Dieter Uckelmann)
• User interface displaying sensor data and control elements for user input (Prof. Volker Coors, Prof. Gero Lückemeyer)
• Data security and data protection (Prof. Jan Seedorf)

Using three example scenarios, possibilities for augmented reality applications will be presented:

• Facility management: display sensor data and derived information for monitoring machinery
• Industry modelling: such as the detection and modelling of industrial building equipment and load carriers based on laser point clouds and photogrammetry
• Indoor navigation: A specialist needs support at a machine. Using a smartphone app with indoor localization, the employee with the shortest route can be alerted. Alternatively, remote assistance can be used to provide help in troubleshooting.

Four different use cases were identified and processed, covering a range of technological requirements: objects and sensors from stationary to mobile, required display technology from conventional to AR. The four use cases are RFID measurement chamber, factory layout, load carrier and e-bike. Prototypes were implemented and evaluated for these use cases, which successfully linked the technology components required for the respective use case, from AI-supported object recognition to sensor data exchange and user interface to security requirements. In particular, the RFID measuring chamber and the prototype of an AR-supported operating aid and route navigation to the measuring chamber created for it turned out to be a particularly representative use case. On the one hand, it represents common industrial scenarios such as route guidance to and status display of machines and also requires the linking of a large number of different technology components.

The transfer events, which included demonstrations and discussions in the form of workshops, made it possible to obtain feedback from outside in general and from invited SMEs in particular. This feedback confirmed the prototypical character of the use cases and triggered an exchange of ideas and knowledge. However, the feedback also showed that, as expected, very specific questions arise in industry and especially among SMEs, which require a specifically coordinated implementation in each case. However, this coordination is much easier to implement due to the openly implemented basis and the previous focus on generalisable processes. Within the project, the feasibility of an AR-based visualisation of sensor data in the Industrie 4.0 environment was demonstrated, which outlines the path for future product developments in this area. The goal of providing SMEs with a low-threshold entry into the topic was thus achieved. Further details, such as a film on the path or the final report with the individual results of the work packages, can be found at sensar.hft-stuttgart.de

The following publications/contributions in the form of conference participation, workshops and publications have resulted from the project:

• Jensen, M. P.; Uckelmann, D. (2020) Standards für das Internet der Dinge: Heterogenität, Interoperabilität und Herausforderungen. ZWF Zeitschrift für wirtschaftlichen Fabrikbetrieb, 115 (3), S. 171-174. DOI: 10.3139/104.112245
• Abbasi Zadeh Behbahani, E; Adineh, H.; Uckelmann, D.; Jensen, M. P.  (2022). Digitalization of an Indoor-Positioning Lab Using a Mobile Robot and IIoT Integration. In D. Uckelmann, G. Romagnoli, J. Baalsrud Hauge, V. Kammerlohr (Hrsg.) Online-Labs in Education, Proceedings of the 1st International Conference on Online-Labs in Education, 10.-12. 03. 2022, Stuttgart. Nomos, Baden-Baden, ISBN 978-3-98542-036-0.
• Stefan Knauth: Evaluation of Geomagnetic Matching Algorithms for Indoor Positioning, Vortrag und Poster, IPIN 2021 Conference , November 29 – December 2, 2021, Lloret de Mar, Spain, http://ceur-ws.org/Vol-3097/paper31.pdf
• Gemeinsame Veröffentlichung eines Whitepapers mit dem VDC, der Holo-Light GmbH sowie der Trumpf Tracking Technologies GmbH in Business Geometrics 1/22 „Standardisierte Indoor-Ortung mit omlox. Exakte Lokalisierung für Augmented-Reality-Anwendungen im Bereich Industrie 4.0“, für die HFT Stuttgart Volker Coors, Dieter Uckelmann, Stefan Knauth , 31.03.2022 (https://www.vdc-fellbach.de/nachrichten/2022/01/31/whitepaper-zur-standardisierten-indoor-ortung-mit-omlox/ (abgerufen am 30.11.2022))
• Lars Sören Obrock und Colien Schreiber: Posterpräsentation zum Thema Objekterkennung auf dem Dialogforum Region Stuttgart zum Thema „Künstliche Intelligenz: Technologien begreifen – Potenziale nutzen“, 22.11.2019 an der Hochschule für Technik Stuttgart. https://www.hft-stuttgart.de/forschung/news/dialogforum-region-stuttgart-kuenstliche-intelligenz  (abgerufen am 7.3.2022)
• Stefan Knauth: Evaluation of Geomagnetic Matching Algorithms for Indoor Positioning, Vortrag und Poster, IPIN 2021 Conference , November 29 – December 2, 2021, Lloret de Mar, Spain, http://ceur-ws.org/Vol-3097/paper31.pdf
• Stefan Knauth: Evaluation of Geomagnetic Matching Algorithms for Indoor Positioning, oral presentation and poster, IPIN 2021 Conference , November 29 – December 2, 2021, Lloret de Mar, Spain, http://ceur-ws.org/Vol-3097/paper31.pdf
• Marc-Philipp Jensen: “Digitalization of an Indoor-Positioning Lab Using a Mobile Robot and IIoT Integration”, oral presentation at the 1st International Conference on Online-Labs in Education, 10.-12. März 2022, Stuttgart.
• Max Pengrin: „Functional Encryption & Homomorphic Encryption for RIOT“, oral presentation at the RIOT Summit 2020
• IHK-Webinare mit HFT-Forschenden zum Projekt „SensAR“. https://www.hft-stuttgart.de/forschung/news/ihk-webinar-mit-hft-forschenden-zum-projekt-sensar-augmented-reality-loesungen-fuer-kleinere-und-mittlere-unternehmen. (abgerufen am 7.3.2022)
  • 24.03.2021: Inhaltlicher Fokus auf die Objekterkennung und Visualisierung, Vorstellung der Arbeiten und bisherigen Ergebnisse durch die beteiligten Professor:innen und Mitarbeiter:innen
  • 14.04.2021: Inhaltlicher Fokus auf Lokalisation, Sensoren und Datensicherheit, Vorstellung der Arbeiten und bisherigen Ergebnisse durch die beteiligten Professor:innen und Mitarbeiter:innen

• Hightechsummit2021. https://hightech-summit.de/tag2/ (retrieved 7.3.2022)
  • 19.10.2021: Presentation of the project and results to date at the Techbreakfast under the title "Sensory Data and AR in Manufacturing", presentation by the professors and staff involved
• XR Week 2022:
  • 14.09.2022: oral presentation as part of the congress programme, "The SensAR project and further work on AR in urban planning / participation", Prof. Dr. V. Coors
  • 14-15.10.2022: XR Expo exhibition booth on the SensAR project, stand supervision by all participating professors