HFT TransferBot

The overall aim of the project is to develop and investigate a ChatBot for HFT Stuttgart,
focusing on the use case of making the research results created at HFT Stuttgart more easily accessible to the public.
With regard to this application purpose, existing Large Language Models (LLMs) and their parametrisations will be validated,
the relevant data determined and aggregated, and a TransferBot prototype will be implemented and evaluated.

1. Validation of existing LLMs and determining suitable parametrisations for their use as a TransferBot on the HFT Stuttgart homepage
2. Definition and aggregation of the relevant data (e.g. from the homepage, from research management, etc.) in cooperation with the relevant experts at the university and preparation in a suitable database
3. Implementation of a running prototype, experiments with licence-based, freely available and, if necessary, self-operated language models, as well as various processing methods for the information to be covered
4. Evaluation of the TransferBot for correctness and coverage of the answers
5. Practical creation of the technical resources identified as necessary (knowledge database, LLM operation if necessary) and integration of the TransferBot into the HFT Stuttgart homepage

ChatBots based on the latest technical developments in the field of LLMs promise easy access to information. In contrast to predefined texts, e.g. on a project homepage, or a keyword search, LLMs formulate customisable and individual, tailor-made answers to user queries. The technology is therefore also suitable as a channel for communicating research results to the scientific community and society in general in an easily accessible way, as envisioned at HFT Stuttgart.

An important challenge in the development of such a TransferBot is the tendency of LLMs to hallucinate information and present it in such a credible manner that the misinformation is not recognisable at first glance. Such a hallucinating TransferBot would of course be damaging and counterproductive. Therefore, the Retrieval Augmented Generation (RAG) approach will be chosen, in which relevant and reliable information is identified in an upstream step and the LLM only has the task of presenting this information in a fluent and comprehensible way. The RAG approach not only ensures that the TransferBot's output is correct in terms of content – it also protects it to a great extent against malicious Prompt injection. This is a strategy of generating inappropriate or offensive content through fictitious requests, on which of course no information exist in the ChatBot’s trusted data collection. This makes the choice and pre-processing of the reliable documents key to the method’s output quality.

Another important consideration is the provenance and quality of the LLM used. Licence-based models are guaranteed to be of high quality and easy to integrate, but cause operating costs and are questionable from the perspective of data and knowledge protection. Therefore, if possible, a freely available (open source) LLM should be used that can be operated at a trustworthy location (possibly also by HFT Stuttgart itself). However, this requires experiments to compare quality, for example by extrinsic evaluation of the model’s reaction to user prompts.

Reliable knowledge about the quality and usefulness of the LLMs under consideration as well as their parametrization (e.g. through prompting) 
in the context of the planned use case has been gained. 
An initial database has been filled with sufficient amounts of data for the prototype, and requirements for further data collection is derived. 
The prototype has been evaluated and has been integrated into the HFT Stuttgart homepage structure.