Chair of Business Informatics
Saarland University

Lecture on Data Science at Universität des Saarlandes is going to start this week. Don't miss it. This lecture targets students in business informatics, computer science, bioinformatics, computer linguistics, and beyond. After last year collaboration with Bosch and HYDAC Group, this time projects will be conducted in collaboration with the Boston Consulting Group (BCG).

Streamlining LLMs: Adaptive Knowledge Distillation for Tailored Language Models

Prajvi Saxena, Sabine Janzen, Wolfgang Maass:

Large language models (LLMs) like GPT-4 and LLaMA-3 offer transformative potential across industries, e.g., enhancing customer service, revolutionizing medical diagnostics, or identifying crises in news articles. However, deploying LLMs faces challenges such as limited training data, high computational costs, and issues with transparency and explainability. Our research focuses on distilling compact, parameter-efficient tailored language models (TLMs) from LLMs for domain-specific tasks with comparable performance. Current approaches like knowledge distillation, fine-tuning, and model parallelism address computational efficiency but lack hybrid strategies to balance efficiency, adaptability, and accuracy. We present ANON - an adaptive knowledge distillation framework integrating knowledge distillation with adapters to generate computationally efficient TLMs without relying on labeled datasets. ANON uses cross-entropy loss to transfer knowledge from the teacher's outputs and internal representations while employing adaptive prompt engineering and a progressive distillation strategy for phased knowledge transfer. We evaluated ANON's performance in the crisis domain, where accuracy is critical and labeled data is scarce. Experiments showed that ANON outperforms recent approaches of knowledge distillation, both in terms of the resulting TLM performance and in reducing the computational costs for training and maintaining accuracy compared to LLMs for domain-specific applications.

Energy-efficient artificial intelligence for sustainable data centers

The research project ESCADE is presented at Hannover Messe 2025 from March 31 to April 4 at the booth of the Federal Ministry for Economics and Climate Change (Hall 2, Booth A18).

The climate neutrality goals, rising energy costs and the limitations of traditional hardware are jeopardizing the economic viability of German data centers, which are vital for the digital transformation using AI. Their massive energy consumption (16 billion kWh in 2020) is set to be reduced through the use of neuromorphic hardware and energy-efficient AI algorithms.

ESCADE optimizes the energy efficiency of AI in data centers. Approaches such as knowledge distillation, quantization and neural architecture search result in smaller, more energy-efficient AI models that nevertheless deliver a comparable performance. When combined with neuromorphic hardware, these models can achieve energy savings of up to 90%.

An ESCADE AI agent optimizes the energy efficiency of algorithms throughout their development, training and deployment. By making data-driven decisions, it enables resource-efficient solutions, for example in scrap sorting and software development, without performance loss. Small and medium-sized enterprises, in particular, benefit from AI that is both economically and environmentally sustainable.

The ESCADE exhibit visualizes the project results in the form of a physical table demonstrator. The technical prototype EAVE - Energy Analytics for Cost-Effective & Sustainable AI Operations empowers decision-makers with real-time insights into the energy efficiency of AI workloads. The tool supports measuring, predicting and optimizing operational costs, energy-efficiency and CO2 emissions of AI models including the optimization of data center power usage effectiveness (PUE). The EAVE measurement module analyzes current cost- and energy-efficiency, while the prediction module enables analysis of future AI workloads with respect to cost and energy. The optimization module analyses in-depth AI model optimization, focusing on compression techniques for Large Language and Vision Models in the context of industry use cases.

ESCADE is funded from 01.05.2023 - 30.04.2026 by Federal Ministry for Economics and Climate Change (BMWK). The research team of Prof. Maaß at German Research Center for Artificial Intelligence (DFKI) is coordinating the project and collaborates with the funded partners NT Neue Technologie AG, Stahl-Holding-Saar GmbH & Co. KGaA, SEITEC GmbH, Technical University Dresden, University Bielefeld and the Austrian Partner Salzburg Research.