Publications
How Spatial Information Connects Visual Perception and Natural Language Generation in Dynamic Environments: Towards a Computational Model
Maass, W.
In: Frank, A. & Kuhn, W. (Hrsg.), A Theoretical Basis for GIS, Proc. of the Int. Conf. COSIT-95 Spatial Information Theory, Lecture Notes in Computer Science, Springer, 1995, 988, pp. 223-240.
Presents a computational model linking visual perception and natural language generation in spatial contexts.
Visual Grounding of Route Descriptions in Dynamic Environments
Maass, W., Baus, J. & Paul, J.
In: R. Srihari (Hrsg.), Proc. of the AAAI Fall Symposium on Computational Models for Integrating Language and Vision, Cambridge, MA, 1995.
Explores visual grounding for route descriptions in dynamic environments.
From visual perception to multimodal communication: Incremental route descriptions
Maass, W.
Artificial Intelligence Review Journal, 1994, 8(2-3), pp. 159-174.
In the last few years, within cognitive science, there has been a growing interest in the connection between vision and natural language. The question of interest is: How can we discuss what we see. With this question in mind, we will look at the area of incremental route descriptions. Here, a speaker step-by-step presents the relevant route information in a 3D-environment. The speaker must adjust his/her descriptions to the currently visible objects. Two major questions arise in this context:1. How is visually obtained information used in natural language generation? and 2. How are these modalities coordinated? We will present a computational framework for the interaction of vision and natural language descriptions which integrates several processes and representations. Specifically discussed is the interaction between the spatial representation and the presentation representation used for natural language descriptions. We have implemented a prototypical version of the proposed model, called MOSES.
In the last few years, within cognitive science, there has been a growing interest in the connection between vision and natural language. The question of interest is: How can we discuss what we see. With this question in mind, we will look at the area of incremental route descriptions. Here, a speaker step-by-step presents the relevant route information in a 3D-environment. The speaker must adjust his/her descriptions to the currently visible objects. Two major questions arise in this context:1. How is visually obtained information used in natural language generation? and 2. How are these modalities coordinated? We will present a computational framework for the interaction of vision and natural language descriptions which integrates several processes and representations. Specifically discussed is the interaction between the spatial representation and the presentation representation used for natural language descriptions. We have implemented a prototypical version of the proposed model, called MOSES.
VITRA GUIDE: Multimodal route descriptions for computer assisted vehicle navigation
Wolfgang Maaß; Peter Wazinski; Gerd Herzog; P. Chung (Hrsg.); G. L. Lovegrove (Hrsg.); Moonis Ali (Hrsg.)
P. Chung; G. L. Lovegrove; Moonis Ali (Hrsg.). Industrial and Engineering Applications of Artificial Intelligence and Expert Systems, Proceedings of the Sixth International Conference. International Conference on Industrial, Engineering & Other Applications of Applied Intelligent Systems (IEA / AIE-93), 6th, Edinburgh, United Kingdom, ISBN 2-88124-604-4, Gordon and Breach Publishing, 1993.
We present a system that generates multimodal route descriptions for computer assisted vehicle navigation. The presentation modes available in Vitra Guide are natural language, maps and perspective views. Depending on situationspecific constraints, e.g., fully specifed route descriptions for pre-trip planning or incremental route descriptions during driving, Vitra Guide allows the coordinated distribution of the route information over the available presentation modes. The coordinated use of the presentation modes is achieved by a plan-based approach. The output for all presentation modes relies on one common 3-dimensional geometric model of the domain.
We present a system that generates multimodal route descriptions for computer assisted vehicle navigation. The presentation modes available in Vitra Guide are natural language, maps and perspective views. Depending on situationspecific constraints, e.g., fully specifed route descriptions for pre-trip planning or incremental route descriptions during driving, Vitra Guide allows the coordinated distribution of the route information over the available presentation modes. The coordinated use of the presentation modes is achieved by a plan-based approach. The output for all presentation modes relies on one common 3-dimensional geometric model of the domain.
VITRA GUIDE: Utilisation du Langage Naturel et de Représentation Graphiques pour la Description d'Iténitaires
Herzog, G., Maass, W. & Wazinski, P.
Colloque Interdisciplinaire du Commitée National 'Images et Langages: Multimodalitée et Modélisation Cognitive', 1993, pp. 243-251.
Describes the VITRA GUIDE system for multimodal route descriptions using natural language and graphical representations.
Constraint-basierte Plazierung in multimodalen Dokumenten am Beispiel des Layout-Managers in WIP..
Wolfgang Maaß
DFKI, DFKI Documents (D), Vol. 92-02, 1992.
Bei innovativen intelligenten Benutzerschnittstellen, wie im Beispiel des multimodalen Präsentationssystems WIP, spielt insbesondere die automatische Plazierung von Graphiken und Texten eine wichtige Rolle. Das komplexe Plazierungsproblem läßt sich dabei als Constraint-Satisfaction-Problem auffassen. Zu dessen Lösung haben wir das System CLAY, welches ein integraler Bestandteil des Layout-Managers von WIP ist, entwickelt. Das Constraint-Solver-Modell CLAY, basierend auf der Kopplung zweier dedizierter Constraint-Solver, erlaubt die effiziente Verarbeitung komplexer graphischer Beziehungen, wie sie besonders im funktionalen Layout vorherrschen. CLAY stellt einen effizienten und flexiblen Mechanismus zur deklarativen Spezifikation und Lösung graphischer Gestaltungsprobleme dar. In dieser Arbeit werden dem Constraint-Solver-Modell zugrundeliegende abstrakte Algorithmen sowie den Constraint-Definitionssprachen vorgestellt und an Hand von Beispielen illustriert.
Bei innovativen intelligenten Benutzerschnittstellen, wie im Beispiel des multimodalen Präsentationssystems WIP, spielt insbesondere die automatische Plazierung von Graphiken und Texten eine wichtige Rolle. Das komplexe Plazierungsproblem läßt sich dabei als Constraint-Satisfaction-Problem auffassen. Zu dessen Lösung haben wir das System CLAY, welches ein integraler Bestandteil des Layout-Managers von WIP ist, entwickelt. Das Constraint-Solver-Modell CLAY, basierend auf der Kopplung zweier dedizierter Constraint-Solver, erlaubt die effiziente Verarbeitung komplexer graphischer Beziehungen, wie sie besonders im funktionalen Layout vorherrschen. CLAY stellt einen effizienten und flexiblen Mechanismus zur deklarativen Spezifikation und Lösung graphischer Gestaltungsprobleme dar. In dieser Arbeit werden dem Constraint-Solver-Modell zugrundeliegende abstrakte Algorithmen sowie den Constraint-Definitionssprachen vorgestellt und an Hand von Beispielen illustriert.
LAYLAB: Ein System zur automatischen Plazierung von Text-Bild-Kombinationen in multimodalen Dokumenten
Maass, W., Schiffmann, T., Soetopo, D. & Graf, W.
1992.
Develops LAYLAB, a system for automatic placement of text-image combinations in multimodal documents.
Constraint-basierte Verarbeitung graphischen Wissens
Graf, W. & Maass, W.
In: Brauer, W. & Hernández, D. (Hrsg.), Proc. of 4. Internationaler GI-Kongreß Wissensbasierte Systeme – Verteilte KI und kooperatives Arbeiten, Springer, 1991, pp. 243-253.
Explores constraint-based processing of graphical knowledge in distributed AI systems.