Carnegie Mellon University

A variety of colorpul pieces of chalk stand upright on a dark grey surface

Language Models with Combined Data Modalities for AI Systems

By Teruko Mitamura

This collaborative project between CMU and the National Institute of Advanced Industrial Science and Technology, Japan (AIST) seeks to develop multimodal systems for advanced human-machine interactions through discussion and implementation. Especially, the CMU team focuses on the temporal reasoning capabilities of multiple modalities. Multimodal systems with advanced temporal reasoning capability will be beneficial in analyzing and understanding different types of data jointly.