Distinguished Seminar Series on Data Science & Artificial Intelligence - "Unlocking multimodal intelligence with large language model"
Research Seminar
-
Date
01 Apr 2026
-
Organiser
Department of Computing
-
Time
10:00 - 11:30
-
Venue
Online via Zoom / HJ305
Speaker
Dr Jinyu Li
Summary
Large Language Models (LLMs) have made a notable impact on natural language processing (NLP) and are beginning to shape the field of speech processing. This presentation explores how LLMs are influencing multimodal intelligence by tracing the evolution from spoken language models (SLMs) to multimodal LLMs (MLLMs), with examples from Microsoft's model development journey. The talk covers differences in model architectures, recent progress in speech generation and comprehension, and how speech and text are being integrated. Finally, it highlights the latest advancements aimed at enhancing the open-source Phi-4-multimodal model.
Keynote Speaker
Dr Jinyu Li
Partner Applied Science Manager
Microsoft
USA
Dr Jinyu Li is currently the Partner Applied Science Manager at Microsoft in Redmond, WA, USA, where he oversees a scientific team focused on developing and advancing speech modeling algorithms and technologies. Dr. Li is an IEEE Fellow, for contributions to deep-learning-based speech technology innovation and commercialization. Dr. Li served as a member of the IEEE Speech and Language Processing Technical Committee from 2018 to 2023 and as Vice Chair starting in 2026. He was an Associate Editor for the IEEE/ACM Transactions on Audio, Speech and Language Processing from 2015 to 2020 and acted as Technical Program Chair for IEEE SLT in 2021 and 2026, as well as IEEE ASRU in 2023 and 2025. He is a Distinguished Industry Speaker of the IEEE Signal Processing Society in 2025. He also received the IEEE SPS Best Paper Award in 2025. Additionally, Dr Li was honored as the Industrial Distinguished Leader at the Asia-Pacific Signal and Information Processing Association (APSIPA) in 2021 and received the APSIPA Sadaoki Furui Prize Paper Award in 2023.
