Mengshi Qi

I am currently a Professor in School of Computer Science at Beijing University of Posts and Telecommunications (BUPT) where I lead the Multimedia Intelligent Computing Lab (MICLAB@BUPT).

Prior to that, I have ever worked as a post-doc research scientist in CVLab at École polytechnique fédérale de Lausanne (EPFL) from 2019 to 2021, working closely with Prof. Pascal Fua and Dr. Mathieu Salzmann. And I have worked in Baidu Research, where I focus on computer vision and deep learning collaborated with Prof. Yi Yang in 2019.

I did my PhD and Master at Beihang University (BUAA) in 2019 and 2014, respectively, where I was advised by Prof. Yunhong Wang. Especially, I had been a visiting PhD at University of Rochester supervised by Prof. Jiebo Luo from 2017 to 2018. I did my bachelors at Beijing University of Posts and Telecommunications (BUPT) in 2012.

Email  /  CV  /  Google Scholar  /  LinkedIn  /  DBLP

I'm interested in computer vision and machine learning, especially scene understanding, 3D reconstruction and multimedia analysis. Most of my research are about how to understand the semantic content and infer the physical information from images and videos.

[2026.10.4] One paper is accepted by IEEE TPAMI.

[2026.9.25] One paper is accepted by NeurIPS 2026 Evaluations and Datasets Track.

[2026.8.30] One paper is accepted by IEEE TMM.

[2026.8.10] One paper is accepted by IEEE TIP.

[2026.7.10] One paper is accepted by ACM Multimedia 2026.

[2026.6.18] One paper is accepted by ECCV 2026.

[2026.6.2] One paper is accepted by IEEE TIP.

[2026.5.1] One paper is accepted by ICML 2026.

[2026.2.21] Two papers are accepted by CVPR 2026.

[2026.2.5] Honor to recive the 2026 AAAI Program Committe Award.

[2025.12.17] One paper is accepted by IEEE TPAMI.

[2025.11.8] One paper is accepted by AAAI 2026.

[2025.10.27] One paper is accepted by IEEE TMC.

[2025.10.25] One paper is accepted by IEEE TPAMI.

[2025.9.19] One paper is accepted by NeurIPS 2025.

[2025.9.18] One paper is accepted by IEEE TIP.

[2025.7.5] One paper is accepted by ACM Multimedia 2025.

[2025.2.27] Two papers are accepted by CVPR 2025.

[2024.12.9] Two papers are accepted by AAAI 2025.

[2024.7.1] Two papers are accepted by ECCV 2024.

[2024.5.15] Call for papers on IEEE TMM Special Issue on Large Multi-modal Models for Dynamic Visual Scene Understanding, please refer to https://signalprocessingsociety.org/blog/ieee-tmm-special-issue-large-multi-modal-models-dynamic-visual-scene-understanding.

In this paper, we propose a novel diffusion-based trajectory prediction framework that integrates group-level interactions into a conditional diffusion model, enabling the generation of diverse trajectories aligned with specific group activity.

In this paper, we propose the Dual Consistency SAM (DC-SAM) method based on prompt-tuning to adapt SAM and SAM2 for in-context segmentation of both images and videos..

In this paper, we propose a new Robust Disentangled Counterfactual Learning (RDCL) approach for physical audiovisual commonsense reasoning.

In this paper, we propose a new few-shot temporal action localization method by Chain-of-Thought textual reasoning to improve localization performance. We introduce the first dataset named Human-related Anomaly Localization and explore the application of the TAL task in human anomaly detection.

In this paper, we define a new Human Action Form Assessment (AFA) task, and introduce a new diverse dataset CoT-AFA, which contains a large scale of fitness and martial arts videos with multi-level annotations for comprehensive video analysis.

In this paper, we introduce a novel balanced continual multi-modal learning method for 3D HPE, which harnesses the power of RGB, LiDAR, mmWave, and WiFi.

In this paper, we introduce a novel method named Robo-SGG, i.e., Layout-Oriented Normalization and Restitution for Robust Scene Graph Generation. Compared to the existing SGG setting, the robust scene graph generation aims to perform inference on a diverse range of corrupted images, with the core challenge being the domain shift between the clean and corrupted images.

In this work, we formalize Multimodal Fine-grained Traffic ElementDetection (MFTED), a practical task dedicated to identifying traffic targets bysynergizing nuanced textual and visual prompts.

In this work, we propose VTO, a process-supervised reinforcement learning framework. Moving beyond static tool usage, VTO enables the agent to dynamically explore and interact with the environment.

Inspired by pigeons’ building and exploiting cognitive maps for navigation, in this work we propose a novel agentic pipeline for spatial reasoning with VLMs.

In this paper, we propose CritiqueDriveVLM, a unified three-stage autonomous driving framework internalizing reasoning directly into the VLM.

In this paper, we propose a novel semi-supervised 2D human pose estimation method by utilizing a newly designed Teacher-Reviewer-Student framework.

In this paper, we introduce a LearnableBN layer based on Generalized-search Entropy Minimization (GSEM) method, and a new semantic-consistency based dual-stage-adaptation strategy.

In this paper, we propose a novel action quality assessment method through hierarchically pose-guided multi-stage contrastive regression.

In this paper, we propose a new synergistic tensor and pipeline parallelism schedule that simultaneously reduces both types of bubbles.

In this paper, we introduce a novel physics-informed temporal network~(PITN) with adversarial contrastive learning to enable precise Blood Pressure estimation with very limited data.

In this paper we instantiate benchmark (SafeDrive228K) and propose a VLM-based baseline with knowledge graph-based retrieval-augmented generation (SafeDriveRAG) for visual question answering (VQA).

This paper considers 3D indoor scene generation as a planning problem subject to spatial and layout common sense constraints. To solve the problem with a VLM, we propose a new global-local tree search algorithm.

In this paper, we defined a novel Traffic Topology Scene Graph, a unified scene graph explicitly modeling the lane, controlled and guided by different road signals (e.g., right turn), and topology relationships among them, which is always ignored by previous high-definition (HD) mapping methods.

In this paper, to address the challenges posed by the fine-grained and interrelated vision tool usage of VIoT, we build VIoTGPT, the framework based on LLMs to correctly interact with humans, query knowledge videos, and invoke vision models to accomplish complicated tasks.

In this paper, we pioneer a cloud-edge collaborative inference framework for ReID systems and particularly propose a distribution-aware correlation modeling network (DaCM) to make the desired image return to the cloud server as soon as possible via learning to model the spatial-temporal correlations among instances.

In this paper, we propose a novel semi-supervised method, which can be utilized for better assessment of the AQA task by exploiting a large amount of unlabeled data and a small portion of labeled data.

In this paper, we propose a novel Decomposed Vector-Quantized Variational Autoencoder~(DVQ-VAE) to address this limitation by decomposing hand into several distinct parts and encoding them separately.

In this paper, we propose the semantic graph Transformer (SGT) for 3D scene graph generation. The task aims to parse a cloud point-based scene into a semantic structural graph, with the core challenge of modeling the complex global structure.

In this paper, we propose a novel weakly supervised temporal action localization method by inferring salient snippet-feature.

In this paper, we design a novel two-branch end-to-end fusion network named RDFC-GAN, which takes a pair of RGB and incomplete depth images as input to predict a dense and completed depth map.

In this paper, we propose a novel approach, Mutual Distillation Learning For Person Re-identification (termed as MDPR), which addresses the challenging problem from multiple perspectives within a single unified model, leveraging the power of mutual distillation to enhance the feature representations collectively.

In this paper, we propose a Disentangled Counterfactual Learning~(DCL) approach for physical audiovisual commonsense reasoning. The task aims to infer objects' physics commonsense based on both video and audio input, with the main challenge is how to imitate the reasoning ability of humans.

In this paper, we are the first to introduce an unsupervised way to predict self-driving attention by uncertainty modeling and driving knowledge integration.

In this paper, we propose GaitReload, a post-processing adversarial defense method to defend against AWP for the gait recognition model with sequenced inputs.

In this paper, we design a novel two-branch end-to-end fusion network, which takes a pair of RGB and incomplete depth images as input to predict a dense and completed depth map.

In this paper, we propose a novel binary representation learning framework, named Semantics-aware Spatial-temporal Binaries (S²Bin), which simultaneously considers spatial-temporal context and semantic relationships for cross-modal video retrieval. By exploiting the semantic relationships between two modalities, S²Bin can efficiently and effectively generate binary codes for both videos and texts. In addition, we adopt an iterative optimization scheme to learn deep encoding functions with attribute-guided stochastic training.

In this paper, we present a novel Latent Memory-augmented Graph Transformer (LMGT), a Transformer based framework including a designed graph encoding module and a latent memory unit for visual story generation.

In this paper, we proposed a novel Imitative Non-Autoregressive Modeling method to bridge the performance gap between autoregressive and non-autoregressive models for temporal sequence forecasting and imputation. Our proposed framework leveraged an imitation learning fashion including two parts, i.e., a recurrent conditional variational autoencoder (RC-VAE) demonstrator and a nonautoregressive transformation model (NART) learner.

In this paper, we address few-shot video classification by learning an ensemble of SlowFast networks augmented with memory units. Specifically, we introduce a family of few-shot learners based on SlowFast networks which are used to extract informative features at multiple rates, and we incorporate a memory unit into each network to enable encoding and retrieving crucial information instantly.

In this paper, we present a novel Generative Adversarial Networks-based model (i.e., STC-GAN) for predictive scene parsing. STC-GAN captures both spatial and temporal representations from the observed frames of a video through CNN and convolutional LSTM network. Moreover, a coupled architecture is employed to guide the adversarial training via a weight-sharing mechanism and a feature adaptation transform between the future frame generation model and the predictive scene parsing model.

In this study, we present a novel hierarchical recurrent neural network based framework with an attention mechanism for sports video captioning, in which a motion representation module is proposed to capture individual pose attribute and dynamical trajectory cluster information with extra professional sports knowledge, and a group relationship module is employed to design a scene graph for modeling players’ interaction by a gated graph convolutional network.

In the paper, we present a novel attentive semantic recurrent neural network (RNN), namely stagNet, for understanding group activities and individual actions in videos, by combining the spatio-temporal attention mechanism and semantic graph modeling. Specifically, a structured semantic graph is explicitly modeled to express the spatial contextual content of the whole scene, which is afterward further incorporated with the temporal factor through structural- RNN.

In this study, we propose a novel Attentive Relational Network that consists of two key modules with an object detection backbone to approach this problem. The first module is a semantic transformation module utilized to capture semantic embedded relation features, by translating visual features and linguistic features into a common semantic space. The other module is a graph self-attention module introduced to embed a joint graph representation through assigning various importance weights to neighboring nodes.

In this paper, we propose a novel Knowledge Embedded Generative Adversarial Networks, dubbed as KE-GAN, to tackle the challenging problem in a semi-supervised fashion. KE-GAN captures semantic consistencies of different categories by devising a Knowledge Graph from the large-scale text corpus.

We propose a novel attentive semantic recurrent neural network (RNN), dubbed as stagNet, for understanding group activities in videos, based on the spatio-temporal attention and semantic graph.

In this paper, we present a novel hierarchical recurrent neural network (RNN) based framework with an attention mechanism for sports video captioning. A motion representation module is proposed to extract individual pose attribute and group-level trajectory cluster information.

We propose a new framework for online cross-modal scene retrieval based on binary representations and semantic graph. Specially, we adopt the cross-modal hashing based on the quantization loss of different modalities. By introducing the semantic graph, we are able to extract wealthy semantics and measure their correlation across different modalities.

In this paper, we introduce a novel framework towards scene classification using category-specific salient region(CSSR) with deep CNN features, called Deep-CSSR.

In this work, we introduce a novel self-training pipeline using unpaired real all-weather data through multi-teacher distillation and robust radar fusion.

To address the practical challenge of incremental 3D Scene Graph Generation, we equip SGFormer++ with a novel Spatial-guided Feature Adapter.

In this paper, we propose a novel Global-Local Monte Carlo Tree Search in Vision-Language Models for Text-to-3D Indoor Scene Generation.

In this paper, we propose a novel improved Decomposed Vector-Quantized Variational Autoencoder (DVQ-VAE-2) with a new Mesh UFormer as the backbone network to extract the hierarchical structural representations from the mesh and propose a new normal vector-guided position encoding to simulate the hand-object deformation.

In this paper, we propose a novel robust unsupervised attention prediction method, which includes an Uncertainty Mining Branch, Knowledge Embedding Block, and RoboMixup, a novel data augmentation method to improve robustness against corruption and central bias.

In this paper, we propose a sim-to-real reinforcement learning method that leverages dense tactile feedback combined with joint torque sensing to explicitly regulate physical interactions.

In this paper, we introduce RoboDriveBench, the first robustness benchmark focused on end-to-end trajectory prediction tasks, and propose a novel VLM-based autonomous driving framework which enhances robustness by mapping more multimodal data.

In this paper, we propose a novel Multi-Modal Scene Graph with Kolmogorov-Arnold Expert Network for Audio-Visual Question Answering (SHRIKE).

In this paper, we present a novel continual learning framework that combines VLMs with selective memory replay and knowledge distillation, reinforced by task-specific projection layer regularization.

In this paper, we introduce a Target-driven Self-Distillation method (TSD) for motion forecasting. Our method leverages predicted accurate targets to guide the model in making predictions under partial observation condition.

In this paper, we introduce a novel memory-based method, named Motion Pattern Priors Memory Network. Our method involves constructing a memory bank derived from clustered prior knowledge of motion patterns observed in the training set trajectories.

In this paper, we propose a Dual-stage Spatial-Channel Transformer for coarse-to-fine video denoising, which inherits the advantages of both Transformer and CNNs.

In this paper, we introduce an effective approach to exploit 3D geometric constraints within a cycle generative adversarial network (CycleGAN) to perform domain adaptation. Furthermore, we propose to enforce short- and long-term temporal consistency to fine-tune the domain-adapted model in a self-supervised fashion.

Thanks to this awesome guy.