Tiancheng Gu | 顾天承

UniME-V2 leverages Multimodal LLMs as judges to identify high-quality hard negatives and learns fine-grained semantic distinctions through soft-label distribution alignment, achieving SOTA performance in universal multimodal retrieval.

DanQing is a curated dataset of 100 million Chinese image-text pairs sourced from 2024–2025 web data. By implementing a rigorous multi-stage filtering pipeline that retains only the top 9.54% of raw data, it achieves superior quality and temporal relevance over existing benchmarks. Models pre-trained on DanQing achieve state-of-the-art (SOTA) performance in zero-shot classification, cross-modal retrieval, and Chinese-centric multimodal reasoning.

UniME is a novel two-stage framework that empowers Multimodal Large Language Models (MLLMs) to learn universal and discriminative representations for diverse downstream tasks. By leveraging textual discriminative knowledge distillation and hard negative enhanced instruction tuning, UniME overcomes the limitations of traditional models (like CLIP’s token truncation and bag-of-words behavior), achieving state-of-the-art (SOTA) performance on the MMEB benchmark and various retrieval tasks.

RealSyn is a scalable paradigm designed to unlock the potential of unpaired multimodal interleaved documents for vision-language pre-training. By integrating a hierarchical retrieval of realistic texts with LLM-based synthetic caption generation, it constructs a high-quality, semantically balanced dataset of up to 100 million pairs. Models pre-trained on RealSyn consistently achieve state-of-the-art (SOTA) performance across zero-shot transfer, robustness, and retrieval benchmarks, outperforming traditional datasets like LAION and YFCC.

CLIP-CID is an efficient distillation framework for vision-language models that utilizes semantic balancing to prune redundant training data and cluster-instance discrimination to achieve state-of-the-art performance with lower computational costs.

We propose ORID, an Organ-Regional Information Driven framework that enhances radiology report generation by fusing organ-specific diagnostic descriptions with visual features and employing a GNN-based importance analysis to focus on clinically relevant regions while filtering out noise.

RWKV-CLIP is an efficient and robust vision-language model that replaces traditional Transformer backbones with an RWKV-driven architecture and incorporates an LLM-based data refinement framework to achieve SOTA performance with significantly reduced memory and inference costs.