Defu Cao

Defu Cao

He/Him

Machine Learning PhD

Publications

You can also find my articles on my Google Scholar profile.

Selected Research Map

Research StackRepresentative PapersPositioning
Post-training and agentic systemsTSOrchestra, HILA, TS-ReasonerR1-style fine-tuning, metacognitive policy optimization, tool use, feedback loops, and human-agent collaboration.
LLM safety and agent security“Someone Hid It!”, Topology MattersRobustness and privacy risks in LLM retrieval, RAG, agent memory, and multi-agent communication topology.
Foundation models for structured and scientific dataTEMPO, ClimateLLM, TimeDiT, PINFDiTPretraining and generative modeling for complex numerical, spatiotemporal, weather, and physics-constrained data.
Physics-embedded mobility calibrationPhysics-Aware VAEDifferentiable traffic physics for sample-efficient urban travel demand calibration with physics guidance.
Numerical interfaces for LLMsSpeaking Numbers to LLMs, GPT4MTSRepresentation and prompting layers that let language models consume continuous values, multimodal signals, and numerical structure.
Benchmarks and high-stakes evaluationTSAIA, TemporalBench, ECG benchmark, waveform reasoningDiagnostic evaluation for whether LLM/agent systems reason reliably under contextual, event-driven, and clinical constraints.
AI policy and governance collaborationCooperative AI policymaking platformOpen-source collaboration on policy interfaces, economic forecasting, and public-value elicitation.
Surveys and research frameworksReasoning and agentic systems survey, physics-informed ML survey, Toward Evolutionary IntelligenceField-level organization, taxonomies, and design principles.

Post-Training and Agentic Systems

  1. Conversational Time Series Foundation Models: Towards Explainable and Effective Forecasting
    Defu Cao, Michael Gee, Jinbo Liu, Hengxuan Wang, Wei Yang, Rui Wang, Yan Liu
    arXiv 2025 | paper | code
    Positioning: TSOrchestra; R1-style fine-tuned LLM judge that explains and orchestrates foundation-model ensembles through multi-turn optimization.

  2. Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learning
    Wei Yang, Defu Cao, Jiacheng Pang, Muyan Weng, Yan Liu
    ICLR 2026 | paper |code
    Positioning: HILA; post-training for human-agent collaboration through GRPO-based metacognitive deferral and continual learning from expert feedback.

  3. TS-Reasoner: Domain-Oriented Time Series Inference Agents for Reasoning and Automated Analysis
    Wen Ye, Wei Yang, Defu Cao, Yizhou Zhang, Lumingyuan Tang, Jie Cai, Yan Liu
    TMLR, March 2026 | OpenReview | code
    Positioning: domain-specialized inference agent accepted by TMLR after Action Editor confirmation; combines LLM reasoning, computational tools, and an error-feedback loop for multi-step analytical workflows.

LLM Safety and Agent Security

  1. “Someone Hid It!”: Query-Agnostic Black-Box Attacks on LLM-Based Retrieval
    Jiate Li, Defu Cao, Li Li, Wei Yang, Yuehan Qin, Chenxiao Yu, Tiannuo Yang, Ryan A. Rossi, Yan Liu, Xiyang Hu, Yue Zhao
    ICML 2026 | paper
    Positioning: query-agnostic, black-box adversarial attacks for LLM-based retrieval, including RAG, dense IR, and agent memory retrieval.

  2. Topology Matters: Measuring Memory Leakage in Multi-Agent LLMs
    Jinbo Liu, Defu Cao, Yifei Wei, Tianyao Su, Yuan Liang, Yushun Dong, Yan Liu, Yue Zhao, Xiyang Hu
    arXiv 2025 | paper
    Positioning: multi-agent privacy and security; measures how communication graph topology affects PII leakage from agent memory.

Foundation Models for Structured and Scientific Data

  1. TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting
    Defu Cao, Furong Jia, Sercan O Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, Yan Liu
    ICLR 2024 | paper | code | huggingface
    Positioning: GPT-style pretraining with decomposition and prompting for transferable numerical sequence modeling.

  2. ClimateLLM: Efficient Weather Forecasting via Frequency-Aware Large Language Models
    Shixuan Li, Wei Yang, Peiyu Zhang, Xiongye Xiao, Defu Cao, Yuehan Qin, Xiaole Zhang, Yue Zhao, Paul Bogdan
    arXiv 2025 | paper
    Positioning: frequency-aware LLM foundation model for weather, combining Fourier decomposition, MoE routing, and cross-spatial/cross-temporal prompting.

  3. TimeDiT: General-purpose Diffusion Transformers for Time Series Foundation Model
    Defu Cao, Wen Ye, Yizhou Zhang, Yan Liu
    ICML 2024 Workshop on Foundation Models in the Wild | paper
    Positioning: diffusion-transformer proto-foundation model for probabilistic generation, imputation, forecasting, anomaly detection, and model editing.

  4. PINFDiT: Energy-Based Physics-Informed Diffusion Transformers for General-purpose Time Series Tasks
    Defu Cao, Wen Ye, Yizhou Zhang, Sam Griesemer, Yan Liu
    ICLR 2026 | paper
    Positioning: physics-guided inference for diffusion foundation models, using calibrated Langevin correction to enforce physical consistency without retraining.

Numerical Interfaces for LLMs

  1. Speaking Numbers to LLMs: Multi-Wavelet Number Embeddings for Time Series Forecasting
    Defu Cao, Zijie Lei, Jiao Sun, Yan Liu
    IJCAI 2026 | paper
    Positioning: not a standalone foundation-model pretraining paper; it is a numerical representation layer. Multi-Wavelet Number Embedding (MWNE) decomposes continuous values with wavelet bases so LLMs preserve digit recovery, numeracy, multi-scale structure, and normalization robustness.

  2. GPT4MTS: Prompt-based Large Language Model for Multimodal Time-series Forecasting
    Furong Jia, Kevin Wang, Yixiang Zheng, Defu Cao, Yan Liu
    AAAI/EAAI 2024 | paper | Multimodal Dataset
    Positioning: prompt-based adaptation of LLMs for multimodal numerical forecasting.

AI Policy and Governance Collaboration

  1. Creating a Cooperative AI Policymaking Platform through Open Source Collaboration
    Aiden Lewington, Alekhya Vittalam, Anshumaan Singh, Anuja Uppuluri, Arjun Ashok, Ashrith Mandayam Athmaram, Austin Milt, Benjamin Smith, Charlie Weinberger, Chatanya Sarin, Christoph Bergmeir, Cliff Chang, Daivik Patel, Daniel Li, David Bell, Defu Cao, et al.
    arXiv 2024 | paper | organization
    Positioning: large open-source collaboration with Humanity Unleashed on a cooperative AI policymaking platform; contributed to the foundation-model/pretraining component.

Benchmarks and High-Stakes Evaluation

  1. When LLM Meets Time Series: Can LLMs Perform Multi-Step Time Series Reasoning and Inference
    Wen Ye, Jinbo Liu, Defu Cao, Wei Yang, Yan Liu
    arXiv 2025 | paper
    Positioning: TSAIA benchmark for evaluating LLMs as domain assistants over multi-step numerical analysis tasks.

  2. TemporalBench: A Benchmark for Evaluating LLM-Based Agents on Contextual and Event-Informed Time Series Tasks
    Muyan Weng, Defu Cao, Wei Yang, Yashaswi Sharma, Yan Liu
    arXiv 2026 | paper
    Positioning: diagnostic benchmark for contextual and event-conditioned reasoning, separating forecasting accuracy from temporal understanding.

  3. Position: Beyond Prediction: Toward Verifiable Physiological Waveform Reasoning with Foundation Models and Agentic LLMs
    Xiaoda Wang, Ching Chang, Defu Cao, Kaiqiao Han, Fang Sun, Yue Huang, Minxiao Wang, Chang Xu, Xiao Luo, Runze Yan, Xiangliang Zhang, Xiao Hu, Yan Liu, Yizhou Sun, Wei Wang, Carl Yang
    ICML 2026 | paper
    Positioning: high-stakes Physiological Waveform Reasoning; argues for verifiable Plan-Act-Verify systems that connect localized signal evidence to clinical decisions.

  4. EnECG: Efficient Ensemble Learning for Electrocardiogram Multi-task Foundation Model
    Yuhao Xu, Xiaoda Wang, Jiaying Lu, Sirui Ding, Defu Cao, Huaxiu Yao, Yan Liu, Xiao Hu, Carl Yang
    arXiv 2025 | paper
    Positioning: efficient ensemble and MoE adaptation for ECG multi-task foundation models.

  5. An Electrocardiogram Multi-task Benchmark with Comprehensive Evaluations and Insightful Findings
    Yuhao Xu, Jiaying Lu, Sirui Ding, Defu Cao, Xiao Hu, Carl Yang
    arXiv 2025 | paper
    Positioning: benchmark asking whether language, general time-series, and ECG foundation models are useful for ECG analysis.

Surveys and Research Frameworks

  1. A Survey of Reasoning and Agentic Systems in Time Series with Large Language Models
    Ching Chang, Yidan Shi, Defu Cao, Wei Yang, Jeehyun Hwang, Haixin Wang, Jiacheng Pang, Wei Wang, Yan Liu, Wen-Chih Peng, Tien-Fu Chen
    arXiv 2025 | paper | repo
    Positioning: actual survey; organizes reasoning topologies, tool use, evidence grounding, benchmarks, and agent loops for temporal data.

  2. Toward Evolutionary Intelligence: LLM-based Agentic Systems with Multi-Agent Reinforcement Learning
    Wei Yang, Muyan Weng, Jiacheng Pang, Defu Cao, Heng Ping, Peiyu Zhang, Shixuan Li, Yue Zhao, Qiang Yang, Mengdi Wang, Yan Liu
    SSRN 2026 | paper
    Positioning: research framework for learning paradigms in LLM-based multi-agent systems, especially MARL-driven adaptive and evolutionary agent societies.

  3. When Physics Meets Machine Learning: A Survey of Physics-Informed Machine Learning
    Chuizheng Meng, Sam Griesemer, Defu Cao, Sungyong Seo, Yan Liu
    Machine Learning for Computational Science and Engineering, 2025 | journal | arXiv
    Positioning: actual survey of physics-informed machine learning; ranked #1 in the journal’s “Top 10 Downloaded Articles in 2025”.

Selected Publications

2026

  1. TS-Reasoner: Domain-Oriented Time Series Inference Agents for Reasoning and Automated Analysis
    Wen Ye, Wei Yang, Defu Cao, Yizhou Zhang, Lumingyuan Tang, Jie Cai, Yan Liu
    TMLR, March 2026 | OpenReview | code

  2. “Someone Hid It!”: Query-Agnostic Black-Box Attacks on LLM-Based Retrieval
    Jiate Li, Defu Cao, Li Li, Wei Yang, Yuehan Qin, Chenxiao Yu, Tiannuo Yang, Ryan A. Rossi, Yan Liu, Xiyang Hu, Yue Zhao
    ICML 2026 | paper

  3. Position: Beyond Prediction: Toward Verifiable Physiological Waveform Reasoning with Foundation Models and Agentic LLMs
    Xiaoda Wang, Ching Chang, Defu Cao, Kaiqiao Han, Fang Sun, Yue Huang, Minxiao Wang, Chang Xu, Xiao Luo, Runze Yan, Xiangliang Zhang, Xiao Hu, Yan Liu, Yizhou Sun, Wei Wang, Carl Yang
    ICML 2026 | paper

  4. Speaking Numbers to LLMs: Multi-Wavelet Number Embeddings for Time Series Forecasting
    Defu Cao, Zijie Lei, Jiao Sun, Yan Liu
    IJCAI 2026 | paper

  5. PINFDiT: Energy-Based Physics-Informed Diffusion Transformers for General-purpose Time Series Tasks
    Defu Cao, Wen Ye, Yizhou Zhang, Sam Griesemer, Yan Liu
    ICLR 2026 | paper

  6. Physics-Aware Variational Autoencoder for Urban Travel Demand Calibration
    Defu Cao, et al.
    TMLR 2026 | OpenReview
    Positioning: physics-embedded OD calibration for urban travel demand, combining conditional variational inference with differentiable traffic guidance for sample-efficient simulator-based learning.

  7. Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learning
    Wei Yang, Defu Cao, Jiacheng Pang, Muyan Weng, Yan Liu
    ICLR 2026 | paper

  8. Orthogonalized estimation of difference of q-functions
    Defu Cao, Angela Zhou
    AISTATS 2026 | paper

2024-2025

  1. Neuron-based Multifractal Analysis of Neuron Interaction Dynamics in Large Models
    Xiongye Xiao, Heng Ping, Chenyu Zhou, Defu Cao, Yaxing Li, Yizhuo Zhou, Shixuan Li, Nikos Kanakaris, Paul Bogdan
    ICLR 2025

  2. When Splitting Makes Stronger: A Theoretical and Empirical Analysis of Divide-and-Conquer Prompting in LLMs
    Yizhou Zhang*, Defu Cao*, Yan Liu
    CoLM 2025

  3. TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting
    Defu Cao, Furong Jia, Sercan O Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, Yan Liu
    ICLR 2024 | paper | code | huggingface

  4. GPT4MTS: Prompt-based Large Language Model for Multimodal Time-series Forecasting
    Furong Jia, Kevin Wang, Yixiang Zheng, Defu Cao, Yan Liu
    AAAI/EAAI 2024 | paper

  5. Active Sequential Posterior Estimation for Sample-Efficient Simulation-Based Inference
    Sam Griesemer, Defu Cao, Zijun Cui, Carolina Osorio, Yan Liu
    NeurIPS 2024

  6. An Empirical Examination of Balancing Strategy for Counterfactual Estimation on Time Series
    Qiang Huang, Chuizheng Meng, Defu Cao, Biwei Huang, Yi Chang, Yan Liu
    ICML 2024

  7. Neuro-Inspired Hierarchical Multimodal Learning
    Xiongye Xiao, Gengshuo Liu, Gaurav Gupta, Defu Cao, Shixuan Li, Yaxing Li, Tianqing Fang, Mingxi Cheng, Paul Bogdan
    ICLR 2024

  8. MUGSI: Distilling GNNs with Multi-Granularity Structural Information for Graph Classification
    Tianjun Yao, Jiaqi Sun, Defu Cao, Kun Zhang, Guangyi Chen
    WWW 2024 | paper

  9. Coupled Multiwavelet Neural Operator Learning for Coupled Partial Differential Equations
    Xiongye Xiao, Defu Cao, Ruochen Yang, Gaurav Gupta, Gengshuo Liu, Chenzhong Yin, Radu Balan, Paul Bogdan
    ICLR 2024 | paper

2023 and Earlier

  1. Large Scale Financial Time Series Forecasting with Multi-Faceted Model
    Defu Cao, Yixiang Zheng, Parisa Hassanzadeh, Simran Lamba, Xiaomo Liu, Yan Liu
    ICAIF 2023 | Oral | paper

  2. SVGformer: Representation Learning for Continuous Vector Graphics using Transformers
    Defu Cao, Zhaowen Wang, Jose Echevarria, Yan Liu
    CVPR 2023 | paper

  3. Time-delayed Multivariate Time Series Predictions
    Hao Niu, Guillaume Habault, Roberto Legaspi, Chuizheng Meng, Defu Cao, Shinya Wada, Chihiro Ono, Yan Liu
    SDM 2023

  4. Estimating Treatment Effects from Irregular Time Series Observations with Hidden Confounders
    Defu Cao, James Enouen, Yujing Wang, Xiangchen Song, Chuizheng Meng, Hao Niu, Yan Liu
    AAAI 2023 | paper

  5. Counterfactual Neural Temporal Point Process for Estimating Causal Influence of Misinformation on Social Media
    Yizhou Zhang, Defu Cao, Yan Liu
    NeurIPS 2022 | paper | code

  6. Mu2ReST: Multi-Resolution Recursive Spatio-Temporal Transformer for Long-Term Prediction
    Hao Niu, Chuizheng Meng, Defu Cao, Guillaume Habault, Roberto Legaspi, Shinya Wada, Chihiro Ono, Yan Liu
    PAKDD 2022 | paper

  7. Enhancing Self-Attention with Knowledge-Assisted Attention Maps
    Jiangang Bai, Yujing Wang, Hong Sun, Ruonan Wu, Tianmeng Yang, Pengfei Tang, Wei Shen, Defu Cao, Mingliang Zhang, Yaming Yang, Jing Bai, Yunhai Tong, Hao Sun, Ruofei Zhang
    NAACL 2022 | paper

  8. Spectral Temporal Graph Neural Network for Trajectory Prediction
    Defu Cao, Jiachen Li, Hengbo Ma, Masayoshi Tomizuka
    ICRA 2021 | paper

  9. Spectral temporal graph neural network for multivariate time-series forecasting
    Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Conguri Huang, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, Qi Zhang
    NeurIPS 2020 | paper | code | video | Spotlight (2%)

  10. Multivariate time-series anomaly detection via graph attention network
    Hang Zhao, Yujing Wang, Juanyong Duan, Congrui Huang, Defu Cao, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, Qi Zhang
    ICDM 2020 | paper | code

Workshop Papers:

  1. TimeDiT: General-purpose Diffusion Transformers for Time Series Foundation Model
    Defu Cao, Wen Ye, Yizhou Zhang, Yan Liu
    ICML 2024 Workshop on Foundation Models in the Wild | paper

  2. DSLOB: A Synthetic Limit Order Book Dataset for Benchmarking Forecasting Algorithms under Distributional Shift
    Defu Cao, Yousef El-Laham, Loc Trinh, Svitlana Vyetrenko, Yan Liu
    NeurIPS 2022 DistShift Workshop | paper | Proposed Dataset

  3. Estimating Treatment Effects in Continuous Time with Hidden Confounders
    Defu Cao, James Enouen, Yan Liu
    ICML 2022 | video | paper

Patents:

  1. Representation Learning for Continuous Vector Graphics
    Defu Cao, Zhaowen Wang, Jose Echevarria US Patent Application US20240303870A1 | Status: Published Patent Application, United States, 2024 | USPTO

Publication Impact