Peihao's Homepage

Peihao Wang

I am a PhD student at the Department of Electrical and Computer Engineering, The University of Texas at Austin. I am doing scientific research in the areas of machine learning and computer vision, under the supervision of Prof. Atlas Wang at the VITA Group. Prior to that, I obtained my bachelor's degree from ShanghaiTech University in 2021. My current research focuses on theories and applications of geometric learning, language models, generative models, and 3D computer vision.

Mail Scholar GitHub LinkedIn

News

01/2025 One paper accepted to ICLR 2025.
01/2025 One paper accepted to AISTATS 2025.
12/2024 Our work UDC AR Mirror won Best Paper at SIGGRAPH Asia 2024.
09/2024 Two papers accepted to NeurIPS 2024.
09/2024 One paper accepted to SIGGRAPH Asia 2024.
02/2024 One paper accepted to ECCV 2024.

[Older]
- 02/2024 Two papers accepted to CVPR 2024.
- 01/2024 One paper accepted to ICLR 2024.
- 09/2023 One paper accepted to NeurIPS 2023.
- 07/2023 Two papers accepted to ICCV 2023.
- 04/2023 One paper accepted to ICML 2023.
- 03/2023 MIT News covered our work "Learning to Grow Pretrained Models".
- 02/2023 One paper accepted to CVPR 2023.
- 01/2023 Four papers accepted to ICLR 2023.
- 09/2022 Two papers accepted to NeurIPS 2022.
- 07/2022 Two papers accepted to ECCV 2022.
- 05/2022 One paper accepted to ICML 2022.
- 04/2022 One paper accepted to TPAMI.
- 03/2022 Two papers accepted to CVPR 2022.
- 12/2021 One paper accepted to ICLR 2022.
- 09/2021 One paper accepted to NeurIPS 2021.

Selected Publication

(A superscript * denotes equal contribution)

Understanding and Mitigating Bottlenecks of State Space Models through the Lens of Recency and Over-smoothing

Peihao Wang, Ruisi Cai, Yuehao Wang, Jiajun Zhu, Pragya Srivastava, Atlas Wang, Pan Li

International Conference on Learning Representations (ICLR), 2025

We theoretically and empirically reveal and tackle locality and over-smoothing bottlenecks for state space models.
[Paper] [Code]
Polynomial Width is Sufficient for Set Representation with High-dimensional Features

Peihao Wang, Shenghao Yang, Shu Li, Atlas Wang, Pan Li

International Conference on Learning Representations (ICLR), 2024

We theoretically show that polynomial many neurons are sufficient for set representation with the DeepSets architecture.
[Paper]
Learning to Grow Pretrained Models for Efficient Transformer Training

Peihao Wang, Rameswar Panda, Lucas Torroba Hennigen, Philip Greengard, Leonid Karlinsky, Rogerio Feris, David Cox, Atlas Wang, Yoon Kim

International Conference on Learning Representations (ICLR), 2023, Spotlight

This paper proposes to accelerate transformer training by re-using pretrained models via a learnable, linear and sparse model growth operator.
[MIT News] [Paper] [Code]
Is Attention All That NeRF Needs?

Mukund Varma T*, Peihao Wang*, Xuxi Chen, Tianlong Chen, Subhashini Venugopalan, Atlas Wang

International Conference on Learning Representations (ICLR), 2023

We present Generalizable NeRF Transformer (GNT), a pure, unified transformer-based architecture that efficiently reconstructs Neural Radiance Fields (NeRFs) on the fly.
[Project Page] [Paper] [Code]
Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice

Peihao Wang, Wenqing Zheng, Tianlong Chen, Atlas Wang

International Conference on Learning Representations (ICLR), 2022

We prove that self-attention is no more than low-pass filter, and propose two simple yet effective methods to counteract excessive smoothening.
[Paper] [Code]

Teaching & Service

Conference Reviewer of ICML 2022, ECCV 2022, NeurIPS 2022, CVPR 2023, ICML 2023, ICCV 2023, NeurIPS 2023, ICLR 2024, CVPR 2024, ICML 2024, ECCV 2024, NeurIPS 2024, ICLR 2025.
10/25/2024, Talk on "Signal Processing with Implicit Neural Fields: Editing, Manipulation, and Understanding" at SIAM MDS - Machine Learning Advances in Scientific Computing and Applications .
12/01/2023, Talk on "Rethinking Text-to-3D Score Distillation: Variance and Entropy" at IFML Workshop on Generative AI.
04/18/2023, Talk on "Neural Field Editing, Manipulation, and Learning" at Extrality Seminars.
04/10/2023, ECE 381V - Advanced Topics in Computer Vision: Guest lecture in "Neural Radiance Fields".
08/19/2022, Talk on "Signal Processing for Implicit Neural Representation" at MLOPT Research Idea Seminar.
Spring 2021, SI363 - Multi-Scale Imaging for Structural Biology: Teaching assistant.
Fall 2020, CS276 - Computational Photography: Guest lecture in "Camera and Lens".
Aug. 2020, UTech Academy AI Camp: Teaching assistant of computer vision track.
Spring 2020, CS110 - Computer Architecture: Teaching assistant.
Fall 2019, CS100 - Introduction to Programming: Teaching assistant.

Contact

Address

2501 Speedway
EER 3.854
Austin, TX
United States
78712

Email

Academic
peihaowang [at] utexas.edu

Other Use
wangpeihao [at] gmail.com

About Me

Keep thy heart with all diligence; for out of it are the issues of life.

Proverbs 4:23, KJV

My journey in computer programming began at ten years old when my father introduced me to Scratch programming. I spent my leisure time during high school developing a database management software myBase 7. At my college, I was fortunate to work with Prof. Jingyi Yu on 3D vision and computational imaging, and studied algebra from Prof. Manolis C. Tsakiris. I also worked with Prof. Jianbo Shi on graph learning and spectral graph theory. It is always hard to be a starter, hereby I would express my sincere gratitude to those who guided me walk through the novice village of academia.