Peihao's Homepage

Peihao Wang

I am a PhD student at the Department of Electrical and Computer Engineering, The University of Texas at Austin. I am doing scientific research in the areas of machine learning and computer vision, under the supervision of Prof. Atlas Wang. My research investigates the underlying geometric, symbolic, and topologic structures of data, physics, and inference to develop theoretically sound neural architectures, training paradigms, and reasoning algorithms for language models, generative models, and physically grounded computer vision.

Mail Scholar GitHub LinkedIn

News

05/2025 One paper accepted to ICML 2025.
05/2025 Our work on Theoretical Foundation of Neurosymbolic AI won Disruptive Idea Awards at NeuS 2025.
04/2025 I'm selected to receive Texas ECE Graduate Achievement Award.
02/2025 Two papers accepted to CVPR 2025.
02/2025 I'm selected to receive UT Graduate Recognition Fellowship.
01/2025 One paper accepted to ICLR 2025, and one paper accepted to AISTATS 2025.

[Older]
- 12/2024 Our work UDC AR Mirror won Best Paper at SIGGRAPH Asia 2024.
- 09/2024 Two papers accepted to NeurIPS 2024.
- 09/2024 One paper accepted to SIGGRAPH Asia 2024.
- 02/2024 One paper accepted to ECCV 2024.
- 02/2024 Two papers accepted to CVPR 2024.
- 01/2024 One paper accepted to ICLR 2024.
- 09/2023 One paper accepted to NeurIPS 2023.
- 07/2023 Two papers accepted to ICCV 2023.
- 04/2023 One paper accepted to ICML 2023.
- 03/2023 MIT News covered our work "Learning to Grow Pretrained Models".
- 02/2023 One paper accepted to CVPR 2023.
- 01/2023 Four papers accepted to ICLR 2023.
- 09/2022 Two papers accepted to NeurIPS 2022.
- 07/2022 Two papers accepted to ECCV 2022.
- 05/2022 One paper accepted to ICML 2022.
- 04/2022 One paper accepted to TPAMI.
- 03/2022 Two papers accepted to CVPR 2022.
- 12/2021 One paper accepted to ICLR 2022.
- 09/2021 One paper accepted to NeurIPS 2021.

Selected Publication

(A superscript * denotes equal contribution)

Rethinking Addressing in Language Models via Contextualized Equivariant Positional Encoding

Jiajun Zhu*, Peihao Wang*, Ruisi Cai, Jason D. Lee, Pan Li, Atlas Wang

International Conference on Machine Learning (ICML), 2025

We enhance the addressing ability of large language models via contextualizing positional encoding with equivariance constraints.
[Paper] [Code]
Why Neural Network Can Discover Symbolic Structures with Gradient-based Training: Theoretic Foundation for Neurosymbolic AI

Peihao Wang*, Atlas Wang*

International Conference on Neuro-symbolic Systems (NeuS), 2025, Disruptive Idea Awards

We theoretically demonstrate that symbolic structures are inherent in neural weight space and gradient descent under geometric constraints can find these solutions.
[Paper]
Polynomial Width is Sufficient for Set Representation with High-dimensional Features

Peihao Wang, Shenghao Yang, Shu Li, Atlas Wang, Pan Li

International Conference on Learning Representations (ICLR), 2024

We theoretically show that polynomial many neurons are sufficient for set representation with the DeepSets architecture.
[Paper]
Learning to Grow Pretrained Models for Efficient Transformer Training

Peihao Wang, Rameswar Panda, Lucas Torroba Hennigen, Philip Greengard, Leonid Karlinsky, Rogerio Feris, David Cox, Atlas Wang, Yoon Kim

International Conference on Learning Representations (ICLR), 2023, Spotlight

This paper proposes to accelerate transformer training by re-using pretrained models via a learnable, linear and sparse model growth operator.
[MIT News] [Paper] [Code]
Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice

Peihao Wang, Wenqing Zheng, Tianlong Chen, Atlas Wang

International Conference on Learning Representations (ICLR), 2022

We prove that self-attention is no more than low-pass filter, and propose two simple yet effective methods to counteract excessive smoothening.
[Paper] [Code]

Teaching & Service

10/19/2025-10/23/2025, Co-organizer of ICCV 2025 Curated Data for Efficient Learning Workshop.
Conference Reviewer of ICML 2022, ECCV 2022, NeurIPS 2022, CVPR 2023, ICML 2023, ICCV 2023, NeurIPS 2023, ICLR 2024, CVPR 2024, ICML 2024, ECCV 2024, NeurIPS 2024, ICLR 2025.
5/28/2025, Oral Presentations on "Why Neural Network Can Discover Symbolic Structures with Gradient-based Training: A Theoretic Foundation for Neurosymbolic AI" at NeuS 2025.
10/25/2024, Talk on "Signal Processing with Implicit Neural Fields: Editing, Manipulation, and Understanding" at SIAM MDS - Machine Learning Advances in Scientific Computing and Applications .
12/01/2023, Talk on "Rethinking Text-to-3D Score Distillation: Variance and Entropy" at IFML Workshop on Generative AI.
04/18/2023, Talk on "Neural Field Editing, Manipulation, and Learning" at Extrality Seminars.
04/10/2023, ECE 381V - Advanced Topics in Computer Vision: Guest lecture in "Neural Radiance Fields".
08/19/2022, Talk on "Signal Processing for Implicit Neural Representation" at MLOPT Research Idea Seminar.
Spring 2021, SI363 - Multi-Scale Imaging for Structural Biology: Teaching assistant.
Fall 2020, CS276 - Computational Photography: Guest lecture in "Camera and Lens".
Aug. 2020, UTech Academy AI Camp: Teaching assistant of computer vision track.
Spring 2020, CS110 - Computer Architecture: Teaching assistant.
Fall 2019, CS100 - Introduction to Programming: Teaching assistant.

Contact

Address

2501 Speedway
EER 3.854
Austin, TX
United States
78712

Email

Academic
peihaowang [at] utexas.edu

Other Use
wangpeihao [at] gmail.com

About Me

Keep thy heart with all diligence; for out of it are the issues of life.

Proverbs 4:23, KJV

My journey in computer programming began at ten years old when my father introduced me to Scratch programming. I spent my leisure time during high school developing a database management software myBase 7. At my college, I was fortunate to work with Prof. Jingyi Yu on 3D vision and computational imaging, and studied algebra from Prof. Manolis C. Tsakiris. I also worked with Prof. Jianbo Shi on graph learning and spectral graph theory. It is always hard to be a starter, hereby I would express my sincere gratitude to those who guided me walk through the novice village of academia.