Peihao's Homepage

Peihao Wang

I am a PhD student at the Department of Electrical and Computer Engineering, The University of Texas at Austin. I study and research machine learning and computer vision, under the supervision of Prof. Atlas Wang. My research explores "geometric primitives" for enhancing language model reasoning, understanding neural architectures, discovering new scaling paradigms, and grounding generative vision models with physics.

My work is recognized with the DARPA Disruptive Idea Paper Award and the SIGGRAPH Asia 2024 Best Paper Award. I'm a recipient of the Google Fellowship 2025 in ML Foundations and named as Stanford Rising Star in Data Science.

Mail Scholar GitHub LinkedIn

News

10/2025 I'm selected to receive Google Fellowship 2025 in ML and ML Foundations.
09/2025 Three papers accepted to NeurIPS 2025.
09/2025 I'm named as Stanford Rising Star in Data Science.
06/2025 One paper accepted to ICCV 2025.
05/2025 One paper accepted to ICML 2025.
05/2025 Our work on Symbolic Emergence Theory won Disruptive Idea Awards at NeuS 2025.
04/2025 I'm selected to receive Texas ECE Graduate Achievement Award.

[Older]
- 02/2025 Two papers accepted to CVPR 2025.
- 02/2025 I'm selected to receive UT Graduate Recognition Fellowship.
- 01/2025 One paper accepted to ICLR 2025.
- 01/2025 One paper accepted to AISTATS 2025.
- 12/2024 Our work UDC AR Mirror won Best Paper at SIGGRAPH Asia 2024.
- 09/2024 Two papers accepted to NeurIPS 2024.
- 09/2024 One paper accepted to SIGGRAPH Asia 2024.
- 02/2024 One paper accepted to ECCV 2024.
- 02/2024 Two papers accepted to CVPR 2024.
- 01/2024 One paper accepted to ICLR 2024.
- 09/2023 One paper accepted to NeurIPS 2023.
- 07/2023 Two papers accepted to ICCV 2023.
- 04/2023 One paper accepted to ICML 2023.
- 03/2023 MIT News covered our work "Learning to Grow Pretrained Models".
- 02/2023 One paper accepted to CVPR 2023.
- 01/2023 Four papers accepted to ICLR 2023.
- 09/2022 Two papers accepted to NeurIPS 2022.
- 07/2022 Two papers accepted to ECCV 2022.
- 05/2022 One paper accepted to ICML 2022.
- 04/2022 One paper accepted to TPAMI.
- 03/2022 Two papers accepted to CVPR 2022.
- 12/2021 One paper accepted to ICLR 2022.
- 09/2021 One paper accepted to NeurIPS 2021.

Selected Publications

(A superscript * denotes equal contribution)

Rethinking Addressing in Language Models via Contextualized Equivariant Positional Encoding

Jiajun Zhu*, Peihao Wang*, Ruisi Cai, Jason D. Lee, Pan Li, Atlas Wang

International Conference on Machine Learning (ICML), 2025

We enhance the algorithmic reasoning ability of large language models via contextualizing positional encoding with equivariance constraints.
[Paper] [Code]
Why Neural Network Can Discover Symbolic Structures with Gradient-based Training: An Algebraic and Geometric Foundation for Neurosymbolic Reasoning

Peihao Wang*, Atlas Wang*

International Conference on Neuro-symbolic Systems (NeuS), 2025, Disruptive Idea Awards

We theoretically demonstrate that symbolic structures are inherent in neural weight space and gradient descent under geometric constraints can find these solutions.
[Paper]
Polynomial Width is Sufficient for Set Representation with High-dimensional Features

Peihao Wang, Shenghao Yang, Shu Li, Atlas Wang, Pan Li

International Conference on Learning Representations (ICLR), 2024

We theoretically show that polynomial many neurons are sufficient for set representation with the DeepSets architecture.
[Paper]
Learning to Grow Pretrained Models for Efficient Transformer Training

Peihao Wang, Rameswar Panda, Lucas Torroba Hennigen, Philip Greengard, Leonid Karlinsky, Rogerio Feris, David Cox, Atlas Wang, Yoon Kim

International Conference on Learning Representations (ICLR), 2023, Spotlight

This paper proposes to accelerate transformer training by re-using pretrained models via a learnable, linear and sparse model growth operator.
[MIT News] [Paper] [Code]
Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice

Peihao Wang, Wenqing Zheng, Tianlong Chen, Atlas Wang

International Conference on Learning Representations (ICLR), 2022

We prove that self-attention is no more than low-pass filter, and propose two simple yet effective methods to counteract excessive smoothening.
[Paper] [Code]

Teaching & Services

10/19/2025-10/23/2025, Co-organizer of ICCV 2025 Curated Data for Efficient Learning Workshop.
10/19/2025, Tutorial at ICCV 2025 Learning Deep Low-Dimensional Models from High-Dimensional Data: From Theory to Practice.
Conference Reviewer of ICML 2022, ECCV 2022, NeurIPS 2022, CVPR 2023, ICML 2023, ICCV 2023, NeurIPS 2023, ICLR 2024, CVPR 2024, ICML 2024, ECCV 2024, NeurIPS 2024, ICLR 2025, CVPR 2025, ICML 2025, ICCV 2025, NeurIPS 2025.
9/15/2025, Guest lecture on CMU 10747 - Neuro-Symbolic AI: "How neural networks represent and learn symbols" at Carnegie Mellon University.
8/14/2025, Tutorial on "Why Neural Networks Can Discover Symbolic Structures" at Neuro-Symbolic AI Summer School 2025.
5/28/2025, Talk on "A Theory of Symbolic Emergence in Neural Networks" at 2nd International Conference on Neuro-symbolic Systems (NeuS).
10/25/2024, Talk on "Signal Processing with Implicit Neural Fields: Editing, Manipulation, and Understanding" at SIAM MDS - Machine Learning Advances in Scientific Computing and Applications .
12/01/2023, Talk on "Rethinking Text-to-3D Score Distillation: Variance and Entropy" at IFML Workshop on Generative AI.
9/27/2023, Guest lecture on ECE 381V - Advanced Topics in Computer Vision: "Neural Radiance Fields" at University of Texas at Austin.
04/18/2023, Talk on "Neural Field Editing, Manipulation, and Learning" at Extrality Seminars.
04/10/2023, Guest lecture on ECE 381V - Advanced Topics in Computer Vision: "Neural Radiance Fields" at University of Texas at Austin.
08/19/2022, Talk on "Signal Processing for Implicit Neural Representation" at MLOPT Research Idea Seminar.
Spring 2021, Course founder and teaching assistant of SI363 - Multi-Scale Imaging for Structural Biology at ShanghaiTech University.
Fall 2020, Guest lecture on CS276 - Computational Photography: "Camera and Lens" at ShanghaiTech University.
Spring 2020, Teaching assistant of CS110 - Computer Architecture at ShanghaiTech University.
Fall 2019, Teaching assistant of CS100 - Introduction to Programming at ShanghaiTech University.

Contact

Address

2501 Speedway
EER 3.854
Austin, TX
United States
78712

Email

Academic
peihaowang [at] utexas.edu

Other Use
wangpeihao [at] gmail.com

About Me

Keep thy heart with all diligence; for out of it are the issues of life.

Proverbs 4:23, KJV

My journey in computer programming began at ten years old when my father introduced me to Scratch programming. I spent my leisure time during high school developing a database management software myBase 7. At my college, I was fortunate to work with Prof. Jingyi Yu on 3D vision and computational imaging, and studied algebra from Prof. Manolis C. Tsakiris. I also worked with Prof. Jianbo Shi on graph learning and spectral graph theory. It is always hard to be a starter, hereby I would express my sincere gratitude to those who guided me walk through the novice village of academia.