Siyuan Chen (陈思元)

Siyuan Chen (陈思元)

CS Ph.D. student at Carnegie Mellon University

Siyuan Chen (陈思元)

I am a second year Ph.D. student at computer science department at Carnegie Mellon University, happily advised by Phil Gibbons and Heather Miller, and I am closely working with Ben Titzer. I am a member of PDL and Catalyst at CMU. I finished my undergraduate at Turning Class of Peking University, where I was fortunately advised by Yun (Eric) Liang.

My research interests broadly focus on developing system support for machine learning. Specifically, I am interested in exploring system solutions for the efficient execution of irregular machine learning tasks. This includes optimizing the batching of dynamic neural networks (ED-Batch, ICML23’) and automating scheduling techniques for large language model (LLM) serving. Additionally, I am concentrating on machine learning at the edge, which involves memory-efficient and high-precision fine-tuning on commodity hardware through compressed offloading (LSP-Offload, Under Submission), as well as enabling machine learning deployment in the browser via WebGPU.

Previously, I am deeply engaged in hardware-aware optimizations for machine learning. I worked on performance modeling of data movement for tensor programs (Chimera, HPCA23’), mapping mechanism from DNN to SoC (COMB, DAC23’), and analytical-based simulator for fused program on general hardware (TileFlow, MICRO23’).

News

  • Jun. 2024 I begin the internship as student researcher in SystemResearch@Google (SRG) working on LLM serving, hosted by Samira Khan.
  • Sept. 2023 I am starting CS PhD at Carnegie Mellon University, co-advised by Phil Gibbons and Heather Miller. Hope for a stimulating and fruitful journey at Pittsburgh!
  • Aug. 2023 TileFlow is publicly available!
  • Apr. 2023 ED-Batch is accepted to ICML23’, thanks for the mentor and professors!

Publications

(*Equal Contribution)

  • TileFlow: A Framework for Modeling Fusion Dataflow via Tree-based Analysis. Size Zheng, Siyuan Chen, Siyuan Gao, Liancheng Jia, Guangyu Sun, Runsheng Wang, Yun Liang. MICRO 2023. PDF

  • ED-Batch: Efficient Automatic Batching of Dynamic Deep Neural Networks via Finite State Machine. Siyuan Chen, Pratik Fegade, Tianqi Chen, Phillip B. Gibbons, Todd C. Mowry. ICML 23’. PDF Code Poster Video.

  • Memory and Computation Coordinated Mapping of DNNs onto Complex Heterogeneous SoC. Size Zheng, Siyuan Chen, Yun Liang. The 60th Design Automation Conference (DAC), July 2023. PDF

  • Chimera: An Analytical Optimizing Framework for Effective Compute-intensive Operators Fusion. Size Zheng*, Siyuan Chen*, Pedi Song, Renze Chen, Xiuhong Li, Shengen Yan, Dahua Lin, Jingwen Leng, Yun Liang. 29th international symposium on High Performance Computer Architecture (HPCA), February 2023. PDF.

Under Submission

  • Practical Offloading for Fine-Tuning LLM on Commodity GPU via Learned Sparse Projectors Siyuan Chen, Zhuofeng Wang, Zelong Guan, Yudong Liu, Phillip B. Gibbons. PDF