Junbo Li

myphoto.jpg

I’m currently a first-year Ph.D. student in Computer Science at UT Austin, fortunately co-advised by Prof. Atlas Wang and Prof. Qiang Liu. I’m broadly interested in the intersection between machine learning, optimization, and generative AI, where I strike a balance between theoretical understanding and practical application. Most recently, I study:

  1. AI reasoning for math / code: Post-training (SFT & RL) and test-time scaling.
  2. Math for AI: General / efficient / distributed large-scale neural network optimization.

My works include publications on conferences such as ICLR[1], NeurIPS[2, 3], ECCV[4] and COLM[5, 6]. Additionally, I have contributed to large open-source projects like LLM360.

I was a visiting student with Prof. Eric Xing from 2023 to 2024 and also with Prof. Ruqi Zhang in 2023. I earned my master’s degree in computer science from UC Santa Cruz in 2023 and my bachelor’s degree in mathematics and applied mathematics from Fudan University in 2021. During my undergraduate, I worked with Prof. Zhaoran Wang and Zhuoran Yang. In high school, I studied math competitions for one year, and won a silver medal in the China Mathematical Olympiad (CMO).

news

Sep 26, 2024 Our Web2Code is accepted by NeurIPS 2024.
Jul 10, 2024 Our LLM360 and CrystalCoder are accepted by COLM 2024.
Jan 16, 2024 My first-author paper Sparse Subspace Variational Inference is accepted by ICLR 2024.
Dec 14, 2023 We release LLM360, including two 7B base models training from scratch: Amber and CrystalCoder, as well as the fine-tuned versions: AmberChat, AmberSafe, and CrystalChat. Our CrystalChat outperforms both Llama 2 and CodeLlama on both English and code benchmarks.
Sep 21, 2023 My first-author paper FedNAR is accepted by NeurIPS 2023, and is featured in the recent spotlight track at CPAL 2024.
Jun 27, 2023 Glad to join Sailing-MBZUAI lab as a visiting student.
Sep 14, 2022 My co-first-author paper Score Matching for RL is accepted by NeurIPS 2022 as the oral presentation.
Jul 08, 2022 My first-author paper ViP is accepted by ECCV 2022.

selected publications

  1. crystal.png
    Crystal: Illuminating LLM Abilities on Language and Code
    Tianhua TaoJunbo LiBowen TanHongyi Wang ,  William Marshall ,  Bhargav M Kanakiya ,  Joel Hestness ,  Natalia Vassilieva ,  Zhiqiang ShenEric P. Xing ,  and  Zhengzhong Liu
    Conference on Language Modeling, 2024
  2. llm360.png
    LLM360: Towards Fully Transparent Open-Source LLMs
    Zhengzhong Liu ,  Aurick Qiao ,  Willie NeiswangerHongyi WangBowen TanTianhua TaoJunbo Li ,  Yuqi Wang ,  Suqi Sun ,  Omkar Pangarkar ,  Richard Fan ,  Yi Gu ,  Victor Miller ,  Yonghao Zhuang ,  Guowei He ,  Haonan Li ,  Fajri Koto ,  Liping Tang ,  Nikhil Ranjan ,  Zhiqiang Shen ,  Xuguang Ren ,  Roberto Iriondo ,  Cun Mu ,  Zhiting Hu ,  Mark Schulze ,  Preslav Nakov ,  Tim Baldwin ,  and  Eric P. Xing
    Conference on Language Modeling, 2024
  3. ssvi.png
    Training Bayesian Neural Networks with Sparse Subspace Variational Inference
    Junbo LiZichen MiaoQiang Qiu ,  and  Ruqi Zhang
    ICLR, 2024
  4. fednar.png
    FedNAR: Federated Optimization with Normalized Annealing Regularization
    Junbo LiAng Li ,  Chong Tian ,  Qirong HoEric P Xing ,  and  Hongyi Wang
    NeurIPS, 2023
  5. smrl.png
    Exponential Family Model-Based Reinforcement Learning via Score Matching
    Gene LiJunbo LiAnmol KabraNati SrebroZhaoran Wang ,  and  Zhuoran Yang
    NeurIPS (Oral presentation), 2022
  6. vip.png
    ViP: Unified Certified Detection and Recovery for Patch Attack with Vision Transformers
    Junbo LiHuan Zhang ,  and  Cihang Xie
    ECCV, 2022