Puneesh Deora

Hi, I'm Puneesh! I am a first year PhD student at UBC, fortunate to be advised by Prof. Christos Thrampoulidis. I am broadly interested in understanding foundations of deep learning optimization and generalization. More recently, I've been working on empirics and theory of phenomena fundamental to LLMs.

I recently finished my Master's (from UBC) where I worked on optimization and generalization theory on transformer models. Before that, I received my Bachelor's from IIT Roorkee in ECE, where I had the good fortune to receive guidance from Prof. Saumik Bhattacharya and Prof. P.M. Pradhan . My undergrad research was mainly focused on GANs for compressed sensing MRI reconstruction.

Email | Google Scholar | CV | Github

selected papers

(* denotes equal contribution)

  1. Implicit Bias and Convergence Rates for Self-attention
    Bhavya Vasudeva*, Puneesh Deora*, and Christos Thrampoulidis.
    Preprint.
  1. On the Optimization and Generalization of Multi-head Attention
    Puneesh Deora*, Rouzbeh Ghaderi*, Hossein Taheri*, and Christos Thrampoulidis.
    TMLR 2024, and HiLD at ICML 2023.
  1. Fast Test Error Rates for Gradient-based Algorithms on Separable Data
    Puneesh Deora*, Bhavya Vasudeva*, Vatsal Sharan, Christos Thrampoulidis.
    HiLD at ICML 2023, and ICASSP 2024.