Seminar| Institute of Mathematical Sciences
Time: Thursday, November 19th, 2026,15:00-16:00
Location: IMS S518
Speaker: Difeng Cai, The Chinese University of Hong Kong, Shenzhen
Abstract: Gaussian process is a fundamental tool in uncertainty quantification and surrogate modeling. However, it is computationally prohinitive for large scale data due to the high complexity in training that involves dense matrices, nonconvex optimization, hyperparameter updates, etc. We focus on the optimization problem in GPs and introduce stochastic estimators for preconditioning hyperparameter optimization to accelerate convergence. The estimators are based on stochastic Krylov subspace methods and can be applied in more general settings beyond GPs. The theory reveals connection between preconditioning in matrix computations and variance reduction in stochastic algorithms. The proposed stochastic estimators, termed Preconditioned Truncated Single-Sample (PTSS) estimators, offer reduced variance compared to commonly used unbiased estimators. The new methodology can be used to accelerate various matrix operations such as linear system solutions, log-determinant, as well as parameter-gradient calculations in stochastic learning. Experiments show improved computational efficiency as well as convergence behavior compared to existing methods.