Nikita Doikov

Welcome! I work on optimization for machine learning and AI.

I am an assistant professor in the School of Operations Research and Information Engineering (ORIE) at Cornell University.

My research combines algorithm design with convergence and complexity analysis. I am interested in how problem structure, curvature, and randomness can make training large AI models more efficient.

I received my PhD in 2021 at UCLouvain, Belgium, supervised by Yurii Nesterov. My thesis is New second-order and tensor methods in Convex Optimization. After that, I was a postdoctoral researcher at EPFL, Switzerland, working in the Machine Learning and Optimization Laboratory with Martin Jaggi.

Recent Research Directions

All papers ↓

Accelerated and adaptive Newton methods

Comparison of first-order and second-order optimization trajectories

Fast Newton methods with global guarantees and simple updates. This includes primal acceleration with one linear solve per iteration, global linear convergence for quasi-self-concordant objectives, and adaptive super-universal methods.

Primal acceleration · 2026 Quasi-self-concordance · Math. Prog. 2025 Super-universal Newton · SIOPT 2024

Curvature, non-convexity, and geometry of training

Non-convex landscape with a marked convex direction

How problem geometry affects optimization, and how to exploit it through global curvature bounds, affine-invariant methods, and inexpensive Hessian information.

Global curvature bounds · 2026 Lazy Hessians · ICML 2023 Graded non-convexity · ICML 2024 Contracting Newton · NeurIPS 2020

Shuffling, stochasticity, and data unlearning

Convergence and complexity guarantees for shuffled and stochastic optimization, and for removing the influence of training data through certified unlearning.

Certified unlearning · ICML Workshop 2026 Momentum · AISTATS 2025 Variance reduction · TMLR 2024 Shuffling & data ordering · ICML 2024

Collaborative and causal learning

Collaborative work on personalized on-device large language models (LLMs) with mixtures of experts (CoMiGS), and causal discovery from interventional data (PACER).

CoMiGS · ICML 2025 PACER · ICML 2026

Teaching & Lecture Notes

ORIE 3300 / 5300

Optimization I

Canvas

Fall 2026

ORIE 6365

Continuous Optimization: Algorithms and Complexity

Lecture notes (PDF)

Spring 2026

All Papers

Google Scholar ↗

Preprints

Primal Acceleration of Newton's Method.
Nikita Doikov, 2026 (arXiv)
On Optimization Complexity of Second-Order Certified Unlearning.
Nikita Doikov, Anastasia Koloskova, 2026 (arXiv, ICML Workshop)
Universal Reduced-Operator Method and High-Order Global Curvature Bounds.
Nikita Doikov, Yurii Nesterov, 2025 (arXiv)

2026

On the Complexity of Lower-Order Implementations of Higher-Order Methods.
Nikita Doikov, Geovani Nunes Grapiglia (Math. Prog, arXiv)
Complexity of Minimizing Regularized Convex Quadratic Functions.
Daniel Berg Thomsen, Nikita Doikov (SIOPT, arXiv)
PACER: Acyclic Causal Discovery from Large-Scale Interventional Data
Ramon Viñas Torné, Sílvia Fàbregas Salazar, Soyon Park, Ivo Alexander Ban,
Artyom Gadetsky, Nikita Doikov, Maria Brbić (ICML, arXiv)
Gradient-Normalized Smoothness for Optimization with Approximate Hessians.
Andrei Semenov, Martin Jaggi, Nikita Doikov (ICLR, arXiv)

2025

Minimizing Quasi-Self-Concordant Functions by Gradient Regularization of Newton Method.
Nikita Doikov (Math. Prog, arXiv)
Lower Complexity Bounds for Minimizing Regularized Functions.
Nikita Doikov (Optim. Lett, arXiv)
On-Device Collaborative Language Modeling via a Mixture of Generalists and Specialists.
Dongyang Fan, Bettina Messmer, Nikita Doikov, Martin Jaggi (ICML, arXiv)
Improving Stochastic Cubic Newton with Momentum.
El Mahdi Chayti, Nikita Doikov, Martin Jaggi (AISTATS, arXiv)
Cubic regularized subspace Newton for non-convex optimization.
Jim Zhao, Aurelien Lucchi, Nikita Doikov (AISTATS, arXiv)
First and zeroth-order implementations of the regularized Newton method with lazy approximated Hessians.
Nikita Doikov, Geovani Nunes Grapiglia (Sci. Comp, arXiv)

2024

Spectral Preconditioning for Gradient Methods on Graded Non-convex Functions.
Nikita Doikov, Sebastian U. Stich, Martin Jaggi (ICML, arXiv)
On Convergence of Incremental Gradient for Non-Convex Smooth Functions.
Anastasia Koloskova, Nikita Doikov, Sebastian U. Stich, Martin Jaggi (ICML, arXiv)
Unified Convergence Theory of Stochastic and Variance-Reduced Cubic Newton Methods.
El Mahdi Chayti, Martin Jaggi, Nikita Doikov (TMLR, arXiv)
Super-Universal Regularized Newton Method.
Nikita Doikov, Konstantin Mishchenko, Yurii Nesterov (SIOPT, arXiv, code)

2023

Linearization Algorithms for Fully Composite Optimization.
Maria-Luiza Vladarean, Nikita Doikov, Martin Jaggi, Nicolas Flammarion (COLT, arXiv)
Polynomial Preconditioning for Gradient Methods.
Nikita Doikov, Anton Rodomanov (ICML, arXiv)
Second-order optimization with lazy Hessians.
Nikita Doikov, El Mahdi Chayti, Martin Jaggi (ICML, arXiv)
Gradient Regularization of Newton Method with Bregman Distances.
Nikita Doikov, Yurii Nesterov (Math. Prog, arXiv)

2022

High-Order Optimization Methods for Fully Composite Problems.
Nikita Doikov, Yurii Nesterov (SIOPT, arXiv)
Affine-invariant contracting-point methods for Convex Optimization.
Nikita Doikov, Yurii Nesterov (Math. Prog, arXiv, code)

2021

Local convergence of tensor methods.
Nikita Doikov, Yurii Nesterov (Math. Prog, arXiv)
Minimizing Uniformly Convex Functions by Cubic Regularization of Newton Method.
Nikita Doikov, Yurii Nesterov (JOTA, arXiv)

2020

Convex optimization based on global lower second-order models.
Nikita Doikov, Yurii Nesterov (NeurIPS, arXiv, code)
Stochastic Subspace Cubic Newton Method.
Filip Hanzely, Nikita Doikov, Peter Richtárik, Yurii Nesterov (ICML, arXiv)
Inexact Tensor Methods with Dynamic Accuracies.
Nikita Doikov, Yurii Nesterov (ICML, arXiv, code)
Contracting Proximal Methods for Smooth Convex Optimization.
Nikita Doikov, Yurii Nesterov (SIOPT, arXiv)

2018

Randomized Block Cubic Newton Method.
Nikita Doikov, Peter Richtárik (ICML, arXiv)

Talks & slides

June 18, 2026: Lower Complexity Bounds for Minimizing Regularized Quadratic Functions, UCLouvain, Louvain-la-Neuve (slides)
June 11, 2026: Gradient-Normalized Smoothness for Optimization with Approximate Hessians, VOCAL, Mosonmagyaróvár (slides)
May 4, 2025: Cubic regularized subspace Newton for non-convex optimization, AISTATS, Phuket (slides)
April 9, 2025: Stochastic second-order optimization: global bounds, subspaces, and momentum, UCLouvain, Louvain-la-Neuve (slides)
December 3, 2024: Stochastic second-order optimization: global bounds, subspaces, and momentum, WIAS, Berlin (slides)
August 27, 2024: Spectral Preconditioning for Gradient Methods on Graded Non-convex Functions, ALGOPT, Louvain-la-Neuve (slides)
July 1, 2024: Minimizing quasi-self-concordant functions by gradient regularization of Newton method, EURO, Copenhagen (slides)
June 26, 2024: Spectral Preconditioning for Gradient Methods on Graded Non-convex Functions, EUROPT, Lund (slides)
June 20, 2024: Polynomial Preconditioning for Gradient Methods, FGS-24, Gijón (slides)
April 9, 2024: Minimizing quasi-self-concordant functions by gradient regularization of Newton method, NOPTA, University of Antwerp (slides)
August 25, 2023: Super-Universal Regularized Newton Method, EUROPT, Budapest (slides)
July 20, 2023: Second-Order Optimization with Lazy Hessians, ICML, Hawaii (slides, poster, video)
July 19, 2023: Polynomial Preconditioning for Gradient Methods, ICML, Hawaii (poster)
June 3, 2023: Second-Order Optimization with Lazy Hessians, SIAM Conference on Optimization, Seattle (slides)
September 27, 2022: Super-Universal Regularized Newton Method, TML Laboratory, EPFL (slides)
July 29, 2022: Affine-invariant contracting-point methods for Convex Optimization, EUROPT, Lisbon (slides)
June 3, 2022: Second-order methods with global convergence in Convex Optimization, the research team of Panos Patrinos, KULeuven (slides)
May 5, 2022: Optimization Methods for Fully Composite Problems, FGP-22, Porto (slides)
February 21, 2022: Second-order methods with global convergence in Convex Optimization, MLO Laboratory, EPFL (slides)
July 7, 2021: Local convergence of tensor methods, EUROPT, online (slides)
March 4, 2021: Affine-invariant contracting-point methods for Convex Optimization, Symposium on Numerical Analysis and Optimization (invited by Geovani Nunes Grapiglia), UFPR, online (slides)
October 28, 2020: Convex optimization based on global lower second-order models, NeurIPS, online (slides, poster)
June 17, 2020: Inexact Tensor Methods with Dynamic Accuracies, ICML, online (slides, poster, video)
October 8, 2019: Proximal Method with Contractions for Smooth Convex Optimization, ICTEAM seminar, Louvain-la-Neuve
September 23, 2019: Proximal Method with Contractions for Smooth Convex Optimization, Optimization and Learning for Data Science seminar (invited by Dmitry Grishchenko) Université Grenoble Alpes, Grenoble (slides)
September 18, 2019: Complexity of Cubically Regularized Newton Method for Minimizing Uniformly Convex Functions, FGS-19, Nice (slides)
August 5, 2019: Complexity of Cubically Regularized Newton Method for Minimizing Uniformly Convex Functions, ICCOPT, Berlin
July 5, 2019: Randomized Block Cubic Newton Method, Summer School on Optimization, Big Data and Applications, Veroli
June 28, 2019: Complexity of Cubically Regularized Newton Method for Minimizing Uniformly Convex Functions EUROPT, Glasgow
June 20, 2018: Randomized Block Cubic Newton Method, ICML, Stockholm (slides, poster, video)
June 13, 2018: Randomized Block Cubic Newton Method, X Traditional summer school on Optimization, Voronovo