Analytic Continual Learning

Signature Research Branch

Analytic Continual Learning

Analytic Continual Learning (ACL), pioneered by MIAA Lab, is the first research direction to systematically address continual learning with analytic, closed-form solutions. It starts from the observation that continual gradient updates repeatedly overwrite model parameters carrying old knowledge and are therefore a root cause of catastrophic forgetting. ACL reformulates incremental learning as a recursively solvable analytic problem and uses recursive least squares and related recursive analytic updates to absorb new knowledge while preserving old knowledge without storing or replaying historical exemplars.

30+ Peer-reviewed publications

NeurIPS / ICML / CVPR Published in leading AI conferences

LLMs and Embodied AI Expanding beyond vision continual learning

Core Idea

Analytic Continual Learning begins from the optimization root of catastrophic forgetting: when a model continually learns new tasks through gradient descent, incoming gradients repeatedly overwrite the parameters formed by earlier tasks and damage previously acquired knowledge. ACL replaces this iterative gradient process by formulating incremental learning as an analytic problem with a closed-form solution. The model maintains sufficient statistics such as auto-correlation and cross-correlation matrices and updates them recursively through recursive least squares. Under the corresponding theoretical conditions, the model learned phase by phase is equivalent to the one obtained by joint learning on all data, enabling absolute memorization, exemplar-free learning, privacy preservation, and efficient training.

Method Lineage

2019–2022

Analytic foundations and the birth of ACL

Low-memory kernel projection, correlation projection, and the Blockwise Recursive Moore-Penrose Inverse (BRMP) established the recursive analytic foundation. ACIL then introduced absolute memorization and privacy protection for class-incremental learning.

2023

Few-shot extension: GKEAL

GKEAL embedded analytic learning into Gaussian kernels, extending the framework to few-shot class-incremental learning.

2024

Generalization, online learning, and multimodal learning

DS-AL, GACL, F-OAL, MMAL, RAIL, and REAL improved fitting capacity and extended the branch to generalized, online, multimodal, and vision-language settings.

2025

Foundation models and broader applications

Any-SSR, L3A, AnalyticKWS, CFSSeg, CrossACL, MICIL, ReFu, and 3D-AOCL expanded analytic learning to LLMs, multi-label recognition, speech, segmentation, remote sensing, and 3D perception. AFL further opened the analytic federated learning direction.

2026

Deep analytic learning

PAL, DeepAFL, and new analyses of analytic classifier sub-optimality continue the line through missing-modality learning, deep analytic federated learning, and stronger theoretical understanding.

Representative Works

ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protection

NeurIPS 2022. The founding work of analytic continual learning.

Paper

GKEAL: Gaussian Kernel Embedded Analytic Learning for Few-Shot Class Incremental Task

CVPR 2023. Kernelized analytic learning for few-shot increments.

Paper

GACL: Exemplar-Free Generalized Analytic Continual Learning

NeurIPS 2024. Extends analytic learning to generalized class-incremental settings.

Paper

F-OAL: Forward-only Online Analytic Learning with Fast Training and Low Memory Footprint in Class Incremental Learning

NeurIPS 2024. Fast online analytic learning with low memory footprint.

Paper

MMAL: Multi-Modal Analytic Learning for Exemplar-Free Audio-Visual Class Incremental Tasks

ACM MM 2024. Extends analytic continual learning to audio-visual class-incremental tasks.

Paper

Advancing Cross-domain Discriminability in Continual Learning of Vision-Language Models

NeurIPS 2024. Recursive ridge regression for cross-domain continual learning of vision-language models.

Paper

AFL: A Single-Round Analytic Approach for Federated Learning with Pre-trained Models

CVPR 2025. Analytic aggregation for one-round federated learning.

Paper

Any-SSR: How Recursive Least Squares Works in Continual Learning of Large Language Models

ICCV 2025. Recursive least squares for continual learning of large language models.

Paper

AnalyticKWS: Towards Exemplar-Free Analytic Class Incremental Learning for Small-footprint Keyword Spotting

ACL Findings 2025. Brings analytic continual learning to small-footprint speech tasks.

Paper

DeepAFL: Deep Analytic Federated Learning

ICLR 2026. Extends analytic federated learning toward deep architectures.

Paper

Complete Publication Map

Cross-checked against the public Google Scholar record, DBLP, publisher pages, and the lab publication list. Only formally published versions are listed. Earlier titles such as AOCIL→F-OAL, G-ACIL→GACL, ASR→Any-SSR, and SegACIL→CFSSeg are not listed separately.

Foundations of Analytic and Recursive Learning

• 2019A Low-Memory Learning Formulation for a Kernel-and-Range Network. IJCNN 2019.
• 2020Training a multilayer network with low-memory kernel-and-range projection. Journal of the Franklin Institute.
• 2021Correlation Projection for Analytic Learning of a Classification Network. Neural Processing Letters.
• 2021Training Multilayer Neural Networks Analytically Using Kernel Projection. ISCAS 2021.
• 2022Blockwise Recursive Moore-Penrose Inverse for Network Learning. IEEE TSMC.
• 2025An analytic formulation of convolutional neural network learning for pattern recognition. Information Sciences.

Core Analytic Continual Learning and Task Extensions

• 2022ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protection. NeurIPS 2022.
• 2023GKEAL: Gaussian Kernel Embedded Analytic Learning for Few-Shot Class Incremental Task. CVPR 2023.
• 2024DS-AL: A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learning. AAAI 2024.
• 2024GACL: Exemplar-Free Generalized Analytic Continual Learning. NeurIPS 2024.
• 2024F-OAL: Forward-only Online Analytic Learning with Fast Training and Low Memory Footprint in Class Incremental Learning. NeurIPS 2024.
• 2024MMAL: Multi-Modal Analytic Learning for Exemplar-Free Audio-Visual Class Incremental Tasks. ACM MM 2024.
• 2024Advancing Cross-domain Discriminability in Continual Learning of Vision-Language Models. NeurIPS 2024.
• 2024Class incremental learning with analytic learning for hyperspectral image classification. Journal of the Franklin Institute.
• 2025REAL: Representation enhanced analytic learning for exemplar-free class-incremental learning. Knowledge-Based Systems.
• 2025Analytic Class Incremental Learning for Sound Source Localization With Privacy Protection. IEEE Signal Processing Letters.
• 2025Online Analytic Exemplar-Free Continual Learning With Large Models for Imbalanced Autonomous Driving Task. IEEE Transactions on Vehicular Technology.
• 2025ReFu: Recursive Fusion for Exemplar-Free 3D Class-Incremental Learning. WACV 2025.
• 20253D-AOCL: Analytic online continual learning for imbalanced 3D point cloud classification. Alexandria Engineering Journal.
• 2025L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning. ICML 2025.
• 2025Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-Incremental Learning. ICML 2025.
• 2025CrossACL: Analytic Continual Learning via Feature Cross for Hyperspectral Image Classification. IEEE GRSL.
• 2025Multi-modality integrated class incremental learning networks for 3D object recognition. Knowledge-Based Systems.
• 2025CFSSeg: Closed-Form Solution for Class-Incremental Semantic Segmentation of 2D Images and 3D Point Clouds. ACM MM 2025.
• 2025Analytic Continual Test-Time Adaptation for Multi-Modality Corruption. ACM MM 2025.
• 2025AnalyticKWS: Towards Exemplar-Free Analytic Class Incremental Learning for Small-footprint Keyword Spotting. ACL Findings 2025.
• 2025Any-SSR: How Recursive Least Squares Works in Continual Learning of Large Language Models. ICCV 2025.
• 2026PAL: Prompting analytic learning with missing modality for multi-modal class-incremental learning. Pattern Recognition.
• 2026Rethinking Obscured Sub-Optimality in Analytic Learning for Exemplar-Free Class-Incremental Learning. IEEE TCSVT.

Analytic Federated Learning and Continual Services

• 2025AFL: A Single-Round Analytic Approach for Federated Learning with Pre-trained Models. CVPR 2025.
• 2025CALM: A Ubiquitous Crowdsourced Analytic Learning Mechanism for Continual Service Construction with Data Privacy Preservation. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies.
• 2026DeepAFL: Deep Analytic Federated Learning. ICLR 2026.

Back to Continual Learning View Publications Google Scholar