PC-ALM brings local learning closer to backprop

What PC-ALM actually changes

For me, the key result of PC-ALM is that local learning moves closer to backpropagation with a finite number of internal steps. In its paper, “Augmented Lagrangian Predictive Coding,” Sakana AI presents the method as a layer-local scheme with a separate dual variable for every layer. As of September 2026, I read the work primarily as an engineering case for an alternative learning mechanism.

The mechanics are quite specific. Rather than using an argmin search in a standard predictive-coding step, PC-ALM takes a gradient step on an augmented Lagrangian. The dual variable is updated immediately afterward, and the cycle repeats within a fixed inference budget.

The central experimental setting is T=2L, tying the inference budget to network depth. According to the paper, PC-ALM is comparable with backpropagation across the tested width-and-depth combinations up to 128 layers. The result is especially telling for deep, narrow networks: standard predictive coding falls behind there, while the new scheme keeps performance much closer to BP.

There is also a more extreme test. On MNIST, the method trains residual MLPs up to 1,000 layers deep while remaining roughly two percentage points behind backpropagation. Gains over conventional predictive coding also appear on Fashion-MNIST, CIFAR-10, and Tiny ImageNet, including experiments with ResNet-18.

Why this is more than an optimizer trick

PC-ALM looks like a meaningful step for local learning, though not yet like a ready-made replacement for backpropagation. What matters here is not the biological metaphor itself, but the fact that a finite update budget no longer appears to collapse as depth increases in the demonstrated settings.

I would first test sensitivity to T, the memory cost of storing dual variables, and end-to-end training time. Accuracy close to BP does not yet reveal how the method behaves in terms of memory and compute. Transfer beyond the presented image-classification tasks and architectures is equally important.

Sakana AI has substantially narrowed the main gap: local signals no longer have to imply a sharp loss in quality. The harder open question is now how much computational cost is hidden inside that approximation.

We previously covered how IRT metrics can measure the reliability of LLM-as-a-Judge systems. That evaluation perspective also helps frame what new research such as PC-ALM needs to demonstrate in practice.