Self-Healing Neural Networks in PyTorch: Fix Model Drift in Real Time Without Retraining

has been in production two months. Accuracy is 92.9%.

Then transaction patterns shift quietly.

By the time your dashboard turns red, accuracy has collapsed to 44.6%.

Retraining takes six hours—and needs labeled data you won’t have until next week.

What do you do in those six hours?

TL;DR

Problem: Model drifts, retraining unavailable
Solution: Self-healing adapter layer
Key idea: Update a small component, not the full model

System behavior:

• Backbone stays frozen
• Adapter updates in real time
• Updates run asynchronously (no downtime)
• Symbolic rules provide weak supervision
• Rollback ensures safety

Result: +27.8% accuracy recovery — with an explicit recall tradeoff explained inside.

This article is about a ReflexiveLayer: a small architectural component that sits inside the network and adjusts to shifted distributions while the backbone stays frozen. The adapter updates in a background thread so inference never stops. Combined with a symbolic rule engine for weak supervision and a model registry for rollback, it recovered 27.8 percentage points of accuracy in this experiment without touching the backbone weights once.

The results are honest: recovery is real but comes with a recall tradeoff that matters in fraud detection. Both are explained in full.

Full code, all 7 versions, production stack, monitoring export, all plots: https://github.com/Emmimal/self-healing-neural-networks/

Why standard approaches fall short here

When a model starts degrading, the typical playbook is one of three things: retrain on fresh labeled data, use an ensemble that includes a recently trained model, or roll back to a previous checkpoint.

All standard approaches assume you have something you may not:

• Labeled data
• Time to retrain
• A checkpoint that works on the new distribution

Rollback is especially misleading.

Rolling back to clean weights on a shifted distribution doesn’t fix the problem—it repeats it.

What I wanted was something that could operate in the gap: no new labeled data, no downtime, no rollback to a distribution that no longer exists. That constraint shaped the architecture.

While this experiment focuses on fraud detection, the same constraint appears in any production system where retraining is delayed—recommendation engines, risk scoring, anomaly detection, or real-time personalization.

The architecture: one frozen backbone, one trainable adapter

The key design choice is where to put the trainable capacity. Rather than making the whole network adaptable, I isolate adaptation to a single component, the ReflexiveLayer, sandwiched between the frozen backbone and the frozen output head.

Here’s the architecture in one glance:

class ReflexiveLayer(nn.Module):
    def __init__(self, dim):
        super().__init__()
        self.adapter = nn.Sequential(
            nn.Linear(dim, dim), nn.Tanh(),
            nn.Linear(dim, dim)
        )
        self.scale = nn.Parameter(torch.tensor(0.1))

    def forward(self, x):
        return x + self.scale * self.adapter(x)

The residual connection (x + self.scale * self.adapter(x)) is doing important work here. The scale parameter starts at 0.1, so the adapter begins as a near-zero perturbation. The backbone signal passes through almost unmodified. As healing accumulates, scale can grow, but the original backbone output is always present in the signal. The adapter can only add correction; it cannot overwrite what the backbone learned.

The adapter cannot overwrite the model—it can only correct it.

The full model inserts the ReflexiveLayer between the backbone and output head:

class SelfHealingMLP(nn.Module):
    def __init__(self, input_dim=10, hidden_dim=64):
        super().__init__()
        self.backbone = nn.Sequential(
            nn.Linear(input_dim, hidden_dim), nn.ReLU(),
            nn.Linear(hidden_dim, hidden_dim), nn.ReLU()
        )
        self.reflexive = ReflexiveLayer(hidden_dim)
        self.output_head = nn.Sequential(
            nn.Linear(hidden_dim, 1), nn.Sigmoid()
        )

    def freeze_for_healing(self):
        for p in self.backbone.parameters():
            p.requires_grad = False
        for p in self.output_head.parameters():
            p.requires_grad = False

    def unfreeze_all(self):
        for p in self.parameters():
            p.requires_grad = True

During a heal event, freeze_for_healing() is called first. Only the ReflexiveLayer receives gradient updates. After healing, unfreeze_all() restores the full parameter graph in case a full retrain is eventually run.

One thing worth noting about the parameter counts: the model has 13,250 parameters total, and the ReflexiveLayer holds 8,321 of them (two 64×64 linear layers plus the scalar scale). That is 62.8% of the total. The backbone, which maps 10 input features up through 64 hidden units across two layers, holds only 4,864. So the adapter is not “small” in parameter count. It is architecturally focused: its job is limited to transforming the backbone’s hidden representations, and the residual connection plus frozen backbone ensure it cannot destroy what was learned during training.

The reason this split matters: catastrophic forgetting (the tendency of neural networks to lose previously learned behavior when updated on new data) is limited because the backbone is always frozen during healing. The gradient flow during heal steps only touches the adapter, so the foundational representations cannot degrade regardless of how many heal events occur.

Two signals that decide when to heal

Healing triggered too frequently wastes compute. Healing triggered too late lets degradation accumulate. The system uses two independent signals.

Signal one: FIDI (Feature-based Input Distribution Inspection)

FIDI monitors the rolling mean of feature V14, the feature the network independently identified as its strongest fraud signal in Neuro-Symbolic AI Experiment. It computes a z-score against calibration statistics from training:

FIDI | μ=-0.363  σ=1.323  threshold=1.0

V14 clean | mean=-0.377  pct<-1.5 = 18.8%
V14 drift | mean=-2.261  pct<-1.5 = 77.4%

When the z-score exceeds 1.0, the incoming data no longer looks like the training distribution. In this experiment the z-score crosses the threshold at batch 3 and stays elevated. The drifted V14 distribution has a mean 1.9 standard deviations below calibration, and this drift is applied as a constant shift for all 25 batches. The system correctly detects it and never returns to HEALTHY.

Signal two: symbolic conflicts

The SymbolicRuleEngine encodes one domain rule: if V14 < -1.5, the transaction is likely fraud. A conflict occurs when the neural network assigns a low fraud probability (below 0.30) to a transaction the rule flags. When five or more conflicts appear in a batch, a heal is triggered even without a significant z-score.

The two signals complement each other. FIDI is sensitive to overall distribution shift in V14’s mean. Conflict counting is sensitive to model-rule disagreement on specific samples and can catch localized degradation that a distribution-level z-score might miss. The dataset has 15.0% fraud (150 fraud transactions in the 1,000-sample test set).

Async healing: weight updates that do not interrupt inference

The most production-critical design decision here is that healing never blocks inference. A background thread processes heal requests from a queue. An RLock (reentrant lock) protects the shared model state.

class AsyncHealingEngine:
    def __init__(self, model):
        self.model = model
        self._lock = threading.RLock()
        self._queue = queue.Queue()
        self._worker = threading.Thread(
            target=self._heal_worker, daemon=True
        )
        self._worker.start()

    def predict(self, X):
        with self._lock:            # brief lock, just a forward pass
            self.model.eval()
            with torch.no_grad():
                return self.model(X)

    def request_heal(self, X, y, symbolic, batch_idx, fraud_frac=0.0):
        self._queue.put({           # non-blocking, returns immediately
            "X": X.clone(), "y": y.clone(),
            "symbolic": symbolic,
            "batch_idx": batch_idx,
            "fraud_frac": fraud_frac,
        })

request_heal() returns immediately. The inference thread never waits. The heal worker picks up the job, acquires the lock, runs the gradient steps, and releases. The daemon=True flag ensures the background thread exits when the main process terminates without leaving orphaned threads.

What happens during a heal

The heal combines three loss components into one objective:

total_loss = 0.70 * real_loss + 0.24 * consistency_loss + 0.03 * entropy

(The coefficients come from alpha=0.70 and lambda_lag=0.80, so the consistency term is (1 - 0.70) * 0.80 = 0.24.)

Real data loss (ground truth)

Real data loss is weighted binary cross-entropy against the incoming batch labels. The fraud weight scales with the observed fraud fraction among conflicted samples:

fraud_frac = 0%    ->  pos_weight = 1.0  (no adjustment)
fraud_frac = 10%   ->  pos_weight = 2.0
fraud_frac = 20%   ->  pos_weight = 3.0
fraud_frac >= 30%  ->  pos_weight = 4.0  (cap)

The condition fraud_frac >= 0.10 acts as a gate: below that, the model adapts symmetrically. On batches where conflicted transactions turn out to be mostly legitimate, aggressive fraud weighting would push the adapter in the wrong direction. This gating prevents that.

Consistency loss (symbolic guidance)

Consistency loss is binary cross-entropy against the symbolic rule engine’s predictions. Even without ground-truth labels, the symbolic rule provides a stable weak supervision signal that keeps the adapter aligned with domain knowledge rather than overfitting to whatever pattern happens to dominate the current batch. This is the neuro-symbolic anchor described in Hybrid Neuro-Symbolic Fraud Detection and Neuro-Symbolic AI Experiment.

Entropy minimization (confidence recovery)

Entropy minimization (weight 0.03) pushes predictions toward more confident values. Under drift, models often become uncertain across many transactions rather than confidently wrong about specific ones. Call it decision-boundary paralysis. Minimizing entropy counteracts this without dominating the other loss terms.

Only five gradient steps are taken per heal. A 100-sample batch is not enough data to safely take large gradient steps. Five steps nudge the adapter toward the new distribution without committing to any single batch’s signal.

The shadow model: an honest counterfactual

Any online adaptation system needs an answer to a basic question: is the adaptation actually helping? To measure this, a frozen copy of the baseline model (the “shadow model”) runs in parallel every batch and never adapts. The lift metric is simply:

acc_lift = healed_accuracy - shadow_accuracy

In this experiment, lift is positive on every one of the 25 batches, ranging from +0.050 to +0.360. The shadow model provides the honest baseline: what you would get if you did nothing.

Understanding the full results honestly

The final evaluation runs on the full 1,000-sample drifted test set after all 25 streaming batches:

Stage                              Acc      Prec    Recall    F1
------------------------------------------------------------------
Clean Baseline                    92.9%    0.784    0.727    0.754
Under Drift, No Healing           44.6%    0.194    0.853    0.316
Shadow, Frozen                    44.6%    0.194    0.853    0.316
Production Self-Healed            72.4%    0.224    0.340    0.270

The accuracy recovery is genuine. The healed model reaches 72.4% on data the baseline collapses on, a 27.8 percentage point improvement over any frozen alternative.

As seen in the production logs, the healed model catches fewer total frauds (Recall 0.34) but stops the ‘false positive explosion’ that occurs when a drifted model loses its decision boundary.

But the recall numbers need explanation, because a naive read of this table can be misleading.

What “recall 0.853 at 44.6% accuracy” actually means

The confusion matrix for the no-healing model under drift:

No-Healing:  TP=128  TN=318  FP=532  FN=22
Healed:      TP=51   TN=673  FP=177  FN=99

The no-healing model catches 128 out of 150 fraud cases (recall 0.853). But it also generates 532 false positives, flagging 532 legitimate transactions as fraud. Accuracy is 44.6% because nearly half the predictions are wrong. In a payment fraud system, 532 false positives in a 1,000-transaction batch means the model has effectively lost its decision boundary. It is flagging everything suspicious. Operations teams drowning in false alarms is often the first sign that a production model has drifted badly.

The healed model catches 51 out of 150 fraud cases (recall 0.340) while producing only 177 false positives. It misses more fraud, but its predictions are far more reliable.

F1 does not capture this tradeoff

F1 treats false positives and false negatives symmetrically. The no-healing model’s F1 is 0.316 and the healed model’s F1 is 0.270. By F1 alone, the no-healing model looks better. But F1 does not account for the cost structure of the problem. In most payment fraud systems, the cost of a false positive (a blocked legitimate transaction) is not zero, and the ratio of cost between false positives and false negatives determines which model behavior is preferable.

If missing a fraud transaction costs $5,000 on average and a false positive costs $15 in customer support and churn risk, the no-healing model’s behavior might be worth its 532 false positives to catch more fraud. If your review queue has a hard capacity and a false positive costs closer to $200 in operational overhead, the healed model’s 177 false positives and higher accuracy are clearly better.

The point is: this is a deployment decision, not a model quality decision. The tradeoff exists because the adapter learns that V14’s shifted range is no longer a reliable fraud signal in isolation. That is the correct adaptation for the distribution change applied. Whether it serves your specific deployment context requires knowing your cost structure.

Model registry and rollback: the safety net

Every heal event creates two snapshots: one before the heal and one after. Post-heal snapshots are tagged and form the pool of rollback candidates. The health monitor tracks a rolling window of F1 scores and compares them to a baseline established at the first successful heal.

If rolling F1 drops more than 8 percentage points below that baseline, the rollback engine restores the highest-F1 post-heal snapshot. It targets post-heal snapshots specifically, not the original clean weights.

This distinction matters. In Neuro-Symbolic Fraud Detection: Catching Concept, the drift monitoring approach demonstrated that rolling back to pre-drift weights on a drifted distribution reproduces the same failure. The best available state is whichever post-heal snapshot performed best on the drifted data, not the clean-data baseline.

v21 | batch=10 | acc=0.710 | f1=0.408 | post-heal [BEST]

In this experiment, no rollback was triggered across 25 batches. The rollback_f1_drop threshold is set conservatively at 0.08 and the heal quality was consistently above it. That is a good result but not a test of the rollback path. To exercise it deliberately: set rollback_f1_drop = 0.03 and drift_strength = 3.5. The adapter will start receiving conflicting update signals from noisy late batches, F1 will dip below the tightened threshold, and the engine will restore v21. Running this before any production deployment is worthwhile.

System state over time

The model moves through four states during a production run:

HEALTHY: no drift signal, no symbolic conflicts above threshold. No healing occurs.

DRIFTING: FIDI z-score is elevated or conflict count exceeds the minimum. Healing is triggered each batch.

HEALING: the transient state during an active heal event. Inference continues on the current weights until the background thread completes and the lock is released.

ROLLED_BACK: healing degraded performance beyond the configured threshold and the registry restored a prior snapshot.

In this experiment, the system is HEALTHY for batches 1 and 2, then enters DRIFTING at batch 3 and stays there for the remainder of the run. Given that the synthetic drift is applied as a permanent constant shift (V14 mean moves by 1.9 standard deviations and stays there), the z-score never returns below the threshold. In a real deployment with gradual or intermittent drift, you would expect to see more oscillation between states.

Production monitoring export

After every run, the system exports three files to monitoring_export/:

metrics.csv: one row per batch, with accuracy, F1, precision, recall, z-score, conflict count, acc lift vs shadow, and system state. This format imports directly into Grafana as a CSV data source or loads into pandas for ad-hoc analysis.

events.json: one entry per non-trivial action (heal triggers, rollbacks). Structured for ELK or any log aggregation system.

threshold_config.json: the current rollback thresholds in a standalone file:

{
  "rollback_f1_drop": 0.08,
  "rollback_acc_drop": 0.10,
  "health_window": 5,
  "note": "Edit values and restart to tune risk tolerance"
}

Separating thresholds into their own file means the operations team can adjust risk tolerance without touching model code. Model owners control architecture and training parameters. Operations controls alerting and rollback thresholds. These are different decisions made by different people on different timescales.

What this approach does not solve

It requires at least one symbolic rule. The consistency loss keeps the adapter from overfitting to noisy batches. Without some form of domain anchor (a rule, a soft label, a teacher model), the heal degrades to fitting the adapter on small samples with only the real data loss, which produces unstable updates. If you cannot express even one domain rule, this approach needs a different weak supervision source.

Recovery is bounded by the frozen backbone. The backbone learned representations from clean data. If drift is severe enough that those representations contain no useful signal, the adapter cannot compensate. In this experiment the backbone’s representations remain partially useful because V14 is still the most informative feature, just shifted in mean. A drift that introduces an entirely new fraud mechanism the backbone never saw would exhaust what the adapter can fix. This system buys time on gradual distributional shift. It does not replace retraining.

The recall tradeoff is real and deployment-specific. The healed model reduces false positives substantially but misses more fraud. This is a consequence of the adapter learning that V14’s new range is no longer a clean fraud signal. Whether that tradeoff is acceptable depends on your cost structure.

The rollback system was not stress-tested in this run. Zero rollbacks in 25 batches means the heal quality stayed above the configured threshold throughout. That is not a test of the rollback path. Exercise it explicitly before relying on it in production.

How this fits the series

Hybrid Neuro-Symbolic Fraud Detection embedded analyst-written rules directly into the training loss. The gain over a pure neural baseline was real but smaller than the framing suggested. The symbolic component helps most when training data is noisy or label-sparse.

Neural Network Learned Its Own Fraud Rules reversed the direction: let the gradient discover rules rather than having them provided. The network independently identified V14 as its strongest fraud signal without being told to look for it. That convergence between gradient findings and domain expert knowledge is what makes V14 monitoring meaningful.

Neuro-Symbolic Fraud Detection: Catching Concept Drift Before F1 Drops used learned rule activations as a drift canary, monitoring rule agreement rates to detect distribution shift before model metrics visibly declined. That article left the response question open.

This article is the response. FIDI and symbolic conflict detection trigger healing (developed in Neuro-Symbolic Fraud Detection: Catching Concept Drift Before F1 Drops). The symbolic rule provides the consistency signal during healing (the loss architecture from Hybrid Neuro-Symbolic Fraud Detection and Neural Network Learned Its Own Fraud Rules). The reflexive adapter provides the trainable capacity to absorb the shift.

V14 connects all four articles. It appeared in the hybrid loss in Hybrid Neuro-Symbolic Fraud Detection. The gradient found it without guidance in Neural Network Learned Its Own Fraud Rules. Its distribution change was the drift canary in Neuro-Symbolic Fraud Detection: Catching Concept Drift Before F1 Drops. Here its shift is the drift being recovered from. In real fraud datasets, a small number of features carry most of the discriminative signal, and those features are also the ones that change most meaningfully when fraud patterns evolve.

Running it yourself

The full implementation is a single Python file that uses only a fully synthetic, generic dataset generated on-the-fly inside the script. No external or real-world datasets are loaded. The generator creates a 10-feature tabular problem with a 15% fraud ratio and applies a controlled mean shift to one sensitive feature (called “V14” for continuity across the series) to simulate concept drift.

All code is available at: https://github.com/Emmimal/self-healing-neural-networks/

# 1. Make sure you're in the correct directory
cd production

# 2. Install the required packages (only these three are needed)
pip install torch numpy matplotlib

# 3. Run the script
python self_healing_production_final.py

Expected runtime is under two minutes on CPU. The run generates 8 plots and the three monitoring export files in monitoring_export/.

Key Parameters

To see the rollback system trigger: set rollback_f1_drop = 0.03 and drift_strength = 3.5. The adapter will receive conflicting update signals from noisy late batches, F1 will dip below the tightened threshold, and the rollback engine will restore the best post-heal snapshot (batch 10, F1=0.408). Running this deliberately is the right way to verify the safety net before trusting it.

Key takeaway: You don’t need to retrain the whole model to survive drift—you need a controlled place for adaptation.

Summary

A frozen-backbone architecture with a trainable ReflexiveLayer adapter recovered 27.8 percentage points of accuracy under distribution shift, without retraining, without labeled data, and without blocking inference. The recovery comes from three combined mechanisms: the adapter absorbs the distribution shift, the symbolic rule consistency loss keeps the adapter anchored during healing, and the conditional fraud weighting scales the loss to the fraud rate observed in incoming batches.

The tradeoffs are real. Recall drops from 0.853 to 0.340 because the adapter correctly learns that V14’s shifted range is no longer a clean fraud signal. Whether that tradeoff is acceptable depends on the cost structure of the deployment. For a system where false positive cost is high and review capacity is limited, the healed model’s behavior is clearly preferable. For a system where missing fraud is catastrophic, the numbers need careful evaluation before deploying this approach.

The rollback and registry infrastructure, the monitoring export, and the tunable thresholds are not cosmetic. In a production system affecting real transactions, you need visibility into model behavior, the ability to revert if healing degrades performance, and a clean separation between model tuning and operational threshold tuning. The architecture here tries to provide that infrastructure alongside the core adaptation mechanism.

What the system cannot do: recover from drift that makes the backbone’s representations obsolete, operate without any domain rule for weak supervision, or replace a full retrain when fraud patterns change fundamentally. It buys time on gradual distributional shift. For most production fraud systems, gradual shift is the common case.

The question is no longer whether models can adapt in real time. It is whether we are guiding that adaptation in the right direction.

Disclosure

This article is based on independent experiments using a fully synthetic dataset generated entirely in code. No real transaction data, no external datasets, no proprietary information, and no confidential data were used at any point.

The synthetic data generator creates a simple 10-feature tabular problem with a 15% fraud ratio and applies a controlled mean shift to one feature to simulate concept drift. While the design draws loose inspiration from general statistical patterns commonly observed in public fraud detection benchmarks, no actual data from the ULB Credit Card Fraud dataset (Dal Pozzolo et al., 2015) — or any other real dataset — was loaded, copied, or used.

All results are fully reproducible using the single Python file provided in the repository. The views and conclusions expressed here are my own and do not represent any employer or organization.

GitHub: https://github.com/Emmimal/self-healing-neural-networks/

References

[1] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., Hassabis, D., Clopath, C., Kumaran, D., and Hadsell, R. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13), 3521-3526. https://doi.org/10.1073/pnas.1611835114

[2] Python Software Foundation. (2024). threading: Thread-based parallelism. Python 3 Documentation. https://docs.python.org/3/library/threading.html

[3] Powers, D. M. W. (2011). Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation. Journal of Machine Learning Technologies, 2(1), 37-63. https://arxiv.org/abs/2010.16061

[4] Gama, J., Zliobaite, I., Bifet, A., Pechenizkiy, M., and Bouchachia, A. (2014). A survey on concept drift adaptation. ACM Computing Surveys, 46(4), Article 44. https://doi.org/10.1145/2523813

[5] Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., and Zhang, G. (2018). Learning under concept drift: A review. IEEE Transactions on Knowledge and Data Engineering, 31(12), 2346-2363. https://doi.org/10.1109/TKDE.2018.2876857

[6] Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S. (2019). Parameter-efficient transfer learning for NLP. Proceedings of the 36th International Conference on Machine Learning (ICML). https://arxiv.org/abs/1902.00751

[7] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. (2019). PyTorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems (NeurIPS). https://arxiv.org/abs/1912.01703