nlsq.recovery module¶

Recovery strategies for optimization failures.

This module provides automatic recovery mechanisms for handling optimization failures with multiple retry strategies.

class nlsq.stability.recovery.OptimizationRecovery(max_retries=3, enable_diagnostics=True)[source]

Bases: object

Automatic recovery from optimization failures.

This class provides multiple recovery strategies for handling optimization failures including: - Parameter perturbation - Algorithm switching - Regularization adjustment - Problem reformulation - Multi-start optimization

max_retries

Maximum number of recovery attempts

Type:: int

strategies

List of recovery strategies to try

Type:: list

diagnostics

Diagnostics collector for monitoring

Type:: nlsq.diagnostics.OptimizationDiagnostics

stability_guard

Numerical stability checker

Type:: nlsq.stability.NumericalStabilityGuard

__init__(max_retries=3, enable_diagnostics=True)[source]

Initialize recovery system.

Parameters:

max_retries (int) – Maximum recovery attempts
enable_diagnostics (bool) – Enable diagnostic collection

recover_from_failure(failure_type, optimization_state, optimization_func, **kwargs)[source]

Attempt recovery from optimization failure.

Parameters:

failure_type (str) – Type of failure (‘convergence’, ‘numerical’, ‘memory’, etc.)
optimization_state (dict) – Current state of optimization
optimization_func (callable) – Function to retry optimization
**kwargs – Additional arguments for optimization function

Returns:

success (bool) – Whether recovery succeeded
result (dict) – Recovered optimization result or error info

Return type:

tuple[bool, dict]

Overview¶

The nlsq.recovery module provides automatic recovery mechanisms for handling optimization failures. When curve fitting encounters problems like convergence failures, numerical instabilities, or ill-conditioned systems, the recovery system intelligently applies various strategies to achieve successful optimization.

New in version 0.1.1: Complete recovery system with multi-strategy fallback chains.

Key Features¶

Automatic parameter perturbation to escape local minima
Algorithm switching between TRF, LM, and Dogbox methods
Regularization adjustment for ill-conditioned problems
Problem reformulation with data normalization and bound relaxation
Multi-start optimization from different initial points
Recovery history tracking for diagnostics and debugging
Configurable retry strategies with customizable priorities

Classes¶

OptimizationRecovery([max_retries, ...])

Automatic recovery from optimization failures.

Recovery Strategies¶

The OptimizationRecovery class implements five recovery strategies, tried in order:

Parameter Perturbation: Add Gaussian noise to escape local minima
Algorithm Switching: Try alternative optimization algorithms (TRF → LM → Dogbox)
Regularization Adjustment: Increase regularization for numerical stability
Problem Reformulation: Normalize data and relax bounds
Multi-start: Generate new random starting points

Usage Examples¶

Basic Recovery¶

Use recovery to automatically retry failed optimizations:

from nlsq.recovery import OptimizationRecovery
from nlsq import LeastSquares
import numpy as np

# Create recovery system
recovery = OptimizationRecovery(max_retries=3, enable_diagnostics=True)


# Define optimization function
def optimize(params, regularization=0):
    ls = LeastSquares()
    return ls.least_squares(
        residual_func, params, method="trf", regularization=regularization
    )


# Initial optimization attempt
try:
    result = optimize(initial_params)
    if not result.success:
        # Attempt recovery
        success, recovered_result = recovery.recover_from_failure(
            failure_type="convergence",
            optimization_state={"params": initial_params, "method": "trf"},
            optimization_func=optimize,
        )
        if success:
            print("Recovery succeeded!")
            result = recovered_result
except Exception as e:
    print(f"Optimization failed: {e}")

Advanced Recovery with Custom State¶

Provide detailed optimization state for better recovery:

from nlsq.recovery import OptimizationRecovery

recovery = OptimizationRecovery(max_retries=5)

optimization_state = {
    "params": current_params,
    "p0": initial_guess,
    "method": "trf",
    "xdata": x_data,
    "ydata": y_data,
    "bounds": (lower_bounds, upper_bounds),
    "iteration": 150,
    "cost": 1.5e-3,
    "has_outliers": True,
    "regularization": 1e-8,
}

success, result = recovery.recover_from_failure(
    failure_type="numerical",
    optimization_state=optimization_state,
    optimization_func=my_optimization_function,
    additional_kwarg1=value1,
    additional_kwarg2=value2,
)

if success:
    print(f"Recovered with cost: {result['cost']}")
    print(f"Recovery history: {recovery.recovery_history}")

Recovery from Different Failure Types¶

Handle specific failure modes:

# Convergence failure - try parameter perturbation
success, result = recovery.recover_from_failure("convergence", state, opt_func)

# Numerical issues - increase regularization
success, result = recovery.recover_from_failure("numerical", state, opt_func)

# Ill-conditioned system
success, result = recovery.recover_from_failure("ill_conditioned", state, opt_func)

# Outlier-contaminated data - switch to robust loss
state["has_outliers"] = True
success, result = recovery.recover_from_failure("outliers", state, opt_func)

Diagnostics and Monitoring¶

Track recovery attempts and performance:

from nlsq.recovery import OptimizationRecovery

recovery = OptimizationRecovery(max_retries=3, enable_diagnostics=True)

# After multiple recovery attempts
success, result = recovery.recover_from_failure("convergence", state, opt_func)

# Check recovery history
for attempt in recovery.recovery_history:
    print(f"Failure type: {attempt['failure_type']}")
    print(f"Iteration: {attempt['iteration']}")
    print(f"Cost: {attempt['cost']}")

# Access diagnostics if enabled
if recovery.enable_diagnostics:
    events = recovery.diagnostics.get_events()
    for event in events:
        if event["type"] == "recovery_success":
            print(f"Successful strategy: {event['data']['strategy']}")
            print(f"Retry number: {event['data']['retry']}")

Integration with curve_fit¶

Recovery can be integrated into curve fitting workflows:

from nlsq import curve_fit
from nlsq.recovery import OptimizationRecovery
import jax.numpy as jnp


def fit_with_recovery(f, xdata, ydata, p0, **kwargs):
    """Curve fit with automatic recovery."""
    recovery = OptimizationRecovery(max_retries=3)

    try:
        # Try standard fit
        popt, pcov = curve_fit(f, xdata, ydata, p0=p0, **kwargs)
        return popt, pcov
    except Exception as e:
        # Attempt recovery
        state = {
            "params": p0,
            "p0": p0,
            "xdata": xdata,
            "ydata": ydata,
            "method": kwargs.get("method", "trf"),
            "bounds": kwargs.get("bounds"),
        }

        def opt_func(**state_args):
            merged_kwargs = {**kwargs, **state_args}
            return curve_fit(f, xdata, ydata, **merged_kwargs)

        success, result = recovery.recover_from_failure("convergence", state, opt_func)

        if success:
            return result
        else:
            raise RuntimeError("Recovery failed") from e


# Use it
popt, pcov = fit_with_recovery(
    lambda x, a, b: a * jnp.exp(-b * x), x_data, y_data, p0=[1.0, 0.5]
)

Performance Considerations¶

Recovery adds robustness but increases computation time:

Typical overhead per strategy:

Parameter perturbation: +10-50% (depending on noise level)
Algorithm switching: +20-100% (full re-optimization with new method)
Regularization adjustment: +5-20% (minor overhead)
Problem reformulation: +15-30% (data transformation costs)
Multi-start: +50-200% (random sampling and re-optimization)

Total recovery overhead: Cumulative across strategies tried

Best practices:

Set max_retries based on problem complexity (3-5 is typical)
Enable diagnostics only for debugging (adds 5-10% overhead)
Pre-normalize data to reduce need for reformulation strategy
Provide good initial guesses to minimize recovery attempts

Success Rates¶

Typical success rates with recovery enabled:

No recovery: ~60-70% success on difficult problems
With recovery (1 retry): ~75-80% success
With recovery (3 retries, all strategies): ~85-92% success

Recovery is most effective for:

Poorly conditioned Jacobians
Bad initial parameter guesses
Noisy or outlier-contaminated data
Stiff differential equation systems
Multi-modal objective functions