nlsq.diagnostics package

Model Health Diagnostics System for NLSQ.

This package provides comprehensive diagnostic capabilities for nonlinear least squares curve fitting, including:

  • Identifiability analysis (structural and practical)

  • Gradient health monitoring during optimization

  • Parameter sensitivity spectrum analysis

  • Aggregated health reports with actionable recommendations

  • Plugin system for domain-specific diagnostics

Basic Usage

>>> from nlsq import curve_fit
>>> result = curve_fit(model, x, y, compute_diagnostics=True)
>>> print(result.diagnostics.summary())

Plugin Usage

>>> from nlsq.diagnostics import DiagnosticPlugin, PluginRegistry, PluginResult
>>> class MyPlugin:
...     @property
...     def name(self) -> str:
...         return "my-plugin"
...     def analyze(self, jacobian, parameters, residuals, **context):
...         return PluginResult(plugin_name=self.name, data={}, issues=[])
>>> PluginRegistry.register(MyPlugin())

Exports

Types and Enumerations:

HealthStatus : Overall health status (HEALTHY, WARNING, CRITICAL) IssueSeverity : Issue severity level (INFO, WARNING, CRITICAL) IssueCategory : Issue category (IDENTIFIABILITY, GRADIENT, etc.) DiagnosticLevel : Diagnostic depth (BASIC, FULL) ModelHealthIssue : Single detected issue with recommendation AnalysisResult : Base class for analysis results IdentifiabilityReport : Report from identifiability analysis GradientHealthReport : Report from gradient health monitoring ParameterSensitivityReport : Report from parameter sensitivity analysis PluginResult : Result from a diagnostic plugin ModelHealthReport : Aggregated health report with overall assessment DiagnosticsReport : Aggregated diagnostics report (legacy) DiagnosticsConfig : Configuration for diagnostic computation

Analyzers:

IdentifiabilityAnalyzer : Analyzer for parameter identifiability GradientMonitor : Monitor for gradient health during optimization ParameterSensitivityAnalyzer : Analyzer for parameter sensitivity spectrum

Plugin System:

DiagnosticPlugin : Protocol for custom diagnostic plugins PluginRegistry : Global registry for plugin registration run_plugins : Execute all registered plugins

Factory Functions:

create_health_report : Create aggregated health report from components

Recommendations:

RECOMMENDATIONS : Mapping of issue codes to recommendation text get_recommendation : Get recommendation text for an issue code

class nlsq.diagnostics.AnalysisResult(available=True, error_message=None, computation_time_ms=0.0)[source]

Bases: object

Base class for analysis results.

Provides common attributes for tracking whether an analysis completed successfully, any error messages, and timing information.

available

Whether the analysis completed successfully.

Type:

bool

error_message

Error message if analysis failed.

Type:

str | None

computation_time_ms

Time taken to compute this analysis in milliseconds.

Type:

float

Examples

>>> result = AnalysisResult()
>>> result.available
True
>>> result = AnalysisResult(available=False, error_message="SVD failed")
>>> result.available
False
>>> result.error_message
'SVD failed'
available: bool
error_message: str | None
computation_time_ms: float
__init__(available=True, error_message=None, computation_time_ms=0.0)
class nlsq.diagnostics.DiagnosticLevel(*values)[source]

Bases: Enum

Diagnostic analysis depth level.

BASIC

Fast analysis: identifiability + gradient health.

Type:

auto

FULL

Comprehensive analysis: includes parameter sensitivity analysis.

Type:

auto

BASIC = 1
FULL = 2
class nlsq.diagnostics.DiagnosticPlugin(*args, **kwargs)

Bases: Protocol

Protocol for diagnostic plugins.

Users implement this protocol to create custom diagnostics that integrate with NLSQ’s health report system.

name

Unique plugin name. Should be a short, descriptive identifier. Convention: lowercase with hyphens (e.g., “optical-scattering”).

Type:

str

Protocol
--------
analyze(jacobian, parameters, residuals, \*\*context)

Run the plugin’s analysis and return results.

Example

>>> class OpticalScatteringPlugin:
...     @property
...     def name(self) -> str:
...         return "optical-scattering"
...
...     def analyze(
...         self,
...         jacobian: np.ndarray,
...         parameters: np.ndarray,
...         residuals: np.ndarray,
...         **context
...     ) -> PluginResult:
...         issues = []
...         # Domain-specific analysis...
...         if any(parameters < 0):
...             issues.append(ModelHealthIssue(...))
...         return PluginResult(
...             plugin_name=self.name,
...             data={"my_metric": value},
...             issues=issues,
...         )
...
>>> # Register the plugin
>>> PluginRegistry.register(OpticalScatteringPlugin())
__init__(*args, **kwargs)
analyze(jacobian, parameters, residuals, **context)

Run plugin analysis.

Parameters:

  • jacobian (np.ndarray) – Jacobian matrix at solution (n_residuals x n_params).

  • parameters (np.ndarray) – Fitted parameters.

  • residuals (np.ndarray) – Residuals at solution.

  • **context (Any) – Additional context passed from curve_fit: - xdata: Independent variable data - ydata: Dependent variable data - bounds: Parameter bounds - model: Model function - config: DiagnosticsConfig

Returns:

Analysis results with any detected issues.

Return type:

PluginResult

property name: str

Unique plugin name.

Should be a short, descriptive identifier. Convention: lowercase with hyphens (e.g., “optical-scattering”).

Returns:

The plugin’s unique name.

Return type:

str

class nlsq.diagnostics.DiagnosticsConfig(level=DiagnosticLevel.BASIC, condition_threshold=100000000.0, correlation_threshold=0.95, imbalance_threshold=1000000.0, vanishing_threshold=1e-06, sloppy_threshold=1e-06, gradient_window_size=100, stagnation_window=10, stagnation_tolerance=0.01, verbose=True, emit_warnings=True)[source]

Bases: object

Configuration for diagnostic computation.

This frozen dataclass contains all thresholds and settings used by the diagnostic analyzers. Being frozen ensures configuration immutability during analysis.

level

Diagnostic analysis depth.

Type:

DiagnosticLevel

condition_threshold

FIM condition number threshold for practical identifiability. Default: 1e8.

Type:

float

correlation_threshold

Correlation coefficient threshold for high correlation warning. Default: 0.95.

Type:

float

imbalance_threshold

Gradient imbalance ratio threshold. Default: 1e6.

Type:

float

vanishing_threshold

Relative gradient magnitude threshold for vanishing detection. Default: 1e-6.

Type:

float

sloppy_threshold

Eigenvalue ratio threshold for sensitivity classification. Default: 1e-6.

Type:

float

gradient_window_size

Window size for gradient norm history. Default: 100.

Type:

int

stagnation_window

Number of iterations to check for gradient stagnation. Default: 10.

Type:

int

stagnation_tolerance

Relative tolerance for detecting gradient stagnation. Default: 0.01 (1% change).

Type:

float

verbose

Print diagnostic summary to console. Default: True.

Type:

bool

emit_warnings

Emit Python warnings for critical issues. Default: True.

Type:

bool

Examples

>>> config = DiagnosticsConfig()
>>> config.level
<DiagnosticLevel.BASIC: 1>
>>> config.condition_threshold
100000000.0

>>> config = DiagnosticsConfig(
...     level=DiagnosticLevel.FULL,
...     condition_threshold=1e10,
...     verbose=False,
... )
>>> config.level
<DiagnosticLevel.FULL: 2>
level: DiagnosticLevel
condition_threshold: float
correlation_threshold: float
imbalance_threshold: float
vanishing_threshold: float
sloppy_threshold: float
gradient_window_size: int
stagnation_window: int
stagnation_tolerance: float
verbose: bool
emit_warnings: bool
__post_init__()[source]

Validate configuration values after initialization.

__init__(level=DiagnosticLevel.BASIC, condition_threshold=100000000.0, correlation_threshold=0.95, imbalance_threshold=1000000.0, vanishing_threshold=1e-06, sloppy_threshold=1e-06, gradient_window_size=100, stagnation_window=10, stagnation_tolerance=0.01, verbose=True, emit_warnings=True)
class nlsq.diagnostics.DiagnosticsReport(identifiability=None, gradient_health=None, overall_status=HealthStatus.HEALTHY, computation_time_ms=0.0)[source]

Bases: object

Aggregated diagnostics report containing all analysis results.

This class aggregates results from all diagnostic analyses into a single report. It provides access to individual analysis results and an overall health assessment.

identifiability

Results from identifiability analysis.

Type:

IdentifiabilityReport | None

gradient_health

Results from gradient health monitoring.

Type:

GradientHealthReport | None

overall_status

Overall health status based on all analyses.

Type:

HealthStatus

computation_time_ms

Total time for all diagnostic computations.

Type:

float

Examples

>>> from nlsq.diagnostics.types import IdentifiabilityReport
>>> ident = IdentifiabilityReport(
...     condition_number=1e5,
...     numerical_rank=3,
...     n_params=3,
...     correlation_matrix=np.eye(3),
...     highly_correlated_pairs=[],
...     issues=[],
...     health_status=HealthStatus.HEALTHY,
... )
>>> report = DiagnosticsReport(identifiability=ident)
>>> report.overall_status
<HealthStatus.HEALTHY: 1>
identifiability: IdentifiabilityReport | None
gradient_health: GradientHealthReport | None
overall_status: HealthStatus
computation_time_ms: float
__post_init__()[source]

Compute overall status from individual analyses.

__str__()[source]

Return a human-readable summary of all diagnostics.

summary()[source]

Return a summary string of all diagnostics.

Returns:

Human-readable summary of all diagnostic analyses.

Return type:

str

__init__(identifiability=None, gradient_health=None, overall_status=HealthStatus.HEALTHY, computation_time_ms=0.0)
class nlsq.diagnostics.GradientHealthReport(available=True, error_message=None, computation_time_ms=0.0, n_iterations=0, health_score=1.0, mean_gradient_norm=0.0, final_gradient_norm=0.0, mean_gradient_magnitudes=<factory>, variance_gradient_magnitudes=<factory>, max_imbalance_ratio=1.0, has_numerical_issues=False, vanishing_detected=False, imbalance_detected=False, stagnation_detected=False, issues=<factory>, health_status=HealthStatus.HEALTHY)[source]

Bases: AnalysisResult

Report from gradient health monitoring during optimization.

Contains results from monitoring gradient behavior across iterations, including detection of vanishing gradients, gradient imbalance, and gradient stagnation.

This dataclass extends AnalysisResult to include gradient-specific metrics tracked during optimization using memory-efficient algorithms (sliding window for norms, Welford’s algorithm for running statistics).

Memory usage is bounded at <1KB regardless of iteration count.

n_iterations

Total number of iterations monitored.

Type:

int

health_score

Overall gradient health score in [0, 1]. Higher is healthier.

Type:

float

mean_gradient_norm

Mean gradient norm across all iterations.

Type:

float

final_gradient_norm

Gradient norm at the final iteration.

Type:

float

mean_gradient_magnitudes

Mean gradient magnitude per parameter (from Welford’s algorithm).

Type:

np.ndarray

variance_gradient_magnitudes

Variance of gradient magnitude per parameter (from Welford’s algorithm).

Type:

np.ndarray

max_imbalance_ratio

Maximum ratio between largest and smallest gradient components.

Type:

float

has_numerical_issues

Whether NaN or Inf values were detected in gradients.

Type:

bool

vanishing_detected

Whether vanishing gradients were detected.

Type:

bool

imbalance_detected

Whether gradient imbalance was detected.

Type:

bool

stagnation_detected

Whether gradient stagnation was detected.

Type:

bool

issues

List of detected gradient issues (GRAD-001, GRAD-002, GRAD-003).

Type:

list[ModelHealthIssue]

health_status

Overall health status based on detected issues.

Type:

HealthStatus

Examples

>>> report = GradientHealthReport(
...     n_iterations=100,
...     health_score=0.95,
...     mean_gradient_norm=0.1,
...     final_gradient_norm=0.001,
...     mean_gradient_magnitudes=np.array([0.1, 0.08, 0.12]),
...     variance_gradient_magnitudes=np.array([0.01, 0.01, 0.01]),
...     max_imbalance_ratio=1.5,
...     has_numerical_issues=False,
...     vanishing_detected=False,
...     imbalance_detected=False,
...     stagnation_detected=False,
...     issues=[],
...     health_status=HealthStatus.HEALTHY,
... )
>>> report.available
True
>>> report.health_score
0.95
n_iterations: int
health_score: float
mean_gradient_norm: float
final_gradient_norm: float
mean_gradient_magnitudes: ndarray
variance_gradient_magnitudes: ndarray
max_imbalance_ratio: float
has_numerical_issues: bool
vanishing_detected: bool
imbalance_detected: bool
stagnation_detected: bool
issues: list[ModelHealthIssue]
health_status: HealthStatus
__str__()[source]

Return a human-readable summary of the gradient health report.

summary()[source]

Return a summary string of the report.

Returns:

Human-readable summary of the gradient health analysis.

Return type:

str

__init__(available=True, error_message=None, computation_time_ms=0.0, n_iterations=0, health_score=1.0, mean_gradient_norm=0.0, final_gradient_norm=0.0, mean_gradient_magnitudes=<factory>, variance_gradient_magnitudes=<factory>, max_imbalance_ratio=1.0, has_numerical_issues=False, vanishing_detected=False, imbalance_detected=False, stagnation_detected=False, issues=<factory>, health_status=HealthStatus.HEALTHY)
class nlsq.diagnostics.GradientMonitor(config)[source]

Bases: object

Monitor gradient health during optimization iterations.

This class tracks gradient behavior to detect potential optimization issues such as vanishing gradients, gradient imbalance between parameters, and gradient stagnation. It uses memory-efficient algorithms to ensure memory usage stays below 1KB regardless of iteration count.

Parameters:

config (DiagnosticsConfig) – Configuration containing thresholds and settings for gradient monitoring.

config

Configuration for the monitor.

Type:

DiagnosticsConfig

iteration_count

Total number of iterations recorded.

Type:

int

Examples

>>> from nlsq.diagnostics import DiagnosticsConfig
>>> from nlsq.diagnostics.gradient_health import GradientMonitor
>>> import numpy as np
>>>
>>> config = DiagnosticsConfig()
>>> monitor = GradientMonitor(config)
>>>
>>> # Record gradients during optimization
>>> for i in range(100):
...     gradient = np.array([0.1, 0.08, 0.12]) / (i + 1)
...     monitor.record_gradient(gradient, cost=1.0 / (i + 1))
>>>
>>> report = monitor.get_report()
>>> print(report.health_status)
HealthStatus.HEALTHY

Integration with curve_fit callback:

>>> from nlsq import curve_fit
>>> from nlsq.diagnostics import DiagnosticsConfig, GradientMonitor
>>>
>>> config = DiagnosticsConfig()
>>> monitor = GradientMonitor(config)
>>> callback = monitor.create_callback()
>>>
>>> # Use in curve_fit (gradient is estimated from Jacobian)
>>> # result = curve_fit(model, x, y, p0=p0, callback=callback)
>>> # report = monitor.get_report()

Notes

Memory efficiency is achieved through:

  1. Sliding window: Stores only the last N gradient norms (default 100), using a deque with maxlen for O(1) append/pop.

  2. Welford’s algorithm: Computes running mean and variance in O(1) space per parameter, without storing individual values.

The total memory footprint is approximately: - Sliding window: window_size * 8 bytes (floats) - Per-parameter stats: 3 * n_params * 8 bytes (mean, M2, count) - Overhead: ~100 bytes for scalars and bookkeeping

For 100 window size and 10 parameters: ~900 bytes < 1KB

__init__(config)[source]

Initialize the gradient monitor.

Parameters:

config (DiagnosticsConfig) – Configuration containing monitoring thresholds.

config
iteration_count: int
record_gradient(gradient, cost)[source]

Record a gradient observation from an optimization iteration.

Parameters:

  • gradient (np.ndarray or Sequence[float]) – The gradient vector (partial derivatives w.r.t. each parameter).

  • cost (float) – The current cost/loss value at this iteration.

Raises:

ValueError – If gradient is empty.

Notes

This method uses Welford’s online algorithm to update running statistics for per-parameter gradient magnitudes. This allows computing mean and variance without storing individual values, achieving O(1) memory per parameter.

The algorithm maintains: - mean: Running mean of absolute gradient values - M2: Sum of squared differences from the mean

Variance is computed as M2 / (n - 1) when needed.

create_callback(user_callback=None)[source]

Create a callback function for integration with curve_fit/TRF.

This method creates a callback compatible with NLSQ’s optimization callbacks. The callback extracts gradient information from the optimization state and records it in the monitor.

Parameters:

user_callback (callable, optional) – An optional user callback to chain with the gradient monitor. Will be called after gradient recording with the same arguments.

Returns:

A callback function compatible with curve_fit’s callback parameter.

Return type:

callable

Examples

>>> from nlsq import curve_fit
>>> from nlsq.diagnostics import DiagnosticsConfig, GradientMonitor
>>>
>>> monitor = GradientMonitor(DiagnosticsConfig())
>>> callback = monitor.create_callback()
>>>
>>> # result = curve_fit(model, x, y, p0=p0, callback=callback)
>>> # report = monitor.get_report()

Notes

The callback receives iteration information including: - iteration: Current iteration number - cost: Current cost value - params: Current parameter values - info: Dictionary with gradient_norm, nfev, step_norm, etc.

When the gradient is not directly available, we estimate it from changes in parameters and cost, or use gradient_norm from info.

get_report()[source]

Generate a gradient health report from recorded observations.

Returns:

Report containing gradient health metrics and any detected issues.

Return type:

GradientHealthReport

Notes

The report includes:

  • Overall health score (0-1, higher is healthier)

  • Mean and final gradient norms

  • Per-parameter mean and variance of gradient magnitudes

  • Detection of vanishing gradients, imbalance, and stagnation

  • List of ModelHealthIssue objects for any detected problems

reset()[source]

Reset the monitor to its initial state.

Clears all recorded gradients and statistics. Useful when starting a new optimization run.

class nlsq.diagnostics.HealthStatus(*values)[source]

Bases: Enum

Overall model health status.

HEALTHY

No issues detected, high confidence in results.

Type:

auto

WARNING

Minor issues detected, results may be reliable.

Type:

auto

CRITICAL

Serious issues detected, results may be unreliable.

Type:

auto

HEALTHY = 1
WARNING = 2
CRITICAL = 3
class nlsq.diagnostics.IdentifiabilityAnalyzer(config)[source]

Bases: object

Analyzer for parameter identifiability from Jacobian matrices.

This class analyzes the Fisher Information Matrix (FIM) derived from the Jacobian to assess parameter identifiability. It detects both structural unidentifiability (rank deficiency) and practical unidentifiability (ill-conditioning).

Parameters:

config (DiagnosticsConfig) – Configuration containing thresholds for identifiability analysis.

config

Configuration for the analyzer.

Type:

DiagnosticsConfig

Examples

>>> import numpy as np
>>> from nlsq.diagnostics import DiagnosticsConfig
>>> from nlsq.diagnostics.identifiability import IdentifiabilityAnalyzer
>>> config = DiagnosticsConfig()
>>> analyzer = IdentifiabilityAnalyzer(config)
>>> J = np.random.randn(100, 3)  # 100 data points, 3 parameters
>>> report = analyzer.analyze(J)
>>> print(report.health_status)
HealthStatus.HEALTHY
__init__(config)[source]

Initialize the identifiability analyzer.

Parameters:

config (DiagnosticsConfig) – Configuration containing analysis thresholds.

analyze(jacobian)[source]

Analyze identifiability from a Jacobian matrix.

Computes the Fisher Information Matrix (FIM) as J.T @ J and analyzes it for identifiability issues.

Parameters:

jacobian (np.ndarray) – Jacobian matrix of shape (n_data, n_params).

Returns:

Report containing analysis results and any detected issues.

Return type:

IdentifiabilityReport

Notes

The analysis includes:

  1. FIM computation: FIM = J.T @ J

  2. SVD of FIM for condition number and rank

  3. Correlation matrix extraction

  4. Issue detection based on thresholds

analyze_from_fim(fim)[source]

Analyze identifiability from a pre-computed FIM.

Parameters:

fim (np.ndarray) – Fisher Information Matrix of shape (n_params, n_params).

Returns:

Report containing analysis results and any detected issues.

Return type:

IdentifiabilityReport

class nlsq.diagnostics.IdentifiabilityReport(available=True, error_message=None, computation_time_ms=0.0, condition_number=inf, numerical_rank=0, n_params=0, correlation_matrix=None, highly_correlated_pairs=<factory>, issues=<factory>, health_status=HealthStatus.HEALTHY)[source]

Bases: AnalysisResult

Report from identifiability analysis.

Contains results from analyzing the Fisher Information Matrix (FIM) including condition number, numerical rank, correlation structure, and any detected identifiability issues.

This dataclass extends AnalysisResult to include identifiability-specific information such as condition number, rank, and correlation analysis.

condition_number

Condition number of the FIM. High values (> 1e8) indicate practical unidentifiability.

Type:

float

numerical_rank

Numerical rank of the FIM. If less than n_params, indicates structural unidentifiability.

Type:

int

n_params

Total number of parameters in the model.

Type:

int

correlation_matrix

Parameter correlation matrix derived from FIM. None if computation failed.

Type:

np.ndarray | None

highly_correlated_pairs

List of highly correlated parameter pairs as (i, j, correlation). Only includes pairs with absolute correlation greater than correlation_threshold.

Type:

list[tuple[int, int, float]]

issues

List of detected identifiability issues (IDENT-001, IDENT-002, CORR-001).

Type:

list[ModelHealthIssue]

health_status

Overall health status based on detected issues.

Type:

HealthStatus

Examples

>>> report = IdentifiabilityReport(
...     condition_number=1e5,
...     numerical_rank=3,
...     n_params=3,
...     correlation_matrix=np.eye(3),
...     highly_correlated_pairs=[],
...     issues=[],
...     health_status=HealthStatus.HEALTHY,
... )
>>> report.available
True
>>> report.condition_number
100000.0

>>> # Report with issues
>>> from nlsq.diagnostics.types import ModelHealthIssue, IssueCategory, IssueSeverity
>>> issue = ModelHealthIssue(
...     category=IssueCategory.IDENTIFIABILITY,
...     severity=IssueSeverity.CRITICAL,
...     code="IDENT-001",
...     message="Structural unidentifiability detected",
...     affected_parameters=(0, 1),
...     details={"numerical_rank": 2, "n_params": 3},
...     recommendation="Reparameterize model",
... )
>>> report = IdentifiabilityReport(
...     condition_number=float('inf'),
...     numerical_rank=2,
...     n_params=3,
...     correlation_matrix=None,
...     highly_correlated_pairs=[],
...     issues=[issue],
...     health_status=HealthStatus.CRITICAL,
... )
>>> len(report.issues)
1
condition_number: float
numerical_rank: int
n_params: int
correlation_matrix: ndarray | None
highly_correlated_pairs: list[tuple[int, int, float]]
issues: list[ModelHealthIssue]
health_status: HealthStatus
__str__()[source]

Return a human-readable summary of the identifiability report.

summary()[source]

Return a summary string of the report.

Returns:

Human-readable summary of the identifiability analysis.

Return type:

str

__init__(available=True, error_message=None, computation_time_ms=0.0, condition_number=inf, numerical_rank=0, n_params=0, correlation_matrix=None, highly_correlated_pairs=<factory>, issues=<factory>, health_status=HealthStatus.HEALTHY)
class nlsq.diagnostics.IssueCategory(*values)[source]

Bases: Enum

Category of detected issue.

IDENTIFIABILITY

Parameter identifiability issues.

Type:

auto

GRADIENT

Gradient health issues.

Type:

auto

CORRELATION

Parameter correlation issues.

Type:

auto

CONDITIONING

Numerical conditioning issues.

Type:

auto

CONVERGENCE

Convergence-related issues.

Type:

auto

SENSITIVITY

Parameter sensitivity spectrum issues (wide eigenvalue spread).

Type:

auto

IDENTIFIABILITY = 1
GRADIENT = 2
CORRELATION = 3
CONDITIONING = 4
CONVERGENCE = 5
SENSITIVITY = 6
class nlsq.diagnostics.IssueSeverity(*values)[source]

Bases: Enum

Severity level of a detected issue.

INFO

Informational, no action required.

Type:

auto

WARNING

Potential problem, review recommended.

Type:

auto

CRITICAL

Serious problem, action required.

Type:

auto

INFO = 1
WARNING = 2
CRITICAL = 3
class nlsq.diagnostics.ModelHealthIssue(category, severity, code, message, affected_parameters, details, recommendation)[source]

Bases: object

A single detected model health issue.

This dataclass represents an actionable issue detected during diagnostic analysis, including its category, severity, and a recommendation for addressing it.

category

Category of the issue.

Type:

IssueCategory

severity

Severity level.

Type:

IssueSeverity

code

Unique issue code (e.g., “IDENT-001”, “GRAD-002”).

Type:

str

message

Human-readable description of the issue.

Type:

str

affected_parameters

Indices of affected parameters, if applicable.

Type:

tuple[int, …] | None

details

Additional issue-specific details.

Type:

dict[str, Any]

recommendation

Actionable recommendation for addressing the issue.

Type:

str

Examples

>>> issue = ModelHealthIssue(
...     category=IssueCategory.IDENTIFIABILITY,
...     severity=IssueSeverity.CRITICAL,
...     code="IDENT-001",
...     message="Parameters 0 and 1 are structurally unidentifiable",
...     affected_parameters=(0, 1),
...     details={"numerical_rank": 2, "n_params": 3},
...     recommendation="Consider reparameterizing the model",
... )
>>> issue.code
'IDENT-001'
>>> issue.severity
<IssueSeverity.CRITICAL: 3>
category: IssueCategory
severity: IssueSeverity
code: str
message: str
affected_parameters: tuple[int, ...] | None
details: dict[str, Any]
recommendation: str
__post_init__()[source]

Validate issue attributes after initialization.

__init__(category, severity, code, message, affected_parameters, details, recommendation)
class nlsq.diagnostics.ModelHealthReport(identifiability=None, gradient_health=None, sloppy_model=None, plugin_results=<factory>, status=HealthStatus.HEALTHY, health_score=1.0, all_issues=<factory>, config=None, computation_time_ms=0.0)[source]

Bases: object

Aggregated model health report with overall assessment.

This dataclass aggregates results from all diagnostic components (identifiability, gradient health, parameter sensitivity, and plugins) into a unified health report with overall status, health score, and actionable recommendations.

identifiability

Results from identifiability analysis.

Type:

IdentifiabilityReport | None

gradient_health

Results from gradient health monitoring.

Type:

GradientHealthReport | None

sloppy_model

Results from parameter sensitivity analysis (level=FULL only).

Type:

ParameterSensitivityReport | None

plugin_results

Results from diagnostic plugins, keyed by plugin name.

Type:

dict[str, PluginResult]

status

Overall health status (HEALTHY, WARNING, or CRITICAL).

Type:

HealthStatus

health_score

Overall health score in [0.0, 1.0]. Higher is healthier.

Type:

float

all_issues

Aggregated issues from all components, sorted by severity.

Type:

list[ModelHealthIssue]

config

Configuration used for diagnostics.

Type:

DiagnosticsConfig | None

computation_time_ms

Total computation time for all diagnostics in milliseconds.

Type:

float

Examples

>>> from nlsq.diagnostics.health_report import create_health_report
>>> report = create_health_report(
...     identifiability=healthy_ident_report,
...     gradient_health=healthy_grad_report,
... )
>>> report.status
<HealthStatus.HEALTHY: 1>
>>> report.health_score
1.0
identifiability: IdentifiabilityReport | None
gradient_health: GradientHealthReport | None
sloppy_model: ParameterSensitivityReport | None
plugin_results: dict[str, PluginResult]
status: HealthStatus
health_score: float
all_issues: list[ModelHealthIssue]
config: DiagnosticsConfig | None
computation_time_ms: float
summary(verbose=True)[source]

Generate human-readable summary.

Parameters:

verbose (bool, default=True) – Include detailed issue descriptions and recommendations.

Returns:

Formatted summary string suitable for console output.

Return type:

str

to_dict()[source]

Convert report to dictionary for serialization.

Returns:

Dictionary representation of the report.

Return type:

dict[str, Any]

__str__()[source]

Return summary as string representation.

__init__(identifiability=None, gradient_health=None, sloppy_model=None, plugin_results=<factory>, status=HealthStatus.HEALTHY, health_score=1.0, all_issues=<factory>, config=None, computation_time_ms=0.0)
class nlsq.diagnostics.ParameterSensitivityAnalyzer(config=None)[source]

Bases: object

Analyzer for parameter sensitivity spectrum from Jacobian matrices.

This class analyzes the eigenvalue spectrum of the Fisher Information Matrix (FIM) to identify well-determined vs poorly-determined parameter directions. It detects wide eigenvalue spread when eigenvalues span many orders of magnitude.

Parameters:

config (DiagnosticsConfig | None) – Configuration containing thresholds for sensitivity analysis. If None, uses default configuration.

config

Configuration for the analyzer.

Type:

DiagnosticsConfig

Notes

Detection logic:

  • A model is considered to have wide eigenvalue spread when its eigenvalue range exceeds a threshold derived from the config’s sloppy_threshold.

  • The sloppy_threshold (default 1e-6) defines when a direction is classified as poorly-determined: eigenvalue < threshold * max_eigenvalue.

  • The overall is_sloppy flag is set when eigenvalue range (in orders of magnitude) is significant enough to cause practical issues.

Examples

>>> import numpy as np
>>> from nlsq.diagnostics import DiagnosticsConfig
>>> from nlsq.diagnostics.parameter_sensitivity import ParameterSensitivityAnalyzer
>>> config = DiagnosticsConfig()
>>> analyzer = ParameterSensitivityAnalyzer(config)
>>> J = np.random.randn(100, 3)  # 100 data points, 3 parameters
>>> report = analyzer.analyze(J)
>>> print(report.is_sloppy)
False
DEFAULT_SLOPPY_DETECTION_ORDERS = 2.0
STIFF_RATIO_THRESHOLD = 0.1
SLOPPY_RATIO_THRESHOLD = 0.01
__init__(config=None)[source]

Initialize the parameter sensitivity analyzer.

Parameters:

config (DiagnosticsConfig | None) – Configuration containing analysis thresholds. If None, uses default configuration.

analyze(jacobian)[source]

Analyze parameter sensitivity spectrum from a Jacobian matrix.

Computes the Fisher Information Matrix (FIM) as J.T @ J and analyzes its eigenvalue spectrum for wide eigenvalue spread.

Parameters:

jacobian (np.ndarray) – Jacobian matrix of shape (n_data, n_params).

Returns:

Report containing analysis results and any detected issues.

Return type:

ParameterSensitivityReport

Notes

The analysis includes:

  1. FIM computation: FIM = J.T @ J

  2. Eigenvalue decomposition for spectrum analysis

  3. Eigenvalue range computation (log10 ratio)

  4. Stiff/poorly-determined direction classification

  5. Effective dimensionality using participation ratio

  6. Issue detection based on thresholds

analyze_from_fim(fim)[source]

Analyze parameter sensitivity spectrum from a pre-computed FIM.

Parameters:

fim (np.ndarray) – Fisher Information Matrix of shape (n_params, n_params).

Returns:

Report containing analysis results and any detected issues.

Return type:

ParameterSensitivityReport

class nlsq.diagnostics.ParameterSensitivityReport(available=True, error_message=None, computation_time_ms=0.0, is_sloppy=False, eigenvalues=<factory>, eigenvectors=None, eigenvalue_range=0.0, effective_dimensionality=0.0, stiff_indices=<factory>, sloppy_indices=<factory>, issues=<factory>, health_status=HealthStatus.HEALTHY)[source]

Bases: AnalysisResult

Report from parameter sensitivity spectrum analysis.

Contains results from eigenvalue spectrum analysis to identify well-determined vs poorly-determined parameter directions based on the spread of eigenvalues in the Fisher Information Matrix.

is_sloppy

Whether the model exhibits wide eigenvalue spread (sensitivity spectrum).

Type:

bool

eigenvalues

Eigenvalue spectrum of the Fisher Information Matrix.

Type:

np.ndarray

eigenvectors

Eigenvectors of the FIM (columns are eigenvectors).

Type:

np.ndarray | None

eigenvalue_range

Log10 range of eigenvalues (orders of magnitude).

Type:

float

effective_dimensionality

Effective number of well-determined parameter combinations.

Type:

float

stiff_indices

Indices of stiff (well-determined) directions.

Type:

list[int]

sloppy_indices

Indices of poorly-determined directions.

Type:

list[int]

issues

List of detected sensitivity issues (SENS-001, SENS-002).

Type:

list[ModelHealthIssue]

health_status

Overall health status based on detected issues.

Type:

HealthStatus

is_sloppy: bool
eigenvalues: ndarray
eigenvectors: ndarray | None
eigenvalue_range: float
effective_dimensionality: float
stiff_indices: list[int]
sloppy_indices: list[int]
issues: list[ModelHealthIssue]
health_status: HealthStatus
get_sloppy_combinations()[source]

Get poorly determined parameter combinations.

Returns:

List of (eigenvector, eigenvalue) tuples for poorly-determined directions.

Return type:

list[tuple[np.ndarray, float]]

__init__(available=True, error_message=None, computation_time_ms=0.0, is_sloppy=False, eigenvalues=<factory>, eigenvectors=None, eigenvalue_range=0.0, effective_dimensionality=0.0, stiff_indices=<factory>, sloppy_indices=<factory>, issues=<factory>, health_status=HealthStatus.HEALTHY)
class nlsq.diagnostics.PluginRegistry[source]

Bases: object

Global registry for diagnostic plugins.

Thread-safe singleton for managing plugin registration. Plugins are stored by name and can be registered, unregistered, retrieved, or listed.

This class uses class-level storage and methods, so it acts as a singleton without explicit instantiation.

Example

>>> from nlsq.diagnostics import PluginRegistry
>>> # Register a plugin
>>> PluginRegistry.register(my_plugin)
>>> # Get a plugin by name
>>> plugin = PluginRegistry.get("my-plugin")
>>> # List all plugins
>>> all_plugins = PluginRegistry.all()
>>> # Clear all plugins (for testing)
>>> PluginRegistry.clear()
classmethod register(plugin)[source]

Register a diagnostic plugin.

Parameters:

plugin (DiagnosticPlugin) – Plugin instance to register.

Raises:

ValueError – If a plugin with the same name is already registered.

Example

>>> PluginRegistry.register(MyPlugin())
classmethod unregister(name)[source]

Unregister a plugin by name.

Parameters:

name (str) – Name of the plugin to unregister.

Returns:

True if the plugin was found and removed, False otherwise.

Return type:

bool

Example

>>> PluginRegistry.unregister("my-plugin")
True
classmethod get(name)[source]

Get a plugin by name.

Parameters:

name (str) – Name of the plugin to retrieve.

Returns:

Plugin instance if found, None otherwise.

Return type:

DiagnosticPlugin | None

Example

>>> plugin = PluginRegistry.get("my-plugin")
>>> if plugin is not None:
...     result = plugin.analyze(...)
classmethod all()[source]

Get all registered plugins.

Returns:

List of all registered plugin instances. This is a copy; modifying it does not affect the registry.

Return type:

list[DiagnosticPlugin]

Example

>>> for plugin in PluginRegistry.all():
...     print(plugin.name)
classmethod clear()[source]

Unregister all plugins.

Primarily for testing purposes.

Example

>>> PluginRegistry.clear()
>>> assert len(PluginRegistry.all()) == 0
class nlsq.diagnostics.PluginResult(plugin_name='', available=True, error_message=None, data=<factory>, issues=<factory>, computation_time_ms=0.0)[source]

Bases: object

Result from a diagnostic plugin execution.

plugin_name

Name of the plugin that produced this result.

Type:

str

available

Whether the plugin executed successfully.

Type:

bool

error_message

Error message if plugin execution failed.

Type:

str | None

data

Plugin-specific result data.

Type:

dict[str, Any]

issues

Issues detected by the plugin.

Type:

list[ModelHealthIssue]

computation_time_ms

Time taken for plugin execution.

Type:

float

plugin_name: str
available: bool
error_message: str | None
data: dict[str, Any]
issues: list[ModelHealthIssue]
computation_time_ms: float
__init__(plugin_name='', available=True, error_message=None, data=<factory>, issues=<factory>, computation_time_ms=0.0)
nlsq.diagnostics.create_health_report(identifiability=None, gradient_health=None, sloppy_model=None, plugin_results=None, config=None)[source]

Create aggregated health report from component reports.

This factory function aggregates results from all diagnostic components into a unified ModelHealthReport with: - Overall status determination (HEALTHY/WARNING/CRITICAL) - Health score computation with weighted contributions - Issue aggregation sorted by severity and code - Optional warnings emission for critical issues - Optional verbose console output

Parameters:
Returns:

Complete aggregated report with status, score, and all issues.

Return type:

ModelHealthReport

Examples

>>> from nlsq.diagnostics.health_report import create_health_report
>>> report = create_health_report(
...     identifiability=ident_report,
...     gradient_health=grad_report,
... )
>>> print(report.status)
HealthStatus.HEALTHY
>>> print(report.health_score)
0.95
nlsq.diagnostics.get_recommendation(code)[source]

Get the recommendation text for an issue code.

Parameters:

code (str) – The issue code (e.g., “IDENT-001”, “GRAD-002”).

Returns:

The recommendation text, or a default message if the code is not found.

Return type:

str

Examples

>>> get_recommendation("IDENT-001")
'Structural unidentifiability detected: ...'
>>> get_recommendation("UNKNOWN-999")
'No specific recommendation available for this issue.'
nlsq.diagnostics.run_plugins(jacobian, parameters, residuals, **context)[source]

Execute all registered plugins with exception isolation.

This function runs each registered plugin’s analyze() method and collects the results. If a plugin raises an exception, it is caught, logged via warnings, and the plugin’s result is marked as unavailable. Other plugins continue executing normally (FR-014).

Parameters:

  • jacobian (np.ndarray) – Jacobian matrix at solution (n_residuals x n_params).

  • parameters (np.ndarray) – Fitted parameters.

  • residuals (np.ndarray) – Residuals at solution.

  • **context (Any) – Additional context for plugins: - xdata: Independent variable data - ydata: Dependent variable data - bounds: Parameter bounds - model: Model function - config: DiagnosticsConfig

Returns:

Mapping of plugin name to result. If a plugin fails, its result has available=False with error_message.

Return type:

dict[str, PluginResult]

Example

>>> from nlsq.diagnostics import run_plugins
>>> results = run_plugins(jacobian, params, residuals, xdata=x, ydata=y)
>>> for name, result in results.items():
...     if result.available:
...         print(f"{name}: {len(result.issues)} issues")
...     else:
...         print(f"{name}: FAILED - {result.error_message}")

Overview

The nlsq.diagnostics package provides a comprehensive Model Health Diagnostics System for nonlinear least squares curve fitting. It helps users understand why a fit may have issues and provides actionable recommendations for improving results.

Key Capabilities

  • Identifiability Analysis - Detect structural and practical parameter unidentifiability

  • Gradient Health Monitoring - Track gradient behavior during optimization

  • Parameter Sensitivity Analysis - Identify stiff vs sensitive parameter directions

  • Health Reports - Aggregated diagnostics with severity-based issue categorization

  • Plugin System - Extensible architecture for domain-specific diagnostics

Quick Start

from nlsq import curve_fit
from nlsq.diagnostics import DiagnosticsConfig

# Enable diagnostics during fitting
result = curve_fit(model, x, y, p0=p0, compute_diagnostics=True)

# Access the health report
if result.diagnostics is not None:
    print(result.diagnostics.summary())

Enumerations

HealthStatus(*values)

Overall model health status.

IssueSeverity(*values)

Severity level of a detected issue.

IssueCategory(*values)

Category of detected issue.

DiagnosticLevel(*values)

Diagnostic analysis depth level.

Types and Data Classes

ModelHealthIssue(category, severity, code, ...)

A single detected model health issue.

AnalysisResult([available, error_message, ...])

Base class for analysis results.

IdentifiabilityReport([available, ...])

Report from identifiability analysis.

GradientHealthReport([available, ...])

Report from gradient health monitoring during optimization.

ParameterSensitivityReport([available, ...])

Report from parameter sensitivity spectrum analysis.

PluginResult([plugin_name, available, ...])

Result from a diagnostic plugin execution.

DiagnosticsReport([identifiability, ...])

Aggregated diagnostics report containing all analysis results.

DiagnosticsConfig([level, ...])

Configuration for diagnostic computation.

ModelHealthReport([identifiability, ...])

Aggregated model health report with overall assessment.

Analyzer Classes

IdentifiabilityAnalyzer(config)

Analyzer for parameter identifiability from Jacobian matrices.

GradientMonitor(config)

Monitor gradient health during optimization iterations.

ParameterSensitivityAnalyzer([config])

Analyzer for parameter sensitivity spectrum from Jacobian matrices.

Plugin System

DiagnosticPlugin(*args, **kwargs)

Protocol for diagnostic plugins.

PluginRegistry()

Global registry for diagnostic plugins.

run_plugins(jacobian, parameters, residuals, ...)

Execute all registered plugins with exception isolation.

Factory Functions

create_health_report([identifiability, ...])

Create aggregated health report from component reports.

Recommendations

RECOMMENDATIONS

Mapping of issue codes to recommendation text.

get_recommendation(code)

Get the recommendation text for an issue code.

Issue Codes Reference

The diagnostics system uses structured issue codes to identify specific problems:

Identifiability Issues (IDENT-xxx)

Code

Severity

Description

IDENT-001

CRITICAL

Structural unidentifiability: FIM is rank-deficient

IDENT-002

WARNING

Practical unidentifiability: FIM has high condition number

Correlation Issues (CORR-xxx)

Code

Severity

Description

CORR-001

WARNING

Highly correlated parameters detected

Gradient Issues (GRAD-xxx)

Code

Severity

Description

GRAD-001

WARNING

Vanishing gradients detected during optimization

GRAD-002

WARNING

Gradient imbalance across parameters

GRAD-003

WARNING

Gradient stagnation detected

Parameter Sensitivity Issues (SENS-xxx)

Code

Severity

Description

SENS-001

WARNING

Wide sensitivity spread detected (large eigenvalue range)

SENS-002

INFO

Low effective dimensionality

Usage Examples

Identifiability Analysis

Analyze parameter identifiability from a Jacobian matrix:

import numpy as np
from nlsq.diagnostics import DiagnosticsConfig, IdentifiabilityAnalyzer

# Configure analysis thresholds
config = DiagnosticsConfig(
    condition_threshold=1e8,
    correlation_threshold=0.95,
)

# Create analyzer
analyzer = IdentifiabilityAnalyzer(config)

# Analyze Jacobian (typically from OptimizeResult.jac)
jacobian = result.jac  # Shape: (n_data, n_params)
report = analyzer.analyze(jacobian)

print(f"Condition number: {report.condition_number:.2e}")
print(f"Numerical rank: {report.numerical_rank}/{report.n_params}")
print(f"Health status: {report.health_status.name}")

# Check for issues
for issue in report.issues:
    print(f"[{issue.severity.name}] {issue.code}: {issue.message}")
    print(f"  Recommendation: {issue.recommendation}")

Gradient Health Monitoring

Monitor gradient behavior during optimization using callbacks:

from nlsq import curve_fit
from nlsq.diagnostics import DiagnosticsConfig, GradientMonitor

# Create gradient monitor
config = DiagnosticsConfig(
    vanishing_threshold=1e-6,
    imbalance_threshold=1e6,
    stagnation_window=10,
)
monitor = GradientMonitor(config)

# Create callback for curve_fit
callback = monitor.create_callback()

# Fit with gradient monitoring
result = curve_fit(model, x, y, p0=p0, callback=callback)

# Get gradient health report
grad_report = monitor.get_report()

print(f"Health score: {grad_report.health_score:.2f}")
print(f"Iterations monitored: {grad_report.n_iterations}")
print(f"Vanishing detected: {grad_report.vanishing_detected}")
print(f"Imbalance detected: {grad_report.imbalance_detected}")

Parameter Sensitivity Analysis

Analyze eigenvalue spectrum to identify sensitivity behavior:

from nlsq.diagnostics import (
    DiagnosticsConfig,
    DiagnosticLevel,
    ParameterSensitivityAnalyzer,
)

# Enable full analysis for sensitivity spectrum detection
config = DiagnosticsConfig(
    level=DiagnosticLevel.FULL,
    sloppy_threshold=1e-6,
)

analyzer = ParameterSensitivityAnalyzer(config)
report = analyzer.analyze(jacobian)

print(f"Wide sensitivity spread: {report.is_sloppy}")
print(f"Eigenvalue range: {report.eigenvalue_range:.1f} orders of magnitude")
print(f"Effective dimensionality: {report.effective_dimensionality:.1f}")
print(f"Stiff directions: {report.stiff_indices}")
print(f"Sloppy directions: {report.sloppy_indices}")

Creating Aggregated Health Reports

Combine all analyses into a unified health report:

from nlsq.diagnostics import (
    DiagnosticsConfig,
    IdentifiabilityAnalyzer,
    GradientMonitor,
    ParameterSensitivityAnalyzer,
    create_health_report,
)

config = DiagnosticsConfig(verbose=False, emit_warnings=False)

# Run individual analyses
ident_analyzer = IdentifiabilityAnalyzer(config)
ident_report = ident_analyzer.analyze(jacobian)

sens_analyzer = ParameterSensitivityAnalyzer(config)
sens_report = sens_analyzer.analyze(jacobian)

# Create aggregated report
health_report = create_health_report(
    identifiability=ident_report,
    gradient_health=grad_report,  # From GradientMonitor
    sensitivity=sens_report,
    config=config,
)

# Access aggregated results
print(f"Overall status: {health_report.status.name}")
print(f"Health score: {health_report.health_score:.2f}")
print(f"Total issues: {len(health_report.all_issues)}")

# Get formatted summary
print(health_report.summary())

# Convert to dictionary for serialization
data = health_report.to_dict()

Custom Diagnostic Plugins

Create domain-specific diagnostics that integrate with the health report:

import numpy as np
from nlsq.diagnostics import (
    DiagnosticPlugin,
    PluginRegistry,
    PluginResult,
    ModelHealthIssue,
    IssueCategory,
    IssueSeverity,
)
from nlsq.diagnostics.recommendations import get_recommendation


class OpticalScatteringPlugin:
    """Plugin for optical scattering parameter validation."""

    @property
    def name(self) -> str:
        return "optical-scattering"

    def analyze(
        self,
        jacobian: np.ndarray,
        parameters: np.ndarray,
        residuals: np.ndarray,
        **context,
    ) -> PluginResult:
        issues = []

        # Domain-specific validation: scattering coefficients must be positive
        if any(parameters < 0):
            issues.append(
                ModelHealthIssue(
                    category=IssueCategory.IDENTIFIABILITY,
                    severity=IssueSeverity.CRITICAL,
                    code="OPTICAL-001",
                    message="Negative scattering coefficient detected",
                    affected_parameters=tuple(np.where(parameters < 0)[0]),
                    details={"negative_values": parameters[parameters < 0].tolist()},
                    recommendation="Ensure bounds enforce non-negative coefficients",
                )
            )

        return PluginResult(
            plugin_name=self.name,
            data={"custom_metric": np.mean(np.abs(parameters))},
            issues=issues,
        )


# Register the plugin
PluginRegistry.register(OpticalScatteringPlugin())

# Plugins are automatically included in health reports
from nlsq.diagnostics import run_plugins

plugin_results = run_plugins(jacobian, parameters, residuals)
for name, result in plugin_results.items():
    print(f"{name}: {len(result.issues)} issues")

Configuration Reference

The DiagnosticsConfig dataclass controls all diagnostic thresholds:

Parameter

Default

Description

level

BASIC

Diagnostic depth: BASIC or FULL (includes sensitivity analysis)

condition_threshold

1e8

FIM condition number threshold for practical identifiability

correlation_threshold

0.95

Correlation coefficient threshold for high correlation warning

imbalance_threshold

1e6

Gradient imbalance ratio threshold

vanishing_threshold

1e-6

Relative gradient magnitude threshold for vanishing detection

sloppy_threshold

1e-6

Eigenvalue ratio threshold for sensitivity classification

gradient_window_size

100

Sliding window size for gradient norm history

stagnation_window

10

Iterations to check for gradient stagnation

stagnation_tolerance

0.01

Relative tolerance for stagnation detection (1%)

verbose

True

Print diagnostic summary to console

emit_warnings

True

Emit Python warnings for critical issues

Memory Efficiency

The diagnostics system is designed for memory efficiency:

  • GradientMonitor: Uses bounded memory (<1KB) regardless of iteration count: - Sliding window (deque) for gradient norm history - Welford’s online algorithm for running mean/variance per parameter

  • IdentifiabilityAnalyzer: Computes SVD once, reuses results

  • ParameterSensitivityAnalyzer: Uses eigenvalue decomposition from SVD

See Also