X Tutup
Skip to content

Parameter Fitting/Estimation Package#205

Open
kwmcbride wants to merge 25 commits intopathsim:masterfrom
kwmcbride:kevin/dev/param_est
Open

Parameter Fitting/Estimation Package#205
kwmcbride wants to merge 25 commits intopathsim:masterfrom
kwmcbride:kevin/dev/param_est

Conversation

@kwmcbride
Copy link
Contributor

This adds several tools to PathSim for parameter fitting and sensitivity analysis using SciPy.optimize.

One of the key features include fitting global and local parameters to accommodate multiple data sets (either as a single optimization problem or using a nested Schur decomposition approach.

It also introduces a TimeSeriesSource block that simplifies using experimental data in a simulation. The TimeSeriesData class is also used to hold data in a convenient container.

There are many examples that show how simple it is to integrate these new tools with existing PathSim models.

kwmcbride and others added 25 commits February 14, 2026 09:46
Style, API, and robustness overhaul of the parameter estimation addon:

opt/ module (parameter_estimator.py, timeseries_data.py, __init__.py):
- Align style with pathsim conventions (ASCII separators, comment headers,
  NumPy-style docstrings)
- Remove legacy constructor params (measurement/outputs/sigma)
- Make `runners` a live @Property instead of a stale field
- Add `transform=` to add_local_block_parameter (was missing)
- Remove num_inputs hack from apply(); remove dead SimRunner.__call__
- fit(): use res.fun instead of re-evaluating objective; fix L-BFGS-B
  deprecation warning by omitting `disp` for unsupported methods
- plot_fit(): iterate all outputs per experiment (was only output_idx=0)
- display(): show bounds alongside parameter values
- Add EstimatorResult.__repr__
- Add add_shared_block_parameter alias for add_global_block_parameter
- Lazy matplotlib import in timeseries_data.plot() and plot_fit()

Robustness fixes (from dual developer/user agent review):
- Parameter.__init__: raise ValueError for inverted bounds (lo > hi);
  warn UserWarning if initial value is outside bounds
- fit(): call _validate_fit_inputs() — raises early with clear messages
  when no experiments, no parameters, or no measurements are configured
- ScopeSignal._extract(): raise ValueError for ambiguous square arrays;
  raise IndexError for out-of-bounds port access
- add_experiment(): warn UserWarning when copy_sim=False and the same
  simulator object is already registered in another experiment

Tests (test_parameter_estimator.py):
- Expand from 96 to 110 tests
- Add TestParameterBoundsValidation, TestScopeSignalExtractRobustness,
  TestPreFitValidation, TestCopySimWarning test classes
- Add TestStress class with 8 convergence/edge-case tests

Examples (example_linear_fit.py, example_cstr_fit.py, example_tritium_fit.py):
- Fix leading-space syntax error in example_cstr_fit.py
- Standardize section separators, rename blocks for clarity
- Move opt imports to top level, replace debug prints with est.display()
API cleanup:
- Remove block_param_to_var / free_param_to_var from __all__ and
  package-level __init__.py (remain accessible as internal helpers)
- Remove add_shared_block_parameter alias; add_global_block_parameter
  is the single canonical name for cross-experiment parameters

Efficiency:
- Cache the flattened parameters list; invalidate on any add_* call
  (eliminates repeated list allocation inside apply() / fit())
- Cache last residuals evaluation keyed on x; skip re-running simulators
  for identical parameter vectors (helps plot_fit() and scipy's finite-
  difference Jacobian which re-evaluates at the same point)
- Cache invalidated on add_timeseries() and all add_*parameter() calls

Clarity:
- Clarify sigma docstring: explicitly states it is the measurement noise
  standard deviation normalizing residuals as (y_pred - y_meas) / sigma
- Constructor docstring: adaptive/pre_run apply to auto-created first
  experiment only

TimeSeriesData.plot():
- Add ax= parameter so callers can embed the plot in an existing axes
- Returns the axes object for further customization
- Switch from top-level plt calls to ax.* methods

Tests (120 total, +10):
- TestCaching: 6 tests covering params cache, residuals cache, and
  invalidation on add_* calls
- TestNanInfHandling: 2 tests verifying NaN propagates and clean data
  produces finite residuals
- TestTimeSeriesDataPlot: 2 tests for new ax= parameter
- Update test_add_shared_block_parameter_alias to use canonical name
SharedBlockParameter:
- Replace super().__init__() call with direct attribute initialization to
  eliminate the fragile init-order dependency (self.targets must exist
  before set() is dispatched; previously relied on ordering convention)
- Add bounds validation and out-of-bounds UserWarning matching Parameter

Deferred duration update:
- _update_duration_from_measurements() is no longer called eagerly on
  every add_timeseries() / add_experiment() — was O(n^2) for many datasets
- Replaced with a _duration_dirty flag; _ensure_duration_current() is
  called lazily at the top of residuals(), simulate(), and _validate_fit_inputs()
- est.duration is now a property that triggers the lazy update, so
  reading it after add_timeseries() still returns the correct value

Constructor defaults forwarding:
- add_experiment() uses a _UNSET sentinel to distinguish "not provided"
  from explicit False/None for adaptive= and pre_run=
- When not provided, inherits the values passed to the constructor so
  est = ParameterEstimator(sim, adaptive=True) followed by
  est.add_experiment(sim2, copy_sim=True) automatically gets adaptive=True

Documentation:
- ParameterEstimator class docstring now includes a "Common usage patterns"
  Notes section covering the three main workflows: single-experiment block
  params, multi-experiment global params, and free closure parameters

Tests (137 total, +17):
- TestSharedBlockParameterInit: 5 tests covering init-order safety,
  bounds validation, and interface completeness
- TestConstructorDefaultsForwarding: 3 tests for adaptive/pre_run inherit
- TestDeferredDurationUpdate: 2 tests for dirty-flag deferred update
- TestConstraintFitting: 5 tests for SLSQP/COBYLA/constraint paths
- TestScopeResolution: 2 tests for multi-experiment scope lookup
- Clarify f_scale, loss, and adaptive parameters in fit() docstring
  with practical guidance on when/why to use each setting
- Add 'Choosing how to register parameters' section to class Notes
  to distinguish add_block_parameter vs add_parameters use cases
- Create examples/example_simple_fit.py: minimal single-parameter
  fit (Source → Amplifier → Scope) to serve as entry-point reference
- Replace Python bisect with np.searchsorted (faster binary search in C)
- Unify _interp_scalar into _interp_at handling 1D/2D in one vectorised
  operation; add _zoh_at helper for zero-order hold interpolation
- Add interpolation='linear'|'zoh' parameter for discrete-sampled data
- Add loop=True parameter for cyclic repetition of the signal
- Move channel validation from update() to __init__ for fail-fast errors
- Replace np.clip with max/min for scalar hold-mode clamping
- Fix __len__ docstring: 'no algebraic passthrough', not 'is algebraic'
- Multi-channel output: spread channels across ports via update_from_array
  instead of assigning a vector to a single Register slot
- Add plot() method delegating to TimeSeriesData.plot()
- Add tests/pathsim/blocks/test_timeseries_source.py (50 tests)
TimeSeriesData is a general data container used by both blocks/ and opt/.
Having blocks/ import from opt/ violated the expected layering.

- Move canonical definition to src/pathsim/utils/timeseries_data.py
- Delete opt/timeseries_data.py
- Update all import sites: opt/__init__.py, opt/parameter_estimator.py,
  blocks/timeseries_source.py, and the two test files
New SensitivityResult class (opt/sensitivity.py):
- Fisher Information Matrix (FIM = J^T J) from weighted Jacobian
- Parameter covariance via pseudo-inverse of FIM
- Standard errors, correlation matrix, eigenvalues, condition number
- display(): table of values/std errors/flags and correlated pair report
- plot(): correlation heatmap + FIM eigenvalue spectrum side-by-side

New ParameterEstimator.sensitivity(x=None, *, eps=None) method:
- Jacobian by 2-point finite differences (n_params extra sim runs)
- Defaults x to cached fit result so est.sensitivity() works after fit()
- Model-space parameter values; handles untransformed and transformed params

Export SensitivityResult from pathsim.opt public API
33 new tests in tests/pathsim/param_est/test_sensitivity.py
Add examples/example_sensitivity_pk.py: 1-compartment oral PK model with
three parameters showing different identifiability profiles
…estimation

- ParameterEstimator.fit_nested(): outer Gauss-Newton on global params using
  reduced Jacobian J_red = P_{L,i} J_{G,i}; inner independent least_squares per
  experiment on local params (warm-started). Outer analytic Jacobian avoids a
  second layer of finite differences.

- sensitivity(): computes Schur complement S_G = F_GG - Σ F_{GL,i} pinv(F_{LL,i}) F_{GL,i}^T
  when both global and local parameters are registered; exposed as SensitivityResult.schur.

- SchurResult (sensitivity.py): covariance, std_errors, correlation, eigenvalues,
  condition_number, display(), plot() for the effective global-param FIM.

- _deepcopy_sim() (parameter_estimator.py): fixes lambda closure references after
  deepcopy. Python treats functions as atomic during deepcopy, leaving Operator._func
  closures pointing at original blocks. _rebind_func_closure() rebuilds affected
  functions using the deepcopy memo dict; _fix_sim_operator_closures() applies this
  to op_alg/_jac and op_dyn/_jac_x/_jac_u across all blocks. add_experiment(copy_sim=True)
  now uses _deepcopy_sim() instead of copy.deepcopy().

- residuals() cache guard: added `and self._cached_residuals is not None` to prevent
  AttributeError when fit_nested() sets cached_x but clears cached_residuals.

- tests/pathsim/param_est/test_schur.py: 35 tests covering SchurResult construction,
  display, plot, Schur complement math, SensitivityResult.schur attachment, and
  ParameterEstimator integration.

- examples/example_schur_multiexp.py: three-experiment decay model demonstrating
  fit_nested() with global k and per-experiment amplitudes.
Code review fixes:
- [DOCS] Clarify loss= is only used when method='least_squares' in fit()
  and fit_nested()
- [API] Consolidate duplicate constraint validation in fit() into a single
  pre-branch guard
- [STYLE] display() now uses dynamic column width based on longest param name
- [CORRECTNESS] sensitivity._build_stats() uses np.nan for undefined
  off-diagonal correlation entries (std_error == 0); heatmap shows "N/A"
- [TEST] Add TestFitNestedNoGlobals: fit_nested() raises ValueError when no
  global parameters are registered

New feature:
- fit_nested() outer_method= parameter: switch outer optimizer from the
  default trust-region least_squares to any scipy.optimize.minimize method
  (trust-constr, trust-ncg, L-BFGS-B, etc.). Hessian-based methods receive
  the Gauss-Newton approximation J_red^T J_red automatically.
- outer_constraints= parameter: forward constraint dicts to minimize for
  nonlinear constraints on global parameters (trust-constr / SLSQP / COBYLA)
Part 1 (unchanged): default TRF outer via least_squares with soft_l1 loss.

Part 2 (new): explains and demonstrates outer_method=:
- trust-ncg: Hessian-based drop-in, auto-supplied Gauss-Newton H = J_red^T J_red
- trust-constr: nonlinear constraint on global k enforcing a half-life upper
  bound (tau <= 2 s  =>  k >= ln(2)/2), showing the outer_constraints= API

Header comments updated to document the scalar conversion
  f = 0.5*||r_red||^2, grad = J_red^T r_red, hess = J_red^T J_red
and explain when to switch from the default least_squares path.

Adds a comparison bar chart: recovered k and outer nfev across all three methods.
- fit_nested(): add validation that outer_constraints is only used with
  constraint-supporting methods (SLSQP, trust-constr, COBYLA), matching
  the equivalent guard already present in fit()
- Fix two-space misalignment of x_G_opt assignment in the least_squares
  results block
@codecov
Copy link

codecov bot commented Mar 4, 2026

Codecov Report

❌ Patch coverage is 84.02310% with 166 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/pathsim/opt/parameter_estimator.py 80.82% 145 Missing ⚠️
src/pathsim/opt/sensitivity.py 93.42% 10 Missing ⚠️
src/pathsim/utils/timeseries_data.py 78.72% 10 Missing ⚠️
src/pathsim/blocks/timeseries_source.py 98.73% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@milanofthe
Copy link
Member

Awesome, I will check it. Its a lot of code though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

X Tutup