Parameter Fitting/Estimation Package#205
Open
kwmcbride wants to merge 25 commits intopathsim:masterfrom
Open
Conversation
Style, API, and robustness overhaul of the parameter estimation addon: opt/ module (parameter_estimator.py, timeseries_data.py, __init__.py): - Align style with pathsim conventions (ASCII separators, comment headers, NumPy-style docstrings) - Remove legacy constructor params (measurement/outputs/sigma) - Make `runners` a live @Property instead of a stale field - Add `transform=` to add_local_block_parameter (was missing) - Remove num_inputs hack from apply(); remove dead SimRunner.__call__ - fit(): use res.fun instead of re-evaluating objective; fix L-BFGS-B deprecation warning by omitting `disp` for unsupported methods - plot_fit(): iterate all outputs per experiment (was only output_idx=0) - display(): show bounds alongside parameter values - Add EstimatorResult.__repr__ - Add add_shared_block_parameter alias for add_global_block_parameter - Lazy matplotlib import in timeseries_data.plot() and plot_fit() Robustness fixes (from dual developer/user agent review): - Parameter.__init__: raise ValueError for inverted bounds (lo > hi); warn UserWarning if initial value is outside bounds - fit(): call _validate_fit_inputs() — raises early with clear messages when no experiments, no parameters, or no measurements are configured - ScopeSignal._extract(): raise ValueError for ambiguous square arrays; raise IndexError for out-of-bounds port access - add_experiment(): warn UserWarning when copy_sim=False and the same simulator object is already registered in another experiment Tests (test_parameter_estimator.py): - Expand from 96 to 110 tests - Add TestParameterBoundsValidation, TestScopeSignalExtractRobustness, TestPreFitValidation, TestCopySimWarning test classes - Add TestStress class with 8 convergence/edge-case tests Examples (example_linear_fit.py, example_cstr_fit.py, example_tritium_fit.py): - Fix leading-space syntax error in example_cstr_fit.py - Standardize section separators, rename blocks for clarity - Move opt imports to top level, replace debug prints with est.display()
API cleanup: - Remove block_param_to_var / free_param_to_var from __all__ and package-level __init__.py (remain accessible as internal helpers) - Remove add_shared_block_parameter alias; add_global_block_parameter is the single canonical name for cross-experiment parameters Efficiency: - Cache the flattened parameters list; invalidate on any add_* call (eliminates repeated list allocation inside apply() / fit()) - Cache last residuals evaluation keyed on x; skip re-running simulators for identical parameter vectors (helps plot_fit() and scipy's finite- difference Jacobian which re-evaluates at the same point) - Cache invalidated on add_timeseries() and all add_*parameter() calls Clarity: - Clarify sigma docstring: explicitly states it is the measurement noise standard deviation normalizing residuals as (y_pred - y_meas) / sigma - Constructor docstring: adaptive/pre_run apply to auto-created first experiment only TimeSeriesData.plot(): - Add ax= parameter so callers can embed the plot in an existing axes - Returns the axes object for further customization - Switch from top-level plt calls to ax.* methods Tests (120 total, +10): - TestCaching: 6 tests covering params cache, residuals cache, and invalidation on add_* calls - TestNanInfHandling: 2 tests verifying NaN propagates and clean data produces finite residuals - TestTimeSeriesDataPlot: 2 tests for new ax= parameter - Update test_add_shared_block_parameter_alias to use canonical name
SharedBlockParameter: - Replace super().__init__() call with direct attribute initialization to eliminate the fragile init-order dependency (self.targets must exist before set() is dispatched; previously relied on ordering convention) - Add bounds validation and out-of-bounds UserWarning matching Parameter Deferred duration update: - _update_duration_from_measurements() is no longer called eagerly on every add_timeseries() / add_experiment() — was O(n^2) for many datasets - Replaced with a _duration_dirty flag; _ensure_duration_current() is called lazily at the top of residuals(), simulate(), and _validate_fit_inputs() - est.duration is now a property that triggers the lazy update, so reading it after add_timeseries() still returns the correct value Constructor defaults forwarding: - add_experiment() uses a _UNSET sentinel to distinguish "not provided" from explicit False/None for adaptive= and pre_run= - When not provided, inherits the values passed to the constructor so est = ParameterEstimator(sim, adaptive=True) followed by est.add_experiment(sim2, copy_sim=True) automatically gets adaptive=True Documentation: - ParameterEstimator class docstring now includes a "Common usage patterns" Notes section covering the three main workflows: single-experiment block params, multi-experiment global params, and free closure parameters Tests (137 total, +17): - TestSharedBlockParameterInit: 5 tests covering init-order safety, bounds validation, and interface completeness - TestConstructorDefaultsForwarding: 3 tests for adaptive/pre_run inherit - TestDeferredDurationUpdate: 2 tests for dirty-flag deferred update - TestConstraintFitting: 5 tests for SLSQP/COBYLA/constraint paths - TestScopeResolution: 2 tests for multi-experiment scope lookup
- Clarify f_scale, loss, and adaptive parameters in fit() docstring with practical guidance on when/why to use each setting - Add 'Choosing how to register parameters' section to class Notes to distinguish add_block_parameter vs add_parameters use cases - Create examples/example_simple_fit.py: minimal single-parameter fit (Source → Amplifier → Scope) to serve as entry-point reference
- Replace Python bisect with np.searchsorted (faster binary search in C) - Unify _interp_scalar into _interp_at handling 1D/2D in one vectorised operation; add _zoh_at helper for zero-order hold interpolation - Add interpolation='linear'|'zoh' parameter for discrete-sampled data - Add loop=True parameter for cyclic repetition of the signal - Move channel validation from update() to __init__ for fail-fast errors - Replace np.clip with max/min for scalar hold-mode clamping - Fix __len__ docstring: 'no algebraic passthrough', not 'is algebraic' - Multi-channel output: spread channels across ports via update_from_array instead of assigning a vector to a single Register slot - Add plot() method delegating to TimeSeriesData.plot() - Add tests/pathsim/blocks/test_timeseries_source.py (50 tests)
TimeSeriesData is a general data container used by both blocks/ and opt/. Having blocks/ import from opt/ violated the expected layering. - Move canonical definition to src/pathsim/utils/timeseries_data.py - Delete opt/timeseries_data.py - Update all import sites: opt/__init__.py, opt/parameter_estimator.py, blocks/timeseries_source.py, and the two test files
New SensitivityResult class (opt/sensitivity.py): - Fisher Information Matrix (FIM = J^T J) from weighted Jacobian - Parameter covariance via pseudo-inverse of FIM - Standard errors, correlation matrix, eigenvalues, condition number - display(): table of values/std errors/flags and correlated pair report - plot(): correlation heatmap + FIM eigenvalue spectrum side-by-side New ParameterEstimator.sensitivity(x=None, *, eps=None) method: - Jacobian by 2-point finite differences (n_params extra sim runs) - Defaults x to cached fit result so est.sensitivity() works after fit() - Model-space parameter values; handles untransformed and transformed params Export SensitivityResult from pathsim.opt public API 33 new tests in tests/pathsim/param_est/test_sensitivity.py Add examples/example_sensitivity_pk.py: 1-compartment oral PK model with three parameters showing different identifiability profiles
…estimation
- ParameterEstimator.fit_nested(): outer Gauss-Newton on global params using
reduced Jacobian J_red = P_{L,i} J_{G,i}; inner independent least_squares per
experiment on local params (warm-started). Outer analytic Jacobian avoids a
second layer of finite differences.
- sensitivity(): computes Schur complement S_G = F_GG - Σ F_{GL,i} pinv(F_{LL,i}) F_{GL,i}^T
when both global and local parameters are registered; exposed as SensitivityResult.schur.
- SchurResult (sensitivity.py): covariance, std_errors, correlation, eigenvalues,
condition_number, display(), plot() for the effective global-param FIM.
- _deepcopy_sim() (parameter_estimator.py): fixes lambda closure references after
deepcopy. Python treats functions as atomic during deepcopy, leaving Operator._func
closures pointing at original blocks. _rebind_func_closure() rebuilds affected
functions using the deepcopy memo dict; _fix_sim_operator_closures() applies this
to op_alg/_jac and op_dyn/_jac_x/_jac_u across all blocks. add_experiment(copy_sim=True)
now uses _deepcopy_sim() instead of copy.deepcopy().
- residuals() cache guard: added `and self._cached_residuals is not None` to prevent
AttributeError when fit_nested() sets cached_x but clears cached_residuals.
- tests/pathsim/param_est/test_schur.py: 35 tests covering SchurResult construction,
display, plot, Schur complement math, SensitivityResult.schur attachment, and
ParameterEstimator integration.
- examples/example_schur_multiexp.py: three-experiment decay model demonstrating
fit_nested() with global k and per-experiment amplitudes.
Code review fixes: - [DOCS] Clarify loss= is only used when method='least_squares' in fit() and fit_nested() - [API] Consolidate duplicate constraint validation in fit() into a single pre-branch guard - [STYLE] display() now uses dynamic column width based on longest param name - [CORRECTNESS] sensitivity._build_stats() uses np.nan for undefined off-diagonal correlation entries (std_error == 0); heatmap shows "N/A" - [TEST] Add TestFitNestedNoGlobals: fit_nested() raises ValueError when no global parameters are registered New feature: - fit_nested() outer_method= parameter: switch outer optimizer from the default trust-region least_squares to any scipy.optimize.minimize method (trust-constr, trust-ncg, L-BFGS-B, etc.). Hessian-based methods receive the Gauss-Newton approximation J_red^T J_red automatically. - outer_constraints= parameter: forward constraint dicts to minimize for nonlinear constraints on global parameters (trust-constr / SLSQP / COBYLA)
Part 1 (unchanged): default TRF outer via least_squares with soft_l1 loss. Part 2 (new): explains and demonstrates outer_method=: - trust-ncg: Hessian-based drop-in, auto-supplied Gauss-Newton H = J_red^T J_red - trust-constr: nonlinear constraint on global k enforcing a half-life upper bound (tau <= 2 s => k >= ln(2)/2), showing the outer_constraints= API Header comments updated to document the scalar conversion f = 0.5*||r_red||^2, grad = J_red^T r_red, hess = J_red^T J_red and explain when to switch from the default least_squares path. Adds a comparison bar chart: recovered k and outer nfev across all three methods.
- fit_nested(): add validation that outer_constraints is only used with constraint-supporting methods (SLSQP, trust-constr, COBYLA), matching the equivalent guard already present in fit() - Fix two-space misalignment of x_G_opt assignment in the least_squares results block
…o return a TimeSeriesData instance.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Member
|
Awesome, I will check it. Its a lot of code though. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds several tools to PathSim for parameter fitting and sensitivity analysis using SciPy.optimize.
One of the key features include fitting global and local parameters to accommodate multiple data sets (either as a single optimization problem or using a nested Schur decomposition approach.
It also introduces a TimeSeriesSource block that simplifies using experimental data in a simulation. The TimeSeriesData class is also used to hold data in a convenient container.
There are many examples that show how simple it is to integrate these new tools with existing PathSim models.