Pod Simulation and V&V Enhancement Plan

Last updated: 2026-06-23
Owner: BionicLoop engineering
Status: Phase 1 core simulator scaffold implemented; Phase 2+ evidence campaign still open

Goal

Reduce real Omnipod DASH consumption by moving every repeatable fallback, reconciliation, meal-announcement, relaunch, and pod-replacement edge case into deterministic simulation wherever the behavior can be credibly modeled without live BLE hardware.

The intended V&V posture is:

automated simulation owns repeatable runtime policy, state, reconciliation, evidence, and UI-state correctness
hardware verification is reserved for behavior that depends on real pod radio, pairing, firmware timing, iOS background behavior, or live device faults
simulation evidence is traceable enough to support formal or working STR packages, not just ad hoc developer confidence

Existing Leverage

The repository already has useful foundations:

BionicLoopCore/Tests/BionicLoopCoreTests/SimulationHarnessSupport.swift provides deterministic time, CGM events, pump status events, meal announce events, and doWork execution.
BionicLoopCore/Tests/BionicLoopCoreTests/TestSupport.swift provides a protocol-conformant TestPumpService with bolus delay/error injection, fallback maintenance calls, temp basal calls, pod identity, and pending attribution restore hooks.
BionicLoopTests/BionicLoopPumpServiceAdapterFallbackTests.swift exercises mapping from OmniBLE manager/pod state into app/core pump state and fallback reconciliation snapshots.
OmniBLE/OmniBLETests/PodCommsSessionTests.swift already has a small MockMessageTransport for lower-level session testing.
OmniBLE contains real dose-progress and pod-state concepts that can anchor simulator math, including PodDoseProgressEstimator, StatusResponse, PodInsulinMeasurements, PodState.unfinalizedBolus, PodState.unfinalizedTempBasal, finalized doses, and cancel handling.
Docs/Quality/STP/STP-SIM-001.md, Scripts/run_sim_harness_verification.sh, and Scripts/check_sim_merge_gate.sh already define a deterministic simulation evidence lane.

Before this scaffold, these were mostly fixture/status mocks. They did not provide one stateful simulated pod whose delivery ledger evolved across time, disconnect, reconnect, app relaunch, mask expiry, pod expiry, and replacement.

Implemented foothold as of 2026-06-23:

BionicLoopCore/Tests/BionicLoopCoreTests/SimulatedDashPodSupport.swift adds a test-only stateful pod ledger, ScriptedPodPumpService, and protocol-conformant pump-service scripting.
BionicLoopCore/Tests/BionicLoopCoreTests/SimulatedDashPodScenarioSupport.swift adds PodScenarioBuilder and shared pod scenario invariant assertions.
BionicLoopCore/Tests/BionicLoopCoreTests/SimulatedDashPodTests.swift and BionicLoopCore/Tests/BionicLoopCoreTests/SimulatedDashPodScenarioTests.swift cover bolus progress/cancel, persisted pending-dose restoration, fallback mask expiry/renewal, pod expiry/replacement identity transitions, meal-before-fallback-baseline partition safety, meal/fallback overlap partitioning, force-quit/relaunch restoration of in-flight meal delivery knowledge before the next CGM, lost-final-response disconnect before status refresh with same-pod reconnect, same-pod completed-meal live attribution without replay, reservoir-capped meal delivery feeding actual delivered units without replay, different/new-pod assumed-delivered replay with a replacement-pod live dose, user-escaped unavailable-pod assumed evidence into replacement-pod live dosing, canceled and consecutive-canceled meal evidence feeding subsequent meal announcements without replay or stale evidence reuse, stale cached idle rejection when fresh refresh fails, fallback-maintenance command-block suppression of a live bolus, stale fallback evidence replay suppression while disconnected, schedule-weighted fallback replay allocation after mask expiry, reservoir-capped fallback replay using observed pump delta instead of modeled exposure, meal/fallback overlap coordinator replay that merges the partitioned meal dose into the first fallback-active row, relaunch fallback replay with unavailable CGM after step 0, assumed old-pod fallback replay that allows replacement-pod dosing, explicit nonreplayable fallback-plan clearing without modeled-dose injection, pending meal-progress cleanup when matching issued-dose evidence is consumed on same-pod reconnect, fallback-merge replay, or user-escaped old-pod recovery, missing-status/nonreplayable fallback-plan clearing without modeled-dose injection, and runtime replay paths through the production LoopRuntimeCoordinator.
Scripts/run_sim_harness_verification.sh now emits a separate pod-sim suite for the simulator scaffold under STR-SIM-001.
Scripts/check_sim_merge_gate.sh now treats fallback/reconciliation domain paths as high-risk simulation-gated paths.
BionicLoopTests/BionicLoopRecentDoseTimelineTests.swift now covers the app-layer recent-dose persistence/display seam for merged meal/fallback replay evidence, including source, disposition, request step, requested units, and delivered units after reload.

Remaining limitation: this first scaffold/scenario slice is not yet a complete table-driven scenario campaign, emits suite-level STR artifacts but not detailed per-scenario pod ledger traces, and does not yet cover every POD-SIM-* matrix row.

Proposed Test Architecture

Layer 1: Core Stateful Pod Simulator

Add test-only types under BionicLoopCore/Tests/BionicLoopCoreTests/:

SimulatedDashPod
ScriptedPodPumpService
PodScenarioBuilder
PodScenarioAssertions

Responsibilities:

maintain a pod identity, reservoir, total delivered units, and delivery state
track bolus requests by request step, requested units, start time, expected completion, delivered units, and cancellation state
track temp basal mask requests, active mask expiry, fallback programmed schedule, and fallback basal delivery after mask expiry
model pod continuity: same pod, different/new pod, missing pod identity, no active pod, expired pod, retired pod
model communication state: connected, disconnected, stale cached status, no response, uncertain delivery, command accepted then response lost
generate PumpStatus and FallbackBasalReconciliationPumpSnapshot directly from the simulated ledger
implement PumpService, PumpBolusCommandStartReporting, AlgorithmStepTaggingPumpService, PendingIssuedDoseAttributionRestoring, and PendingMealDeliveryAttributionRestoring

This layer should use the production LoopRuntimeCoordinator and production runtime policies. It must not contain alternative policy logic beyond simulated pod physics/state.

Layer 2: App/Adapter Mapping Simulator

Extend app tests to cover the PumpServiceAdapter boundary with richer OmniBLE-like manager state:

build OmniBLEPumpManagerState and PodState fixtures from the same simulated pod ledger
verify adapter mapping for:
pod identifier
lastInsulinMeasurements.delivered
bolusNotDelivered
active bolus vs idle status
active 0.0 temp basal mask vs nonzero temp basal
finalized dose history
stale or missing basal delivery state
keep these tests focused on adapter mapping, not coordinator policy

This layer proves that what core thinks the pump reported matches what the OmniBLE adapter would produce from persisted pod state.

Layer 3: Optional OmniBLE Message Transport Scripts

Use lower-level OmniBLE scripted transport tests only where the failure mode is about command/session behavior rather than runtime policy:

no response after command write
pod acknowledges but returns no final status
nonce/session failure around command recovery
status response with active bolus and nonzero bolusNotDelivered
fault/expired response while unfinalized bolus or temp basal exists

This should remain a smaller layer. It is not the first place to test complete fallback/reconciliation workflows.

Scenario Matrix

The simulator should support a table-driven scenario campaign. Each scenario must assert runtime state, algorithm inputs, pump commands, recent dose rows, telemetry/evidence payloads, and user-visible blocking/unblocking state where app tests are involved.

ID	Scenario	Simulator expectation	Hardware residual
POD-SIM-001	Nominal bolus completes before next step	Pending issued-dose attribution clears on the attribution step; requested/delivered/requestStep are correct	None
POD-SIM-002	Bolus canceled before completion	Partial delivered amount is input to the next eligible step; meal UI shows partial/canceled state; no unresolved block remains	One live confirmation of cancel behavior
POD-SIM-003	Meal announce bolus starts, app force-quits before pump response, relaunch before next CGM	Pending attribution is restored from persistence; UI does not lose the in-flight dose; no step `0` artifact; resolution can occur on status refresh or next `doWork`	iOS force-quit/background persistence timing spot check
POD-SIM-004	Meal announce bolus starts, disconnect mid-delivery, reconnect before fallback mask expiry	Same-pod evidence resolves the issued dose; no fallback replay rows are created; next live step receives correct delivered input	BLE reconnect timing spot check
POD-SIM-005	Meal announce bolus starts, disconnect mid-delivery, fallback mask expires, reconnect same pod	Pump total delta is partitioned into issued dose plus fallback basal residual; issued dose is merged into the first fallback-active replay row when appropriate; no pre-fallback replay rows	Live pod confirmation of measured total behavior
POD-SIM-006	Meal announce bolus starts, original pod expires before reconciliation	Assumed-delivered meal evidence is recorded per clinical policy; modal/UI can escape; no dosing block on replacement pod	Expired-pod UI/OmniBLE state transition spot check
POD-SIM-007	Meal announce bolus starts, user replaces pod before reconciliation	Existing algorithm session is preserved; old-pod dose is assumed delivered; new-pod dosing is not blocked; old-pod fallback is not reconciled using new-pod counters	Replacement workflow confirmation
POD-SIM-008	Correction bolus or basal microdose unresolved across relaunch	Fresh unresolved issued dose is assumed delivered when policy allows; future dosing is not blocked solely because user cannot recover the old evidence	None beyond one live smoke
POD-SIM-009	Fallback armed, 0.0 mask expires offline, same pod reconnects with credible pump-total delta	Fallback-active missed steps replay with `CGM=-1`, delivered units allocated by schedule weights, no pump commands during replay, current live step runs only after replay	Live fallback replay smoke
POD-SIM-010	Fallback armed, same pod reconnects with missing or non-credible delta	No confirmed replay from non-credible evidence; evidence labels explain why; no aggregate missing fallback dose is added to live step; explicit nonreplayable replay plans are cleared without row emission	One live ambiguous-evidence observation if feasible
POD-SIM-011	Fallback armed, old pod inactive/retired/service-stopped before evidence can be recovered	Persisted-schedule modeled exposure can be recorded as assumed delivered when policy allows; assumed-vs-confirmed distinction is explicit; replacement-pod live dosing is allowed after assumed replay	Expired/retired pod transition
POD-SIM-012	Fallback maintenance blocked by active bolus while mask still active	Maintenance defers; step command policy follows current safety rule; fallback state records defer reason	None
POD-SIM-013	Fallback maintenance remask fails after schedule write	Recovery-required state is entered; ordinary closed-loop step is blocked until recovery path resolves	One live no-response/unacknowledged command drill if possible
POD-SIM-014	Status cache says idle but fresh refresh fails or reports unknown	Runtime does not trust stale cached status for maintenance or dosing; no false connected/idle behavior	Live BLE disconnect smoke
POD-SIM-015	App relaunch with pending fallback recovery before next CGM	State is restored; recovery can be processed by reconnect/status refresh path when available; if waiting for `doWork`, replay still covers all eligible rows	iOS lifecycle confirmation
POD-SIM-016	Reconnect/status refresh closes meal progress UI before next algorithm step	UI unblocks after reconciliation evidence is stored; next step still consumes the correct issued-dose evidence	UI smoke
POD-SIM-017	Multiple canceled meal announces in close sequence	No synthetic replay steps; request step remains the actual requesting step; cadence message shows next open time	None
POD-SIM-018	Meal/correction dose completes before fallback baseline window	Do not subtract it from fallback pump delta; do not replay pre-fallback issued-dose-only rows	None
POD-SIM-019	Meal/correction dose overlaps fallback baseline-to-recovery window	Subtract only covered issued-dose evidence from raw pod delta; residual drives fallback replay only if credible	None
POD-SIM-020	Local recent dose rows vs cloud telemetry vs algorithm output files	Same logical replay and reconciliation evidence appears consistently across local UI, local files, and telemetry payloads	Cloud upload path remains separate if real cloud is in scope

Current Automated Result Matrix

This table is the current human-readable status of the safety matrix. "Pass" means an automated test exists today and passed in the latest focused or full local harness run. "Partial" means the core behavior is covered, but UI, adapter, cloud, or formal evidence artifacts still need expansion before the row is closure-ready.

ID	What it does	Current automated result	Remaining gap
POD-SIM-001	Nominal/completed bolus evidence and request-step preservation	Pass: simulator restores completed pending-dose evidence and feeds it into replay/live attribution paths; same-pod completed meal and reservoir-capped completed meal preserve the original request step; runtime event mapping preserves algorithm input snapshots; recent-dose display now shows completed requested/delivered/request-step attribution	None for app/core evidence; hardware not required
POD-SIM-002	Partial bolus delivery, including cancel and reservoir cap	Pass for app/core evidence: simulator ledger caps delivered amount and runtime pod scenarios prove canceled meal evidence and reservoir-capped completed meal evidence feed the next eligible step without replay, stale evidence reuse, or unresolved block; recent-dose display now shows partial delivered amount and original request step	One live confirmation of real pod cancel/progress semantics remains useful
POD-SIM-003	Meal starts, app force-quits before final pump response, app relaunches before next CGM	Pass: pending issued-dose attribution survives relaunch and no step `0` artifact is emitted	UI/modal relaunch smoke remains app-layer work
POD-SIM-004	Meal starts, pump disconnects before a final status refresh, same pod reconnects	Pass: disconnected recovery sends no command, reconnect reconciles old meal evidence before any live command, and any new live dose is separately tracked	BLE reconnect timing remains one hardware spot check
POD-SIM-005	Meal delivery overlaps fallback mask expiry and reconnects same pod later	Pass: simulator-backed coordinator scenario partitions the same-pod pump delta, merges the meal dose into the first fallback-active replay row, replays only fallback-active rows, sends no replay pump commands, clears matching meal progress state, and app recent-dose persistence/display, local step CSV export, and runtime cloud step-event emission preserve the replay evidence source, disposition, request step, requested units, and delivered units	Add real upload/Dynamo observation and deeper local algorithm-artifact parity coverage if those artifacts are claimed
POD-SIM-006	Meal starts, old pod expires before reconciliation	Pass for app/core policy: simulator can model expired pod/unknown status; runtime scenarios assume old-pod meal delivery when the original pod is no longer active, clear pending meal modal state, preserve the attribution/evidence for the next eligible step, and avoid blocking replacement-pod live dosing	Add visual modal escape/replacement smoke and expired-pod adapter mapping
POD-SIM-007	User replaces pod before old-pod dose can be reconciled	Pass: old-pod dose is assumed delivered, session is preserved, no pending replay plan remains for old pod, and replacement-pod live dosing proceeds	Hardware replacement workflow confirmation
POD-SIM-008	Unresolved correction/basal microdose across relaunch	Pass: simulator-backed runtime scenarios explicitly cover correction-only and basal-only unresolved doses without recoverable pod identity; both are assumed delivered when policy allows, replayed with the original request step, and do not block replacement-pod live dosing	One live smoke remains useful but not required for core logic
POD-SIM-009	Fallback mask expires offline and same pod reconnects with credible delta	Pass: fallback-active missed steps replay with schedule-weighted delivered units, reservoir-capped replay uses the lower observed pump delta, and no replay pump commands are sent	Live fallback replay smoke remains hardware residual
POD-SIM-010	Fallback reconnect has missing or non-credible pump-total delta	Pass for app/core evidence: simulator confirms disconnected stale pump-total evidence does not authorize replay, direct reconciler tests cover unavailable/non-credible deltas, a coordinator relaunch scenario proves ambiguous restore creates no replay rows and injects no aggregate modeled fallback dose into the live step, missing-status and explicit nonreplayable fallback plans are consumed with `skipped_status_not_replayable`, and recent-dose fallback rows explicitly show restored/resumed-without-reconciliation wording without claiming a reconciled residual	One live ambiguous-evidence observation remains optional if feasible
POD-SIM-011	Old pod inactive/retired/service-stopped before fallback evidence can be recovered	Partial pass in app-runtime and pod-sim coverage: retired/no-active pod recovery records modeled fallback exposure as assumed delivered per clinical policy; simulator-backed relaunch replays the assumed fallback rows with unavailable CGM, sends no replay pump commands, and allows a replacement-pod live dose	Add pod-sim adapter-level expired/retired mapping if future transport fixtures expose those exact OmniBLE states
POD-SIM-012	Fallback maintenance blocked by active bolus while mask still active	Pass in app-runtime unit coverage: pre-execution fallback maintenance defers renewal/refresh when a bolus blocks the mask and the mask is still active	Add pod-sim adapter-level active-bolus fixture only if a future transport harness needs it
POD-SIM-013	Fallback schedule write succeeds but remask fails	Partial pass: simulator-backed runtime scenario records a failed fallback maintenance event and blocks the live bolus command before any pod ledger delivery	Add real recovery-required adapter/UI state assertion and live no-response drill if feasible
POD-SIM-014	Cached status says idle but fresh pump refresh fails	Pass: live step is skipped, no bolus command is sent, and cached idle is not trusted as fresh proof	Live BLE disconnect smoke remains hardware residual
POD-SIM-015	App relaunch with pending fallback recovery before next CGM	Partial pass: simulator-backed runtime scenario proves pending fallback replay survives persisted relaunch state and executes with unavailable CGM inputs after step 0 without replay pump commands	Add app lifecycle smoke for foreground/relaunch trigger routing
POD-SIM-016	Reconnect/status refresh closes meal progress UI before next algorithm step	Pass for app/core policy: runtime assertions cover matching issued-dose evidence closing the meal progress UI before the next algorithm step, same-pod reconnect evidence, fallback-merge replay, user-escaped old-pod assumed recovery, and launch-time reconciliation with no active pod	Add visual SwiftUI smoke for the presented sheet if UI automation scope is opened
POD-SIM-017	Multiple canceled meal announces in close sequence	Pass for app/core policy: simulator-backed runtime scenario covers two consecutive canceled meals feeding subsequent meal announcements without synthetic replay, stale evidence reuse, or step `0` artifacts; presentation helper covers the visible next-open-time cadence message	Add visual SwiftUI smoke only if UI automation scope is opened
POD-SIM-018	Meal/correction completes before fallback baseline	Pass: completed meal before fallback baseline does not pollute fallback pump delta	None for core math; evidence/UI parity still separate
POD-SIM-019	Meal/correction overlaps fallback baseline-to-recovery window	Pass: covered issued-dose amount is subtracted from raw pod delta and residual drives fallback replay	Add end-to-end telemetry/recent-dose parity
POD-SIM-020	Recent-dose rows, local algorithm files, and cloud telemetry agree	Partial pass: app recent-dose tests persist and render nominal completed, partial/canceled, and merged meal/fallback replay evidence with source, disposition, request step, requested units, and delivered units after reload; local step CSV export includes replay evidence source/disposition/failure columns for replay rows; runtime cloud step-event emission publishes replay rows before the live row with the same replay evidence fields	Add real upload/Dynamo observation and deeper local algorithm-artifact parity tests if those artifacts are claimed; not owned by core simulator alone

Required Scenario Invariants

Every scenario in the campaign should assert the relevant subset of these invariants:

algorithm stop/start remains explicit user action only
algorithm session and cadence anchor are preserved across recovery unless the user explicitly resets/stops
replay never sends catch-up pump commands
replay rows are bounded to eligible missed steps before the current due step
fallback replay is limited to fallback-active delivery windows, not pre-activation disconnected gaps
issued dose evidence is input on the correct attribution step or merged into the first fallback-active replay row when it predates that row
no unresolved meal state can permanently trap the UI or block pod replacement
replacement-pod dosing is not blocked solely because old-pod evidence is unrecoverable
assumed-delivered, pump-last-delivery, partitioned-pump-delta, and confirmed/corrected fallback evidence remain distinguishable in state, telemetry, and UI
stale cached pump status is not used as fresh proof for dosing or fallback maintenance
same-pod pump counters are never mixed with new-pod counters
no replay row uses requestStep = 0 unless the original request truly belongs to step 0
local recent-dose rows, algorithm input/output files, and telemetry payloads agree on step, requestStep, requestedUnits, deliveredUnits, evidence source, and reconciliation disposition

Refactor Qualification Gates

The simulator/refactor work should not be considered qualified just because the new helper types compile. Qualification requires all of the following:

Matrix coverage gate: every POD-SIM-* row must be classified as automated, hardware-residual, or clinically/policy deferred. No row should remain implicit or unowned.
Invariant gate: each automated scenario must use shared assertions for the invariants above, especially session preservation, replay bounds, no replay pump commands, same-pod/new-pod counter separation, no requestStep = 0 artifacts, and telemetry/UI/local-output agreement.
Boundary gate: the automated campaign must include timing boundaries around the five-minute step edge, mask expiry edge, dose completion edge, midnight, six-hour fallback schedule boundaries, app relaunch before/after next CGM, and pod expiry/replacement before and after reconciliation.
Fault-injection gate: scenarios must include command accepted/no final response, command rejected, status refresh failure, stale cached idle state, active bolus during fallback maintenance, disconnected active bolus, unavailable/expired/retired original pod, and replacement pod with unrelated delivery state.
Cross-layer gate: for high-risk scenarios, core runtime evidence must be reconciled with app adapter mapping, recent-dose UI rows, local algorithm output files, and cloud telemetry payloads. A pass in core alone is not sufficient for UI/evidence claims.
Evidence gate: the STR runner must emit per-scenario result artifacts, including pod ledger, runtime state before/after, algorithm input/output rows, pump commands attempted, recent-dose/evidence rows, and clinical disposition (confirmed, assumed, ambiguous, different_or_new_pod, or deferred).
Residual hardware gate: real Omnipod testing is reserved for behavior that simulation cannot credibly model: BLE transfer timing, real firmware dose progress/status semantics, pod expiration/fault response, pairing/transfer lifecycle, iOS background wake, and cloud upload path if formal cloud evidence is claimed.
Regression gate: Scripts/check_sim_merge_gate.sh must require the deterministic simulation evidence lane for changes to runtime, pump-service, fallback, reconciliation, persistence, and relevant Home UI evidence paths.

V&V Evidence Strategy

Proposed Verification Buckets

Add a new deterministic simulation sub-campaign under STP-SIM-001:

TV-SIM-POD-001: stateful pod simulator determinism and ledger accuracy
TV-SIM-POD-002: issued-dose reconciliation across disconnect/relaunch
TV-SIM-POD-003: fallback mask expiry and same-pod replay reconciliation
TV-SIM-POD-004: new/expired/retired pod assumed-evidence paths
TV-SIM-POD-005: UI unblock and recent-dose/evidence consistency

These can be mapped back to existing requirements and risks rather than creating new product behavior requirements:

SRS-PUMP-006, SRS-PUMP-008, SRS-PUMP-009, SRS-PUMP-010
SRS-RUN-006
SRS-MEAL-* affected meal lifecycle rows
SRS-STATE-* persistence/relaunch rows
SRS-LOG-* telemetry/evidence rows
RA-003, RA-006, RA-008, RA-017
TV-PUMP-008, TV-PUMP-009, TV-STATE-004, TV-LOG-009, TV-MEAL-* affected rows

Exact IDs should be confirmed when updating RTM/SVVP. The implementation should avoid claiming formal closure until those trace links are updated.

Evidence Artifacts

Extend Scripts/run_sim_harness_verification.sh or add a sibling pod-sim runner to emit:

run context: git SHA, simulator schema version, scenario set, seed/timezone
simulator qualification summary: deterministic re-run hash and math basis
scenario input file or generated scenario JSON
expected vs actual output for each scenario
state transition trace
pump ledger trace
algorithm input/output trace
pump command trace
recent dose timeline trace
telemetry payload trace
pass/fail summary
trace map from scenario ID to SRS/RA/TV rows

Evidence should land under:

working: Docs/Quality/Evidence/Working/STR-SIM-001/<run-label>/
formal: Docs/Quality/Evidence/Formal/STR-SIM-001/<run-label>/

Simulator Qualification Criteria

Before relying on the pod simulator for V&V evidence:

deterministic replay: identical scenario inputs produce identical outputs
no wall-clock dependency: all time comes from virtual clock or explicit dates
dose math references OmniBLE constants or an explicitly documented copy of the same pulse/rate rules
same scenario can be replayed through direct core PumpService and, where applicable, app adapter mapping fixtures with consistent pump evidence
at least one real hardware trace is used as a calibration fixture for:
bolus progress/cancel semantics
fallback total-delivery counter behavior
replacement/expired pod state semantics
negative controls prove the simulator can fail when policy is wrong, such as:
new-pod counters incorrectly accepted for old-pod fallback
stale cached idle status incorrectly accepted as fresh
pre-fallback disconnected steps incorrectly replayed
aggregate fallback insulin incorrectly added to the live step

Hardware Testing Reduction Rule

For every proposed hardware case, ask:

Does the case depend on real BLE/radio behavior, physical pod firmware, iOS background delivery, or live user interaction timing?
Can the expected pump state be generated from a deterministic simulator with the same public app/core interfaces?
Is hardware needed for initial calibration only, or for every regression?

Default disposition:

if the behavior is runtime policy, reconciliation math, persistence, evidence, or UI state, test it in simulation
if the behavior is OmniBLE mapping from persisted pod state, test it with constructed OmniBLEPumpManagerState/PodState
if the behavior is BLE transport, physical pod ack/fault timing, pairing, iOS background wake, or actual pod replacement workflow, keep one focused hardware test
once a hardware issue is understood, capture it as a simulator regression fixture so it does not need repeated pod burn

Implementation Plan

Phase 0: Design Lock

Add this plan to implementation tracking.
Confirm product/quality agreement that stateful pod simulation can support working/formal simulation evidence but does not replace hardware-only rows.
Decide whether to extend existing TV-SIM-* IDs or add TV-SIM-POD-* subcampaign IDs.
Define scenario JSON/Swift-builder source of truth.

Exit criteria:

approved scenario list
agreed trace/evidence lane
hardware-only boundary accepted

Phase 1: Core Simulator Scaffold

Implement SimulatedDashPod.
Implement ScriptedPodPumpService.
Add unit tests for ledger math:
bolus progress
partial cancel
delivery counter monotonicity
mask expiry
fallback schedule-weighted delivery
pod replacement/expiry identity transitions

Exit criteria:

simulator unit tests pass deterministically
no production code dependency on simulator

Phase 2: Runtime Scenario Harness Integration

Add PodScenarioBuilder events:
advance
cgm
announceMeal
doWork
disconnectPod
reconnectPod
replacePod
expirePod
forceQuitAndRelaunch
commandOutcome
Add common assertions for runtime state, pump commands, algorithm telemetry, replay traces, pending attribution, and fallback events.
Port the highest-risk cases first: POD-SIM-003, POD-SIM-005, POD-SIM-007, POD-SIM-009, POD-SIM-014, POD-SIM-020.

Exit criteria:

high-risk meal/fallback/pod-replacement scenarios pass through production coordinator logic

Phase 3: App/Adapter and UI Evidence

Add adapter mapping fixtures generated from the simulated pod ledger.
Add UI/recent-dose assertions for:
active meal progress text
canceled partial display
resolved/assumed meal state
fallback replay rows
pod replacement escape path
Validate recent-dose display fields:
step
request step
recommended/requested dose
delivered dose
evidence source
reconciliation disposition

Exit criteria:

local UI/evidence display agrees with runtime/telemetry trace for each scenario

Phase 4: Evidence Packaging and Merge Gate

Extend the simulation runner to include the pod-sim campaign.
Emit STR-style artifacts for each scenario.
Update check_sim_merge_gate.sh so changes touching high-risk paths run the pod-sim campaign:
BionicLoopCore/Sources/BionicLoopCore/Runtime/
BionicLoop/Runtime/
BionicLoop/Integrations/Pump/
BionicLoop/Features/Home/
reconciliation/fallback domain files
Update STP/SVVP/RTM trace links after IDs are finalized.

Exit criteria:

one command produces complete pod-sim evidence
high-risk runtime/pump changes cannot merge without pod-sim pass or explicit deviation

Phase 5: Hardware Campaign Pruning

Convert current hardware test list into:
simulator-owned regression cases
adapter-owned mapping cases
hardware-only residual cases
Keep a small hardware smoke set:
pod pairing/restore
live bolus cancel/progress behavior
live fallback mask expiry/reconnect once per major baseline
live replacement/expired-pod state transition
live iOS background/foreground behavior
For any hardware failure, create or update a deterministic pod-sim scenario before retesting repeatedly.

Exit criteria:

hardware protocol only contains cases that are truly unsimmable or required for calibration/confirmation

Risks and Mitigations

Risk	Mitigation
Simulator becomes another policy implementation	Keep policy in production runtime; simulator only models pod state and command outcomes
Simulator delivery math diverges from OmniBLE	Reuse OmniBLE constants/types where possible and add calibration tests from hardware traces
False confidence replaces hardware evidence	Keep `STP-HW-001` for BLE/firmware/iOS-only behavior and document simulator limitations
Scenario matrix becomes too large to maintain	Make table-driven scenarios with shared assertions and stable evidence schema
Tests become flaky	Virtual clock only, no wall-clock sleeps except existing async polling tests where unavoidable
Quality trace is unclear	Emit scenario-to-SRS/RA/TV maps in STR-SIM artifacts and update RTM/SVVP with final IDs

Immediate Next Step

Phase 2 is no longer the primary blocker: the deterministic core-sim and pod-sim lanes now run in the combined harness, and the full local harness run passed with core-sim, pod-sim, and alert-sim all reporting status 0 when executed with normal CoreSimulator access.

Continue in this order:

Close the remaining app-side evidence parity gap:
POD-SIM-020: local recent-dose, local output, and runtime cloud-event parity for claimed fields; real cloud/Dynamo observation only if cloud upload is claimed.
Keep UI lifecycle smoke constrained to the rows that require actual presentation/lifecycle confirmation rather than core policy:
POD-SIM-006: visual Replace Pod escape from an unresolved meal sheet.
POD-SIM-015: relaunch/foreground trigger routing for pending fallback recovery.
POD-SIM-016 and POD-SIM-017: optional SwiftUI smoke only; app/core policy is already covered by deterministic tests.
Add adapter-state mapping checks where deterministic transport fixtures can cover expired/retired/replacement pod semantics without consuming hardware: POD-SIM-006, POD-SIM-011, and POD-SIM-013.
Promote the current harness run into a working evidence bundle only after the intended app-side evidence gaps are either closed or explicitly marked as hardware/UI residuals.
Prune the hardware campaign to residual behavior only:
live bolus cancel/progress semantics
BLE reconnect timing before fallback expiry
one fallback mask-expiry/replay calibration smoke
replacement/expired-pod state transition
iOS background/foreground/force-quit timing

This keeps the simulator doing the repeatable policy/regression work and reserves real pods for firmware, BLE, lifecycle, and calibration behavior that the simulator cannot credibly prove.