Skip to content

STP-AUTO-001 Automated Verification Protocol

Status: Final draft prepared for handoff (pending review)
Version: 0.9
Owner: BionicLoop engineering
Prepared by: BionicLoop engineering Reviewer: ____
Approver: ____
Decision date: ____
Effective date: ____
Baseline freeze SHA: ____
Last updated: 2026-04-06 12:37 EDT

Revision History

Version Date Author Summary of Changes
0.1 2026-03-27 Engineering Initial automated verification protocol draft
0.9 2026-04-06 BionicLoop engineering Added handoff-ready document-control metadata and explicit TV-SEC-001 ownership/execution linkage for the software package

1. Purpose

Define the standard protocol for automated software verification executed on a development host using simulator-based unit, integration, and UI automation.

2. Scope

This protocol owns automated verification that does not require live G7/DASH hardware.

Primary coverage includes:

  • TV-RUN-001..006
  • TV-CGM-001..004
  • TV-BG-001..012
  • TV-CLIN-001..013
  • TV-PUMP-001..003, TV-PUMP-006
  • TV-MEAL-001..009
  • TV-STATE-001..002
  • TV-LOG-001..008
  • TV-SEC-001
  • TV-UI-001..010
  • automated/unit/integration subsets of TV-ALERT-*

Exclusions:

  • deterministic simulation campaign (STP-SIM-001)
  • real-device / hardware verification (STP-HW-001)
  • manual/system alert drill execution (STP-ALERT-001)
  • Algo2015 coverage/replay campaign (STP-ALG-001)
  • cloud/device-to-cloud verification (deferred from the current software handoff package)

3. References

4. Roles

  • Author: BionicLoop engineering
  • Executor: engineering or QA delegate
  • Reviewer: quality / design assurance
  • Approver: submission-quality owner

5. Prerequisites

  • Checked-out submission-baseline branch or review SHA
  • Clean or intentionally controlled worktree
  • Current Xcode and simulator runtime installed
  • Required local packages resolve successfully
  • Required test fixtures and deterministic defaults available

6. Environment

  • Host: macOS development workstation
  • Toolchain: checked-in Xcode project + local Swift package resolution
  • Device class for UI automation: iPhone 17 family simulator unless protocol-specific exception is recorded

7. Procedure

7.1 Setup

  1. Record git SHA, branch, Xcode version, simulator runtime, and test destination.
  2. Confirm source packages resolve and the checked-in .xcodeproj is used.
  3. Confirm required environment flags/fixtures for deterministic UI tests are set.

7.2 Execution

  1. Run focused suites for the affected workstream first.
  2. Run broader impacted suites at minimum:
  3. BionicLoopTests
  4. relevant BionicLoopUITests lanes when user-facing behavior changed
  5. If cloud-log review support is part of the run, start an Integration Log Session in Home Settings before execution and stop it immediately after the scenario completes.
  6. Capture command output and xcresult paths.
  7. Record any skipped tests with rationale.
  8. If TV-SEC-001 is part of the freeze run, execute it using Cybersecurity_TV_SEC_001_Freeze_Execution_Checklist.md and archive the result under the formal STR-SEC-001 path.

7.3 Failure / Deviation Handling

  1. If simulator-only infrastructure fails, rerun once after resetting simulator state.
  2. If failure persists, record blocking defect or protocol deviation before claiming pass.
  3. Do not replace hardware-required evidence with simulator evidence.

8. Expected Results

  • All selected automated TV-* rows pass or are dispositioned explicitly.
  • No unexpected regression failures appear in broader impacted suites.
  • Output artifacts identify exact SHA, environment, and executed destinations.

9. Pass / Fail Criteria

  • Pass when all protocol-selected automated suites pass and artifacts are archived under the planned STR path.
  • Fail when any required TV-* row fails or required evidence is incomplete.

10. Evidence to Capture

  • test commands used
  • console/test logs
  • xcresult paths
  • screenshots for UI automation where relevant
  • git SHA and test environment metadata
  • if cloud-log review support was used:
  • test_run_id
  • UTC time window
  • selected upload threshold
  • explicit note that the session was started before execution and stopped after execution
  • STR target path, usually:
  • Docs/Quality/Evidence/Formal/STR-UI-AUTO-001/<run-label>/ for formal UI automation
  • family-specific formal STR paths for broader automated execution

11. Traceability Summary

Group TV-* Coverage
Runtime automated TV-RUN-001..006
CGM/BG automated TV-CGM-001..004, TV-BG-001..012
Clinical/UI automated TV-CLIN-001..013, TV-UI-001..010
Meal/log/state automated TV-MEAL-001..009, TV-STATE-001..002, TV-LOG-001..008
Security automated TV-SEC-001
Alert automated subsets unit/integration/UI subsets of TV-ALERT-*