# Verification and Validation Methodology

Turn customer specifications into traceable test data, a deterministic runner,
human-readable test documentation, and retained execution evidence. Keep these
four artifacts separate so a specification revision produces reviewable diffs.

## Derive the test model

1. Inventory every source with revision and applicability: communication matrix,
   DBC, diagnostic survey, CDD/ODX, enterprise CAN rules, network management,
   bootloader flow, memory map, security requirements, and ECU variant.
2. Extract atomic requirements with source section, preconditions, stimulus,
   expected response, timing/tolerance, repetition, cleanup, and safety class.
3. Normalize IDs, frame flags, signal scaling/ranges, invalid values, cycle and
   timeout behavior, UDS session/security, NRCs, addressing, padding, transfer
   limits, and variant conditions. Record contradictions instead of guessing.
4. Build coverage using boundary values, equivalence classes, state transitions,
   timing edges, negative cases, and pairwise/orthogonal combinations. Avoid an
   unbounded Cartesian product; justify reduced combinations.
5. Assign stable requirement and test-case IDs. Preserve source traceability in
   generated data, names, reports, and evidence.

For LIN, inventory the LDF or communication matrix revision and normalize node
roles, schedule tables, frame publisher/subscribers, protected IDs, DLC,
signal packing, checksum model, response timing, unconditional/event-triggered
semantics, diagnostic frames, sleep/wakeup, pull-up, and supply ownership. BMCLI
currently executes reviewed raw primitives; the runner must retain traceability
from every generated master-write/header request/slave response to its source row.

## Generate maintainable automation

- Store normalized cases in CSV/JSON/YAML and keep execution logic generic.
- Use shell only for short smoke sequences. Use Python for parameterized cases,
  timing assertions, retries, JSONL, reports, and artifact management.
- Query runtime help for every command family used by the runner.
- Dry-run request construction where supported, then execute one authorized
  representative case before a matrix.
- Use DBC message and signal semantics only after checking network and variant.
  Without a DBC, degrade to explicit IDs and bytes and record the limitation.
- Never generate or infer an OEM seed-key algorithm. Load only the approved
  external library named by the project configuration.

## Execute and classify

Run preflight doctor and bench validation, capture ECU identity/software DIDs,
then execute each test independently with its own setup and cleanup. Distinguish:

- pass;
- requirement assertion failure;
- valid UDS negative response;
- transport or BUSOFF failure;
- test-infrastructure failure; and
- not run because a precondition or authorization was absent.

Record exact argv, exit code, stdout/stderr, timestamps, analyzer serial/port and
firmware, BMCLI build identity, ECU identity, configuration hashes, captures,
and generated report. Add postflight DTC and communication health checks where
the specification permits them.

Randomize independent HIL tests to expose state leakage. A test must not depend
on daemon state, routes, TX tasks, bus mode, files, or ECU state left by an
earlier case. Treat flashing as a reviewed multi-stage workflow and verify ECU
identity/version, DTCs, and functional smoke behavior after reset; TransferExit
alone is not evidence of successful programming.

For each LIN master/slave case, run both role directions when the product permits
it, validate exact ID/DLC/data/checksum/error flags and response counts, then
remove response tasks and stop owned daemon state. A partial response count is a
failure unless a documented firmware limitation is explicitly under acceptance;
never conceal it with an undocumented retry inside a production command.
