# Verification and Validation Methodology

Turn customer specifications into traceable test data, a deterministic runner,
human-readable test documentation, and retained execution evidence. Keep these
four artifacts separate so a specification revision produces reviewable diffs.

## Derive the test model

1. Inventory every source with revision and applicability: communication matrix,
   DBC, diagnostic survey, CDD/ODX, enterprise CAN rules, network management,
   bootloader flow, memory map, security requirements, and ECU variant.
2. Extract atomic requirements with source section, preconditions, stimulus,
   expected response, timing/tolerance, repetition, cleanup, and safety class.
3. Normalize IDs, frame flags, signal scaling/ranges, invalid values, cycle and
   timeout behavior, UDS session/security, NRCs, addressing, padding, transfer
   limits, and variant conditions. Record contradictions instead of guessing.
4. Build coverage using boundary values, equivalence classes, state transitions,
   timing edges, negative cases, and pairwise/orthogonal combinations. Avoid an
   unbounded Cartesian product; justify reduced combinations.
5. Assign stable requirement and test-case IDs. Preserve source traceability in
   generated data, names, reports, and evidence.

For LIN, inventory the LDF or communication matrix revision and normalize node
roles, schedule tables, frame publisher/subscribers, protected IDs, DLC,
signal packing, checksum model, response timing, unconditional/event-triggered
semantics, diagnostic frames, sleep/wakeup, pull-up, and power ownership. Use
`ldf info` and the typed node/frame/signal/schedule views as the normalized input
when an LDF is available. Inspect a candidate with `ldf schedule show` before
using `ldf schedule activate`. Activate only reviewed schedules that contain
supported unconditional or diagnostic entries; classify event-triggered, sporadic, and
configuration entries as policy-required rather than silently approximating them.
The runner must retain traceability from every runtime `system` entry or raw
master-write/header request/slave response to its source table and entry.

## Generate maintainable automation

- Store normalized cases in CSV/JSON/YAML and keep execution logic generic.
- Use shell only for short smoke sequences. Use Python for parameterized cases,
  timing assertions, retries, JSONL, reports, and artifact management.
- Query runtime help for every command family used by the runner.
- Dry-run request construction where supported, then execute one authorized
  representative case before a matrix.
- Use DBC message and signal semantics only after checking network and variant.
  Without a DBC, degrade to explicit IDs and bytes and record the limitation.
- Never generate or infer an OEM seed-key algorithm. Load only the approved
  external library named by the project configuration.
- For BMCLI-backed white-box integration tests, use A2L names in test data and
  resolve them at runtime. Use XCP characteristic writes to inject reviewed
  internal state, synchronous measurement reads for assertions, and DAQ when
  timing or a sequence matters. Correlate those observations with CAN/LIN
  signals or UDS results while keeping each protocol's response ID distinct.
  Validate the transport against an independent XCP Slave implementation, not
  only a BMCLI-authored mock. Acceptance must include an A2L symbolic
  read/write/readback round trip and a bounded DAQ run while UDS and logging
  remain active on the same analyzer channel.

## Execute and classify

Run preflight doctor and bench validation, capture ECU identity/software DIDs,
then execute each test independently with its own setup and cleanup. Distinguish:

- pass;
- requirement assertion failure;
- valid UDS negative response;
- transport or BUSOFF failure;
- test-infrastructure failure; and
- not run because a precondition or authorization was absent.

Record exact argv, exit code, stdout/stderr, timestamps, analyzer serial/port and
firmware, BMCLI build identity, ECU identity, configuration hashes, captures,
and generated report. Add postflight DTC and communication health checks where
the specification permits them.

Randomize independent HIL tests to expose state leakage. A test must not depend
on daemon state, routes, TX tasks, bus mode, files, or ECU state left by an
earlier case. Treat flashing as a reviewed multi-stage workflow and verify ECU
identity/version, DTCs, and functional smoke behavior after reset; TransferExit
alone is not evidence of successful programming.

For each LIN master/slave case, run both role directions when the product permits
it, validate exact ID/DLC/data/checksum/error flags and response counts, then
remove response tasks and stop owned daemon state. A partial response count is a
failure unless a documented firmware limitation is explicitly under acceptance;
never conceal it with an undocumented retry inside a production command.
