How Evaluation Works

Overview

VERA is evaluated through a deterministic, buyer-run, offline process.

Evaluation does not require:

  • deployment

  • integration

  • runtime access

  • vendor trust

All evaluation activity occurs locally, under the buyer’s sole control.

What Is Being Evaluated

VERA evaluates governance behavior, not model performance.

The evaluation examines:

  • deterministic decision boundaries

  • explicit refusal and failure conditions

  • proposal versus commit separation

  • version-lock integrity

  • reproducible evidence generation

VERA does not evaluate:

  • model accuracy

  • real-world outcomes

  • production behavior

  • live system performance

Evaluation Preconditions

To evaluate VERA, the buyer requires:

  • a standard local execution environment

  • macOS or Linux

  • a local Python 3 interpreter

  • no vendor system access

  • no credentials

  • no integration with existing systems

The artifact is delivered as a version-locked evaluation archive.

Evaluation Steps

1. Artifact Receipt

The buyer receives the VERA evaluation artifact as two files:

  • a version-locked evaluation archive (.zip)

  • a corresponding cryptographic checksum (.sha256)

Download note: Evaluation is delivered as a version-locked ZIP + SHA256; if attachments are blocked by your email security policy, the same files will be provided via download link. Files may be saved to Downloads or Desktop.

2. Integrity Verification

Before execution, the buyer verifies artifact integrity using the provided checksum.

The checksum allows the buyer to independently confirm:

  • artifact integrity

  • absence of modification

  • correspondence to the stated version lock

No vendor credentials or external services are required.

3. Local Execution

The buyer runs the evaluation suite locally using a single, copy-paste execution block.

  • No file editing

  • No variable substitution

  • No path configuration

The execution block automatically:

  • locates the matching .zip and .sha256 pair

  • stages the evaluation safely

  • verifies integrity

  • extracts the archive

  • runs the deterministic evaluation suite

The evaluation script is repo-relative and executes exactly as delivered.

During execution:

  • no external services are contacted

  • no runtime access is granted

  • no data leaves the buyer’s environment

4. Evidence Generation

During execution, VERA generates:

  • deterministic governance behavior

  • reproducible evaluation results

  • refusal and boundary traces

  • manifests and version identifiers

  • evaluation logs

All evidence is generated locally and remains under buyer control. Successful evaluation substantiates specific governance claims for the evaluated artifact version. These claims are defined in the Governance Claims Substantiated document.

5. Independent Verification (Optional)

The same evaluation may be run on a second machine (Device-B).

Matching outputs confirm:

  • determinism

  • integrity

  • absence of hidden state or vendor dependency

Any discrepancies are directly inspectable.

6. Acceptance Decision

Acceptance is based solely on evidence.

Successful execution produces:

  • a clean deterministic result

  • no test failures

  • reproducible outputs

There is:

  • no subjective scoring

  • no vendor interpretation

The artifact either behaves as claimed, or it does not.

What Evaluation Proves

A successful evaluation demonstrates that:

  • governance behavior is deterministic

  • refusal conditions are explicit and testable

  • evaluation results are reproducible

  • claims can be independently verified

Evaluation does not imply:

  • runtime enforcement

  • future behavior guarantees

  • safety outside evaluated conditions

What Evaluation Does Not Require

Evaluation does not require:

  • trust in the vendor

  • access to production systems

  • ongoing services

  • subscriptions

  • future commitments

Evaluation precedes trust.

Evaluation Outcomes

There are only two valid outcomes:

  • Accept — based on evidence

  • Reject — based on evidence

Rejection is expected and valid.

What the Evaluation Produces

A VERA evaluation produces a locally generated governance evidence record suitable for internal audit, risk review, legal assessment, and board oversight.

The evaluation yields deterministic pass/fail results, cryptographic integrity verification, reproducible second-machine confirmation, explicit governance boundary enforcement, and governance claims substantiated directly by evidence.

All evidence is generated by the evaluating organization. No runtime access, services, or vendor dependency are required.

Governance Continuity

Evaluation validates governance behavior at a specific point in time.

VERA artifacts are intentionally version-locked to preserve audit integrity.

Continued reliance on updated governance evidence requires evaluation of a new artifact release under an active governance continuity renewal.

Renewal is optional and never implied.

After Evaluation

If accepted, licensing terms apply only to the specific evaluated artifact version, including its manifest, bundle snapshot, and generated evidence.

If rejected, no further action is required.

VERA does not persist access, collect data, or maintain ongoing visibility.