Product roadmap¶
DevS69 SDETKit is an evidence-first, local-first release-confidence and repository-diagnosis layer for SDET, QA, DevOps, release, security, and platform teams. It reads repository-owned evidence, explains the first meaningful failure, recommends exact proof commands, and remains review-first whenever action is not proven safe.
Machine-readable authority:
docs/contracts/platform-capability-matrix.v1.jsondocs/contracts/failure-vector-support-matrix.v1.jsondocs/contracts/safety-gate-policy-matrix.v1.jsondocs/contracts/remediation-research.v1.jsondocs/contracts/formatter-policy-proposal-observation.v1.jsonbuild/formatter-policy-proposal-observation/formatter-policy-proposal-observation.jsonbuild/sdetkit/adoption-product-kpi-report.jsonbuild/sdetkit/product-maturity-radar-portfolio.json
Product promise¶
What kind of repository is this?
Which commands and artifact contracts does it own?
What failed first beneath wrapper noise?
Which workspace or workflow surface owns the failure?
What should a human run next?
Which actions are blocked?
What evidence should be retained?
SDETKit is ambitious in understanding and conservative in action. It does not replace CI, test runners, compilers, scanners, build systems, or release systems.
Stable public path¶
python -m sdetkit gate fast
python -m sdetkit gate release
python -m sdetkit doctor
python -m sdetkit adoption-surface --root . --out build/sdetkit/adoption-surface.json --format report
Target-repository commands remain advisory until a human runs them in a trusted environment.
Shared operator journey¶
repository discovery
→ workspace-scoped proof recommendations
→ saved failure evidence
→ FailureVector
→ SafetyGate
→ ProtectedVerifier
→ Doctor / PR Quality / merge readiness
→ trajectory / RepoMemory
A detector alone is not a complete product capability. A complete vertical must preserve shared evidence, diagnosis, safety, verification, reporting, and learning contracts.
Current architecture¶
| Plane | Responsibility | Implemented examples | Boundary |
|---|---|---|---|
| Evidence | Collect repository, CI, artifact, security, release, topology, workspace, and exact-head evidence. | Gate artifacts, adoption surface, failure bundles, manifests, provenance, reviewed KPI records. | Evidence does not authorize mutation. |
| Diagnosis | Extract the first meaningful failure and likely owner surface. | FailureVector, ecosystem adapters, C++ and workspace-owned saved failures. | Unknown diagnosis remains review-first. |
| Safety | Decide eligibility and block unsafe categories or scope. | SafetyGate and policy matrix. | Eligibility is not patch authority. |
| Verification | Validate scope, proof, and anti-cheat boundaries. | ProtectedVerifier, proof chain, replay benchmark. | Verification does not authorize merge. |
| Reporting | Render status, uncertainty, and one next human action. | Doctor, PR Quality, workflow governance, merge readiness, maturity radar, reviewed KPI portfolio. | Recommendations and projections are not proof. |
| Learning | Retain reviewed outcomes. | TrajectoryStore, RepoMemory, and reviewed real-repository KPI observations. | History does not authorize the current change. |
| Orchestration | Run deterministic local jobs over file-backed artifacts. | Diagnostic queue and bounded workers. | Orchestration remains non-authorizing. |
| Adapters | Translate ecosystems and providers into shared contracts. | Python, JS/TS, Go, Rust, Java, .NET, C++, GitHub Actions, GitLab, Jenkins, CircleCI, Azure DevOps. | Target repositories stay read-only by default. |
Current implementation truth¶
| Surface | Current truth | Highest-value remaining gap |
|---|---|---|
| Release | sdetkit==1.2.0 is published from immutable tag v1.2.0, with provenance, matching public distribution digests, and a clean public install independently verified. |
Keep future release verification exact-tag, fail-closed, and regression-guarded. |
| Ecosystems | Python, JS/TS, Go, Rust, Java, .NET, and C++ have supported evidence paths. | Continue shared-contract consistency audits. |
| Providers | GitHub Actions, GitLab, Jenkins literal shell, CircleCI literal run, and Azure DevOps literal script evidence exist. | Add reviewed provider observations and continue shared-contract consistency audits. |
| Mixed workspaces | A complete mixed-language monorepo operator vertical preserves root/nested commands, workspace ownership, ambiguity, Doctor, verifier, and memory evidence. | Add reviewed mixed-workspace observations to the KPI denominator. |
| Safety | SafetyGate, ProtectedVerifier, isolation, scope, and anti-cheat checks exist. | Authenticated human decision and audit contracts must precede execution. |
| Reporting | Doctor, PR Quality, maintenance, release, governance, exact-head merge readiness, maturity radar, and reviewed KPI portfolio reporting exist. | Keep every projection freshness-bound and source-backed. |
| Learning | TrajectoryStore, RepoMemory, benchmark, patterns, and two reviewed source-backed KPI observations exist. | Retain one real reviewed formatter proposal observation before later execution research. |
| UI | GitHub comments and artifacts are the current operator surface. | Thin UI only after decision and audit contracts stabilize. |
Detailed released-versus-main truth: Current product delta.
Completed execution chain¶
| Completed lane | Evidence |
|---|---|
| Protected diagnosis spine | FailureVector, SafetyGate, TrajectoryStore, RepoMemory, benchmark, ProtectedVerifier, PatchScorer, PRReporter, and JobQueue are connected. |
| Rust, Java, and .NET verticals | Discovery, saved failures, security evidence, and end-to-end Doctor proof are implemented for supported slices. |
| C++ vertical | Discovery, compiler/linker/test normalization, explicit quality/security evidence, and complete operator proof are implemented. |
| Jenkins proof-command discovery, CircleCI proof-command discovery, and Azure DevOps proof-command discovery | Literal repository-owned command evidence preserves source context; dynamic behavior remains review-first. |
| Nested workspaces | Workspace path and working-directory identity are preserved across supported ecosystems. |
| Mixed-language monorepo operator vertical | Root and nested commands, workspace failure ownership, ambiguity, SafetyGate, ProtectedVerifier, Doctor, and observation-only RepoMemory are composed deterministically. |
| Review visibility | PR Quality, release handoff, post-merge verification, and exact-head merge readiness are implemented. |
| Reviewed KPI baseline | Two source-backed reviewed observations produce eleven reviewed pass outcomes, three explicit not_applicable outcomes, seven measured metrics, and zero unavailable metrics in adoption-product-kpi-report.json. |
| KPI portfolio projection | The maturity radar, KPI report, capability matrix, roadmap, and operator guidance are projected without duplicating source authority. |
| Remediation research contract | A versioned local-first contract validates formatter-only candidate evidence, exact scope, inventories, proof, rollback, reviewer state, scenarios, limitations, and false-authority counts without granting mutation authority. |
| Candidate benchmark: formatter-only | A disposable local fixture executes the pinned formatter, retains exact before/after digests and diff evidence, proves all six research scenarios, verifies rollback, reports false positives explicitly, and grants no mutation or promotion authority. |
| Verifier and trajectory proof | The retained formatter packet is independently checked for exact scope, artifact digests, evidence shadowing, proof capture, rollback bytes, and authority boundaries before review-first trajectory and RepoMemory artifacts are written. |
| Formatter policy proposal eligibility | Provider-bound human approval is tied to the exact verifier digest, source PR, source commit, reviewer identity, and timestamp; formatter-only becomes proposal-eligible while execution, patching, merge, publication, security dismissal, and semantic-equivalence authority remain denied. |
| Formatter policy proposal observation | A versioned reporting-only contract retains digest-bound reviewer decisions and proposal-quality metrics. |
| Public 1.2.0 release | Exact wheel qualification, Trusted Publishing, provenance, public digest comparison, clean install, and GitHub Release evidence are complete. |
Selection rule¶
- Red main, install, security, package, release, or required-check blocker.
- Stale product-control evidence that could cause duplicate or unsafe work.
- Missing link in an active ecosystem or provider vertical.
- Missing reviewed denominator for an already implemented KPI metric.
- Cross-report consistency and operator decision quality.
- Human-approved bounded action contracts.
- One narrowly proven mechanical automation family.
- Packaging, documentation, UI, and future public-release polish.
Active executable PR ladder¶
| Order | Slice | Product value | Exit criteria |
|---|---|---|---|
| 1 | Formatter proposal observation | Observe the quality and usefulness of provider-bound formatter policy proposals before considering any execution research. | Reviewed proposal records remain digest-bound and review-first with zero false authority; formatter_policy_proposal_reviewed_evidence stays reporting-only and no branch execution lane is active. |
Executable roadmap¶
Phase 0 — Keep main trustworthy¶
small_scope=true
current_head_verified=true
focused_proof=true
required_checks_green_or_explicitly_nonblocking=true
compatibility_preserved=true
rollback_known=true
protected_surfaces_not_weakened=true
Phase 1 — Maintain the verified public release path¶
The v1.2.0 publication gate is complete.
- Preserve the immutable tag and source-SHA record.
- Keep the GitHub
pypienvironment and matching PyPI Trusted Publisher exact. - Build distributions once and retain manifest, qualification, and provenance artifacts.
- Check out the exact tag before repository-owned post-publish verification scripts run.
- Verify public filenames, SHA-256 digests, and clean installation before GitHub Release creation.
- Update public references only after independent verification.
- Treat changes after
v1.2.0as unreleased until a later qualified publication.
SDETKit must not silently tag or publish.
Phase 2 — Ecosystem consistency¶
C++, Rust, Java, .NET, and mixed-monorepo proofs are implemented for supported slices. Audit these shared dimensions without creating parallel schemas:
workspace_identity
source_context
working_directory
proof_purpose
confidence
manual_or_auto_run_policy
executes_untrusted_code
failure_class
failure_tool
first_failure
owner_paths
security_evidence_source
authority_boundary
Phase 3 — Reviewed product KPI evidence¶
The reviewed real-repository KPI baseline is complete.
The reviewed real-repository product KPI evidence capability is implemented and tested.
The current adoption-product-kpi-report.json is source-backed, exact-head bound, and contains two reviewed observations. Across the seven contracted metrics it records eleven reviewed pass outcomes and three explicit not_applicable outcomes. Every metric now has an applicable reviewed denominator, so measured_metric_count=7 and unavailable_metric_count=0.
The portfolio projection consumes this verified artifact without becoming a new source of authority. Denominator expansion is complete, but two observations still do not support a broad maturity claim. The versioned remediation-research contract is implemented and tested. Its machine-readable authority is docs/contracts/remediation-research.v1.json; it validates local evidence only and grants no SafetyGate policy change or patch authority. The formatter-only candidate benchmark and its verifier/trajectory proof are implemented through disposable evidence, ProtectedVerifier, ReplayableBenchmarkHarness, TrajectoryStore, and read-only RepoMemory. The active guarded_remediation_promotion lane now advances only to one source-backed reviewed proposal observation; SafetyGate policy, patch authority, merge authority, and semantic-equivalence claims remain unchanged.
Phase 4 — Provider depth¶
The conservative Azure DevOps proof discovery slice is complete. Root azure-pipelines.yml and azure-pipelines.yaml files may contribute literal script, bash, pwsh, and powershell evidence with file, job, and task-key context.
Templates, expressions, variables, variable groups, strategies, matrices, tasks, deployments, service connections, external resources, multiline scripts, and pipeline-file precedence remain unresolved and review-first. No discovered target command is executed.
Phase 5 — Trust every decision report¶
Require deterministic input digests, generator/schema provenance, exact-head or snapshot binding, missing/stale/malformed/unavailable classifications, cross-report consistency, no authoritative zero from failed collection, and one canonical next action.
Phase 6 — Operator decision cards¶
Classifications: current_valid, stale, duplicate, already_fixed, already_present, false_positive, insufficient_evidence, unsafe, unsupported, valuable_now, valuable_later.
Human choices: accept, reject, defer, request_more_evidence, mark_duplicate, mark_already_done, prepare_patch_proposal.
Every choice records actor, target, exact head, reason, evidence, and timestamp.
Phase 7 — Patch proposals without application¶
patch_application_allowed=false
direct_main_mutation=false
merge_authorized=false
A proposal includes target files, expected diff shape, risk, rollback, focused/full proof, and verifier requirements.
Phase 8 — Human-approved bounded execution¶
Require authenticated operator, explicit approval, exact target/scope, reason, dry-run preview, rollback, audit record, ProtectedVerifier, no direct-main mutation, no merge authority, and no automatic dismissal. Initial execution may create or update a branch only.
Phase 9 — One mechanical automation family¶
Promote only a repeatedly proven reversible family such as formatter-only output, deterministic metadata alignment, or docs-link hygiene. Security, dependency, release, workflow-permission, public-API, and unknown failures remain excluded.
Phase 10 — Productization¶
Deliver stable installation, canonical CLI journeys, versioned schemas, compatibility/deprecation policy, docs, support boundaries, multi-language/multi-CI proof, and rollback guidance.
Phase 11 — Supply chain and thin UI¶
Require security policy, threat model, least privilege, full-SHA pins, dependency review, Trusted Publishing, SBOM, provenance, attestations, rollback, and post-publication verification. UI must visualize existing contracts rather than create a parallel control plane.
Phase 12 — Public product release¶
stable_package=true
canonical_journeys=true
versioned_schemas=true
compatibility_policy=true
security_and_support_docs=true
trusted_release_pipeline=true
multi_ecosystem_proof=true
human_decision_UX=true
bounded_action_contract=true
known_limitations=true
external_adoption_evidence=true