RDA Vibeware · Production Readiness Review
Read 1 of 2 · The Scorecard
Readiness Scorecard: 46% (F)
Each category is scored 0–5 and weighted by its influence on shipping safely. Bars show today's score against the maximum — the remediation roadmap later in this report is sequenced to move them.
Code & Architecture (w 2.5)
3.0
AI / ML Components (w 1.5)
2.5
Integration & Legacy Map (w 1.0)
2.0
Security & Compliance (w 1.0)
1.0
Operability & Observability (w 1.0)
2.0
| Category | Weight | Score (0–5) | Weighted |
|---|
| Code & architecture review | 2.5 | 3.0 | 1.50 |
| AI/ML component assessment | 1.5 | 2.5 | 0.75 |
| Integration & legacy systems map | 1.0 | 2.0 | 0.40 |
| Security & compliance review | 1.0 | 1.0 | 0.20 |
| Operability & observability review | 1.0 | 2.0 | 0.40 |
| Overall | 7.0 | — | 3.25 / 7.0 = 46% |
Swipe the table for more →
Code & architecture (3.0) blends backend core (3.0), frontend admin console (3.0), and test quality (3.5); the backend dominates the weight. AI/ML and Ops sit at the low-middle because the abstractions are sound but operational controls are mostly unbuilt. Security at 1.0 is the anchor: with no authentication anywhere in the backend, nothing else can pull the composite above a failing grade.
RDA Vibeware · Production Readiness Review
From Findings to Action
The Four Ship Blockers
Four hard blockers must be cleared before any non-localhost deployment. Each one independently forces a “Not Ready” verdict. The prioritized remediation plan on the next page sequences all twenty items by severity and effort.
The four ship blockers
- No authentication or authorization anywhere in the backend. Any network client has full anonymous admin control — create/deploy/delete agents, run workflows, upload/delete documents, introspect the cluster. Fix: register AddAuthentication (JWT/OIDC) + AddAuthorization, [Authorize] on admin/Orleans controllers; at minimum gate behind an authenticating reverse proxy. (Tracked #65, #86, #45, PR #32.)
- Broken multi-tenancy / IDOR. No per-user identity exists; even after adding auth, any principal can read/mutate any tenant's resources by id. Fix: introduce a real authenticated principal, key user-scoped grains by it, add ownership checks before every grain lookup. (Partially tracked #65, #86.)
- Orleans cluster-management endpoints exposed anonymously. OrleansController (in the shippable library) leaks silo addresses, topology, and live grain stats. Fix: require an admin authorization policy, feature-flag/remove in production, move to authenticated clustering. (Untracked — new issue.)
- No durable event-store or read-model — persistence is in-memory only. An event-sourced platform with no durable journal/views loses every agent, thread, deployment, and projection on restart. Fix: ship at least one durable IJournalStorageProvider + persistent view repository + durable clustering; forbid in-memory in Production. (Partially tracked #13/#100/#14/#58/#57; journal WRITE path untracked.)
RDA Vibeware · Production Readiness Review
Backlog Alignment
Issue Alignment & Backlog
Findings were aligned against the GitHub issues at df8adf6. Of 56 findings: 2 well-tracked, 22 partially-tracked, 32 untracked.
Tracked
No-auth blocker → #65 (RBAC contract), #86 (first-admin/self-registration), #45 (endpoint authz visibility), PR #32 — all M0-MVP. MCP-tools-dropped-at-chat → #59/#102 + PR #103.
Partially tracked
View-store durability has a full set (#13 Mongo, #100 Postgres, #14 SQL, #58 Cosmos, #57 HorizonDb) — but the durable Orleans journal/clustering WRITE path has no issue. Observability rails → #43, #98/#99, PR #31 — but per-call cost/token capture and product instrumentation rollout are unscoped. Audit → #85. Secrets → #51. IDOR/CORS hardening sit adjacent to the auth issues but aren't explicitly scoped.
Untracked (new-issue candidates)
Anonymous Orleans admin lockdown; event versioning/upcasting; [GrainType] ×4; tool-loop bound; LLM resilience/timeout/429; max-output + model params; prompt-injection guardrails; MCP URI/SSRF validation + bearer-leak redaction; blob path-traversal + upload limits; security headers/HSTS/exception handler; CORS lock-down; health probes in prod; deployment artifact/IaC/rollback; frontend CI + lockfile + package build; controller/authZ/frontend tests + coverage gate; dead IMetricsWriteProvider + mock metrics UI; publish.yml tag injection; empty-plugin packability.
Milestone assessment — M0-MVP is not a usable prod-readiness gate
M0 mixes hard blockers (#65/#86/#45 auth; #13 persistence) with feature-expansion that is not a prod gate — and whole blocker categories have no issue in any milestone (durable grain/journal storage, security headers/CORS/exception handler, production health endpoints, deployment/IaC, frontend CI, LLM resilience/cost/loop bounds). Recommendation: carve an explicit “prod-gate” subset out of M0 (auth #65/#86/#45 + tenant-scoping; durable persistence #13 + a new durable-journal issue; secrets #51) and file new issues for the untracked blocker-class gaps before treating M0 completion as ship-ready.
