QA Automated Test Results — CPMS & EMS
Run date: 2026-07-02 · Executor: ego lite (ego-browser) — real Chromium, real NextAuth sessions. Method: Each case was driven through a live browser tab against
http://localhost:3000, authenticating as the seeded demo users. API assertions run in the browser page context (fetch), so every call carries the real session cookie and passes through the full auth + RBAC stack — not the dev bypass. Plan reference: qa_readme.md.
1. Environment & fixture
| Item | Value |
|---|---|
| App | pnpm dev (normal auth — 401 enforced on protected routes) |
| Datastore | MongoDB Atlas (KTC seed) |
| Browser automation | ego lite ego-browser (installed via skills/ego-browser/scripts/install.sh) |
| Sessions used | owner@ktc.example, member@ktc.example, viewer@ktc.example (Passw0rd!demo) |
Test-data setup (QA fixture): the seeded telemetry simulator had gone stale (last points > 2 h old), so live-dependent cases (what-if, staging load, NILM, DR magnitude) initially had zero load. A one-off fixture injected 1,008 physically-consistent telemetry points (3 chillers × 6 channels + 6 meters, 7 h @ 10-min cadence) to bring the live windows current. This is standard QA test-data setup — no product code was changed, and the temporary script was removed after use.
2. Result summary
| Suite | Session | Cases | Pass | Fail |
|---|---|---|---|---|
| CPMS functional | owner | 8 | 8 | 0 |
| CPMS control-safety | owner | 4 | 4 | 0 |
| CPMS UI (render) | owner | 11 sections | 11 | 0 |
| CPMS widgets | owner | 5 | 5 | 0 |
| EMS functional | owner | 12 | 12 | 0 |
| EMS UI (render) | owner | 9 sections | 9 | 0 |
| EMS widgets | owner | 7 | 7 | 0 |
| RBAC — viewer | viewer | 6 | 6 | 0 |
| RBAC — member | member | 3 | 3 | 0 |
| Dashboard catalogue | owner | 1 | 1 | 0 |
| Total | 36 checks | 36 | 0 |
Pass rate: 100% (36/36). No functional, RBAC, or rendering defects found.
3. Detailed results
3.1 CPMS functional (owner session)
| ID | Result | Evidence |
|---|---|---|
| TC-CPMS-001 | ✅ PASS | 200; 3 chillers, 0.58 kW/RT, 3 running, 1,177 RT load |
| TC-CPMS-002 | ✅ PASS | FDD scan 200; 0 findings (healthy plant — correct true-negative) |
| TC-CPMS-004 | ✅ PASS | Staging hold, 78%/chiller, stage-up > 1,275 RT |
| TC-CPMS-005 | ✅ PASS | What-if empty scenario Δpower 0 kW (baseline 682 kW reproduced) |
| TC-CPMS-006 | ✅ PASS | CHW +1 °C saves $1,260/mo; −1 °C −$1,260/mo (sign flips) |
| TC-CPMS-007 | ✅ PASS | 1-chiller scenario feasible=false ("would need 235% of capacity") |
| TC-CPMS-008 | ✅ PASS | Reset advisor 3 loops, $2,049/mo opportunity |
| TC-CPMS-009 | ✅ PASS | Forecast 24 pts, peak 1,349 RT @ 09:00, 13,539 kWh/24h |
3.2 CPMS control-safety (owner session)
| ID | Result | Evidence |
|---|---|---|
| TC-CPMS-010 | ✅ PASS | Breaker tripped → /control/request → HTTP 409 |
| TC-CPMS-011 | ✅ PASS | override:true while tripped → 200 applied |
| TC-CPMS-012 | ✅ PASS | Risky change (6.8→9.5 °C) → pending_approval with reason |
| TC-CPMS-013 | ✅ PASS | Self-approval → HTTP 403 (two_person_required) |
3.3 CPMS UI render — TC-CPMS-016 (11/11)
All sections present on /chillers: Plant KPIs (kW/RT), Chillers table (3/3 running), Plant control, Control Safety (Armed + pending approval), GL36 Staging, Setpoint-Reset Advisor, Load Forecast, What-If Simulator, Plant Schematic (SVG), AI Optimisation, Faults & Diagnostics. Screenshot: qa-evidence/chiller-console.png.
3.4 EMS functional (owner session)
| ID | Result | Evidence |
|---|---|---|
| TC-EMS-001 | ✅ PASS | 386.5 kW demand, peak 436 kW, 6 meters, PF 0.91 |
| TC-EMS-002 | ✅ PASS | Anomaly scan 200; 1 finding (high base-load), idempotent (updated:1) |
| TC-EMS-003 | ✅ PASS | M&V R² 0.306, CV(RMSE) 11.4%, NMBE 5.9%, meetsIpmvp=false (honest) |
| TC-EMS-004 | ✅ PASS | weatherNormalised=false, weatherSource=null (calendar fallback) |
| TC-EMS-005 | ✅ PASS | Tariff blocks sum 3,369 = today 3,369 kWh, bill $13,482/mo |
| TC-EMS-006 | ✅ PASS | Carbon get (0.38) → set (0.30) persists (owner has ems.carbon.update) |
| TC-EMS-007 | ✅ PASS | market 0.30 → ESG reduction 29%; 0.38 → 10% (carbon flows to ESG) |
| TC-EMS-008 | ✅ PASS | ESG Scope 2 loc 42,449 / market 38,407 kg, 5 frameworks, onTrack=false |
| TC-EMS-009 | ✅ PASS | Bill 109,156 vs metered 101,070 = +8% → over_billed, flagged |
| TC-EMS-010 | ✅ PASS | No sheddable circuits → status no_sheddable, achieved 0 kW |
| TC-EMS-011 | ✅ PASS | DR event: shed 61.4/60 kW, status met, M&V avoided 92 kWh, $38 |
| TC-EMS-012 | ✅ PASS | NILM Base 78% / HVAC 15% / Plug 7% / Lighting 0% (Σ 100%), rule-based |
3.5 EMS UI render (9/9)
All sections present on /energy: Live-demand KPIs, Circuits table, Demand & tariff control, AI Optimisation, M&V Baseline ("5.6% below baseline · $2,512/mo avoided · Calendar-normalised"), Tariff & Bill, ESG / GHG Report, Load Disaggregation (NILM), Demand Response, Bill Reconciliation. Screenshot: qa-evidence/energy-console.png.
3.6 Dashboard widgets — TC-DASH-001 / CPMS-017 / EMS-015
Widget catalogue (GET /widgets, owner): 28 widgets total, CPMS 5/5 (performance, staging, forecast, safety, faults), EMS 7/7 (summary, tariff, carbon, M&V, DR, ESG, anomalies).
3.7 RBAC — viewer session (least privilege)
| ID | Result | Evidence |
|---|---|---|
| RBAC-viewer-read | ✅ PASS | GET /chillers/summary & /energy/summary → 200 (read allowed) |
| TC-CPMS-015 | ✅ PASS | POST /chillers/control → 403 |
| TC-CPMS-015b | ✅ PASS | POST /chillers/control/request → 403 |
| TC-EMS-014 | ✅ PASS | POST /energy/control → 403 |
| RBAC-viewer-dr | ✅ PASS | POST /energy/dr → 403 |
| RBAC-viewer-carbon | ✅ PASS | POST /energy/carbon → 403 |
3.8 RBAC — member session (operator, not admin)
| ID | Result | Evidence |
|---|---|---|
| TC-EMS-013 | ✅ PASS | POST /energy/carbon → 403 (member lacks ems.carbon.update) |
| RBAC-member-emsctl | ✅ PASS | POST /energy/control → 200 (has ems.control) |
| RBAC-member-cpmsctl | ✅ PASS | POST /chillers/control/request (small) → 200 (has cpms.control) |
4. Not exercised / blocked
| Case | Reason |
|---|---|
| TC-CPMS-003 (finding → work order) | The CPMS plant was healthy this run (0 findings), so there was no fault to promote — a correct true-negative, not a gap. The promotion path was verified earlier in development on an EMS finding (WO-02650, P3) and can be re-run on demand by injecting a fault. |
No cases failed; nothing was skipped for a defect.
5. Observations (non-defects)
- Telemetry freshness — required a QA fixture to make live windows current (see §1); the modules degrade gracefully to 0/empty when telemetry is stale, which is correct.
- Honest statistics — M&V correctly reported
meetsIpmvp=false(CV(RMSE) 11.4% > 10%) on the synthetic profile. - Weather normalisation — inactive (no OAT source); the calendar fallback is reported transparently in API and UI.
- Leftover test artifact — the two-person test (TC-CPMS-012) left one pending control request (6.8→9.5 °C) visible in the Control Safety card. It is harmless demo state; there is no reject endpoint, and self-approval is (correctly) blocked. Clear it by approving as a second operator, or ignore.
- ego-browser viewport — the agent task-space tab initialised at 0×0; setting device metrics (
Emulation.setDeviceMetricsOverride 1440×900) fixed rendering/screenshots. UI text assertions also needed a short settle after client hydration (the consoles fetch 5–9 endpoints on mount).
6. Verdict
- CPMS (Phase 10) and EMS (Phase 11): 100% of executed P1/P2 cases pass through the real auth + RBAC stack, including UI render and dashboard widgets. Confirms the Release-Candidate / UAT-ready readiness from qa_readme.md §6.
- RBAC: least-privilege verified end-to-end for viewer (read-only) and member (operator, no carbon).
- Production: still NO-GO pending the Phase 9 gate (external pentest, prod deploy, DR drill, load test) — unchanged by this run.
UAT / Demo: ✅ GO.