For Researchers src/api/dashboard_analytics.py

The researcher dashboard gives you a real-time view of the study. This section explains what each view shows and how to interpret the numbers — not just what the data is, but what to look for.

Overview (Study Health)

The Overview shows enrolment count, active sessions, completion rate, attrition, and condition balance. Watch the condition balance. If one condition is accumulating more sessions than the others, the study protocol may have a condition assignment bug — conditions should be randomised at session creation and balance over time. A persistent imbalance after 20+ sessions is worth investigating.

Overview (Study Health) — episode count

The Overview now includes an Episodes Completed stat alongside session counts. This is the total number of scenario episodes across all participants and sessions. Low episode counts relative to completed sessions indicate participants are not reaching Episode 2 or 3 — check abandonment patterns and task difficulty.

Sessions view

Live and recent sessions with readiness state, last governance decision, and study participant ID. Key things to look at:

• Readiness state distribution. If most live sessions are OVERLOAD, the tasks may be too complex, or the calibration isn't clearing correctly.
• Last decision distribution. A high rate of WITHHOLD decisions is expected (it means governance is correctly identifying sessions that don't need help). A very high rate of WITHHOLD_ALWAYS suggests most sessions are in the control condition — check condition balance.
• Sessions without calibration cleared. If sessions are lingering without calibration_cleared = true after several minutes, something is wrong with baseline event submission from the participant app.
• Episode progress badge. Each session row shows EP 2/3 and the current ability score. A session stuck at EP 1/3 for a long time may indicate the participant is struggling to complete the first scenario goal.

Sessions view — Episodes tab

Clicking any session row opens a detail drawer. The Episodes tab shows the full within-session scenario trajectory.

• Ability trajectory chart. A bar chart showing ability before and after each episode. An upward trajectory across 3 episodes indicates the participant is learning within the session. A flat or declining trajectory in the governed_adaptive condition is a signal worth investigating.
• Episode cards. Each card shows: scenario ID, difficulty badge (D1–D5), performance score bar (green ≥0.70, amber ≥0.40, red below), readiness at start / floor / end grid, help event count, governance flags (worsened outcome, L4 HANDOFF, L3+).
• Governance flags. A worsened pill means the participant's outcome measurably declined after an adaptation — the system penalised this episode's score and applied a 60-second passive-help-only cooldown. A L4 HANDOFF flag means the session reached the human handoff threshold during this episode.
• What to look for. If Episode 3 difficulty is lower than Episode 1 difficulty across many sessions, the ability estimation is working — participants who struggled are receiving easier final scenarios. If difficulty is consistently rising, the model may be over-estimating ability from low-help easy completions.

Governance view

Rule hit rates and cooldown effectiveness. Tells you which rules are firing most often and whether cooldowns are behaving correctly.

• R12 hit rate (explicit help) should be relatively low — if it's high, participants aren't getting proactive help early enough.
• FR49 hit rate (abandonment risk) tracks how close participants are getting to giving up. A rising FR49 rate over the study period may indicate task difficulty is increasing (later tasks may be harder).
• Cooldown effectiveness — after a governed intervention, did the next event show improvement? Low effectiveness rate for a specific rule is a signal that the content for that rule isn't working.

Signals view

Signal completeness, confidence histogram, baseline quality. Important for data quality assessment:

• Signal completeness — what percentage of events had full signal payloads vs sparse (few signals, low confidence). High sparsity may indicate connectivity issues or app problems.
• Baseline quality — were calibration events high-confidence? Low-quality calibrations produce unreliable personal baselines and make all downstream readiness scores less trustworthy for that session.
• Device type split — are the proportions of desktop vs mobile sessions as expected for your study population?

Sequencing view (Analytics)

The Sequencing tab in Analytics shows ability distribution and episode patterns across all participants.

• Ability score distribution. Histogram of all participant ability scores in 5 buckets (0–19, 20–39, 40–59, 60–79, 80–100). Color-coded green ≥ 60, amber 40–59, red below 40. A skewed distribution can indicate floor effects (tasks too hard) or ceiling effects (tasks too easy).
• Episodes by difficulty tier. Stacked bar (completed vs incomplete) for each D-tier. If D3 shows high incompletion relative to D2, the difficulty gap between tiers may be too steep.
• Scenario performance table. Per-scenario: attempts, completion rate, avg performance score, avg help events. Sorted by difficulty then scenario ID. Use this to identify poorly-performing scenarios for content review.

Participants view — ability estimate

The participant drawer now includes an Adaptive Ability Estimate section showing:

• Current ability score (color-coded: emerald ≥60, amber ≥40, red <40) with ZPD label (e.g., "D3–D4 zone")
• Attempts count — how many episodes have contributed to this estimate
• Last scenario ID (most recent scenario attempted)

A low attempts count (1–3) means the ability estimate is still noisy. Treat it as a prior rather than a precise measurement until at least 5 episodes have contributed.

Outcomes view

Completion rates by condition, task, and age group. This is your primary outcome measure. Compare:

• governed_adaptive vs control — does adaptive support improve completion rates?
• governed_adaptive vs always_adaptive — does governance quality matter, or is any intervention better than none?
• Task-by-task completion rates — which tasks have high abandonment? These may need content improvements or governance threshold adjustments (in calibration phase only).

Alerts view

Four alert categories. Check this daily during active data collection:

Content management (admin)

The Content view lists all adaptation content by step, language, and level. Every content update creates a version history entry and writes to the audit log. You can see when content was changed and by whom.

When to update content: A high rate of Ignored failures on a specific step type means participants aren't engaging with the content — it may be too long, unclear, or in the wrong register. A high rate of Worsened failures may mean the content is confusing rather than helpful. Use the Governance view to identify which steps have the highest failure rates, then review the content for those steps.

Exports

Seven export views available in CSV, XLSX, SPSS, and R formats. All exports are logged in the audit trail. Dashboard and export data is always in English regardless of participant language.

Analysis cautions

• Completion rate is the primary outcome measure; intervention count is explanatory, not automatically positive.
• WITHHOLD is not a failure. It often means governance preserved autonomy because the participant was coping.
• Low signal confidence should reduce certainty in readiness interpretation for that event or session.
• Capture mode matters. Compare proxy, extension, limited, and offline-replayed sessions when investigating data quality.

Participant management

You can withdraw a participant at any time. Withdrawal flags their record but retains all historical data — this is required for research integrity. Excluded sessions are excluded from primary analysis exports but remain in the raw exports. Exclusion and reinstatement are both audit-logged.