Back to Dashboard
What We Measure src/pipeline/signal_normaliser.py
Every time a participant event arrives, the signal normaliser runs first. It converts raw interaction data into structured scores. These scores — not the raw events — are what the readiness model and governance system work with.
The pipeline entry point
Raw event
Clicks, keystrokes, timing, device sensors
→
10 signal categories
Grouped by behaviour type
→
Conflict detection
4 contradiction patterns
→
5 sub-scores
0–100 integers (clamped)
→
9 enrichment flags
Context signals computed after scores
Desktop vs. mobile signals
Device signal separation is enforced at the schema level. Cursor and keystroke signals (
keystroke_count, backspace_ratio, inter_key_interval_avg_ms) are null on mobile payloads. Touch and orientation signals (touch_duration, touch_ratio, scroll_reversal_count, device_flat, device_held) are null on desktop payloads. The API returns 422 if the wrong signals are submitted. This is not a soft validation — a mobile session that sends keystroke data is rejected entirely.
Why enforce this strictly? Mixing signals across device types produces nonsensical sub-scores. A desktop session with null cursor signals would score artificially low on extraneous load. More importantly, including null signals as zeroes would silently corrupt the readiness calculation in ways that are hard to detect in post-hoc analysis.
The five sub-scores (all 0–100 integers)
Extraneous Load (EL)
wrong_clicks×12 + backtracking×10 + field_clears×8 + language_mismatch×15 + long_pause×10
Measures avoidable cognitive load caused by interface confusion or mismatch. High EL means the participant is doing extra work because the environment is working against them. This is the heaviest-weighted input to the cognitive load formula (50%).
Progression
Base 60 (completed) or 20 (not) − wrong_clicks×5 − backtracking×8 − invalid_actions×3 ± conflict adjustment
A proxy for persistence and forward momentum. Used as the persistence input to motivational readiness. Steps completed cleanly score high; steps with multiple retries and errors score low.
Error Control
100 − adjusted_EL, with additional penalty for fast-but-wrong conflict
Inversely related to extraneous load, with extra weight for the specific pattern of rushing and making errors simultaneously. High score = few recoverable errors relative to interactions.
Temporal Stability
time_band_score − min(long_idle_count×10, 40) + slow-but-correct adjustment
Captures whether the participant's pace is unusual. Unusually fast, unusually slow, or punctuated by long idle periods all reduce this score. The slow-but-correct conflict adjustment prevents penalising deliberate, careful participants.
Help Autonomy
100 − help_requests×10 − dismissed×20 − explicit_help×10 + accepted_help×10 ± conflict
Interprets help-seeking as information, not failure. A participant who accepts help and then succeeds is different from one who dismisses every intervention and keeps failing. Dismissals are penalised more than requests.
Context Fit
100 − offline×60 − unstable×30 − language_mismatch×30 − device_flat×20 + assisted×5
Environmental readiness — is the participant's context supporting or working against them? Offline status is the heaviest penalty (60 points). Low-quality signal confidence down-weights this score's contribution to the overall readiness formula.
Floats never leave the normaliser. All five sub-scores are clamped to 0–100 integer values before being passed downstream. If a formula produces 103.7, it becomes 100. If it produces -4.2, it becomes 0. The readiness model and governance system rely on this guarantee — floating-point scores would produce unpredictable comparisons against thresholds.
Conflict detection — four patterns that change scoring
Conflict detection runs after derived signals but before sub-scores are computed. These four patterns identify contradictions between different signal streams and trigger score overrides, because a simple averaging would produce a misleading result.
| Pattern | What it means | What changes |
|---|---|---|
| fast-but-wrong | High interaction speed combined with high error rate. The participant is rushing without checking their work — a pattern often associated with anxiety or overconfidence. | Error control score is penalised further. Temporal stability is not boosted by the fast pace. |
| slow-but-correct | Low pace but clean, accurate completion. A deliberate participant who takes their time. Without this detection, temporal stability would incorrectly flag them as struggling. | Temporal stability receives a positive adjustment. Governance is less likely to trigger an intervention despite the slow pace. |
| accepts-help-but-fails | The participant accepted an adaptation (didn't dismiss it) but their error rate remained high or worsened. The help content may not be working for this person. | Flags an adaptation failure candidate. Help autonomy score is not boosted by the acceptance. |
| ignores-help-but-progresses | Dismissed adaptations, but task is advancing cleanly. The participant is coping independently and found the interventions intrusive. | Help autonomy is not penalised for dismissals. Governance will be more cautious about firing further interventions. |
The 9 enrichment signals
These are computed after sub-scores and are used by the governance system and adaptation layer. They provide context that the five scores alone don't capture.
readiness_trajectory
Is the overall readiness score trending up, down, or flat across recent events? A rapidly declining trajectory triggers R0_PREDICTIVE even if the current score hasn't crossed a threshold yet.
error_cluster_flag
True if errors are appearing in bursts (multiple errors in a short window) rather than spread evenly. Bursts are more indicative of acute confusion than a steady background error rate.
recovery_latency_ms
How long it took the participant to recover from their last error. Long recovery latency on a short step suggests the error caused genuine confusion.
help_dwell_ms
How long the participant spent reading the help overlay before dismissing or acting. Very short dwell (<500ms) on an L2 message suggests they didn't read it — a dismissed-but-not-read signal.
pace_outlier
Is the participant's pace significantly faster or slower than their own personal baseline (established during calibration)? A ±2σ deviation from personal baseline is more meaningful than an absolute pace threshold.
connectivity_change_flag
True if the participant went offline and came back during this session. A connectivity disruption mid-task is correlated with abandonment risk even after reconnection.
abandonment_risk_score
A composite score (0–1) estimating how likely the participant is to abandon the session in the next few events. Inputs: time on step, error rate, help dismissal pattern, and session momentum. Triggers FR49 in governance when it crosses the threshold.
session_momentum_score
How much of the session has been completed relative to time elapsed. Low momentum late in a session is a disengagement signal distinct from per-step struggle.
signal_reliability_context
A quality-of-evidence score for this event's signals. Low reliability (poor connectivity, very short step, device sensor gaps) down-weights environmental readiness in the overall formula and makes the governance system more conservative.
touch_ratio is null until baseline is established. If calibration hasn't cleared, we don't have the participant's personal touch baseline, so we can't meaningfully interpret their touch ratio. Rather than using a global average (which would skew younger), the signal is null and contributes 0 to fatigue scoring until the baseline is ready.