Task Sequencing src/pipeline/sequencer.py

Within each practice session, AGS presents up to 3 episodes. Each episode is a distinct scenario on the same simulation site. The scenario is not fixed — it is selected at runtime to match the participant's current ability, targeting the Zone of Proximal Development (ZPD): just beyond what they have demonstrated, not so far beyond as to cause defeat.

Why adaptive sequencing for this population

Older adults show substantially higher within-person variability than younger adults — fatigue, time of day, health, and emotional state all affect task performance more strongly than in younger groups. A fixed scenario order that works well for one participant may be either trivially easy or impossibly hard for another with nominally the same digital literacy level.

Traditional mastery-learning approaches require many trials to converge on the right difficulty. AGS uses a small number of episodes per session (3) combined with calibration data and a persistent ability estimate to get close to the right difficulty from the first scenario.

Ability estimate — ELO-inspired model

Each participant has a single persistent ability_score (0–100, default 50.0) stored in the participant_ability table. It is updated after every completed episode.

Performance score

The performance score for an episode is computed from three weighted components plus governance penalties:

Governance penalties applied after the weighted sum:

• −0.10 if a Worsened adaptation failure was recorded during the episode
• −0.10 if the readiness floor fell below 20 (OVERLOAD reached at any point)
• −0.15 if a Level 4 HANDOFF intervention fired during the episode

The final score is clamped to [0.0, 1.0]. All constants live in src/config.py (TASK_HELP_BUDGET, TASK_READINESS_DELTA_WEIGHT).

Ability update formula

After scoring, ability is updated using an ELO-inspired rule:

expected = 1 / (1 + 10^((difficulty_level − ability/20 − 0.5) / TASK_ELO_BAND))
K = 32  if attempts < 5   (high plasticity — early estimates)
    20  if attempts < 15  (settling)
    12  otherwise          (stable estimate)
new_ability = clamp(ability + K × (performance − expected), 0, 100)

The K-factor shrinks as the system accumulates data, reducing oscillation once the estimate has converged. A participant at ability 50 facing a D3 scenario has an expected performance of ~0.5 — a score of 1.0 pushes ability up by 16 points (K=32 × 0.5); a score of 0.0 pulls it down by 16.

Worked example

ZPD targeting — next scenario selection

After scoring, the next scenario is selected from the participant's Zone of Proximal Development:

1. Compute target difficulty: round(ability / 20) → ability 63.8 → D3
2. If end-of-episode readiness < 40 (LOW/OVERLOAD): shift target down by 1
3. Filter to scenarios not yet visited this session
4. Apply D4/D5 gate: if no D3 scenario completed yet, restrict candidates to D1–D3
5. From remaining candidates, prefer those within ±1 of target difficulty
6. Within that set, sort by (|difficulty − target|, scenario order) and take the first

If all scenarios are exhausted, session_done: true is returned and the participant proceeds to the survey.

Difficulty tiers and the D4/D5 gate

D4 and D5 scenarios involve privacy judgment and scam detection — tasks where a mistake has a meaningful real-world analogue. The gate ensures participants encounter these only after demonstrating they can navigate multi-step tasks reliably. Additionally, D4/D5 are never the starting scenario in any session.

Starting difficulty algorithm

At the start of each session, three adjustments are applied to the persistent ability before selecting the first scenario:

1 — Time-gap decay

Older adults experience cognitive deconditioning between study visits. Ability is reduced before applying today's calibration:

• Gap > 30 days: −10.0 (significant deconditioning)
• Gap > 14 days: −5.0 (moderate gap)
• Gap ≤ 14 days: no decay

These are the TASK_DECAY_14D and TASK_DECAY_30D constants in src/config.py.

2 — Calibration delta

The 3 baseline interactions provide a same-day readiness signal before the first scenario loads. The calibration delta is computed from:

• IKI ratio — actual inter-keystroke interval / participant's personal baseline IKI mean. Ratio > 1.3 (typing 30% slower than usual) → −5.0. Ratio < 0.8 → +3.0. Skipped on mobile (no keyboard).
• Error rate — wrong clicks / total clicks during baseline. Each 0.1 of error rate → −0.8 (max −8.0)
• Help rate — help events during baseline (0–3). Each event → −5.0 (max −15, then clamped)

Total calibration delta is clamped to [−10.0, +10.0] and applied to the decayed ability.

3 — D3 cap

Starting difficulty is capped at D3 regardless of adjusted ability. A participant whose persistent ability is 85 (D5 zone) starts at D3 and earns their way back to harder scenarios through today's performance. This protects against overestimation from prior sessions on good days.

Scenario catalogue

Scenarios are defined in src/config/task_profiles.json. Each profile specifies the scenario ID, difficulty, expected completion time, the template step to start at, and the expected step sequence. All five practice sites (WhatsApp sim, SHA service form, KRA iTax, online shopping, health search) share the same sequencing API.

Participant-facing episode UX

Each episode is presented to the participant in three distinct moments:

1. Scenario intro card — shown before the sim becomes interactive. Displays the task number ("Task 1 of 3"), a scenario icon, the task title, the goal in plain language, and a single-sentence success condition ("You'll know when you send the reply"). The participant taps Start to begin. The sim is already loaded behind the card — there is no loading delay on tap.
2. Step tracker — a named progress bar in the teal header bar, visible throughout the episode. Steps are labelled with short action phrases (e.g. "Open Sarah → Read message → Send reply"). The current step is highlighted; completed steps show a checkmark. The tracker advances automatically as the participant progresses.
3. Transition screen — shown immediately on goal completion. Displays a success confirmation ("Umefanikiwa!") and waits for the server's next-scenario response. When the response arrives, the screen reveals the next task title, the episode counter ("Task 2 of 3"), and a one-line difficulty signal ("↑ A bit more challenging" / "Similar difficulty" / "↓ A little easier"). The participant taps Continue to proceed to the next scenario intro card.

The difficulty signal uses plain language only — no numeric scores or algorithm details are ever shown to the participant. "Easier" means the system detected lower readiness or poorer performance; "more challenging" means the participant demonstrated readiness for harder material.

Architecture: two separate layers

The sequencer and the pipeline are distinct layers that coexist without interference:

When a new episode starts, task_service.start_episode() writes the scenario's first_step as the new current_step_id in Redis. This is the only coupling point: it ensures governance and adaptation content resolution start at the correct step for each episode. The pipeline itself remains untouched.

Redis keys for episode tracking

ep_readiness_floor is written by events.py after every pipeline run — it's a lightweight Redis write (<1ms) that does not affect the synchronous pipeline path. All episode keys are cleared by task_service.complete_episode() after being flushed to the scenario_episodes DB table.

Researcher guidance — what to look for

• Ability score distribution. A bimodal distribution (many participants at D1–D2 and many at D4–D5, few in D3) may indicate the D3 scenarios are too difficult — a clear threshold effect. Check individual episode cards for D3 completion rates.
• Consistently declining trajectories. If ability drops episode-over-episode within sessions in the governed_adaptive condition, the governance system may be rescuing participants through D3 scenarios they haven't genuinely mastered. Cross-reference with adaptation failure data.
• Zero D4/D5 exposure. If no participant ever reaches D4 or D5 scenarios, either D3 completion rates are very low (tasks too hard) or the ability model is under-estimating performance (check K-factor and performance scores).
• High calibration delta variance. A wide spread of baseline_delta_used values across sessions indicates participants show significant day-to-day variation — this is expected and confirms the need for same-day calibration adjustment.
• Episode count vs session count mismatch. If many sessions show only 1 episode completed, look at whether participants are abandoning or whether session_done: true is being returned prematurely. Check the scenario catalogue — if the starting scenario is consistently too hard, participants may not be completing Episode 1 goals.