Lessons learned
The Challenge measures itself — this page is the record. These are findings and lessons learned by running the Challenge with real groups of educators, and the iterative improvements each one produced in the foundation and the design. Nothing here is hidden as failure: a design that claims to be evidence-honest owes the public its own evolution.
Rehearsal with educators — 2026-08-18
4 teams × 5 educators, in person, ~3 hours — a full rehearsal of the format (sealed brief, Human-First Window, Divergence Gate, build with perturbation, no-AI defense) run with the demo kit, instrumented by its validation matrix: every question tied to an instrument and to the real-event decision it feeds.
Challenges were too loose
What we observed. The nine AI-resistance properties guarantee that a generic AI answer is insufficient — but they do not, by themselves, give a real team enough structure. Teams wanted more definition, detail and guidance.
The lesson. Structure is scaffolding, not ceremony. A brief can stay open in its solution space while being concrete about its deliverable.
What changed. Every rehearsal brief gained a "Deliverable, format and route" block: 2–4 format options, 3–5 concrete deliverable contents, a 4–6 step suggested route. The challenge generator now produces all three by rule.
No format commitment before building
What we observed. Teams reached the build phase knowing their concept but not what would exist at the end. Work drifted because the target had no shape.
The lesson. Before touching the AI, a team should know WHAT it is creating — not just the idea, the format. Divergence can even be three alternative formats of one concept.
What changed. New rule: each Divergence Gate approach names its concept AND candidate format, and passing the Gate includes declaring the chosen one. Round 1 (Anticipate) now includes format definition.
Learning objectives were missing
What we observed. A challenge could be designed, and its solution built, without touching any curriculum or learning objective — activity without learning impact.
The lesson. The canon already states that a brief IS a Learning Object (ruling R73). Without an explicit field, that principle silently drops out in practice.
What changed. Every brief now names its learning objectives; the authoring worksheet leads with them; and the quality bar asks: can this challenge be solved without touching the objective? If yes, redesign it.
The stages of knowledge work were indistinguishable
What we observed. Teams experienced the work as one undifferentiated mass of "using the AI". Stages blurred — and validation was the first thing to disappear.
The lesson. The rounds are the rhythm of the work; the knowledge stages are its anatomy. Both must be visible: research, source validation, data preparation (extract, analyze, validate, select, structure), insight generation, insight validation, presentation, and AI-output audit as a transversal activity.
What changed. A new stages section in the kit; the creation record gained a stage column so every consequential moment is tagged; the discernment test is explicit: a team can order its own work and name what was researching, extracting, validating, analyzing, manipulating, presenting, auditing.
Solutions did not reflect their makers
What we observed. Left to the tools, solutions drift toward the model’s average. Nothing asked teams to put themselves into the work.
The lesson. Personal relevance is not decoration — it is another divergence countermeasure, and it converges with the rubric’s fit-to-context dimension.
What changed. New rule: the solution carries its makers’ seal — each team incorporates perspectives of relevance and pertinence to them. A solution any other team could have produced identically is incomplete.
AI-output validation was treated as a formality
What we observed. The rubric already reads human-AI direction and iteration — but the briefs never said out loud that auditing AI output is core work.
The lesson. What is assessed must be announced. Reviewing, auditing and adjusting AI output — errors, inadequacies, drift from instructions and sources — is central process, not an afterthought.
What changed. Declared in the rules of play as core, assessed process; carried into the generator as a binding rule; proposed to the canon as brief-level language.
Instance vs. canon — an honesty note. The changes above are applied in this site's rehearsal kit and challenge generator. The Challenge's canonical design lives in the Smoother SSoT, which decides separately whether each finding becomes rule; proposals for all six have been handed to it. Where the canon rules differently, this instance follows the canon.
The aiLearning Challenge is an initiative of aiLearning.global, CEMI.ai and CEMI Labs — created by Carlos Miranda Levy, applying the Smoother learning methodology. This log is part of its commitment to measuring itself.