How the System Grades Its Own Writing
How CMS-Agent reviews a draft before it moves on, how rubrics and model judges score agent output, how people take part, and why a score is evidence rather than permission.
3 min read By Platform Documentation Guide #Publishing #Testing
Quality
An agent wrote it. Is it any good?
A finished draft can look polished and still be wrong, vague or unconvincing. Kugel Platform answers that question at two levels. Inside each article's workflow, separate reviewers check the draft before it can move on. Outside any one article, an evaluation layer grades the agents' own output, so a change to how an agent works is tested before it is adopted.
Reviewers inside the workflow
In the CMS-Agent article workflow, four reviewers read the draft at the same time, each with one job: whether it reads like a person wrote it, whether every claim still matches the research, whether the emotional tone lands as intended, and how an unfamiliar reader would react on a first read. An aggregator merges their findings into one verdict with specific revision instructions. Only after that verdict is the article shaped into the site's structure, with its images planned and verified.
On the article itself, claims and their sources can be recorded. A claim marked high risk cannot publish while it is unverified, disputed or unsourced.
Rubrics and judges
Each workflow step can have a rubric: a written standard with weighted criteria. A separate model acts as the judge and scores the step's output against it, so the writer never marks its own work. Several judges can score the same output as a panel, people can record which of two outputs they prefer, and saved sets of past cases can be re-run as a regression test whenever something changes.
How grades change the agents
Scores feed improvement, but never directly. An optimizer can propose a new version of a step's instructions and test it against the current one on the same saved cases. Adopting the better version is a human decision, recorded as a versioned change that can be rolled back.
Lessons can also be written into a step's playbook, a short set of notes the agent reads on every run. Automatic reflection after each run, and automatic promotion of changes for low-risk steps, both exist but are switched off by default.
What a score does not mean
A score is evidence, not permission. Passing a rubric publishes nothing; the publish rules still apply. Rubrics are versioned and keep changing as they are calibrated, so a score is only comparable with scores from the same rubric version.
Reader analytics and sales results can be fed in as feedback, but today they do not change agent behaviour automatically. A person still decides what the numbers mean.