One post tagged with "workflows"

TL;DR: Loops are winning in prompting because you can trace outcome quality back to the prompt and improve it. We’ve deconstructed agentic SDLC work into <workflow> + <task description> + <behaviour guidance>, catalogued ~12 modular QA workflows (plus composites like run-qa and upkeep), and put the behaviour piece in version-controlled policies in your repo. Every workflow execution records policy file · version · git SHA—so outcome data can feed back into the policy. From ad-hoc gut-feel prompting → traceable, structured, modular workflows.

Loops win when you can close them​

Loops are all the rage in prompting today—and rightly so.

If you can trace the quality of an agent’s work back to the prompt, you get a feedback loop.

Better prompts → better outcomes.

We’ve been cooking something along those lines. But instead of treating every agent invocation as a raw blob of prompt text, we deconstructed agentic interactions a bit first.

Most agent work is three parts​

Look carefully at how agents are actually used in the SDLC. Most interactions are of the form:

<workflow> + <task description> + <behaviour guidance>

What’s interesting is that the first part has a surprisingly small vocabulary.

Author Story. Implement Task. Write Tests. Fix Issue. Fix Test Failure…

For QA, we found these distill into around 12 core workflows—plus a couple of composites for shorthand (run-qa, upkeep). The Workflows catalog is that vocabulary: modular playbooks the skill and the platform share.

From there, outcome quality mostly depends on two things:

Today, both still happen ad-hoc—with gut feel. Someone pastes a long prompt, tweaks a line that “felt” important last time, and hopes the next run is better. There is no durable artifact to improve. There is no evidence trail tying this outcome to that guidance.

We already hardened the task side​

We’ve been attacking the task-description lever for a while:

• DeFOSPAM requirement quality checks — score and fix ambiguous specs before agents spend tokens implementing or automating them
• Rich context for failures — fix-test-execution and related workflows pull execution detail instead of “tests are red, please fix”
• Test Planning as Code — stories and scenarios as markdown in Git, so the task itself is structured and agent-readable

That closes one half of the loop: better inputs into the agent.

The other half—how the agent should approach the work for your team—was still mostly vibes in a system prompt.

Introducing policy-traceable workflows​

Today we’re making that second lever first-class: policy-traceable workflows.

Each catalog workflow is backed by an optional policy—a Markdown file that defines how the agent should approach the task for your project. Policies are not secret sauce buried in a chat window. They live in your repository:

plans/knowledge/policies/*.policy.md

Version-controlled. Reviewable in PRs. Shared team-wide. Same files local /testchimp runs and Automations use.

---
workflow-id: implement
version: 1.2.0
---

Playbooks stay generic (the battle-tested skill steps). Policies hold the project choices: scoping rules, environment strategy, which composite subflows to run or skip, quality bars, exclusions, domain quirks.

Author or refresh them with /testchimp create policy <workflow-id>. Bump version whenever guidance changes—that version is what makes the loop measurable.

Full model: Workflows · Policies.

Trace every execution to policy · version · SHA​

A policy you can’t attribute is just another prompt.

For Plan → approve → Execute runs, the agent mints a stable workflow_execution_id, then reports mutative actions with:

• which policy file was used
• the policy version
• the git SHA it came from

…alongside workflow id, actor, branch, and entity identity. That lands on the Workflows execution timeline in TestChimp—not only in chat history that evaporates when the session ends.

The result is data you can use to improve policies. Data you can feed to an agent alongside the existing policy—to iterate on it.

Instead of tweaking prompts based on gut feel, you iterate on policies using evidence.

Ad-hoc prompting → modular, traceable workflows​

Put it together:

1. Small vocabulary of workflows — implement, create-tests, fix-issue, fix-test-execution, run-explorechimp, … plus composites
2. Context-rich tasks — governed requirements, execution detail, scoped branch diffs
3. Versioned behavioural guidance — *.policy.md in Git
4. Attribution on every run — policy file + version + git SHA on the execution timeline
5. Evidence → next policy version — close the loop

That is the same philosophy as skills as SaaS distribution (the playbook travels with the agent) and boiling the QA lake (agents in a continuous feedback loop)—applied to the behaviour contract itself.

From ad-hoc prompting → to traceable, structured, modular workflows.

Frequently asked questions​

What is a workflow policy?​

A policy is a project-owned Markdown file (plans/knowledge/policies/*.policy.md) that tells the agent how to run a catalog workflow for your team—scoping, env, quality bar, composite subflows, exclusions—without rewriting the skill playbook.

How is this different from a system prompt or CLAUDE.md?​

Those are useful, but usually opaque and hard to A/B. Policies are per-workflow, semver’d, synced to the platform, and recorded on every execution with file name, version, and git SHA—so you can compare runs and improve the guidance with evidence.

Do I need a policy for every workflow?​

Defaults are seeded for common composites (run-qa, upkeep) on /testchimp init. Some workflows (notably connect-to-test-env) need an explicit policy before dependent automation stays healthy. Atomic workflows can fall back to broader instructions—but named policies are what make optimization and auditability real.

How do I try an alternate policy without changing the default?​

Keep implement.policy.md as the team default and pass a variant:

/testchimp implement US-181 --policy implement-strict.policy.md

Same playbook, different behavioural contract. Compare outcomes on the execution timeline.

Where do I see policy traceability?​

Executions → Workflow Executions (and the Workflows UI timeline). After mutative actions, reported runs show the policy file and version used for that execution.

Try it​

1. Install the TestChimp skill and map plans/tests in Git
2. Run /testchimp init (or author policies with /testchimp create policy <id>)
3. Open plans/knowledge/policies/*.policy.md, encode one team convention, bump version
4. Run a workflow (/testchimp implement …, /testchimp run QA, …)
5. Check Workflow Executions for policy file · version · SHA—then improve the policy from what you see

Start here:

• Workflows overview
• Create policy
• Automations (same policies, event-driven)
• TestChimp skill

Better policies → better outcomes. Now you can prove which version got you there.

Further reading​

TestChimp

• Workflows
• Create policy
• Run QA
• Upkeep
• Implement Requirement
• Automations

Related posts

• Put QA on Auto-Pilot with Workflow Automations
• Requirement quality governance (DeFOSPAM)
• SKILLs as SaaS distribution
• Boiling the QA lake
• Test planning as code

Concepts

• Agent Skills
• TestChimp skills repository