Converter feedback: carry the agent's got-vs-expected, and how often may it ask to send?
The questions: the opt-in --share flow sanitizes each conversion into a structural-only JSON and
POSTs it to a Google Form. (1) Will an external developer's AI agent send us something we can actually
use to improve the converter? (2) What's the right consent cadence — does it send cumulatively or per
bug; does one "yes" cover the rest of the site; and is it OK to keep asking after a "no"?
Context
The share report is auto-generated from the converter's own trace — the agent's only real decision is
whether to run --share, not what to put in it. The payload carries per-element role / detected / mapped-shortcode / fallback+opportunity flags / the converter's own "why" / tag / class tokens, section
decisions + styling-drop property names, aggregate stats, and a salted host hash. Anonymization is solid
(no URL, content, images, links, PII).
Two gaps surfaced in the audit:
- Got, but not expected. The build protocol tells the agent to flag misses as "element → got vs.
expected," but the payload had no
expectedfield and no free-text — so the agent's diagnosis (the single most actionable signal) was dropped, and a confident-but-wrong mapping (no fallback flag, so the row looks clean — exactly the class of bug we fixed all week) was invisible. - Consent cadence was unspecified. The protocol said only "get consent," leaving room for an over-eager agent to fire per bug, re-ask after a yes, or nag after a no.
Options considered
- Add many Google Form fields per finding. Rejected — the
payloadfield is an opaque JSON blob; more fields = field-id churn and a brittle schema. A single JSON blob + one human summary is the right shape. - Leave the payload as-is; rely on aggregate mining. Rejected — good for ranking recurring misses, but never recovers "what it should have been" for a specific/novel miss, and can't see silent-wrong maps.
- Add an agent-supplied
findings[]inside the existing payload JSON (no Form change), enrich the summary with a findings count, and codify the consent cadence in the protocol. Chosen.
Decision
findings[]rides inside the payload JSON ({ ref, got, expected, note, systematic }), written by the agent toshare-findings.jsonand merged by--share.noteis capped at 120 chars and auto-redacted of URLs/emails/quoted content, so it stays structural. Thesummaryfield gains anN agent-findingscount for at-a-glance Sheet triage. No new Google Form field is required — the data is a new key in the opaque blob; an optional human-readable "findings" field is a Tier-2 ergonomics nice-to-have, not needed for the mechanism.- Consent cadence: ASK ONCE → then auto-send each bug with a notification (explicit in
site-build-protocol.md). The agent asks exactly once, upfront; a yes authorizes streaming for the WHOLE site, and it then sends each finding immediately via a newsend-finding.mjs(a lean{ hostHash, version, one finding }≈ a few hundred bytes, throttled ≥1s apart) and notifies the user concisely each time (⚑ reported: code_block → special_heading). A stats-only--summarygoes once per site. Never ask per bug; a "no" is final (never re-prompt); consent is the site owner's so it does not carry to a different site. A batch alternative (share-findings.json+--share, one report at the end) remains for those who prefer it. - Framing: "improvement," not "bug." The user-facing ask + per-item notification are worded as "a place the Site Converter could translate better", not "a bug found" — it's an accurate frame (nothing broke; a translation gap was spotted) and a more inviting one (contributing to a better tool vs. reporting a failure), attributed to the converter (the tool), not the Dev Kit (the docs). Crucially the positive wording still states what is sent (anonymized, structural only, no content/URL) — a rosy frame that hid the data-send would be a soft consent dark pattern.
Why
The auto report is a good aggregate signal and a safe pipe, but it isn't a precise bug channel until the agent's got-vs-expected can travel — so we gave it a sanitized field inside the existing blob rather than restructuring the transport.
On cadence we iterated. A pure end-of-run batch keeps the Sheet clean, but (a) a very large page's full
report can approach the 50k Google-Sheets cell limit, and (b) a single silent send is less transparent —
the owner consents to an unseen blob. Per-bug sending fixes size but risks Sheet noise, burst
rate-limiting, and the ask-per-bug dark pattern. The chosen ask-once → stream + notify takes the best of
both: one informed yes (no nagging), lean per-bug payloads that can never hit the cell limit however many
bugs a site has, and a per-bug notification that makes the single consent genuinely informed (the user
watches what goes out and can stop it). Throttling neutralizes the rate-limit risk; a once-per-site stats
summary preserves the aggregate signal without repeating the report. Consent still belongs to the
site/owner, and a "no" is still final — re-asking would be a coercive dark pattern that erodes an opt-in
feature. (Superseded the initial "batch once per site" conclusion after weighing size + transparency.)
