Skip to content
HyoDo

Research

Status — research-only program, updated 2026-09-08. This page describes research in progress, not a shipped HyoDo capability or runtime. It is not a claim of peer review, submission, or acceptance by the Association for Computational Linguistics or any other venue.

Reader map. This page is the broader empirical program for support allocation research: benchmark status, related work, evaluation design, ablations, and publication boundary. For the focused strategy-prior + collaboration-topology hypothesis, including corpus governance, null baselines, and threats to validity, see the support-allocation field note. For the local privacy-minimized sensor contract, see Friction Contribution.

Working title: Evidence-Grounded Support Allocation for Human–AI Agents

HyoDo is the local-first evidence and gate substrate. The support-allocation research layer asks: what support profile should a human–AI workflow receive in this context?

Three questions, three responsibilities

Section titled “Three questions, three responsibilities”
Layer Question Responsibility
Authorization policy Whether? Is this action permitted at all?
Support allocation What support profile? What oversight, verification, explanation, exploration, or budget is appropriate?
Evidence Gate Done? Is completion actually proven by evidence?

This separation is a hard research invariant:

Population evidence may influence support allocation. It must never grant execution authority.

A task can be statistically routine and still require approval or be blocked because the local action is destructive, out of scope, or insufficiently evidenced.

Current support model

Section titled “Current support model”

The working support-allocation model combines four evidence families:

local prior
    +
population prior
    +
context similarity
    +
current evidence
        ↓
support-profile recommendation

Candidate conditioning signals include:

  • prior successful executions;
  • missing or incomplete evidence;
  • task risk and reversibility;
  • task novelty;
  • confidence and calibration signals;
  • cognitive-load proxies;
  • retries and rework;
  • verification failures;
  • human interventions;
  • approval wait;
  • resource conflicts; and
  • orchestration pattern.

The output is a support recommendation, not an authorization token. The research should not assume that support can be compressed into one scalar until data shows that doing so preserves the useful distinctions among oversight, verification, explanation, exploration, and resource budget.

Current claim lock

Section titled “Current claim lock”

The public product boundary is fixed below. The Measured Run #1 receipt records the observed host/HyoDo execution; it is not a claim of support-allocation effectiveness.

Capability Status Evidence boundary
gates / ledger / friction preview SHIPPED Local preview/export; ledger.
Graph v1 SHIPPED (site DEMO FIXTURE) Local dashboard; fixed demo site.
Graph v2 join SHIPPED Multi-parent runtime/viewer with v1 compatibility; public site remains fixture-only.
Codex host adapter SHIPPED / LIVE UNOBSERVED Native adapter shipped; a fresh canonical live canary is separate evidence.
Cursor host adapter SHIPPED / LIVE UNOBSERVED Native adapter shipped; fresh live host observation is not yet sealed.
IFA v0 SHIPPED Observer-only information-flow attestation; never execution authority.
remote MCP / ChatGPT CONTRACT ONLY Hosted contract; runtime unobserved.
Support-allocation runtime / strategy-prior hypothesis RESEARCH Research-only hypothesis; no shipped runtime or automatic router.
friction collector NOT BUILT No collector/uploader; transport disabled.

Instrument version boundary

Section titled “Instrument version boundary”

HyoDo’s local evidence, gate, policy, report, and inspection surfaces predate the friction instrument. The hyodo friction command is introduced in HyoDo 4.17.0; HyoDo 4.16.x and earlier do not expose that command.

That distinction matters for reproducibility: a paper, benchmark, or measured run must record the exact installed HyoDo version rather than treating the development branch and the latest published package as interchangeable.

Friction Contribution v1 defines a local-only, explicit-opt-in derived measurement surface before any population collector exists. Raw prompts, model responses, source code, diffs, credentials, file paths, email bodies, and raw event bodies are not population features by default.

Before any population signal is used for support allocation, it must pass validation, aggregation, versioning, deduplication, and bias / poisoning checks.

Research status

Section titled “Research status”

The research program has completed three internal stages:

  1. Research framing and terminology. The system was translated from internal operating language into human-centered agent research terms, with KRO as a typed runtime/process backbone.
  2. Runtime experiments. ACT / ASK / ABSTAIN scenarios and evidence-layer traces were exercised to establish a reproducible trace format.
  3. Trace-based evaluation baseline. A sealed benchmark baseline established a versioned dataset, scoring metrics, negative controls, and append-only benchmark evidence.

The current stage is a working-paper refresh plus measured-runtime expansion: update the research claim against the 2025–2026 literature, connect HyoDo friction evidence to support allocation, and produce a broader measured empirical results table before any venue submission claim.

Current empirical baseline — useful, but not submission-grade

Section titled “Current empirical baseline — useful, but not submission-grade”

The existing sealed baseline is intentionally small:

Provenance n Current result What it actually proves
Synthetic 14 ACT / ASK / ABSTAIN precision = 1.0 in the sealed fixture set The benchmark machinery is deterministic against its specification-derived fixtures; not real-world system quality.
Measured 3 All three are external-write cases labeled ASK The measured trace path works, but there is no measured ACT or ABSTAIN coverage yet.

The benchmark integrity layer also passed its sealed regression suites (9/9 vessel checks and 6/6 trace-experiment checks) and surfaced one definition-to-implementation design debt around an unreachable evidence-missing decision path.

That is enough to say the evaluation vessel exists and catches at least one real design gap. It is not enough to say the support-allocation policy or strategy-prior hypothesis has been empirically validated.

Before submission, the measured set must grow across task classes, risk levels, orchestration patterns, models/environments, and support profiles, with genuine ACT / ASK / ABSTAIN diversity and failure cases. Synthetic perfect scores must remain separated from measured evidence.

Updated research claim

Section titled “Updated research claim”

Recent work already studies when agents should defer to humans, how autonomy levels can be classified, how authorization can be enforced, how multi-agent topology interacts with task structure, and how metacognitive strategies can be evaluated. The claim therefore cannot simply be “agents should know when to ask” or “more agents are better.”

The working hypothesis is narrower:

Human–AI agent systems can allocate support more effectively when local experience, population experience, context similarity, current evidence, and task structure are combined, while execution authority and proof-of-completion remain independently governed.

The strategy-prior hypothesis is an additional hypothesis, not an assumed ingredient: a provenance-governed strategy prior must outperform simpler task-structure and semantic-routing baselines after accounting for cost, or it has not earned a place in the model.

Section titled “Related work we have to beat or complement”

KRO remains useful as the typed runtime/process representation that makes intent, plan, tool use, evidence, decision, and memory inspectable. It is a method backbone, not the sole novelty claim.

Evaluation plan

Section titled “Evaluation plan”

The next public-quality evaluation should report, at minimum:

Metric What it tests
Support-profile quality Did the model recommend an appropriate combination of oversight, verification, explanation, exploration, and budget?
Human intervention rate Did the system reduce unnecessary intervention?
Human cognitive burden / retention Did support allocation preserve the operator’s ability to notice, judge, and recover from agent errors rather than merely reduce clicks?
Rework / retry rate Did lower support increase recovery work?
Verification failure rate Did support allocation preserve evidence quality?
Evidence completeness Was the claimed outcome backed by traceable evidence?
Coordination overhead Did topology gains exceed communication / synchronization cost?
Wall-clock and token / compute cost Was any improvement worth the extra resources?
Authority violations Did population/local priors ever bypass policy? Target: zero.
Calibration by cohort Does the recommendation remain reliable across task, model, environment, and orchestration cohorts?

The friction taxonomy on the support-allocation field note — necessary, productive, avoidable — is an analysis target, not an automatically observed label in Friction Contribution v1. Any result using those classes must document who or what labeled them and how outcome leakage was controlled.

Required ablations

Section titled “Required ablations”

The support-evidence ablation should compare at least:

current evidence only
vs local prior + current evidence
vs population prior + current evidence
vs local + population + similarity + current evidence

The strategy-prior ablation should separately compare:

fixed orchestration / support
vs task-structure heuristic only
vs generic semantic retrieval
vs wisdom retrieval without counter-principle
vs wisdom + counter-principle + task structure

Population observations must be weighted by context similarity and uncertainty rather than treated as universal truth. A wisdom corpus must likewise be treated as provenance-governed hypotheses, not identity-derived user norms.

Non-negotiable invariants

Section titled “Non-negotiable invariants”
Population evidence → support recommendation  ✅
Population evidence → execution authority         ❌
Population evidence → override local policy       ❌
Population evidence → override Evidence Gate      ❌

Likewise, HyoDo’s Integrity Score remains a review signal, never approval.

Publication status

Section titled “Publication status”

Not submitted. Not peer reviewed. Not accepted.

The intended path is a human-centered NLP / LLM-agent research submission through the Association for Computational Linguistics Rolling Review once the refreshed empirical evidence is strong enough. Venue timing is deliberately not treated as product truth; the research should earn submission readiness from measurements first.

Reproducibility direction

Section titled “Reproducibility direction”

The public artifact should ultimately include:

  • versioned task and trace schemas;
  • exact HyoDo / model / environment versions for every result;
  • a documented support-profile labeling protocol;
  • a documented friction-class labeling protocol;
  • benchmark fixtures that do not contain private user data;
  • evaluation and ablation scripts;
  • latency / token / compute accounting;
  • human cognitive-burden measures where human oversight is part of the claim;
  • failure cases, null results, and contradictory evidence; and
  • corpus provenance plus counter-principle links for any strategy-prior experiment.

Until those artifacts exist and have been read back successfully, this page remains a working research program, not a finished paper.