How cys-terminal Compares
JavisRadio vs AgentRadio — where we lead, where we fall short#
cys's passive-awareness layer (Design Principle 6 · T5-20) is a re-implementation of the three primitives of AgentRadio (arXiv:2607.28430) by Coral Protocol, hardened with machine gates. The original research showed that letting four coding agents listen while they work lifts SWE-Atlas QnA task accuracy from 32.3% (single agent) to 62.1% (four agents, McNemar p=0.0023), with a DeepSeek replication (29.0%→50.8%, p=0.0026) and a B1 budget control (one agent with 6× budget still reaches only 37.9% — blocking the "you just spent more compute" objection by design): textbook experimental work. Below is the verdict of a full source-level survey of the original paper and repo (2026-08-14; triple-verified — two independent sessions + two adversarial reviewers + number re-execution), scored on 10 axes — wins and losses first.
At a glance — from cys/Jarvis's side: 8 ahead · 1 conditional · 1 behind#
| # | Axis | Verdict | One-line reason |
|---|---|---|---|
| 1 | Communication (passive awareness) | ⚠️ conditional lead | ports the 3 primitives + 14 defense commands · zero-loss surfacing (duplicates are audited exceptions) · retraction with contamination-cascade closure · idempotent queue — but the concept and the field data belong to AgentRadio |
| 2 | Role topology | ✅ ahead | heterogeneous three-vendor reviewers (claude · agy · codex) block correlated errors vs four same-model agents (a shared blind spot goes uncaught) |
| 3 | Verification & quality gates | ✅ ahead | completion claims without evidence are machine-rejected via exit codes + a four-party convergence gate vs a pipeline whose only machine gate is checking that answer.txt exists — unanimity is a prompt sentence ("count the APPROVEs yourself") |
| 4 | Recovery & durability | ✅ ahead | multi-layer recovery canon (SESSION_STATE · RECOVERY · persistent todos) + repairs born from real incidents (message-loss bug AA20 → single critical section; a 72%-quota burn → mission gate) vs single-layer process resume — server death = team state gone, token expiry = spin |
| 5 | Resource control | ✅ ahead | pre-start resource gate · process ledger · group cleanup vs none (relies on container disposal) |
| 6 | Human-in-the-loop | ✅ ahead | Approval Feed (exit 0/2/3) · kill-switch · denylist boundary vs "do NOT ask for human input" as the spec |
| 7 | Everyday generality | ✅ ahead | daily operation + 114 skills + departments + offline-local (zero network listeners) vs a single-domain benchmark reproduction requiring Docker + Modal cloud + pinned Harbor (17 days of repo activity) |
| 8 | Shipping maturity | ✅ ahead | notarization · dual-channel signed auto-update · 6 platform targets · release-gate CI vs no packaging · hardcoded version '0.1.0' · a checksum-less 106MB JAR from Google Drive |
| 9 | Measured performance proof | ❌ behind | AgentRadio proved its method on a public benchmark — 124 tasks × 4 configs × 2 model families with statistical testing — we have no system-level accuracy measurement (remediation started: JAVIS-BENCH, a pilot on the same task set) |
| 10 | Ecosystem | ✅ ahead | 86 deterministic tools + 114 skills + heterogeneous CLI adapters already running in-house vs 3 primitives (an MCP open-protocol agent-ecosystem ambition exists on their side) |
Fairness note: AgentRadio is a research artifact built to prove one hypothesis, so the absence of axes 5·6·8 is outside its design goal. Read the per-axis evidence, not the totals; the axis-9 loss is our named next task. On judging (a same-vendor AI judge): the same judge is fixed across all configurations, so the bias cancels out in the L2→L3 relative comparison — what it does threaten is the absolute numbers and the "leaderboard #1" narrative (the current single-agent leader at 63.17% exceeds 62.1%, though the ~±5 confidence intervals overlap so neither direction is statistically settled; AgentRadio is self-reported, not on the leaderboard). Cost, per the authors' own figures: $2.96 → $19.45 per task (6.6×).
The quantitative scale — size and depth, all re-measured#
| Metric | AgentRadio | cys/Jarvis stack |
|---|---|---|
| Code size | ~3,300 lines (Python 2,017 + shell 1,301) | ~169,000 lines (Rust 63,371 + pack Python 105,833 + more) = ~50 : 1 |
| Self tests | 0 (no tests or CI for its own harness code) | ~1,700 — Rust #[test] 883 (src) · 920 (whole repo) + pack 531 + radio 297 (incl. 23 red-team cases; re-run same-day, all PASS) + 16 UI test files |
| Communication surface | 3 primitives | 66 CLI subcommands (incl. 17 radio subcommands · a 10-code exit contract) |
| CI | none (1 visible commit) | 5 lanes + flaky-test gate + notarization regression check |
| Benchmark assets | 124 tasks · 1,306 rubrics · contamination canaries · statistical testing (their strongest suit) | none — JAVIS-BENCH started to close this |
The 50:1 ratio cuts both ways — evidence of our depth, and of our complexity; their smallness (fully auditable in an afternoon) is a scientific virtue, though one undercut by an unauditable 106MB server binary.
❌ Where we fall short — all of it, from the same survey#
| # | Gap | Fact |
|---|---|---|
| 1 | Zero public benchmark evidence | we have no outcome-level proof that our orchestration raises task scores — JAVIS-BENCH (single agent vs Jarvis-style orchestration on the same SWE-Atlas QnA tasks) has been started to close this |
| 2 | Intellectual priority is theirs | passive awareness and the 3 primitives are AgentRadio's; our own spec declares the port. Ours is a hardened port |
| 3 | Single machine, no node auth | radio is single-machine and unauthenticated (the name 'master' is always trusted) — their MCP server aims at cross-framework, multi-host reach |
| 4 | Other known gaps | no published cost figures (they publish theirs, down to the 6.6×) · Windows binaries not Authenticode-signed · radio's own docs admit full exactly-once and zero deaf-windows are not guaranteed (the hardening is partial) |
Detail — the radio layer 1:1 (the evidence behind the structural lead)#
| AgentRadio (original research) | JavisRadio (cys pack) | |
|---|---|---|
| Surface | 3 primitives (create_thread / send_message / wait_for_mention) | those 3 + 14 defense commands = 17 subcommands |
| Broadcast truth | none — content relayed as-is | FACT claims machine-verified against evidence (file · line · snippet); failures auto-demoted to hypothesis/unverified |
| Duplication / loss | no idempotency keys, acks, or sequence numbers (mention delivery itself is a server push; the timeout-fallback detection is grep string-counting) — dedup delegated to LLM cognition | monotonic seq + separate surfacing/acceptance cursors — invariant hierarchy "never zero > never twice" |
| Retracting a false broadcast | no concept | retract — closes the contamination cascade, including broadcasts that cited it |
| Completion gate | the only machine check is that answer.txt exists (2-hour polling) — unanimity is a prompt sentence | done-check rejects unsurfaced/unresolved broadcasts via exit codes (10-code contract) |
| Infrastructure | resident 106MB message server (auth key 'test') — server death = team state gone | no resident server — append-only files are the source of truth (rotation keeps seq continuity, archive after close) |
| Abuse defense | none | cooldowns · per-sender circuit breaker · secret masking · record cap · pause isolation |
| Verifiers | four same-model agents agreeing with each other | heterogeneous three-vendor reviewers combined with producer≠evaluator gates |
JavisRadio's quality evidence is 297 checks across 73 sealed cases — including 23 red-team regressions, adversarial tests locking "a violation must be stopped by the exact exit code". The axis-9 gap is being closed through the JAVIS-BENCH main experiment.
Jarvis stack vs Hermes Agent — head-to-head with a heavyweight platform#
We compared NousResearch Hermes Agent (230,268 stars · 1,377,316 lines of executable code —
measured 2026-08-14 / GitHub API) via a full-repo survey plus independent adversarial reviews. It is
the bigger system (~9× code, ~20× test cases). Nine-axis verdict: Jarvis ahead on 1 · Hermes ahead
on 4 · split on 3; a same-layer comparison of 14 capabilities: Jarvis ahead on 6 · Hermes ahead
on 5 · even on 3. The six where Jarvis is ahead are all about making results trustworthy (trust
& control); the five where Hermes is ahead are all about running anywhere, cheaply and
conveniently. In one line: Hermes broadened capability; Jarvis locked down procedure. Neither
side has public benchmark scores (our JAVIS-BENCH is underway). Full evidence: the lane report
JAVIS-vs-Hermes-Agent-종합성능비교보고서-2026-08-14.md (adversarial-review-hardened).
✅ Where Jarvis is ahead — trust & control (6)#
| # | Capability | In plain words (all measured) |
|---|---|---|
| 1 | A broadcast channel between agents | Jarvis has a radio (JavisRadio): news from teammates flows in while you keep working. Hermes core has no such broadcast (verified exhaustively — its closest thing is a board that agents poll) |
| 2 | Workers reporting on their own | Hermes child agents have their send capability stripped; the boss must open their logs. Jarvis workers push reports straight to the master |
| 3 | Machine-checked "done" | Jarvis rejects an evidence-free "done" by machine. Hermes has checking tools too, but the AI must choose to use them — and its always-on guard describes itself as an advisory that "never blocks completion" |
| 4 | A review bench from different vendors | Jarvis seats Claude, Gemini and Codex reviewers full-time, wired into pass/fail gates (so they don't share the same blind spots). Hermes has multi-vendor advice and delegate-to-other-AI skills — but not a verdict gate |
| 5 | Agents organizing their own team | The Jarvis master issues tickets, launches workers and checks convergence entirely through tools. Hermes's team topology (verifier/synthesizer) can only be set up by a human at the command line |
| 6 | Braking before work + signed releases | Jarvis blocks work before it starts if resources fail the check, and refuses to install a release whose signature fails. Hermes has no signing/notarization in CI, and its scripts quietly skip signing when credentials are absent |
❌ Where Hermes is ahead — all of it#
| # | Capability | Fact | Why the gap exists (report §7-1) |
|---|---|---|---|
| 1 | Size & breadth | 9.1× code · ~20× tests · 87 tools · 197 skills · 22 messengers · 7 execution backends | Half choice, half homework — Jarvis built messaging too, but narrowly: two approval/report-only bridges (see 'Channel bridge' below). We kept "data never leaves the machine" instead of broadening; the width itself is genuinely behind |
| 2 | Command safety checks | A 4,919-line dangerous-command engine (111 patterns · de-obfuscation · an AI judge) | Half choice, half homework — Jarvis puts its checks before work starts and after results come out; the thinner mid-run check is homework we intend to import |
| 3 | Cost tracking & worker policy | Auto-sums what child agents spend / risky child commands denied by default | Cost rollup is homework for Jarvis. The approval-policy difference is half choice, half homework |
| 4 | Conversations that follow you · open standards | The same conversation continues across terminal↔app↔messengers / connects to outside AIs via three standards (A2A · MCP · ACP) | For a conversation to follow you across devices, data must leave the machine — a direct conflict with our principle (choice). Standards are half choice, half homework |
| 5 | Age & adoption | 230,268 vs 27 stars · 398 contributors vs effectively one person · 388 vs 62 days | Neither philosophy nor homework — a function of calendar and headcount |
Where the differences come from — one paragraph#
The two systems aim at different things. Hermes aims at "a capable assistant that goes everywhere with you" and broadened its surfaces (their design doc: "capability lives at the edges"). Jarvis aims at "on my machine, with results I can trust" and locked its boundary (our design doc: "local-first — data never leaves the machine"). Taking the 20 axes/items where Hermes is ahead one by one: 8 are homework Jarvis must do (public benchmark, Windows signing, supply-chain checks, …), 11 are differences born of the different aim (wholly or partly), 1 is a function of time. We applied the same yardstick to Hermes's own weak spots. Item-by-item evidence (file:line) and the full table: report §7-1.
Jarvis stack vs OpenClaw — the layer the world's #1 wrote down as "will not build"#
OpenClaw (386,229 stars · 81,179 forks · 372 registered contributors — measured 2026-08-14 / GitHub API) is the most-starred software project in GitHub history. It is a personal AI assistant: message it on WhatsApp, Telegram or 24 other channels and it works on your machine. We cloned the whole repository (commit db4379bd) and compared it across 10 areas — survey, scoring and rebuttal were done by different agents, and the rebuttal pass returned 0 false claims · 10 corrections, all applied. Verdict: Jarvis ahead on 4 · even on 1 · OpenClaw ahead on 5, and they are the bigger system (~86× production code, ~14,000× stars).
In one line — the two lines they put on their "What We Will Not Merge" list (VISION.md:134-135),
"agent-hierarchy frameworks" and "heavy orchestration layers", are exactly what Jarvis is. Full
evidence: the lane report 자비스-vs-OpenClaw-전수조사-최종보고서-2026-08-14.md
(adversarial-review-hardened).
✅ Where Jarvis is ahead — command, verification, control (4)#
| # | Capability | In plain words (all measured) |
|---|---|---|
| 1 | Running many AIs as an organization | Jarvis checks by machine that four seats are alive — chief of staff, worker, and two reviewers from different vendors — and can clone a whole department. OpenClaw agents spawn children at depth 1 by default, with no master/worker/reviewer roles |
| 2 | Filtering results before trusting them | Jarvis machine-rejects an evidence-free "done", and reviewers use four closed outcomes (accept/revise/block/escalate) instead of scores, with a counter-argument required before passing. In OpenClaw, verifying a child's result is one sentence of guidance, and there is no verifier module |
| 3 | Security (narrow lead) | Jarvis opens no inbound door, persists risky-command approvals with unforgeable signatures, and has a kill switch plus a pre-flight resource check. OpenClaw's own security engineering is top tier, but in early 2026 it saw 135,000+ instances exposed without a password, a one-click remote-execution flaw (patched next day), 341→824 malicious skills, and a Chinese government usage restriction |
| 4 | Release & supply-chain integrity | Jarvis notarizes automatically, signs app and pack separately, and refuses to install if even one file is missing from the signed manifest. OpenClaw's dependency hygiene is exemplary, but its open skill marketplace actually shipped malware |
Even (1) — recovery: they are stronger at protecting the conversation store; Jarvis is stronger at restoring organizational state and at blocking a convincing but false restore (claims are reconciled against measurement right after recovery).
❌ Where OpenClaw is ahead — all of it#
| # | Capability | Fact | Why the gap exists |
|---|---|---|---|
| 1 | Messaging channels | 26 channels + duplicate suppression across the delivery path / Jarvis has 2 | Half choice, half time — the owner locked the scope ("exclude Telegram, Slack and Discord only", 2026-07-04) and two is what one person can maintain. The inconvenience is real: if you only use Telegram, you cannot approve from your phone. On duplicates, 23 of their 26 channels (88%) are in the same position as Jarvis |
| 2 | Tests & CI | ~120,000 test calls · 86 CI lanes / Jarvis has 1,664 · 5 | Mostly a size difference — our production code is 1/86th, so per thousand lines it is 41 vs 27 (1.6×). But having zero tooling to measure which code never runs is homework with no excuse |
| 3 | Feature breadth | 152 extensions · iOS/Android apps · UI in 21 languages / Jarvis has 114+ skills · macOS and Windows | Mostly choice — 83 of their 149 manifests (56%) are per-vendor adapters, so the two sides count different things. Having no marketplace is also a choice (our installer rejects any file absent from the signed manifest, which cannot coexist with an open one) — the cost is that you cannot install someone else's add-on. No Linux, however, is homework: the code is ready and only the release matrix has no slot |
| 4 | Users & community | 386k stars · hundreds of contributors · major press / Jarvis has 28 stars · 53 forks | Mostly direction and time — publishing externally is blocked without owner approval, so going public is the exception (the repo and releases are public; 20,624 asset downloads across the last 12 releases). But not having proven "is this better than a single AI?" is homework — we ran the same tasks and our pipeline lost 2–0 to a single agent (raw evidence was summarized away between stages) |
| 5 | Documentation | 770 documents · auto-translated into 20 languages / Jarvis has 254, Korean | Choice — the primary readers of those documents are programs, not people (directives are injected into each agent, and a failed injection stops it from starting). An English README exists. Having no documentation site is homework |
Where the differences come from — one paragraph#
The two systems aim in opposite directions. OpenClaw aims at "an assistant anyone can use anywhere" and broadened its surfaces — which is why its own vision document says it will not build agent hierarchies or heavy orchestration layers. Jarvis aims at "running many AIs so the results can be trusted" and made exactly that layer its core. Taking the 12 items where OpenClaw is ahead one by one: 4 are homework Jarvis must do (coverage measurement, performance evidence, Linux, a documentation site); the rest are differences born of the different aim, or of time and headcount. We applied the same yardstick to them — in OpenClaw, extensions run with the same privileges as the core, and damage from installing a bad extension is not even accepted as a security report under their policy. That is the price of breadth and openness. And our own messaging layer is itself what we learned by surveying OpenClaw (no code copied — rules re-implemented, with what we rejected written down too).
Back to Overview