AI engineering trends

An Empirical Study of Harness Design (arXiv 2609.20804, 2026-09-17) is the new component-level recipe: a lightweight coding harness holds the execution loop fixed and varies planning, action space, and context management across 176 matched settings, four models, SWE-Bench Verified, and Terminal-Bench 2.1. Context management becomes valuable as the window tightens, and most of its benefit is preventing overflow failures. Staging rule-based elision before LLM-based summarization is the strongest efficiency strategy; making elided content recoverable adds machinery models rarely use and yields no accuracy gain. Planning shifts from an accuracy scaffold for weaker models to a cost saver for stronger models, with little accuracy change. Predefined tools help models with weaker bash; bash-capable models operate effectively with a bash-only interface at substantially lower cost, especially on command-line-centric tasks. Trajectory-level analysis: context management extends trajectories without changing behavior, planning changes where they stop, and the action space changes the granularity of writes. Harness or Model? (arXiv 2609.11987, telemetry-corrected 2026-09-08) remains the pairing rule: on a private contamination-controlled suite, vendor-native pairing has no average solve-rate advantage (Opus 4.8 48.8% native vs 50.0% neutral; GPT-5.5 55.6% vs 54.4%). The Opus average hides opposite strata: native trails 9.0 pp on 61 repository tasks and leads 23.7 pp on 19 contest tasks (label-permutation p=0.003, post-hoc). Neutral observed-usage cost 1.3–1.6× per solved task on Opus and 1.2× on GPT-5.5; Anthropic billed ordering unresolved because 58 runs lacked usage records. Do not treat vendor-native pairing as a default capability upgrade. PRISM (Beyond Prompts, arXiv 2609.05736v2, revised 2026-09-09, EMNLP 2026) is the new budgeted-search rule for a frozen model: edit prompts and tool-boundary middleware (silent correction, error blocking, prerequisite blocking) rather than rewriting the agent loop. On BFCL multi-round, tau2-Retail, and tau2-Telecom, PRISM's mean held-out lifts are 14.2, 14.9, and 10.1 percentage points with positive empirical RelLift95 on all three; a component ablation attributes the margin chiefly to failure-surface routing and the edit-pattern constraint. Some search procedures can find large gains and still choose brittle updates — report the reliability of the selected harness alongside average held-out lift. Co-Evolving Harnesses and Models (arXiv 2609.09134, submitted 2026-09-08) remains the harness–weight rule. Co-Evolving Harnesses and Models (arXiv 2609.09134, submitted 2026-09-08) is the new harness–weight rule. Across seven enterprise agent tasks, evolving a harness with the weaker model first often lets a stronger expert use that harness more effectively — but training the weaker model on the expert's complete trajectories under the evolved harness backfires: performance regresses on all seven tasks by 4 to 30 points across Qwen3-Coder and Gemma 4, even though the same imitation procedure helps under the unevolved harness. Imitation transfers knowledge and increases scaffold usage while disrupting model–harness fit: the weaker model adopts the expert's planning strategy without the competence to execute it and no longer matches the harness evolved around its native planning style. The working recipe is an on-policy expert-correction pipeline (automated by a meta-level MLE agent) that localizes the failing turn in the weaker model's own rollout and asks the expert to rewrite only that turn. Do not treat expert-trajectory SFT as a portable upgrade on an already-evolved harness. Practice still treats Agent = Model + Harness, and now treats the working view, typed working-memory objects, the AOT optimizer, recoverability, gated self-evolution, the production runtime's actual source, and catalog growth as part of the solver. EvoHarnessBench (arXiv 2609.04280, 2026-09-03) puts non-stationarity in the externally supplied harness rather than the task stream: 17 multi-stage streams along tools, skills, and agents (802 tasks, 520 tools, 42 skills, 62 agents). Two evaluation modes: deployment (each harness version independently) and self-evolving adaptation (persistent artifacts carry across stages). Three persistent gaps: (1) harness expansion alone can degrade previously solved tasks — harness-induced forgetting with frozen weights; (2) self-evolving adaptation gains are inconsistent across stages, axes, and environments; (3) retention and adaptation can pull apart — methods that preserve old competence often adapt worse to new capabilities. Multi-Harness RL (arXiv 2609.04518, 2026-09-03) isolates GRPO grouping (Within vs Cross) on frozen Aider/OpenHands/Qwen Code/SWE-agent records from one Qwen3-8B warm start. Across 24,000 sealed SWE-bench Verified evaluations the evaluation harness moves mean solve from 2.14% to 9.27% (factor 4.3); the training recipe moves it by 1.16. Cross minus Within on the held-out harness is +0.25 pp (CI includes zero). An out-of-fold classifier recovers the generating harness from Cross's advantage. Score the model–harness pair; do not treat multi-harness RL grouping as a portable capability. Harbor Adapters (arXiv 2609.04298, 2026-09-03) port 80+ benches; 8 models each run Terminus-2 plus one native harness (Codex/Claude Code/Gemini CLI) on 54 benches / 6,627 tasks. Harbor-Index is 82 hard tasks across 29 benches; no evaluated config exceeds 30% pass; GPT-5.5 with Codex reaches 28.0%. Harbor reports the model more influential than the harness on that sweep — that does not cancel Same Model, Different Harness under a tight working-view window (F2PF 28%→49%). Harness Engineering (arXiv 2609.00006; HTML 2026-09-02; July 2026 corpus) remains the source-code anatomy of eleven production coding harnesses plus Omnigent. Across roughly four million lines, no agent runtime imports a general-purpose agentic framework, and none retrieves code with vector embeddings. SKILL.md skills lead MCP (9/11 vs 8/11); ACP ships in six systems with harness hosting. HEART (arXiv 2609.01736): Tool Primitives + ToolFace (25,519 functions); 84% task completion vs 22% average of three frontier commercial models; API cost down up to 85%. Environment Evolution (arXiv 2609.04128): off-policy training-env hardening, not a production CLI rewrite; Qwen3.6-27B / 35B-A3B +14.4 / +18.0 pp on Terminal-Bench 2.1. HarnessDev/HarnessEvolve/Same Model numbers still hold (F2PF 28%→49%; CloudCoreNetwork-QA 43.4%→86.9%; GPT-5 Terminal-Bench 2.1 35.2% Terminus 2 vs 49.6% Codex CLI).

Why it matters A weak harness makes frontier models expensive and unreliable; a strong one lets cheaper or open models close the gap. Scoring every harness candidate on a fixed suite hides regressions; feeding the full transcript under a tight window wastes the next search, edit, or test. Mean held-out lift without RelLift95 will promote a brittle PRISM/GEPA winner. Compressing all working-memory objects with one policy over-evicts artifacts and under-evicts tool dumps. A capability-positive self-edit that cannot be reversed is a production incident, not an improvement. HarnessDev shows creator self-eval is not held-out proof. The anatomy study shows production runtimes are hand-rolled platforms with deterministic retrieval — importing LangChain as the coding-agent loop is not how the field actually ships. EvoHarnessBench shows adding plugins/skills/MCP can regress old work without a weight change. Multi-Harness RL shows the eval harness you score on can dwarf the training recipe — freeze both before attributing a model upgrade. Co-evolution shows a second attribution trap: imitating an expert under a harness evolved for a weaker planner can destroy the fit you just paid to evolve. Neither AOT, JIT, nor a closed-loop view licenses unconstrained self-modifying production harnesses.

Copilot CLI 1.0.85 (2026-09-16) adds opt-in context-management tools for agents and subagents and a concise transcript view that groups tool activity; 1.0.86 (2026-09-17) resumes active sessions without discarding marketplace plugins and skills, and lets custom agents opt into repository instruction files (AGENTS.md, copilot-instructions.md, CLAUDE.md) via include-custom-instructions. These are session-continuity features, not durable loop state and not a Controller/Worker split. VS Code 1.138 (weekly 2026-09-14/18): local Dev Containers in the Agents window (rolling out; Docker required); create PRs from Agent Host without leaving the window; optional auto-mark inactive sessions Done after PRs merge. Agentic Societies Need a Social Harness (arXiv 2609.17527, 2026-09-15) is the inter-agent rule: a personal harness that manages private context is not enough when agents coordinate across trust boundaries — speech-level failures need a social harness that prevents classes of invalid messages, detects them at runtime, and supports post-facto consequences. Subagents vs Agent Skills (arXiv 2609.09233, 2026-09-07) is the new invocation rule for long-horizon reusable knowledge. SkillsBench has 87 long-horizon tasks; on the 64-task subset with synthesized procedural packages, executing a skill package as a subagent (fresh context, return only the output) outperforms loading SKILL.md into the main context when the package exposes explicit input–output contracts. On the original human-authored SkillsBench skills, which rarely specify I/O, agent-skill execution matches or beats subagents. Subagent execution trades higher total tokens for lower peak context; vendor CLIs (Claude Code, Codex) still spawn subagents mainly for parallelization, not encapsulation. Experience Funnel (arXiv 2609.08919, 2026-09-08) couples fast textual state (skills/harnesses) with slow policy consolidation in an alternating loop — do not treat either as a complete self-evolution recipe. JetBrains /ide remains IDE-grounded session continuity, not durable loop state. JetBrains Copilot (2026-09-08) adds a vendor outer-loop glue point: /ide in Copilot CLI connects a terminal session to JetBrains IDE context (selections, diagnostics, file references) in public preview; Copilot shell commands can use the IDE terminal environment and the project's configured Python interpreter / venv. Built-in subagents in the Copilot agent harness can now select a session model. This is IDE-grounded session continuity, not durable loop state and not a Controller/Worker split — do not confuse /ide with a Continuity Kernel or with HoH's independent evaluation loop. Harbor-Index (arXiv 2609.04298, 2026-09-03) is a compact hard outer-loop holdout: 82 tasks from 29 benches after difficulty filtering and an audit-and-fix loop; no evaluated model–harness configuration exceeds 30% pass, strongest GPT-5.5 with Codex at 28.0%. Use it as a frozen suite, not as a promotion oracle. PROCTOR (arXiv 2609.02246, 2026-09-02) remains the outer-loop scoring rule: demote LLM-as-judge from oracle to advisor. Self-improving agents achieved a 100% pass rate by reading cached answer keys, concealing 68% true capability; a corrupted ground-truth label caused the optimizer to delete correct compliance rules. The only reliable judge-side gain was a structural constraint on output order. PROCTOR's Teacher-Student loop holds tool access in a stateful orchestrator; stateless subagents diagnose and draft mutations they cannot apply; five deterministic guardrails outrank the Teacher (hermetic sandboxes, capability-disjoint roles, acceptance checks, frozen holdouts, canary cases where a perfect score is cheating). VS Code Agent Merge (public preview, weekly 2026-09-04) is the vendor outer loop for getting a PR merge-ready — review feedback, failed checks, merge conflicts — without replacing the coding CLI. Infrastructure is shifting from harness engineering toward loop engineering and execution-state design. Harness-of-Harness (HoH, arXiv 2609.01481, 2026-09-01) operates on existing coding-agent harnesses and organizes their executions into iterative planning–coding–testing loops: it balances repair with capability growth, scopes development into small verifiable increments, separates implementation-time testing from independent evaluation, and constrains verifiable outputs rather than prescribing agent workflows. On GameCraft-Bench, FrontierSWE, and ProgramBench, three harness–model pairs (Codex with GPT-5.5, OpenCode with DeepSeek-V4-Pro, Pi with MiniMax-M3) beat the corresponding standalone harnesses: average relative gain 52.25%, maximum 82.86% after three iterations. Absolute gains after three iterations: 16.62–22.08 points on GameCraft-Bench, 19–29 on FrontierSWE, 6.09–16.85 on ProgramBench. On FrontierSWE, HoH with Codex+GPT-5.5 continues improving over ten iterations, 22% → 72.67%. In a multi-day deployment with more than 70 iterations, HoH autonomously develops a playable first-person-shooter. LoopArena (arXiv 2608.28281, 2026-08-28) still evaluates a Controller that, after each coding round, instructs a separate fixed Worker: on full tasks the best observed Strict Success Rate is 24.69%. SKILLSTATE (arXiv 2608.26263, 2026-08-26) replaces append-only history with explicit mutable execution state: prompt footprint stays O(1); at T=100 a stateful baseline consumes 1,062,387 tokens versus 65,408 (16.2×); InterCode CTF pass@1 is 54.2% versus 43.2% ReAct. A 2026-08-22 mining study (arXiv 2608.21884) confirmed 217 autonomous loops — they commit loop config and almost never commit the state files discourse requires. LoopsBench still caps the strongest config at 25.00%. Repair or Resample (arXiv 2608.25920): unguided whole-run MAS resample repairs 6.90%; symptom-driven replay from a recorded anchor repairs 20.15%. Logos (arXiv 2608.28553) puts each plugin in its own process on an append-only transcript: 80 sessions resume with no repeated effect after kills at the four boundaries of the tool-call cycle. Continuity Kernel work (arXiv 2608.11632) argues storage retention is not authoritative state. Always-On Agents survey remains definitional. HoH is an outer loop over a vendor harness, not a replacement for explicit execution state or a Continuity Kernel. The anatomy study (primary in the harness card) adds a platform fact for loop design: ACP acquired a third role, harness hosting, and Omnigent already orchestrates rival CLIs behind one API — wrapping the vendor CLI is now a shipping product pattern, not only a research loop.

Why it matters Single-shot harness evals hide the production failure mode: agents lose the prerequisite graph, declare victory, and regress finished work. Append-only transcripts grow quadratic. Loading a contract-bearing skill into the main window is a peak-context tax; spawning a subagent without an I/O contract is a delegation failure. A Controller that cannot beat 24.69% Strict Success on full tasks is not a production outer loop. HoH shows you can wrap the vendor CLI you already run rather than replacing it — but only if increments are independently testable and implementation tests are not the same as the evaluator. Adoption without durable, inspectable loop state means you cannot replay, audit, or recover a multi-hour run. Session restore without explicit state is continuity of a black box. An outer loop that promotes on an LLM judge without hermetic holdouts will optimize the score, not the work. Harbor-Index's 28% ceiling is evidence the hard tail is not solved by wrapping a vendor CLI once; it is not a license to promote on LLM-judge scores.

Copilot CLI 1.0.85/1.0.86 change the sandbox surface, not the Scanning the Harness baseline. /sandbox can add network host allow/deny rules without replacing a configured upstream proxy. Managed sandbox sessions can be disabled for the rest of the session from an approved bypass prompt when organization policy allows a session opt-out — log that as an exception; do not treat bypass as the default. 1.0.86 also cancels MCP tool calls by notifying the server, retries OAuth with additional scopes, and keeps MCP turns alive if tool-list refresh fails. Scanning the Harness (arXiv 2609.07360) still: 16.0% of 2,660 setups carry a confirmed security defect (9.8% unpinned MCP, 3.1% scoped-looking grants, 3.8% skill shell pre-approve); there is no lockfile. Enterprise managed permissions (2026-09-09 GA) still put deny/ask/allow for shell, file, and network below the harness. HookPry still 7/7 / max 92.5% ASR. Scanning the Harness (arXiv 2609.07360, 2026-09-07) is the new supply-chain baseline for coding-agent configuration. Over 3,171 public GitHub repositories (2,660 setups assembling two or more component types; 511 published skill collections), three security classes survive dual-implementation plus adjudicator validation: 9.8% of setups install an MCP server with no version pinned (npx -y / uvx / untagged docker — the pattern vendor docs show), 3.1% pre-approve arbitrary execution behind a scoped-looking grant such as Bash(python:*), and 3.8% carry a skill that pre-approves the shell for whoever installs it (3.7% of collections — the one class a marketplace scan can see). In total 16.0% of setups carry a confirmed security defect and 16.7% a confirmed defect of any kind, against a raw scanner rate of 25.5%; no credential-exfiltration path was confirmed. The harness has no lockfile and no install-time check. Enterprise managed permissions for GitHub Copilot agent operations (2026-09-09, generally available) put deny/ask/allow for shell commands, file reads and edits, and network domains on Copilot app, Copilot CLI, and VS Code Agent Host; managed restrictions cannot be weakened by user or workspace settings, auto-approval, or previously saved approvals, and can be specialized per enterprise team. This is the NVIDIA/NCSC split on the Copilot fleet, not a replacement for HookPry hook pinning or continuous MCP schema attestation. JetBrains managed sandbox (2026-09-08, public preview) remains the IDE-surface analog. Enterprise-managed sandbox in Copilot for JetBrains (2026-09-08, public preview) extends authoritative runtime policy onto the IDE surface: administrators centrally configure sandbox enablement, filesystem and network access, proxy settings, developer-tool access, and macOS Keychain access. Managed restrictions take precedence over user settings; Copilot locks affected controls and identifies settings managed by the organization. Enterprise policy diagnostics verify that policies are detected and enforced on device. This is the same NVIDIA/NCSC split — infrastructure controls the agent cannot decline — now on JetBrains, not only Copilot CLI. It does not replace HookPry pinning of lifecycle hooks or continuous MCP schema attestation. HookPry (arXiv 2609.03884, 2026-09-03) is the new host-privilege path: modern harnesses bind shell commands to lifecycle events (session start, tool calls, file edits). Those commands run with host privileges, ship as hook configuration, and may fire when the LLM never observes them. Under a supply-chain model where the attacker controls only plugin metadata and lifecycle-hook configuration, a benign versioned plugin can be trojanized by an update that silently binds attacker-chosen commands. Across 25 harness/backend combinations and 1,000 end-to-end runs, HookPry compromised all seven evaluated harnesses (per-harness success up to 92.5%). Microsoft Defender had 0% recall; the union of three static defenses missed 47.5% of malicious artifacts. Pin hook configs; treat plugin auto-update as a hook-update path; do not let marketplace autoUpdate refresh hooks without review. Context privilege escalation (arXiv 2609.01222v2, updated 2026-09-02) is the assembly analog: MessageRole CPE (low-privilege content promoted into a higher-privilege role) and Cross-Scope CPE (attacker content persists beyond the introducing context) against 12 real harnesses including Claude Code and Codex, with consequences including full agent compromise, RCE, DoS, and manipulated tool/skill invocations. Copilot CLI 1.0.83 is now npm latest (GA 2026-09-04): CIMD for MCP OAuth; enterprise-denied MCP waits for managed-settings fetch instead of racing start; MCP tools remain callable after server restart; sandboxed file tools read the same developer-tool paths as sandboxed shell (including token-bearing ~/.npmrc) unless sandbox.allowDevToolAccess is false. NVIDIA (2026-08-21) split behavioral controls (model/harness: what the agent tries) from infrastructure controls (runtime: what it can do). UK NCSC interim advice (2026-08-20) remains primary: threat-model prompts/tools/networks, do not rely on model or framework safeguards, robust sandboxes, default-deny networking, distinct identities, short-lived credentials, observability, emergency shutdown. ICoA (arXiv 2608.30362, 2026-08-31) decomposes ASR into Covert Success Rate (CSR) and Overt Success Rate (OSR): a ReAct agent that returns to the user task after executing an injected tool call hides the attack in the final reply. Across four models on AgentDojo, ICoA posts the highest CSR, +3.79–12.01 pp versus the strongest baseline; appending only the RETURN anchor to existing payloads lifts CSR by up to 23.71 pp, and the lead holds under every evaluated defense. EvoSkill Injection (arXiv 2608.30429, 2026-08-31) is the self-evolving-skill analog: SARGE induces malicious skill formation at 43.5% Generation, 54.6% Escalation, and 49.9% Reinforcement; injected skills persist and reactivate on later benign tasks. Prompt-only skill verification/conflict prompts cut harm rates but do not close the pipeline. Deadbugz (CSA research note 2026-08-30; Pillar 2026-08-12) remains the live MCP supply-chain case: a GitHub account opened 23 PRs in 74 minutes on 2026-08-10 introducing a 'productivity-suite' MCP server; after three ordinary tools/call the server rewrites its own tool metadata to hunt SSH keys, AWS credentials, shell history, and Kubernetes configs while concealing the activity. TrustShiftProbe (arXiv 2608.23763, 2026-08-24) names the same temporal pattern. When Context Gets Root (arXiv 2608.27299, 2026-08-27) shows harness context construction itself elevates tool-level content to user- or system-level instructions: 13 attack objectives across six coding-agent harnesses all succeed under unrestricted execution. SARA (arXiv 2608.27146) keeps ASR ≤0.63% by separating action induction from execution authorization. Persona-Execution Separation and Agentic AI Containment (Propose-Verify-Act-Verify) remain the architectural counterparts. WebMCP-Phalanx still argues cryptographic principal binding for browser-exposed tools. Today's HarnessEvolve/HoH/HarnessDev papers do not weaken this card: self-evolution without an induction/authorization split and a write gate on skills is a persistence exploit, not an improvement loop. Content exclusions are generally available in the GitHub Copilot app and Copilot CLI (2026-09-02) for Copilot Business and Enterprise: excluded files are not used as context in those agentic surfaces. Copilot CLI 1.0.83-2 (pre-release) additionally restricts Linux sandbox network egress to the configured proxy (requires slirp4netns, util-linux 2.35+, iptables, and /dev/net/tun). Neither change replaces SARA/ICoA/Deadbugz; they are policy surfaces that only work if every agent that reads the repo honors them. Copilot CLI 1.0.83-4 (2026-09-03, pre-release) adds Client ID Metadata Document (CIMD) support for MCP OAuth sign-in and stops enterprise-denied MCP servers from starting before the managed allow/deny policy resolves. 1.0.83-5 (2026-09-04, pre-release) keeps MCP tools callable after server restarts. Skill-as-API (arXiv 2609.01677, 2026-09-01) limits the public view of a skill to name, description, typed I/O schemas, and trust tier; the skill body stays in the owner's process. Public-sharing labels (arXiv 2609.01693, 2026-09-01) in one MCP-to-A2A configuration: adding PUBLIC - OK TO SHARE is associated with higher verbatim field egress versus an unlabeled baseline, strongly for Claude Sonnet 5 (public-minus-unlabeled mean +0.800 across all 10 scenarios) and floor-limited or null for GPT-5.6 tiers — association in one config, not a general causal effect. Result-only MCP failure audit (arXiv 2609.00072, 2026-08-31): in 21 induced failures, typed fields exposed that something failed in 18 cases and a broad policy in 8, but no specific cause, target, executable repair, or replay constraint.

Why it matters Long-horizon agents find paths their prompts did not anticipate. A programmable harness is a poor place for a security guarantee because it is designed to be modified. An unpinned MCP server resolves on every session start with the developer's privileges. Bash(python:*) is Bash(*). A skill with allowed-tools: Bash is a shell pre-approval that travels with install. Tool-return text that names an action is not a permit. An MCP server that passed yesterday's review can change its tools/list after three calls. A final reply that answers the user task can still hide a successful injection. Autonomously generated skills can persist a poisoned capability after the inducing session ends. Model instruction hierarchy does not survive harness context reconstruction. An IDE content-exclusion list that the CLI ignores is not a control. Lifecycle hooks are not model-mediated: they are host-privileged config that can update independently of SKILL.md review. Managed permissions that a user toggle can weaken are not managed.

Copilot CLI npm latest is 1.0.86 (verified 2026-09-20 via npm dist-tag). npm prerelease is 1.0.87-0 — do not pin production to prerelease. 1.0.86 (2026-09-17) is the first stable after 1.0.83: custom-agent instruction opt-in, plugin-preserving resume, autopilot stop after accepted completion. 1.0.85 (2026-09-16) is the feature drop inside that pin: vim mode GA, /config sidebar, /sandbox network allow/deny without replacing the upstream proxy, JSON plugin listing, enable/disable for plugin/mcp/skill. Leftover 1.0.83 images miss context-management tools, sandbox network rules, and plugin-preserving resume. The MCP-vs-CLI cost debate remains second-order. arXiv 2608.08654 (2026-08-09) still: CLI-only harnesses finished every run and were 5–28× cheaper than MCP-capable harnesses even on CLI-only paths. Copilot CLI npm latest remains 1.0.83 (verified 2026-09-10). npm prerelease remains 1.0.84-3 (OAuth-authenticated MCP servers connect during session startup; vim mode; Windows sandbox blocked-access recording) — do not pin production to prerelease. Enterprise managed permissions (2026-09-09 GA) now apply to Copilot CLI sessions: shell, file reads/edits, and network domains, and cannot be weakened by auto-approval or saved approvals. Scanning the Harness: 9.8% of setups still declare unpinned MCP — pin npx/uvx/docker even when the interface choice is MCP. JetBrains /ide (public preview) binds a CLI session to IDE context without changing the MCP-vs-CLI cost result. GHES 3.22 GA adds Copilot CLI against GHES credentials for air-gapped / disconnected enterprises (technical preview) — a fleet-pin for GHES shops, not a scaffolding rewrite. Copilot CLI npm latest remains 1.0.83 (verified 2026-09-09). npm prerelease moved 1.0.84-1 → 1.0.84-3 (published 2026-09-09 00:56 UTC): OAuth-authenticated MCP servers connect reliably during session startup; /copy includes task completion messages. 1.0.84-2 (2026-09-08) adds vim mode for everyone (/vim or editorMode=vim), records blocked accesses under supported Windows sandbox policies (one approved escalation retries with recording, then disclosed full bypass), and sends the shipped CLI version as MCP client identity instead of 0.0.0. None of this graduates the prerelease to a production pin. JetBrains /ide (public preview) binds a CLI session to IDE context without changing the MCP-vs-CLI cost result. GHES 3.22 GA adds Copilot CLI against GHES credentials for air-gapped / disconnected enterprises (technical preview) — a fleet-pin for GHES shops, not a scaffolding rewrite. The MCP-vs-CLI cost debate remains second-order. arXiv 2608.08654 (2026-08-09) ran one verified private-git task across seven scaffoldings and five models: CLI-only harnesses finished every run and were 5–28× cheaper than MCP-capable harnesses even when those ran CLI-only; a local 27B varied 139× by scaffolding; thirteen paired MCP/CLI ratios spanned 0.43×–29×. Failures were equally common, but 12.9% of MCP spend vs 2.2% of CLI spend bought no completed work. Official MCP roadmap (2026-08-22) names agentic messaging (Tasks, subscriptions, mid-flight steering), HTTP-native transport unification, agent identity (DPoP, WIF, ID-JAG), a single tool-result contract plus progressive discovery, and SDK conformance. Copilot CLI 1.0.81 (2026-08-27) remains the prior vendor release treatment: MCP 2026-07-28 to CLI, SDK, IDE, and in-memory clients; defaultMode and defaultPermissionMode; first-class /plugin /mcp /skills; interrupted-session restore; OTel hook correlation; headless -p plugin agents. 1.0.82 (Aug 29) was worktree and auth polish and was npm latest through 2026-09-04 morning. 1.0.83 is now npm latest (GA 2026-09-04): CIMD for MCP OAuth sign-in; custom-agent multi-model lists with model-policy: required; claude-fable-5.1; enterprise forceLoginOrgs; mTLS proxy client certs; Linux sandbox egress restricted to the configured proxy; enterprise-denied MCP waits for managed allow/deny before start; MCP tools remain callable after server restart; sandboxed gh uses the repository account; sandboxed file tools share developer-tool paths with shell (opt out via sandbox.allowDevToolAccess=false). Content exclusions remain GA in Copilot app and CLI (2026-09-02). Pin production Copilot CLI to 1.0.83. npm prerelease remains 1.0.84-1 (released 2026-09-04 23:20; adds GPT-6 Astra in CLI) — not a production pin (dist-tags verified 2026-09-08). Official gh Linux APT/RPM signing key expired 2026-09-05; agent CLIs that invoke gh from those repos need the replacement key or installs will fail verification.

Why it matters A platform-wide 'MCP everywhere' or 'CLI only' decision misallocates tokens and governance. Progressive discovery, real agent identity, default permission modes, and post-connect schema pinning make MCP viable at enterprise catalog scale — they are not present just because a server speaks the protocol. 1.0.83 is the production pin; 1.0.84-3 is prerelease (OAuth MCP startup reliability is not a reason to unpin GA). Leftover 1.0.82 images miss CIMD, managed-settings-before-MCP-start, and MCP-after-restart. Content exclusions that bind the CLI close a context-leak path that IDE-only policy left open. Unpinned MCP is the docs-shaped hole Scanning the Harness measured. A fleet whose gh packages fail to verify after the 2026-09-05 key expiry will stall every agent that shells out to gh.

Agent Plugins Directory census verified 2026-09-18: 4,664 conformant manifests from 1,419 repositories; after exclusions and deduplication, 2,628 distinct verified plugins remain (11,351 skills, 1,211 MCP servers), up from 2,011 / 10,455 / 969 on Sep 9 (3,869 manifests / 1,247 repos). Nine-day +617 plugins / +896 skills / +242 MCP. Treat the move as catalog-ops load, not quality. Copilot CLI 1.0.86 resume no longer discards marketplace plugins on configuration read failure; 1.0.85 adds --json plugin listing and enable/disable for plugin, MCP, and skill. HookPry still makes autoUpdate a host-privileged hook-update path. Scanning the Harness: 3.7% of published skill collections pre-approve the shell. Agent Plugins Directory census verified 2026-09-09: 3,869 conformant manifests from 1,247 repositories; after exclusions and deduplication, 2,011 distinct verified plugins remain from 1,147 publishing repositories (10,455 skills, 969 MCP servers), up from 1,956 / 10,340 / 957 on Sep 8 (3,783 manifests / 1,225 repos). One-day +55 plugins / +115 skills / +12 MCP. Treat the move as catalog-ops load, not quality. HookPry still makes autoUpdate a host-privileged hook-update path. Scanning the Harness: 3.7% of published skill collections pre-approve the shell — a marketplace scan can see that class. Agent Plugins Directory census verified 2026-09-08: 3,783 conformant manifests from 1,225 repos, 1,956 distinct plugins (10,340 skills, 957 MCP servers), up from 1,932 / 10,262 / 940 on Sep 7 (3,626 manifests / 1,207 repos). One-day +24 plugins / +78 skills / +17 MCP. Treat the move as catalog-ops load, not quality. HookPry still makes autoUpdate a host-privileged hook-update path. Agent Plugins 1.0.0 shipped 2026-08-06 as a vendor-neutral package format: plugin.json, skills/, optional mcp.json, namespaced client dirs. TSC maintainers include AWS, Cursor, Microsoft, OpenAI, Vercel; Google joined as core maintainer the same day. Launch clients: ChatGPT/Codex, Cursor, GitHub Copilot, Kiro, VS Code. GitHub GA'd the spec in VS Code, Copilot CLI, SDK, and app on 2026-08-12 with enterprise managed-settings (enabledPlugins, extraKnownMarketplaces, strictKnownMarketplaces) and MCP allowlists. Census verified 2026-09-07: 3,626 conformant manifests from 1,207 repos, 1,932 distinct plugins (10,262 skills, 940 MCP servers), up from 1,901 / 10,100 / 922 on Sep 5 (3,552 manifests / 1,166 repos). Two-day +31 plugins / +162 skills / +18 MCP. Copilot CLI plugin surface (Update action, headless plugin agents) is covered in the CLI/MCP card — reference only here. On 2026-08-26 GitHub added autoUpdate: true on extraKnownMarketplaces — HookPry (security card) makes auto-update a host-privileged hook-update path, not a convenience flag. Copilot app Customize tab GA (2026-08-25); JetBrains (Aug 24/28) applies enterprise controls for plugins, MCP, telemetry, and permission modes. A 2026-08-28 study of Claude Code plugin marketplaces (arXiv 2608.28497) analyzed 1,926 repositories, 8,351 plugins, and 77,773 commits across 2,018 marketplaces: plugin-touching commit activity grew 8.8× in the six months after the Oct 2025 launch; 61.3% of plugins target software engineering; feature commits run at 39.6% vs 17.2% in conventional OSS; Claude co-authors 34.9% of commits; 78% of Script–Markdown co-changes inside skills/ are functionally coupled. Spec stays deliberately small: discovery of Skills and MCP; installation, distribution, and policy remain client-owned. VS Code August releases (2026-08-31) add one-click install of Agent Plugins 1.0 packages into the Agents window; that is distribution UX, not an allowlist. Anatomy study (harness card): SKILL.md already leads MCP 9/11 vs 8/11 inside production coding harnesses — portable plugins standardize the package, not the runtime policy.

Why it matters Teams were maintaining a separate manifest per client. Portability plus auto-update means a poisoned or over-privileged plugin can travel and refresh itself, including lifecycle hooks the model never sees. A 2,011-plugin catalog without matching runtime policy is a supply-chain surface. Instruction files and scripts that co-evolve as a unit are a new maintenance class — treating SKILL.md as docs will miss the 78% coupled changes. A one-day census move of +55 plugins / +115 skills / +12 MCP is a catalog-ops problem, not a quality signal. EvoHarnessBench: growing that catalog can forget previously solved work.

COBRA-Skills (arXiv 2609.11682, 2026-09-10) is the new skill-optimization budget rule: treat skill search as budgeted sequential optimization over an evolving candidate space; a contextual bandit allocates evaluations and evidence-grounded operators (regeneration, rollout mutation, crossover) refresh the population. Across six agent benchmarks and three target models it is the strongest average among compared methods, cutting optimization cost 55–58% vs SkillOpt while using only 50 unique optimization examples per benchmark, and remaining robust across harnesses including self-teaching. Skill Issue (arXiv 2609.12742, 2026-09-11) is the repository-SKILL measurement rule: mine harder tasks from merged PRs reverted at a single frozen base; score a candidate by whether the same agent does better with it than without it. On three Kotlin repos, GEPA raises that score +4.9 pp and SkillOpt +0.1 pp — neither is statistically separable from the agent's run-to-run variance at 20–24 held-out tasks per repo; a maintainer still found project-specific knowledge in the documents. Do not promote a synthesized SKILL.md on a few-point pass-rate delta. Custom agents in Copilot CLI 1.0.86 can opt into AGENTS.md via include-custom-instructions — that is loading instructions, not Skill Lift. Subagents vs Agent Skills (arXiv 2609.09233, 2026-09-07) is the invocation-and-authoring rule: a skill package is suitable for subagent execution when it is procedural knowledge with explicit input and output contracts; otherwise load it in-context. SkillsBench's human-authored packages rarely specify I/O, and on those packages agent-skill execution matches or beats subagents. Synthesized contract-bearing packages reverse the ranking, with the largest gain on smaller, more bandwidth-limited models. Experience Funnel (arXiv 2609.08919) then argues for funneling validated textual skills into parametric policy slowly — a write to SKILL.md is not yet competence. Skill Following / RAE, SkillShift, EvoHarnessBench, Repo-To-Skill, and PROCTOR are unchanged. Skill Following (arXiv 2609.00549, EMNLP 2026 Findings) is the same-task skill-use rule: Retrieval-Invoked Actual-Use Effect (RAE) scores the outcome difference between matched skill-enabled and skill-disabled runs, conditioned only on tasks where the agent actually retrieved a skill. Across 17 LLMs, models frequently show positive aggregate retrieval lift but negative RAE — on MBPP+, several models that look better system-wide harm the exact tasks where retrieval occurred. Promote on paired with/without trials for retrieved skills, not on catalog-wide averages. SkillShift (arXiv 2609.02564, 2026-09-02) is the covert-policy rule: a third-party skill can keep the declared task and valid output interface while redirecting decisions; attacker-favored selection 81.33% (agentic commerce) and 63.33% (software dependency) at 100% utility-preserving rate. Scan-only Skill Lift is not Skill Policy Integrity. EvoHarnessBench (arXiv 2609.04280) is the catalog-growth rule for skills: adding skills is not a free capability bump; deployment evaluation isolates retention as the library expands, and self-evolving adaptation can stale prior skills. Repo-To-Skill (arXiv 2609.02749, 2026-09-02) distills GitHub repositories into skills in two modes: task-agnostic (field-wide reusable skills) and task-oriented. The AREX-Skill Library claims 5,000+ verified skills distilled from 1,000 widely used ML repositories (20 areas, 178 capability families) — still promote on live Skill Lift, not on distillation volume. PROCTOR (loop card) is the promotion-scoring rule for self-written skills: LLM-as-judge is an advisor; hermetic checks outrank the Teacher. RAMP (arXiv 2608.25241, 2026-08-26) scores 441 repositories on a four-level cumulative maturity model of version-controlled AI configuration. Independent annotation reproduces labels on 97% of a held-out sample. Adoption is set-and-forget: 73.8% of artifacts are committed once and never modified. Agents raise commit volume 28–38% in every stratum, but among agent-first repos those without committed AI configuration show roughly twice the increase in cognitive complexity (+53% vs +27%) and 1.7× the increase in static-analysis warnings (observational). SkillZip Pro (arXiv 2608.30785, 2026-08-31) compresses progressively loaded skill directories, not the root SKILL.md: on a production content-moderation skill it removes 38% of bundle tokens and 10.4% of end-to-end per-run tokens with no quality loss; an unprotected 71% configuration loses up to 26 accuracy points to one-sided false positives. Knowledge-gated task construction (arXiv 2608.30322, 2026-08-31) makes Skill Lift testable as a protocol: one frontier configuration goes from 0% pass without a kilobyte artefact to 68.0% with it on byte-identical instructions; a plausible-but-wrong artefact also yields 0%. WikiSkill (arXiv 2608.27454, 2026-08-27) co-evolves executable skills with a persistent wiki compiled from traces; smaller models with skills can outperform larger models without them. ACES (arXiv 2608.20614) reports mean composite Skill Lift 0.2134 on 947 paired cases (positive in 72.8%); scan-only gates correlate with LLM-judge scores at Spearman ρ = 0.14. HarnessEvolve (primary numbers in the harness card) is the promotion gate for self-written skills: it optimizes the entire skill directory including scripts, not only SKILL.md, and rejects leak/bloat; skills optimized on OpenClaw transferred without re-optimization to Hermes, OpenCode, LAMAgent, and DeepSeek Harness. SKILLSTATE execution pattern is primary in the loop-state card. Plugin marketplace co-evolution (arXiv 2608.28497) is primary in the plugins card: 78% of Script–Markdown co-changes inside skills/ are functionally coupled, so a 'docs' commit can be a behavior change. EvoSkill Injection is primary in the security card: autonomous skill writes are a persistence channel, not a docs folder. Verifiability: production stacks put fitness functions, compilers, linters, LSPs, and schemas in front of the act gate; refuse work whose correctness is unknowable from the artifact. Anatomy study (harness card): SKILL.md is already the majority extensibility surface in production coding harnesses (9/11 vs MCP 8/11) — Skill Lift, Script–Markdown coupling, and leak/bloat gates are therefore fleet defaults, not an optional skill program.

Why it matters Context files are first-class with a failure mode: too much always-on text crowds the task. Flattening a skill bundle into the root SKILL.md can raise every invocation's cost. Loading a contract-bearing procedure into the main window is a peak-context tax; spawning a subagent without an I/O contract is a delegation failure. Skills that only pass a schema scan can still fail live. Aggregate retrieval lift can hide same-task harm (RAE). A skill that preserves utility can still steer vendor or package choice (SkillShift). Oracles turn the agent into a component with a contract; a polished dashboard is not evidence. If obligations are not externalized as gates, review reconstructs them on every change. A SKILL.md that is not tested against its scripts is an unowned interface. A self-evolving skill bank without a write gate is a persistence exploit. A 5,000-skill distilled library without paired with/without trials is inventory, not capability.

Agentic CLI customizations in the usage metrics API (2026-09-17) is the new fleet-observability pin: enterprise and organization reports now include totals_by_skill, totals_by_custom_agent, totals_by_mcp, totals_by_slash_cmd, and totals_by_plugin (top five plus distinct-use counts). Customer-defined names are grouped as other; MCP counts connection attempts, not tool calls; plugin totals are a subset of skill totals — do not add them. VS Code Agents-window metrics are GA (2026-09-11) and stay separate from editor-window Agent Mode. Auto model selection (weekly 2026-09-14) adds efficiency, balance, and intelligence tiers that pick from the same available models in VS Code, CLI, and app. Copilot code review (2026-09-11/18 GA): auto-resolves addressed comments with Won't Fix / Incorrect reasons; Lite reviews use an ensemble (+47% high-severity addressed comments, −8% review cost in GitHub's experiment). Copilot budget-increase requests are GA (2026-09-16) for Business/Enterprise usage-based billing (not EMU). Fleet pin: Copilot CLI 1.0.86. Two deprecation waves: 2026-10-02 Gemini 3.5/3.6 Flash, Kimi K2.7 Code, Claude Opus 4.7; 2026-10-19 Gemini 3.7 Flash, GPT-5.5, GPT-5.4, GPT-5.4 mini, GPT-5 mini, Grok 4.5. Agentic autofix (≤25 Code Quality findings) and enterprise managed permissions remain. Sep 28 still: Balanced default review effort; cloud/web/mobile policy unification. Agentic autofix for GitHub Code Quality (2026-09-09) is the new backlog-burn path: select up to 25 standard findings and assign the set to Copilot; Copilot fixes agentically on a branch, validates its own changes, then opens a pull request. Assign to Copilot replaces Generate fix. It follows existing enterprise Code Quality policy (no separate policy) and consumes AI credits; available on GitHub Team and Enterprise Cloud when Code Quality is enabled. This is still a review-ops surface — the PR remains the human gate, not a self-merge. Enterprise managed permissions (2026-09-09 GA) let administrators set deny/ask/allow for shell, file, and network on Copilot app, CLI, and Agent Host so reviewers are not the only control. Copilot for JetBrains managed sandbox (2026-09-08 public preview), GHES 3.22 Copilot CLI air-gap TP, Copilot CLI npm latest 1.0.83, GPT-6 Astra picker availability, and 2026-10-02 model deprecations are unchanged. Copilot for JetBrains (2026-09-08) is the new fleet-surface pin: enterprise-managed sandbox policies (public preview) lock filesystem/network/proxy/dev-tool/Keychain bounds over user settings; enterprise policy diagnostics confirm enforcement on device; /ide connects Copilot CLI sessions to JetBrains IDE context; OpenTelemetry chat settings are now GA. GHES 3.22 GA (2026-09-08) lets administrators configure Copilot CLI against GHES credentials for disconnected / air-gapped enterprises (technical preview) — treat as a GHES fleet pin, not a harness-eval reset. Copilot CLI npm latest remains 1.0.83 (verified 2026-09-09); prerelease 1.0.84-3 is not a fleet pin. Content exclusions, Copilot approval assessments (off by default), GPT-6 Astra picker availability, and 2026-10-02 model deprecations are unchanged. Sep 28 still: Balanced default review effort; cloud/web/mobile Copilot policy unification. Official GitHub CLI Linux APT/RPM signing key expired 2026-09-05; beginning with the first release after that date, repository metadata and newly published RPMs are signed with the replacement key only. Agent CLIs and CI images that install gh from those repos and still trust only the old key will fail verification — treat as a fleet-pin for every harness that shells out to gh. Copilot CLI npm latest remains 1.0.83 (verified 2026-09-08); prerelease 1.0.84-1 is not a fleet pin. GPT-6 Astra is generally available in GitHub Copilot (2026-09-04) across VS Code, Visual Studio, Copilot CLI, coding agent, app, github.com, Mobile, JetBrains, Xcode, and Eclipse — treat as a managed-settings / default-model pin, not a harness-eval reset; billed at provider list pricing under usage-based billing. GitHub Copilot weekly releases (posted 2026-09-04) also note Agent Merge public preview in VS Code 1.136 (resolve review feedback, failed checks, merge conflicts) and Copilot harness GA in JetBrains. GitHub stacked PRs remain in public preview (2026-07-30) with gh-stack, stack maps, partial merges, and an agent skill — teach agents to decompose giant changes into reviewable layers. Copilot code review (2026-08-27) gives Copilot cloud-agent PRs a full agentic review, can review auto-requested bot-authored PRs when org policy allows, and drops the 300-file / 20,000-LOC size cap; resolution reasons close the feedback loop. On 2026-09-01 Copilot code review gained public-preview approval assessments and optional approvals: every review now includes an assessment of whether the PR is ready to approve; Copilot will not approve unless enterprise/org/repo policy turns it on, and path allowlists can restrict which files it may sign off; an approval assessment alone does not count toward merge requirements; when enabled, a Copilot approval counts toward required-approvals and is dismissed on new commits like a human review. From no earlier than 2026-09-28, default review effort moves Lite→Balanced and Copilot cloud/web/mobile unify to one policy enabled by default. Layered supervision (arXiv 2608.26316, 2026-08-26) distributes work across preventive, executable, and human layers that shift toward architectural reasoning. Visual Studio (2026-08-28) adds a Git agent that reviews uncommitted changes before a PR; org owners can publish custom agents across repositories. Span (103 teams, May–Jul 2026): +1 prompt-clarity → 27.2% lower cost per merged AI line; +1 environment-readiness → 88% higher turn yield; +1 quality-stewardship → 39% fewer review cycles per 1,000 merged AI lines. CURA (arXiv 2608.27808, 2026-08-28) is the oversight gap for computer-use agents: on 361 OSWorld tasks a pipeline scoring 82.9 vs 72.4 human still ends 64 of 71 failures (90%) with a success claim, never using the explicit failure affordance across ~9,100 calls; at α=0.10 an external CUSUM alarm on harness-visible telemetry detects 42.3% of failures a median of 31 steps before termination at realized false-alarm rate 0.066. Observability: OpenTelemetry GenAI conventions standardize chat/tool/invoke_agent spans; Cloudflare agent tracing (2026-08-04) with span billing from Oct 2026; AWS AgentCore Observability via ADOT (2026-08-13); llmmas-otel (arXiv 2608.24271) pairs OTel with targeted fault injection. Repair-or-resample numbers are primary in the loop-state card. Cost-Utility dual-ledger primary in skills/verifiability. Copilot CLI OTel hook correlation primary in CLI/MCP card. Do not treat Copilot's approval assessment as a required review, and do not enable Copilot approvals on paths without an executable oracle. Content exclusions GA in Copilot app and CLI (2026-09-02) is a review-ops control: excluded paths must not appear in agent context or in Copilot-approval evidence. Enterprise-managed settings now support any default model, including per-team overrides via team-mappings.json (2026-09-02), so model choice is an org policy artifact rather than a local picker habit. Copilot fleet policy (2026-09-03): Gemini 3.8 Flash is in the model picker (Pro/Pro+/Max/Business/Enterprise; gradual rollout across VS Code, CLI, cloud agent, app, JetBrains, Xcode, Eclipse). The same day GitHub announced deprecation on 2026-10-02 of Gemini 3.5 Flash, Gemini 3.6 Flash, Kimi K2.7 Code, and Claude Opus 4.7 across all Copilot experiences, with suggested replacements Gemini 3.8 Flash, Kimi K3, and Claude Opus 5. This is managed-settings / default-model / custom-agent list work (enterprise-managed settings already support any default model as of 2026-09-02), not a reason to retune harness evals around a new Flash SKU. Copilot CLI 1.0.83-4 already removes retired Claude and Gemini models from /model picker results — only on the prerelease tag.

Why it matters Agent volume without decomposition recreates the senior-engineer tax. Bulk autofix without an executable oracle and a human merge gate converts a quality backlog into rubber-stamped PRs. Without a shared trace contract, incident response stays anecdotal. Self-report is the cheapest oversight channel and fails where oversight matters. Letting an agent approve its own (or a peer agent's) PR without path policy and an executable oracle converts review throughput into a rubber stamp. Seat licenses without harness ownership leave review and incidents on the most expensive people. A broken gh package feed stops every agent that opens PRs from Linux images.