fix(sidecar): .pyc marshal-note is host-verify only (red 2026-11-12) (#175)

* fix(sidecar): do not claim pyc-sidecar-entangle for a host-only trailer

The TYPE_ASCII marshal note is ignored by importlib, so strip or sibling
delete still execute. Recording pyc-sidecar-entangle (and clearing the
strip⇒fail defenseScore gap) was paper coverage. Keep planting the note
and host verifyPycSidecarMarshalNote; annotate pyc-sidecar-host-verify-only
until an import-time hook lands.

* test(sidecar): assert .pyc marshal notes stay host-verify only

protectTarget must not list pyc-sidecar-entangle; importlib still returns
the same RESULT after trailer strip or sidecar delete, while the host
verifier fails closed. defenseScore keeps cap 93 via the new gap when
the host-only note is enabled.

* docs(skill,state): sidecar vectors require a process-start hook

Skill B: only record a countered strip⇒fail sidecar vector when the
entry hook hard-fails at process start or import. Host-only verifiers
(such as verifyPycSidecarMarshalNote) are binding notes, not coverage.
CHANGELOG/STATE/DESIGN record the 2026-11-12 honesty residual.
This commit is contained in:
Timo Zhang
2026-11-12 14:57:45 +08:00
committed by GitHub
parent 9c41bb53b0
commit 0cd513e08e
9 changed files with 65 additions and 7 deletions
+1
View File
@@ -121,6 +121,7 @@ Also: do **not** ship a stable public WASM ABI without explicit `wasm_preserve_e
- Integrity checkers must be interleaved and mutually covering (coverage matrix `bla-integrity-coverage/v1`); single-point self-seal is refuse-worthy. Lint with `bla_lint_integrity_coverage`.
- Optional bind-scope (domain/date) is a **public** license contract (`bla_bind_scope` → `bind-scope.json` + `BLA_BIND_SCOPE_CHECK`). Default OFF. Never a covert backdoor — `public:true` mandatory; lint with `bla_lint_bind_scope`.
- Coverage claims must be real, not paper: an entry may only count as covering itself when its runtime check hard-fails on its own bytes (canonical self-digest, finalized at write). Check `integrityCoverageSelfVerified` on the harden report; when false the matrix drops self edges and refuses. Keep host-trust values (e.g. certificate digests) inside a covered artifact — unhardened data sidecars like `seal.json` are not covered.
- A countered strip⇒fail sidecar vector (`asar-sidecar-entangle`, or `pyc-sidecar-entangle` once it exists at import time) may be recorded only when the entry hook hard-fails at process start or import. A host-only verifier (for example `verifyPycSidecarMarshalNote` on a `.pyc` TYPE_ASCII trailer that `importlib` ignores) is a binding note, not coverage: do not list the vector, and do not treat enabling it as closing the `sidecar-entangle-not-enabled` defenseScore gap (annotate `pyc-sidecar-host-verify-only` instead).
Held across R8–R12 (NO PRIZE). Keep in gate design; do **not** invent packaging refuse lints for these:
+3
View File
@@ -6,6 +6,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Fixed
- **Honesty: `.pyc` marshal-note is host-verify only.** `protectTarget` no longer records `ReaAttackVectorId` `pyc-sidecar-entangle` for planting a TYPE_ASCII trailer. `importlib` ignores post-root trailers, so strip or sibling delete still execute; host `verifyPycSidecarMarshalNote` remains the offline check. An import-time hook is still TBD (same class as the WASM JS-loader). Enabling the note now annotates defenseScore gap `pyc-sidecar-host-verify-only` (soft-cap 93) instead of clearing the process-start strip⇒fail gap.
## [3.2.5] — 2026-11-11
Patch extending opt-in `sidecarEntangle` to **CPython `.pyc` marshal trailer notes**, plus Counter-REA limit catalog hygiene for NativeAOT and IL2CPP v108.
+1 -1
View File
@@ -106,7 +106,7 @@ CrackMe v8 red team (round-20261106): **NO PRIZE** — key/FLAG not recovered; `
## Nightly roadmap remaining
1. ~~**Self-seal / defenseScore honesty (#40)**~~ — landed soft-caps + opt-in `--self-seal` / MCP `self_seal` / `.blarc selfSeal` (batch 2026-11-02). Remaining: deeper onefile/peel score model.
2. **Sidecar entry entanglement** — JS strip⇒fail landed 2026-11-05; Python preamble landed 2026-11-06 nightly; WASM `bla.sidecar.v1` custom-section + host verify landed 2026-11-07 nightly; Bun `.bla-sidecar.v1.json` companion + defenseScore `sidecar-entangle-not-enabled` gap landed 2026-11-08 nightly; ASAR `.bla/sidecar.v1.json` archive-note landed 2026-11-09 nightly; ASAR main JS preamble landed 2026-11-10 nightly. Remaining: optional JS-loader `verifyWasmSidecarCustomSection`.
2. **Sidecar entry entanglement** — JS strip⇒fail landed 2026-11-05; Python preamble landed 2026-11-06 nightly; WASM `bla.sidecar.v1` custom-section + host verify landed 2026-11-07 nightly; Bun `.bla-sidecar.v1.json` companion + defenseScore `sidecar-entangle-not-enabled` gap landed 2026-11-08 nightly; ASAR `.bla/sidecar.v1.json` archive-note landed 2026-11-09 nightly; ASAR main JS preamble landed 2026-11-10 nightly; `.pyc` TYPE_ASCII marshal trailer (host verify) landed 2026-11-11 nightly — **2026-11-12 honesty:** do not record `pyc-sidecar-entangle` / do not clear the strip⇒fail defenseScore gap until an import-time hook lands (`pyc-sidecar-host-verify-only`). Remaining: optional JS-loader `verifyWasmSidecarCustomSection`; `.pyc` import-time hook.
3. **Deception shields → more formats** — WASM export rename + `wasmPreserveExports` landed; DEX/SQLite/model densify and broader numeric template hygiene still open.
4. **Empirical JADX / Android red-team** — real REA mapping+JNI vs BLA-hardened unsigned APK.
5. **ESP flash / esp-image (#8228)** — if REA moved there; UF2 empirical red-team otherwise.
+2 -2
View File
@@ -1,6 +1,6 @@
# DESIGN: Sidecar / entry entanglement (issue #40 / STATE)
Status: **JS + Python + WASM + Bun companion + ASAR archive-note + ASAR main JS preamble + .pyc marshal-note strip⇒fail** — binding digest + JS/Python preambles + WASM `bla.sidecar.v1` custom-section note + Bun sibling `.bla-sidecar.v1.json` companion note (byte-exact binary) + ASAR in-archive `.bla/sidecar.v1.json` note + optional Electron main JS preamble (`generateSidecarAsarMainJsPreamble`) + CPython `.pyc` TYPE_ASCII marshal trailer (`generateSidecarPycMarshalNote` / `verifyPycSidecarMarshalNote`) + protect/harden opt-in landed; defenseScore gap when `sidecarEntangle` is off landed; optional JS-loader `verifyWasmSidecarCustomSection` still TBD.
Status: **JS + Python + WASM + Bun companion + ASAR archive-note + ASAR main JS preamble strip⇒fail; .pyc marshal-note is host-verify only** — binding digest + JS/Python preambles + WASM `bla.sidecar.v1` custom-section note + Bun sibling `.bla-sidecar.v1.json` companion note (byte-exact binary) + ASAR in-archive `.bla/sidecar.v1.json` note + optional Electron main JS preamble (`generateSidecarAsarMainJsPreamble`) + CPython `.pyc` TYPE_ASCII marshal trailer (`generateSidecarPycMarshalNote` / `verifyPycSidecarMarshalNote`, importlib-ignored, no process-start fail) + protect/harden opt-in landed; defenseScore gap when `sidecarEntangle` is off landed; `.pyc` host-only notes annotate `pyc-sidecar-host-verify-only` and must not record `pyc-sidecar-entangle`; optional JS-loader `verifyWasmSidecarCustomSection` and `.pyc` import-time hook still TBD.
## Goal
@@ -44,4 +44,4 @@ checks beat orphan honeypot files.
- ~~Honesty: when entanglement is off, defenseScore gap annotation (like self-seal).~~ — 2026-11-08 nightly: `sidecar-entangle-not-enabled` soft-cap (93) via `sidecarEntangleEnabled` / MCP `sidecar_entangle_enabled`.
- ~~Extend to ASAR archive-note companion~~ — 2026-11-09 nightly: `generateSidecarAsarArchiveNote` / `bindAsarSidecarEntanglement` / `entangleAsarEntryWithSidecars` / `verifyAsarSidecarArchiveNote`; protect + harden ASAR post-pass plant `.bla/sidecar.v1.json` binding `enclave/hardware-tee-bridge.js`.
- ~~ASAR main JS preamble (Electron fs-patched re-measure)~~ — 2026-11-10 nightly: `generateSidecarAsarMainJsPreamble` / `selectAsarMainJsMember` / `entangleAsarMainJsWithSidecars`; protect + harden prepend strip⇒fail preamble to package.json#main (or heuristic entry).
- ~~CPython `.pyc` marshal trailer note~~ — 2026-11-11 nightly: `generateSidecarPycMarshalNote` / `bindPycSidecarEntanglement` / `entanglePycEntryWithSidecars` / `verifyPycSidecarMarshalNote`; TYPE_ASCII trailer after root (importlib-safe); protect + harden `.pyc` post-pass; vector `pyc-sidecar-entangle`; default OFF. Optional JS-loader `verifyWasmSidecarCustomSection` still TBD; next candidates: DEX string pool / SQLite comments / Safetensors·GGUF header metadata.
- ~~CPython `.pyc` marshal trailer note (host verify)~~ — 2026-11-11 nightly: `generateSidecarPycMarshalNote` / `bindPycSidecarEntanglement` / `entanglePycEntryWithSidecars` / `verifyPycSidecarMarshalNote`; TYPE_ASCII trailer after root (importlib-safe). **2026-11-12 honesty:** do not record `pyc-sidecar-entangle` until an import-time hook lands; defenseScore gap `pyc-sidecar-host-verify-only`. Default OFF. Remaining: `.pyc` import-time hook; optional JS-loader `verifyWasmSidecarCustomSection`; next candidates: DEX string pool / SQLite comments / Safetensors·GGUF header metadata.
+13 -1
View File
@@ -18,6 +18,7 @@ export type DefenseScoreGapId =
| "wasm-export-rename-pending"
| "self-seal-not-enabled"
| "sidecar-entangle-not-enabled"
| "pyc-sidecar-host-verify-only"
| "numeric-template-hygiene-partial";
export interface DefenseScoreGap {
@@ -104,7 +105,18 @@ export function collectDefenseScoreGaps(
gaps.push({
id: "sidecar-entangle-not-enabled",
summary:
"Opt-in sidecarEntangle strip⇒fail binding was not enabled (JS/Python preamble, WASM bla.sidecar.v1, Bun .bla-sidecar.v1.json companion, or .pyc marshal note); default remains OFF (issue #40 / DESIGN).",
"Opt-in sidecarEntangle strip⇒fail binding was not enabled (JS/Python preamble, WASM bla.sidecar.v1, Bun companion, or a .pyc import-time hook); a host-only .pyc marshal trailer does not count. Default remains OFF (issue #40 / DESIGN).",
scoreCap: 93,
});
}
// .pyc marshal trailer is host-verify only (importlib ignores it). Enabling
// sidecarEntangle must not clear the process-start strip⇒fail gap.
if (input.detectedFormat === "python-pyc" && input.sidecarEntangleEnabled === true) {
gaps.push({
id: "pyc-sidecar-host-verify-only",
summary:
"sidecarEntangle planted a .pyc TYPE_ASCII marshal trailer verified only by host verifyPycSidecarMarshalNote; importlib still runs after strip or sibling delete. Import-time hook TBD — do not treat this as process-start strip⇒fail (issue #40 / DESIGN).",
scoreCap: 93,
});
}
+5 -1
View File
@@ -950,7 +950,11 @@ function protectTargetInner(options: ProtectionOptions): ProtectionResult {
finalPyc = entangled.buffer;
if (entangled.applied) {
sidecarEntangleApplied = true;
counteredVectors.add("pyc-sidecar-entangle");
// Host-verify trailer only: importlib ignores post-root TYPE_ASCII
// notes, so strip / sibling delete still execute. Do not record
// pyc-sidecar-entangle until an import-time hook lands (paper
// coverage otherwise; same honesty bar as ASAR's process-start
// preamble before asar-sidecar-entangle is claimed).
}
}
writeProtectedOutput(absOutput, finalPyc);
+1 -1
View File
@@ -2018,7 +2018,7 @@ export function bindPycSidecarEntanglement(
...plan,
entryHookKind: "pyc-marshal-note",
status: "bound",
note: "Bound — .pyc TYPE_ASCII marshal trailer note planted; ship sidecars alongside and verify via verifyPycSidecarMarshalNote. Strip/alter fails closed. importlib ignores the trailer.",
note: "Bound — .pyc TYPE_ASCII marshal trailer note planted; ship sidecars alongside and verify via verifyPycSidecarMarshalNote. Host verify fails closed on strip/alter; importlib ignores the trailer (no process-start fail until an import-time hook lands).",
};
return {
plan: boundPlan,
+17
View File
@@ -64,6 +64,23 @@ describe("Issue #40.6: defenseScore honesty", () => {
expect(on.cappedScore).toBe(99);
});
it("keeps a .pyc host-only marshal trailer from closing the strip⇒fail gap", () => {
const off = applyDefenseScoreHonesty(99, {
detectedFormat: "python-pyc",
sidecarEntangleEnabled: false,
});
expect(off.gaps.map((g) => g.id)).toContain("sidecar-entangle-not-enabled");
expect(off.cappedScore).toBe(93);
const hostOnly = applyDefenseScoreHonesty(99, {
detectedFormat: "python-pyc",
sidecarEntangleEnabled: true,
});
expect(hostOnly.gaps.map((g) => g.id)).toContain("pyc-sidecar-host-verify-only");
expect(hostOnly.gaps.map((g) => g.id)).not.toContain("sidecar-entangle-not-enabled");
expect(hostOnly.cappedScore).toBe(93);
});
it("does not raise a low raw score when gaps apply", () => {
const honesty = applyDefenseScoreHonesty(40, {
detectedFormat: "wasm",
+22 -1
View File
@@ -4,6 +4,7 @@ import {
mkdtempSync,
readFileSync,
rmSync,
unlinkSync,
writeFileSync,
} from "node:fs";
import { join } from "node:path";
@@ -209,7 +210,8 @@ describe("sidecar .pyc marshal-note strip⇒fail", () => {
sidecarEntanglePaths: ["honey.txt"],
});
expect(on.sidecarEntangle).toBe(true);
expect(on.counteredReaVectors).toContain("pyc-sidecar-entangle");
// Host-verify only: do not claim the process-start strip⇒fail vector.
expect(on.counteredReaVectors).not.toContain("pyc-sidecar-entangle");
const outBytes = readFileSync(outPyc);
const note = extractPycSidecarMarshalNote(outBytes);
expect(note).not.toBeNull();
@@ -221,6 +223,25 @@ describe("sidecar .pyc marshal-note strip⇒fail", () => {
});
expect(verified.ok).toBe(true);
// Residual: importlib still executes after trailer strip or sibling delete.
const stripped = stripPycSidecarMarshalNote(outBytes);
expect(extractPycSidecarMarshalNote(stripped)).toBeNull();
expect(importPycResult(stripped)).toBe(5);
expect(
verifyPycSidecarMarshalNote({
entryBytes: stripped,
baseDir: dir,
}).ok,
).toBe(false);
unlinkSync(join(dir, "honey.txt"));
expect(importPycResult(outBytes)).toBe(5);
const afterDelete = verifyPycSidecarMarshalNote({
entryBytes: outBytes,
baseDir: dir,
});
expect(afterDelete.ok).toBe(false);
expect(afterDelete.reason).toMatch(/missing/i);
// Honey from protectPythonPycBuffer still present alongside the note.
const honey = protectPythonPycBuffer(entry, {
intensity: "balanced",