mirror of
https://github.com/qybaihe/bla.git
synced 2026-11-20 08:20:59 +08:00
fix(sidecar): .pyc marshal-note is host-verify only (red 2026-11-12) (#175)
* fix(sidecar): do not claim pyc-sidecar-entangle for a host-only trailer The TYPE_ASCII marshal note is ignored by importlib, so strip or sibling delete still execute. Recording pyc-sidecar-entangle (and clearing the strip⇒fail defenseScore gap) was paper coverage. Keep planting the note and host verifyPycSidecarMarshalNote; annotate pyc-sidecar-host-verify-only until an import-time hook lands. * test(sidecar): assert .pyc marshal notes stay host-verify only protectTarget must not list pyc-sidecar-entangle; importlib still returns the same RESULT after trailer strip or sidecar delete, while the host verifier fails closed. defenseScore keeps cap 93 via the new gap when the host-only note is enabled. * docs(skill,state): sidecar vectors require a process-start hook Skill B: only record a countered strip⇒fail sidecar vector when the entry hook hard-fails at process start or import. Host-only verifiers (such as verifyPycSidecarMarshalNote) are binding notes, not coverage. CHANGELOG/STATE/DESIGN record the 2026-11-12 honesty residual.
This commit is contained in:
@@ -121,6 +121,7 @@ Also: do **not** ship a stable public WASM ABI without explicit `wasm_preserve_e
|
||||
- Integrity checkers must be interleaved and mutually covering (coverage matrix `bla-integrity-coverage/v1`); single-point self-seal is refuse-worthy. Lint with `bla_lint_integrity_coverage`.
|
||||
- Optional bind-scope (domain/date) is a **public** license contract (`bla_bind_scope` → `bind-scope.json` + `BLA_BIND_SCOPE_CHECK`). Default OFF. Never a covert backdoor — `public:true` mandatory; lint with `bla_lint_bind_scope`.
|
||||
- Coverage claims must be real, not paper: an entry may only count as covering itself when its runtime check hard-fails on its own bytes (canonical self-digest, finalized at write). Check `integrityCoverageSelfVerified` on the harden report; when false the matrix drops self edges and refuses. Keep host-trust values (e.g. certificate digests) inside a covered artifact — unhardened data sidecars like `seal.json` are not covered.
|
||||
- A countered strip⇒fail sidecar vector (`asar-sidecar-entangle`, or `pyc-sidecar-entangle` once it exists at import time) may be recorded only when the entry hook hard-fails at process start or import. A host-only verifier (for example `verifyPycSidecarMarshalNote` on a `.pyc` TYPE_ASCII trailer that `importlib` ignores) is a binding note, not coverage: do not list the vector, and do not treat enabling it as closing the `sidecar-entangle-not-enabled` defenseScore gap (annotate `pyc-sidecar-host-verify-only` instead).
|
||||
|
||||
Held across R8–R12 (NO PRIZE). Keep in gate design; do **not** invent packaging refuse lints for these:
|
||||
|
||||
|
||||
@@ -6,6 +6,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Fixed
|
||||
- **Honesty: `.pyc` marshal-note is host-verify only.** `protectTarget` no longer records `ReaAttackVectorId` `pyc-sidecar-entangle` for planting a TYPE_ASCII trailer. `importlib` ignores post-root trailers, so strip or sibling delete still execute; host `verifyPycSidecarMarshalNote` remains the offline check. An import-time hook is still TBD (same class as the WASM JS-loader). Enabling the note now annotates defenseScore gap `pyc-sidecar-host-verify-only` (soft-cap 93) instead of clearing the process-start strip⇒fail gap.
|
||||
|
||||
## [3.2.5] — 2026-11-11
|
||||
|
||||
Patch extending opt-in `sidecarEntangle` to **CPython `.pyc` marshal trailer notes**, plus Counter-REA limit catalog hygiene for NativeAOT and IL2CPP v108.
|
||||
|
||||
@@ -106,7 +106,7 @@ CrackMe v8 red team (round-20261106): **NO PRIZE** — key/FLAG not recovered; `
|
||||
## Nightly roadmap remaining
|
||||
|
||||
1. ~~**Self-seal / defenseScore honesty (#40)**~~ — landed soft-caps + opt-in `--self-seal` / MCP `self_seal` / `.blarc selfSeal` (batch 2026-11-02). Remaining: deeper onefile/peel score model.
|
||||
2. **Sidecar entry entanglement** — JS strip⇒fail landed 2026-11-05; Python preamble landed 2026-11-06 nightly; WASM `bla.sidecar.v1` custom-section + host verify landed 2026-11-07 nightly; Bun `.bla-sidecar.v1.json` companion + defenseScore `sidecar-entangle-not-enabled` gap landed 2026-11-08 nightly; ASAR `.bla/sidecar.v1.json` archive-note landed 2026-11-09 nightly; ASAR main JS preamble landed 2026-11-10 nightly. Remaining: optional JS-loader `verifyWasmSidecarCustomSection`.
|
||||
2. **Sidecar entry entanglement** — JS strip⇒fail landed 2026-11-05; Python preamble landed 2026-11-06 nightly; WASM `bla.sidecar.v1` custom-section + host verify landed 2026-11-07 nightly; Bun `.bla-sidecar.v1.json` companion + defenseScore `sidecar-entangle-not-enabled` gap landed 2026-11-08 nightly; ASAR `.bla/sidecar.v1.json` archive-note landed 2026-11-09 nightly; ASAR main JS preamble landed 2026-11-10 nightly; `.pyc` TYPE_ASCII marshal trailer (host verify) landed 2026-11-11 nightly — **2026-11-12 honesty:** do not record `pyc-sidecar-entangle` / do not clear the strip⇒fail defenseScore gap until an import-time hook lands (`pyc-sidecar-host-verify-only`). Remaining: optional JS-loader `verifyWasmSidecarCustomSection`; `.pyc` import-time hook.
|
||||
3. **Deception shields → more formats** — WASM export rename + `wasmPreserveExports` landed; DEX/SQLite/model densify and broader numeric template hygiene still open.
|
||||
4. **Empirical JADX / Android red-team** — real REA mapping+JNI vs BLA-hardened unsigned APK.
|
||||
5. **ESP flash / esp-image (#8228)** — if REA moved there; UF2 empirical red-team otherwise.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# DESIGN: Sidecar / entry entanglement (issue #40 / STATE)
|
||||
|
||||
Status: **JS + Python + WASM + Bun companion + ASAR archive-note + ASAR main JS preamble + .pyc marshal-note strip⇒fail** — binding digest + JS/Python preambles + WASM `bla.sidecar.v1` custom-section note + Bun sibling `.bla-sidecar.v1.json` companion note (byte-exact binary) + ASAR in-archive `.bla/sidecar.v1.json` note + optional Electron main JS preamble (`generateSidecarAsarMainJsPreamble`) + CPython `.pyc` TYPE_ASCII marshal trailer (`generateSidecarPycMarshalNote` / `verifyPycSidecarMarshalNote`) + protect/harden opt-in landed; defenseScore gap when `sidecarEntangle` is off landed; optional JS-loader `verifyWasmSidecarCustomSection` still TBD.
|
||||
Status: **JS + Python + WASM + Bun companion + ASAR archive-note + ASAR main JS preamble strip⇒fail; .pyc marshal-note is host-verify only** — binding digest + JS/Python preambles + WASM `bla.sidecar.v1` custom-section note + Bun sibling `.bla-sidecar.v1.json` companion note (byte-exact binary) + ASAR in-archive `.bla/sidecar.v1.json` note + optional Electron main JS preamble (`generateSidecarAsarMainJsPreamble`) + CPython `.pyc` TYPE_ASCII marshal trailer (`generateSidecarPycMarshalNote` / `verifyPycSidecarMarshalNote`, importlib-ignored, no process-start fail) + protect/harden opt-in landed; defenseScore gap when `sidecarEntangle` is off landed; `.pyc` host-only notes annotate `pyc-sidecar-host-verify-only` and must not record `pyc-sidecar-entangle`; optional JS-loader `verifyWasmSidecarCustomSection` and `.pyc` import-time hook still TBD.
|
||||
|
||||
## Goal
|
||||
|
||||
@@ -44,4 +44,4 @@ checks beat orphan honeypot files.
|
||||
- ~~Honesty: when entanglement is off, defenseScore gap annotation (like self-seal).~~ — 2026-11-08 nightly: `sidecar-entangle-not-enabled` soft-cap (93) via `sidecarEntangleEnabled` / MCP `sidecar_entangle_enabled`.
|
||||
- ~~Extend to ASAR archive-note companion~~ — 2026-11-09 nightly: `generateSidecarAsarArchiveNote` / `bindAsarSidecarEntanglement` / `entangleAsarEntryWithSidecars` / `verifyAsarSidecarArchiveNote`; protect + harden ASAR post-pass plant `.bla/sidecar.v1.json` binding `enclave/hardware-tee-bridge.js`.
|
||||
- ~~ASAR main JS preamble (Electron fs-patched re-measure)~~ — 2026-11-10 nightly: `generateSidecarAsarMainJsPreamble` / `selectAsarMainJsMember` / `entangleAsarMainJsWithSidecars`; protect + harden prepend strip⇒fail preamble to package.json#main (or heuristic entry).
|
||||
- ~~CPython `.pyc` marshal trailer note~~ — 2026-11-11 nightly: `generateSidecarPycMarshalNote` / `bindPycSidecarEntanglement` / `entanglePycEntryWithSidecars` / `verifyPycSidecarMarshalNote`; TYPE_ASCII trailer after root (importlib-safe); protect + harden `.pyc` post-pass; vector `pyc-sidecar-entangle`; default OFF. Optional JS-loader `verifyWasmSidecarCustomSection` still TBD; next candidates: DEX string pool / SQLite comments / Safetensors·GGUF header metadata.
|
||||
- ~~CPython `.pyc` marshal trailer note (host verify)~~ — 2026-11-11 nightly: `generateSidecarPycMarshalNote` / `bindPycSidecarEntanglement` / `entanglePycEntryWithSidecars` / `verifyPycSidecarMarshalNote`; TYPE_ASCII trailer after root (importlib-safe). **2026-11-12 honesty:** do not record `pyc-sidecar-entangle` until an import-time hook lands; defenseScore gap `pyc-sidecar-host-verify-only`. Default OFF. Remaining: `.pyc` import-time hook; optional JS-loader `verifyWasmSidecarCustomSection`; next candidates: DEX string pool / SQLite comments / Safetensors·GGUF header metadata.
|
||||
|
||||
@@ -18,6 +18,7 @@ export type DefenseScoreGapId =
|
||||
| "wasm-export-rename-pending"
|
||||
| "self-seal-not-enabled"
|
||||
| "sidecar-entangle-not-enabled"
|
||||
| "pyc-sidecar-host-verify-only"
|
||||
| "numeric-template-hygiene-partial";
|
||||
|
||||
export interface DefenseScoreGap {
|
||||
@@ -104,7 +105,18 @@ export function collectDefenseScoreGaps(
|
||||
gaps.push({
|
||||
id: "sidecar-entangle-not-enabled",
|
||||
summary:
|
||||
"Opt-in sidecarEntangle strip⇒fail binding was not enabled (JS/Python preamble, WASM bla.sidecar.v1, Bun .bla-sidecar.v1.json companion, or .pyc marshal note); default remains OFF (issue #40 / DESIGN).",
|
||||
"Opt-in sidecarEntangle strip⇒fail binding was not enabled (JS/Python preamble, WASM bla.sidecar.v1, Bun companion, or a .pyc import-time hook); a host-only .pyc marshal trailer does not count. Default remains OFF (issue #40 / DESIGN).",
|
||||
scoreCap: 93,
|
||||
});
|
||||
}
|
||||
|
||||
// .pyc marshal trailer is host-verify only (importlib ignores it). Enabling
|
||||
// sidecarEntangle must not clear the process-start strip⇒fail gap.
|
||||
if (input.detectedFormat === "python-pyc" && input.sidecarEntangleEnabled === true) {
|
||||
gaps.push({
|
||||
id: "pyc-sidecar-host-verify-only",
|
||||
summary:
|
||||
"sidecarEntangle planted a .pyc TYPE_ASCII marshal trailer verified only by host verifyPycSidecarMarshalNote; importlib still runs after strip or sibling delete. Import-time hook TBD — do not treat this as process-start strip⇒fail (issue #40 / DESIGN).",
|
||||
scoreCap: 93,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -950,7 +950,11 @@ function protectTargetInner(options: ProtectionOptions): ProtectionResult {
|
||||
finalPyc = entangled.buffer;
|
||||
if (entangled.applied) {
|
||||
sidecarEntangleApplied = true;
|
||||
counteredVectors.add("pyc-sidecar-entangle");
|
||||
// Host-verify trailer only: importlib ignores post-root TYPE_ASCII
|
||||
// notes, so strip / sibling delete still execute. Do not record
|
||||
// pyc-sidecar-entangle until an import-time hook lands (paper
|
||||
// coverage otherwise; same honesty bar as ASAR's process-start
|
||||
// preamble before asar-sidecar-entangle is claimed).
|
||||
}
|
||||
}
|
||||
writeProtectedOutput(absOutput, finalPyc);
|
||||
|
||||
@@ -2018,7 +2018,7 @@ export function bindPycSidecarEntanglement(
|
||||
...plan,
|
||||
entryHookKind: "pyc-marshal-note",
|
||||
status: "bound",
|
||||
note: "Bound — .pyc TYPE_ASCII marshal trailer note planted; ship sidecars alongside and verify via verifyPycSidecarMarshalNote. Strip/alter fails closed. importlib ignores the trailer.",
|
||||
note: "Bound — .pyc TYPE_ASCII marshal trailer note planted; ship sidecars alongside and verify via verifyPycSidecarMarshalNote. Host verify fails closed on strip/alter; importlib ignores the trailer (no process-start fail until an import-time hook lands).",
|
||||
};
|
||||
return {
|
||||
plan: boundPlan,
|
||||
|
||||
@@ -64,6 +64,23 @@ describe("Issue #40.6: defenseScore honesty", () => {
|
||||
expect(on.cappedScore).toBe(99);
|
||||
});
|
||||
|
||||
it("keeps a .pyc host-only marshal trailer from closing the strip⇒fail gap", () => {
|
||||
const off = applyDefenseScoreHonesty(99, {
|
||||
detectedFormat: "python-pyc",
|
||||
sidecarEntangleEnabled: false,
|
||||
});
|
||||
expect(off.gaps.map((g) => g.id)).toContain("sidecar-entangle-not-enabled");
|
||||
expect(off.cappedScore).toBe(93);
|
||||
|
||||
const hostOnly = applyDefenseScoreHonesty(99, {
|
||||
detectedFormat: "python-pyc",
|
||||
sidecarEntangleEnabled: true,
|
||||
});
|
||||
expect(hostOnly.gaps.map((g) => g.id)).toContain("pyc-sidecar-host-verify-only");
|
||||
expect(hostOnly.gaps.map((g) => g.id)).not.toContain("sidecar-entangle-not-enabled");
|
||||
expect(hostOnly.cappedScore).toBe(93);
|
||||
});
|
||||
|
||||
it("does not raise a low raw score when gaps apply", () => {
|
||||
const honesty = applyDefenseScoreHonesty(40, {
|
||||
detectedFormat: "wasm",
|
||||
|
||||
@@ -4,6 +4,7 @@ import {
|
||||
mkdtempSync,
|
||||
readFileSync,
|
||||
rmSync,
|
||||
unlinkSync,
|
||||
writeFileSync,
|
||||
} from "node:fs";
|
||||
import { join } from "node:path";
|
||||
@@ -209,7 +210,8 @@ describe("sidecar .pyc marshal-note strip⇒fail", () => {
|
||||
sidecarEntanglePaths: ["honey.txt"],
|
||||
});
|
||||
expect(on.sidecarEntangle).toBe(true);
|
||||
expect(on.counteredReaVectors).toContain("pyc-sidecar-entangle");
|
||||
// Host-verify only: do not claim the process-start strip⇒fail vector.
|
||||
expect(on.counteredReaVectors).not.toContain("pyc-sidecar-entangle");
|
||||
const outBytes = readFileSync(outPyc);
|
||||
const note = extractPycSidecarMarshalNote(outBytes);
|
||||
expect(note).not.toBeNull();
|
||||
@@ -221,6 +223,25 @@ describe("sidecar .pyc marshal-note strip⇒fail", () => {
|
||||
});
|
||||
expect(verified.ok).toBe(true);
|
||||
|
||||
// Residual: importlib still executes after trailer strip or sibling delete.
|
||||
const stripped = stripPycSidecarMarshalNote(outBytes);
|
||||
expect(extractPycSidecarMarshalNote(stripped)).toBeNull();
|
||||
expect(importPycResult(stripped)).toBe(5);
|
||||
expect(
|
||||
verifyPycSidecarMarshalNote({
|
||||
entryBytes: stripped,
|
||||
baseDir: dir,
|
||||
}).ok,
|
||||
).toBe(false);
|
||||
unlinkSync(join(dir, "honey.txt"));
|
||||
expect(importPycResult(outBytes)).toBe(5);
|
||||
const afterDelete = verifyPycSidecarMarshalNote({
|
||||
entryBytes: outBytes,
|
||||
baseDir: dir,
|
||||
});
|
||||
expect(afterDelete.ok).toBe(false);
|
||||
expect(afterDelete.reason).toMatch(/missing/i);
|
||||
|
||||
// Honey from protectPythonPycBuffer still present alongside the note.
|
||||
const honey = protectPythonPycBuffer(entry, {
|
||||
intensity: "balanced",
|
||||
|
||||
Reference in New Issue
Block a user