mirror of
https://github.com/qybaihe/bla.git
synced 2026-11-20 08:20:59 +08:00
feat(sidecar): auto-wire GGUF load-time hook into known Python loaders + REA catalog hygiene
When sidecarEntangle is on, protectTarget and the hardenProject post-pass auto-wire a self-contained Python snippet into a same-directory loader.py / __init__.py / __main__.py that already mentions the .gguf basename. The hook verifies the bla.sidecar.v1 STRING metadata note and sibling digest at import; strip or sibling delete fails closed. Tensor infos, relative offsets, and tensor bytes stay bit-exact. Vector gguf-sidecar-entangle is recorded and gguf-sidecar-host-verify-only is cleared only when the hook was actually written. Default remains OFF. Catalogued new REA ceilings on tip c84fde05e (honesty only): FDO_ALLOWANCE, BUNDLE_INFO_PLIST_INPUT_BYTES, INFO_PLIST_DECODE_MIB, MAX_WASM_SECTIONS.
This commit is contained in:
@@ -6,6 +6,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
- Optional GGUF load-time hook: when `sidecarEntangle` is on and a known safe Python loader sits beside the model, stripping the `bla.sidecar.v1` metadata note or a sibling sidecar fails closed at import. Tensor bytes stay bit-exact. Default remains OFF.
|
||||
- REA catalog hygiene on tip `c84fde05e` (honesty only; no new 1:1 densify counters): `FDO_ALLOWANCE`, `BUNDLE_INFO_PLIST_INPUT_BYTES`, `INFO_PLIST_DECODE_MIB`, `MAX_WASM_SECTIONS`.
|
||||
|
||||
## [3.5.1] — 2026-11-18
|
||||
|
||||
Optional `.pyc` import-time hook: when `sidecarEntangle` is on and a known safe Python loader sits beside the module, stripping the marshal note or a sibling sidecar fails closed at import. Default remains OFF.
|
||||
|
||||
+60
@@ -1,5 +1,65 @@
|
||||
# BLA nightly development log
|
||||
|
||||
## 2026-11-19 ~America/Los_Angeles / ~2026-11-20 02:00 Asia/Shanghai
|
||||
|
||||
### REA delta
|
||||
- Pulled `morluto/rea`: `31aaec6b4` → `c84fde05e` (**+345** commits). Dirty
|
||||
`third_party/*` submodule checkouts were left untouched.
|
||||
- No new 1:1 trip-able densify constants. The delta is mostly fix / refactor /
|
||||
test / perf (230 fix / 41 refactor / 22 test / 19 perf / 14 docs / 9 ci /
|
||||
10 feat). Feat commits add inspection capability, not plantable densify
|
||||
targets: FDO packaging/dlopen notes (#7237), Ghidra prototype conflicts
|
||||
(#8161), MIPS32 PIC shared images, RISC-V initializers / PIC, cargo-auditable
|
||||
dependencies from WASM (#5069), DOS COM import, Ghidra 12.2, EM_PPC64
|
||||
classification. New or newly advertised ceilings are admission / decode /
|
||||
format-validation caps, not plantable without breaking the artifact:
|
||||
- `FDO_ALLOWANCE` = 64 KiB (`ElfFdoNote.ts`, #7237) — shared per-report JSON
|
||||
decode cap for ELF FDO packaging / dlopen notes; overflow is left
|
||||
undecoded, and planting dense 64 KiB notes would bloat the ELF.
|
||||
- `BUNDLE_INFO_PLIST_INPUT_BYTES` = 64 MiB and `INFO_PLIST_DECODE_MIB` = 16
|
||||
(`AppBundleExecutable.ts`) — Info.plist admission / decode caps; a 64 MiB
|
||||
property list would bloat or break the app bundle.
|
||||
- `MAX_WASM_SECTIONS` = 65,536 (`RustWasmImage.ts`, #5069) — cargo-auditable
|
||||
WASM walk cap; planting 65,536 sections would bloat or break the module.
|
||||
- Also noted, not catalogued as new trip targets: `MIPS_GNU_ATTRIBUTES_MAX_BYTES`
|
||||
(4096, format validation), PE resource `METADATA_BYTES` (8 MiB),
|
||||
`ASN1_DEPTH_BUDGET` (64, Authenticode nesting), `GUARDIAN_READINESS_BYTES`
|
||||
(4096, JADX workspace). `MAX_VENDOR_HEADER_SIZE` (1 MiB) was already
|
||||
catalogued as `ANDROID_BOOT_MAX_HEADER_SIZE`.
|
||||
- Catalogued the new headline maxima in `REA_INTERNAL_LIMITS` (honesty only).
|
||||
|
||||
### Done tonight
|
||||
- Landed the GGUF load-time hook (`generateSidecarGgufLoaderSnippet` /
|
||||
`prependGgufSidecarLoaderHook` / `entangleGgufLoaderWithSidecars` /
|
||||
`computeGgufSidecarCodeIdentity`; marker `_installGgufSidecarLoaderHook`;
|
||||
`protectTarget` + `hardenProject` post-pass). Auto-wired only into a
|
||||
same-directory Python loader named `loader.py`, `__init__.py`, or
|
||||
`__main__.py` that already mentions the `.gguf` basename
|
||||
(`selectPythonGgufLoader` / `discoverPythonGgufLoader` /
|
||||
`isSafePythonGgufLoader` / `isGgufSidecarLoaderHookWired` /
|
||||
`sourceMentionsGgufFile`). The hook does not rewrite the `.gguf` (tensor
|
||||
infos, relative tensor offsets, and tensor bytes stay bit-exact; the
|
||||
existing alignment-safe `bla.sidecar.v1` STRING metadata note remains the
|
||||
only GGUF mutation). Test / already-entangled / self-seal / unknown helper
|
||||
files are never hooked. Vector `gguf-sidecar-entangle` is recorded and
|
||||
`gguf-sidecar-host-verify-only` is cleared only when that hook was actually
|
||||
written. Default OFF via existing `sidecarEntangle`.
|
||||
- Live BLA tree was pulled `4bb09a6` / `3.5.1` → `1d4d490` (still `3.5.1`,
|
||||
unreleased daytime work already on main: WASM Code-section opaque flower
|
||||
bytecode, decoy-tell scrub, rare-CJK BPE amplification).
|
||||
- Gates: build OK, tests **1296 passed / 5 skipped**, benchmark **14/14**
|
||||
runtime-equivalent.
|
||||
|
||||
### Next step (following night)
|
||||
1. Safetensors / DEX / SQLite load-time hooks (same honesty bar: only record
|
||||
the vector / clear `*-host-verify-only` when a real load-time hook was
|
||||
written). Safetensors Python loaders are the closest remaining analogue.
|
||||
2. Browser WASM loader auto-wire (the hook still needs Node `fs` /
|
||||
`crypto`; a browser snippet would have to verify without those).
|
||||
3. ESM hook that runs before hoisted *user* imports; MCP one-click
|
||||
polish; empirical JADX / Android red-team; ESP #8228; Dependabot majors.
|
||||
|
||||
|
||||
## 2026-11-18 ~America/Los_Angeles / ~2026-11-19 02:00 Asia/Shanghai
|
||||
|
||||
### REA delta
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# DESIGN: Sidecar / entry entanglement (issue #40 / STATE)
|
||||
|
||||
Status: **JS + Python + WASM + Bun companion + ASAR archive-note + ASAR main JS preamble strip⇒fail; .pyc marshal-note + import-time hook (when auto-wired); GGUF STRING metadata note is host-verify only** — binding digest + JS/Python preambles + WASM `bla.sidecar.v1` custom-section note + Bun sibling `.bla-sidecar.v1.json` companion note (byte-exact binary) + ASAR in-archive `.bla/sidecar.v1.json` note + optional Electron main JS preamble (`generateSidecarAsarMainJsPreamble`) + CPython `.pyc` TYPE_ASCII marshal trailer (`generateSidecarPycMarshalNote` / `verifyPycSidecarMarshalNote`, importlib-ignored) + GGUF `bla.sidecar.v1` STRING metadata KV (`generateSidecarGgufMetadataNote` / `verifyGgufSidecarMetadataNote`, alignment-safe rewrite keeps tensor bytes bit-exact; loaders ignore unknown STRING keys) + protect/harden opt-in landed; defenseScore gap when `sidecarEntangle` is off landed; GGUF / Safetensors / DEX / SQLite host-only notes annotate `gguf-sidecar-host-verify-only` / `safetensors-sidecar-host-verify-only` / `dex-sidecar-host-verify-only` / `sqlite-sidecar-host-verify-only` and must not record the process-start vectors; `.pyc` annotates `pyc-sidecar-host-verify-only` until the import-time hook is actually written; optional JS-loader hook `generateSidecarWasmLoaderSnippet` / `prependWasmSidecarLoaderHook` / `entangleWasmLoaderWithSidecars` landed 2026-11-15, auto-wired 2026-11-16 into a same-directory Node CJS loader, and extended 2026-11-17 to a same-directory Node ESM loader with Node-builtin-only static imports (`detectWasmJsLoaderCarrier` / `esmStaticImportsAreNodeBuiltinsOnly`; only then record `wasm-sidecar-entangle` / clear `wasm-sidecar-host-verify-only`); **2026-11-16 honesty:** ESM static `import` / `export-from` of user modules evaluate before the prepended JS IIFE — annotate `esm-import-hoist-before-hooks` (soft-cap 93); do not treat entry-body fail-closed as process-start for the hoisted graph. **2026-11-17 honesty:** Node builtin-only imports are not a user hoisted graph. Optional `.pyc` import-time hook `generateSidecarPycLoaderSnippet` / `prependPycSidecarLoaderHook` / `entanglePycLoaderWithSidecars` landed 2026-11-18, auto-wired into a same-directory (or parent-of-`__pycache__`) Python loader named `loader.py` / `__init__.py` / `__main__.py` that already mentions the module (`selectPythonPycLoader` / `discoverPythonPycLoader` / `isSafePythonPycLoader`; only then record `pyc-sidecar-entangle` / clear `pyc-sidecar-host-verify-only`). The hook does not rewrite the `.pyc`. Remaining: GGUF/DEX/Safetensors/SQLite load-time hooks; browser WASM loader auto-wire; ESM hook that runs before hoisted user imports.
|
||||
Status: **JS + Python + WASM + Bun companion + ASAR archive-note + ASAR main JS preamble strip⇒fail; .pyc marshal-note + import-time hook (when auto-wired); GGUF STRING metadata note + load-time hook (when auto-wired)** — binding digest + JS/Python preambles + WASM `bla.sidecar.v1` custom-section note + Bun sibling `.bla-sidecar.v1.json` companion note (byte-exact binary) + ASAR in-archive `.bla/sidecar.v1.json` note + optional Electron main JS preamble (`generateSidecarAsarMainJsPreamble`) + CPython `.pyc` TYPE_ASCII marshal trailer (`generateSidecarPycMarshalNote` / `verifyPycSidecarMarshalNote`, importlib-ignored) + GGUF `bla.sidecar.v1` STRING metadata KV (`generateSidecarGgufMetadataNote` / `verifyGgufSidecarMetadataNote`, alignment-safe rewrite keeps tensor bytes bit-exact; loaders ignore unknown STRING keys) + protect/harden opt-in landed; defenseScore gap when `sidecarEntangle` is off landed; Safetensors / DEX / SQLite host-only notes annotate `safetensors-sidecar-host-verify-only` / `dex-sidecar-host-verify-only` / `sqlite-sidecar-host-verify-only` and must not record the process-start vectors; `.pyc` annotates `pyc-sidecar-host-verify-only` until the import-time hook is actually written; GGUF annotates `gguf-sidecar-host-verify-only` until the load-time hook is actually written; optional JS-loader hook `generateSidecarWasmLoaderSnippet` / `prependWasmSidecarLoaderHook` / `entangleWasmLoaderWithSidecars` landed 2026-11-15, auto-wired 2026-11-16 into a same-directory Node CJS loader, and extended 2026-11-17 to a same-directory Node ESM loader with Node-builtin-only static imports (`detectWasmJsLoaderCarrier` / `esmStaticImportsAreNodeBuiltinsOnly`; only then record `wasm-sidecar-entangle` / clear `wasm-sidecar-host-verify-only`); **2026-11-16 honesty:** ESM static `import` / `export-from` of user modules evaluate before the prepended JS IIFE — annotate `esm-import-hoist-before-hooks` (soft-cap 93); do not treat entry-body fail-closed as process-start for the hoisted graph. **2026-11-17 honesty:** Node builtin-only imports are not a user hoisted graph. Optional `.pyc` import-time hook `generateSidecarPycLoaderSnippet` / `prependPycSidecarLoaderHook` / `entanglePycLoaderWithSidecars` landed 2026-11-18, auto-wired into a same-directory (or parent-of-`__pycache__`) Python loader named `loader.py` / `__init__.py` / `__main__.py` that already mentions the module (`selectPythonPycLoader` / `discoverPythonPycLoader` / `isSafePythonPycLoader`; only then record `pyc-sidecar-entangle` / clear `pyc-sidecar-host-verify-only`). The hook does not rewrite the `.pyc`. Optional GGUF load-time hook `generateSidecarGgufLoaderSnippet` / `prependGgufSidecarLoaderHook` / `entangleGgufLoaderWithSidecars` landed 2026-11-19, auto-wired into a same-directory Python loader named `loader.py` / `__init__.py` / `__main__.py` that already mentions the `.gguf` basename (`selectPythonGgufLoader` / `discoverPythonGgufLoader` / `isSafePythonGgufLoader`; only then record `gguf-sidecar-entangle` / clear `gguf-sidecar-host-verify-only`). The hook does not rewrite tensor bytes, tensor infos, or relative offsets. Remaining: Safetensors/DEX/SQLite load-time hooks; browser WASM loader auto-wire; ESM hook that runs before hoisted user imports.
|
||||
|
||||
## Goal
|
||||
|
||||
@@ -47,9 +47,9 @@ checks beat orphan honeypot files.
|
||||
- ~~Extend to ASAR archive-note companion~~ — 2026-11-09 nightly: `generateSidecarAsarArchiveNote` / `bindAsarSidecarEntanglement` / `entangleAsarEntryWithSidecars` / `verifyAsarSidecarArchiveNote`; protect + harden ASAR post-pass plant `.bla/sidecar.v1.json` binding `enclave/hardware-tee-bridge.js`.
|
||||
- ~~ASAR main JS preamble (Electron fs-patched re-measure)~~ — 2026-11-10 nightly: `generateSidecarAsarMainJsPreamble` / `selectAsarMainJsMember` / `entangleAsarMainJsWithSidecars`; protect + harden prepend strip⇒fail preamble to package.json#main (or heuristic entry).
|
||||
- ~~CPython `.pyc` marshal trailer note (host verify)~~ — 2026-11-11 nightly: `generateSidecarPycMarshalNote` / `bindPycSidecarEntanglement` / `entanglePycEntryWithSidecars` / `verifyPycSidecarMarshalNote`; TYPE_ASCII trailer after root (importlib-safe). ~~Import-time hook~~ — 2026-11-18 nightly: `generateSidecarPycLoaderSnippet` / `prependPycSidecarLoaderHook` / `entanglePycLoaderWithSidecars`; auto-wired into `loader.py` / `__init__.py` / `__main__.py` (never the `.py` source of the `.pyc`). **Honesty:** record `pyc-sidecar-entangle` / clear `pyc-sidecar-host-verify-only` only when the hook was actually written. Default OFF.
|
||||
- ~~GGUF STRING metadata note (host verify)~~ — 2026-11-12 nightly: `generateSidecarGgufMetadataNote` / `bindGgufSidecarEntanglement` / `entangleGgufEntryWithSidecars` / `verifyGgufSidecarMetadataNote`; `bla.sidecar.v1` STRING KV via alignment-safe `rebuildGguf` (tensor bytes bit-exact). **Honesty:** do not record `gguf-sidecar-entangle` until a load-time hook lands; defenseScore gap `gguf-sidecar-host-verify-only`. Also fixed `protectGgufBuffer` to use the same rewriter so tensor-bearing GGUFs no longer silently shift the data section. Default OFF. Remaining: GGUF/DEX/Safetensors/SQLite load-time hooks.
|
||||
- ~~GGUF STRING metadata note (host verify)~~ — 2026-11-12 nightly: `generateSidecarGgufMetadataNote` / `bindGgufSidecarEntanglement` / `entangleGgufEntryWithSidecars` / `verifyGgufSidecarMetadataNote`; `bla.sidecar.v1` STRING KV via alignment-safe `rebuildGguf` (tensor bytes bit-exact). Also fixed `protectGgufBuffer` to use the same rewriter so tensor-bearing GGUFs no longer silently shift the data section. ~~Load-time hook~~ — 2026-11-19 nightly: `generateSidecarGgufLoaderSnippet` / `prependGgufSidecarLoaderHook` / `entangleGgufLoaderWithSidecars`; auto-wired into a same-directory `loader.py` / `__init__.py` / `__main__.py` that already mentions the `.gguf` basename. **Honesty:** record `gguf-sidecar-entangle` / clear `gguf-sidecar-host-verify-only` only when the hook was actually written. The hook does not rewrite tensor bytes, tensor infos, or relative offsets. Default OFF.
|
||||
- ~~JS runtime carrier honesty~~ — 2026-11-13: `.mjs`, `import`/`export`, top-level `await`, `import.meta`, and `.js` under nearest `package.json` `"type": "module"` are detected via Babel `program.sourceType` + `refineJsCarrierForRuntimePaths` and now receive an ESM-native strip⇒fail preamble (`import { createRequire } from "node:module"` + `import.meta.url`), with matching ESM `self-seal` support.
|
||||
- **ESM import-hoist honesty** — 2026-11-16: the ESM-native preamble fail-closes the entry body, but static `import` / `export-from` bindings are instantiated first, so imported module side effects still run after sidecar strip or hook failure (`jsModuleHasHoistedDependencyGraph`; the BLA `createRequire` preamble import is ignored). Annotate defenseScore gap `esm-import-hoist-before-hooks` (soft-cap 93) when sidecar or self-seal is on. CJS `require()` after the preamble remains process-start. A true pre-import hook (custom ESM loader / `module.register` before the graph loads) is still TBD.
|
||||
- ~~Safetensors `__metadata__` note (host verify)~~ — 2026-11-13: `generateSidecarSafetensorsMetadataNote` / `bindSafetensorsSidecarEntanglement` / `entangleSafetensorsEntryWithSidecars` / `verifySafetensorsSidecarMetadataNote`; `bla.sidecar.v1` `__metadata__` key keeps tensor bytes bit-exact; `defenseScore` annotates `safetensors-sidecar-host-verify-only` and `wasm-sidecar-host-verify-only` (soft-cap 93).
|
||||
- ~~DEX string-pool note (host verify)~~ — 2026-11-13 nightly: `generateSidecarDexStringPoolNote` / `bindDexSidecarEntanglement` / `entangleDexEntryWithSidecars` / `verifyDexSidecarStringPoolNote`; unreferenced sort-last `bla.sidecar.v1=` pool string via `parseDexLayout` / `rebuildDex` (existing string indices + instruction bytes unchanged; trailers preserved). **Honesty:** do not record `dex-sidecar-entangle` until a load-time hook lands; defenseScore gap `dex-sidecar-host-verify-only`. Default OFF.
|
||||
- ~~SQLite schema-comment / reserved-VIEW note (host verify)~~ — 2026-11-14 nightly: `generateSidecarSqliteSchemaComment` / `bindSqliteSidecarEntanglement` / `entangleSqliteEntryWithSidecars` / `verifySqliteSidecarSchemaComment`; unused `bla_sidecar_v1` VIEW (`SELECT '<json>' AS "bla.sidecar.v1" WHERE 0`) keeps user tables, rows, query results, WAL/FK, and application_id/user_version unchanged. **Honesty:** do not record `sqlite-sidecar-entangle` until a load-time hook lands; defenseScore gap `sqlite-sidecar-host-verify-only`. Default OFF. Remaining: GGUF/DEX/Safetensors/SQLite load-time hooks.
|
||||
- ~~SQLite schema-comment / reserved-VIEW note (host verify)~~ — 2026-11-14 nightly: `generateSidecarSqliteSchemaComment` / `bindSqliteSidecarEntanglement` / `entangleSqliteEntryWithSidecars` / `verifySqliteSidecarSchemaComment`; unused `bla_sidecar_v1` VIEW (`SELECT '<json>' AS "bla.sidecar.v1" WHERE 0`) keeps user tables, rows, query results, WAL/FK, and application_id/user_version unchanged. **Honesty:** do not record `sqlite-sidecar-entangle` until a load-time hook lands; defenseScore gap `sqlite-sidecar-host-verify-only`. Default OFF. Remaining: Safetensors/DEX/SQLite load-time hooks.
|
||||
|
||||
@@ -289,6 +289,8 @@ export interface ProtectionResult {
|
||||
readonly wasmSidecarLoaderHookApplied?: boolean;
|
||||
/** Present when a same-directory Python loader was auto-wired with the `.pyc` import-time hook. */
|
||||
readonly pycSidecarLoaderHookApplied?: boolean;
|
||||
/** Present when a same-directory Python loader was auto-wired with the GGUF load-time hook. */
|
||||
readonly ggufSidecarLoaderHookApplied?: boolean;
|
||||
/** Present when WASM export rename actually rewrote at least one non-preserved export (#40.3). */
|
||||
readonly wasmExportRenameApplied?: boolean;
|
||||
/**
|
||||
|
||||
@@ -67,6 +67,12 @@ export interface DefenseScoreHonestyInput {
|
||||
* the .pyc). Clears `pyc-sidecar-host-verify-only`.
|
||||
*/
|
||||
readonly pycSidecarLoaderHookApplied?: boolean;
|
||||
/**
|
||||
* True when a same-directory Python loader was auto-wired with
|
||||
* `entangleGgufLoaderWithSidecars` (or the hook is already present next to
|
||||
* the .gguf). Clears `gguf-sidecar-host-verify-only`.
|
||||
*/
|
||||
readonly ggufSidecarLoaderHookApplied?: boolean;
|
||||
/**
|
||||
* How a SQLite database's generated decoy columns are defined: `"eval"`
|
||||
* when they call the function no stock connection has (opt-in), otherwise
|
||||
@@ -170,12 +176,18 @@ export function collectDefenseScoreGaps(
|
||||
});
|
||||
}
|
||||
|
||||
// GGUF STRING KV is host-verify only (loaders ignore unknown metadata keys).
|
||||
if (input.detectedFormat === "model-gguf" && input.sidecarEntangleEnabled === true) {
|
||||
// GGUF STRING KV is host-verify only (loaders ignore unknown metadata keys)
|
||||
// until a same-directory Python load-time hook is actually auto-wired (or
|
||||
// already present) beside the model.
|
||||
if (
|
||||
input.detectedFormat === "model-gguf" &&
|
||||
input.sidecarEntangleEnabled === true &&
|
||||
input.ggufSidecarLoaderHookApplied !== true
|
||||
) {
|
||||
gaps.push({
|
||||
id: "gguf-sidecar-host-verify-only",
|
||||
summary:
|
||||
"sidecarEntangle planted a GGUF bla.sidecar.v1 STRING metadata note verified only by host verifyGgufSidecarMetadataNote; GGUF loaders still load after strip or sibling delete. Load-time hook TBD — do not treat this as process-start strip⇒fail (issue #40 / DESIGN).",
|
||||
"sidecarEntangle planted a GGUF bla.sidecar.v1 STRING metadata note verified by host verifyGgufSidecarMetadataNote; GGUF loaders still load after strip or sibling delete unless a same-directory Python loader (loader.py / __init__.py / __main__.py) was auto-wired with the load-time hook (issue #40 / DESIGN).",
|
||||
scoreCap: 93,
|
||||
});
|
||||
}
|
||||
|
||||
+116
-1
@@ -95,6 +95,13 @@ import {
|
||||
extractPycSidecarMarshalNote,
|
||||
relativePycPathFromLoader,
|
||||
PYC_SIDECAR_LOADER_MARKER,
|
||||
generateSidecarGgufLoaderSnippet,
|
||||
prependGgufSidecarLoaderHook,
|
||||
selectPythonGgufLoader,
|
||||
computeGgufSidecarCodeIdentity,
|
||||
extractGgufSidecarMetadataNote,
|
||||
relativeGgufPathFromLoader,
|
||||
GGUF_SIDECAR_LOADER_MARKER,
|
||||
verifyAsarSidecarArchiveNote,
|
||||
} from "../shields/runtime/sidecar-entangle.js";
|
||||
import { packAsarBuffer, unpackAsarBuffer } from "../shields/formats/asar.js";
|
||||
@@ -3102,6 +3109,114 @@ function hardenProjectCore(
|
||||
// protectTarget honesty; the loader rewrite above is the act.
|
||||
}
|
||||
|
||||
const ggufPlantedAbs = new Set<string>();
|
||||
const ggufLikeEarly = bindableSummaries.filter(
|
||||
(s) =>
|
||||
s.success &&
|
||||
(s.format === "model-gguf" ||
|
||||
/\.gguf$/i.test(s.relativePath.replaceAll("\\", "/"))),
|
||||
);
|
||||
for (const ggufSum of ggufLikeEarly) {
|
||||
const ggufAbs = join(outputRoot, ggufSum.relativePath);
|
||||
if (!existsSync(ggufAbs)) continue;
|
||||
const ggufDir = dirname(ggufAbs);
|
||||
const searchDirs = [ggufDir];
|
||||
const loaderCandidates: {
|
||||
name: string;
|
||||
source: string;
|
||||
absPath: string;
|
||||
}[] = [];
|
||||
for (const s of pyLike) {
|
||||
const outAbs = join(outputRoot, s.relativePath);
|
||||
if (!searchDirs.includes(dirname(outAbs))) continue;
|
||||
if (!isPythonPycLoaderFilename(s.relativePath)) continue;
|
||||
let predicateSource: string | null = null;
|
||||
try {
|
||||
if (s.inputPath && existsSync(s.inputPath)) {
|
||||
predicateSource = readFileSync(s.inputPath, "utf8");
|
||||
}
|
||||
} catch {
|
||||
predicateSource = null;
|
||||
}
|
||||
if (predicateSource === null && existsSync(outAbs)) {
|
||||
try {
|
||||
predicateSource = readFileSync(outAbs, "utf8");
|
||||
} catch {
|
||||
predicateSource = null;
|
||||
}
|
||||
}
|
||||
if (predicateSource === null) continue;
|
||||
loaderCandidates.push({
|
||||
name: basename(outAbs),
|
||||
source: predicateSource,
|
||||
absPath: outAbs,
|
||||
});
|
||||
}
|
||||
const selectedName = selectPythonGgufLoader(
|
||||
basename(ggufAbs),
|
||||
loaderCandidates,
|
||||
);
|
||||
const found = selectedName
|
||||
? loaderCandidates.find((c) => c.name === selectedName)
|
||||
: undefined;
|
||||
if (!found) continue;
|
||||
const alreadyNoted =
|
||||
extractGgufSidecarMetadataNote(readFileSync(ggufAbs)) !== null;
|
||||
const otherPrimary =
|
||||
(jsEntryAbs !== undefined && found.absPath !== jsEntryAbs) ||
|
||||
(pyEntryAbs !== undefined && found.absPath !== pyEntryAbs);
|
||||
// A foreign JS/Python entry may already bake a digest of this
|
||||
// model. Hooking without a note would fail-closed at import;
|
||||
// planting a new metadata KV would break that digest.
|
||||
if (otherPrimary && !alreadyNoted) continue;
|
||||
|
||||
if (!otherPrimary) {
|
||||
const sidecars: { pathLabel: string; bytes: Buffer }[] = [];
|
||||
for (const s of bindableSummaries) {
|
||||
if (!s.success || s.relativePath === ggufSum.relativePath) continue;
|
||||
const abs = join(outputRoot, s.relativePath);
|
||||
if (jsEntryAbs && abs === jsEntryAbs) continue;
|
||||
if (pyEntryAbs && abs === pyEntryAbs) continue;
|
||||
if (abs === found.absPath) continue;
|
||||
const rel = relative(ggufDir, abs).split(sep).join("/");
|
||||
if (rel.length === 0 || rel.startsWith("..") || !existsSync(abs)) continue;
|
||||
sidecars.push({ pathLabel: rel, bytes: readFileSync(abs) });
|
||||
}
|
||||
const bound = entangleGgufEntryWithSidecars({
|
||||
entryPath: basename(ggufAbs),
|
||||
entryBytes: readFileSync(ggufAbs),
|
||||
sidecars,
|
||||
seedTag: derivePublicSidecarSeedTag(String(options.seed ?? runId)),
|
||||
allowEmptySidecars: true,
|
||||
});
|
||||
if (bound) {
|
||||
writeFileAtomicSync(ggufAbs, bound.bytes, undefined, {
|
||||
allowLinkWriteThrough: false,
|
||||
});
|
||||
ggufPlantedAbs.add(ggufAbs);
|
||||
}
|
||||
}
|
||||
|
||||
const identity = computeGgufSidecarCodeIdentity(readFileSync(ggufAbs));
|
||||
const existing = readFileSync(found.absPath, "utf8");
|
||||
const already = existing.includes(GGUF_SIDECAR_LOADER_MARKER);
|
||||
const ggufRel = relativeGgufPathFromLoader(found.absPath, ggufAbs);
|
||||
const inject = already
|
||||
? null
|
||||
: generateSidecarGgufLoaderSnippet({
|
||||
codeIdentityHex: identity,
|
||||
ggufRelPaths: [ggufRel],
|
||||
});
|
||||
const next = inject
|
||||
? prependGgufSidecarLoaderHook(existing, inject.preamble)
|
||||
: existing;
|
||||
if (next !== existing) {
|
||||
writeFileAtomicSync(found.absPath, next, "utf8", {
|
||||
allowLinkWriteThrough: false,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (jsEntry) {
|
||||
const entryAbs = join(outputRoot, jsEntry.relativePath);
|
||||
if (existsSync(entryAbs)) {
|
||||
@@ -3289,7 +3404,7 @@ function hardenProjectCore(
|
||||
: undefined;
|
||||
if (ggufEntry) {
|
||||
const entryAbs = join(outputRoot, ggufEntry.relativePath);
|
||||
if (existsSync(entryAbs)) {
|
||||
if (existsSync(entryAbs) && !ggufPlantedAbs.has(entryAbs)) {
|
||||
const entryDir = dirname(entryAbs);
|
||||
const sidecars: { pathLabel: string; bytes: Buffer }[] = [];
|
||||
for (const s of bindableSummaries) {
|
||||
|
||||
+71
-4
@@ -116,6 +116,11 @@ import {
|
||||
listPycLoaderSearchDirectories,
|
||||
relativePycPathFromLoader,
|
||||
PYC_SIDECAR_LOADER_MARKER,
|
||||
entangleGgufLoaderWithSidecars,
|
||||
discoverPythonGgufLoader,
|
||||
listGgufLoaderSearchDirectories,
|
||||
relativeGgufPathFromLoader,
|
||||
GGUF_SIDECAR_LOADER_MARKER,
|
||||
} from "../shields/runtime/sidecar-entangle.js";
|
||||
|
||||
/**
|
||||
@@ -455,6 +460,59 @@ function applySidecarEntangleToGguf(
|
||||
return { buffer: bound.bytes, applied: true };
|
||||
}
|
||||
|
||||
function applySidecarGgufLoaderAutoWire(options: {
|
||||
readonly ggufBytes: Uint8Array;
|
||||
readonly ggufBasenames: readonly string[];
|
||||
readonly outputDir: string;
|
||||
readonly sourceDir: string;
|
||||
readonly outputGgufPath: string;
|
||||
}): { applied: boolean; destPath?: string } {
|
||||
const search: string[] = [];
|
||||
const seen = new Set<string>();
|
||||
const addDir = (dir: string): void => {
|
||||
if (!dir || seen.has(dir)) return;
|
||||
seen.add(dir);
|
||||
search.push(dir);
|
||||
};
|
||||
addDir(options.outputDir);
|
||||
if (options.sourceDir !== options.outputDir) addDir(options.sourceDir);
|
||||
for (const extra of listGgufLoaderSearchDirectories(options.outputGgufPath)) {
|
||||
addDir(extra);
|
||||
}
|
||||
|
||||
const names = [...new Set(options.ggufBasenames.filter((n) => /\.gguf$/i.test(n)))];
|
||||
let found: ReturnType<typeof discoverPythonGgufLoader> = null;
|
||||
for (const ggufBasename of names) {
|
||||
found = discoverPythonGgufLoader({
|
||||
ggufBasename,
|
||||
searchDirectories: search,
|
||||
});
|
||||
if (found) break;
|
||||
}
|
||||
if (!found) return { applied: false };
|
||||
let destPath = found.absPath;
|
||||
const foundDir = dirname(found.absPath);
|
||||
if (foundDir !== options.outputDir && options.sourceDir !== options.outputDir) {
|
||||
destPath = join(options.outputDir, found.name);
|
||||
if (existsSync(destPath) && destPath !== found.absPath) {
|
||||
return { applied: false };
|
||||
}
|
||||
}
|
||||
const ggufRel = relativeGgufPathFromLoader(destPath, options.outputGgufPath);
|
||||
const entangled = entangleGgufLoaderWithSidecars({
|
||||
loaderSource: found.source,
|
||||
ggufBytes: options.ggufBytes,
|
||||
ggufRelPath: ggufRel,
|
||||
});
|
||||
if (entangled.code === found.source && destPath === found.absPath) {
|
||||
return found.source.includes(GGUF_SIDECAR_LOADER_MARKER)
|
||||
? { applied: true, destPath }
|
||||
: { applied: false };
|
||||
}
|
||||
writeFileAtomicSync(destPath, entangled.code, "utf8");
|
||||
return { applied: true, destPath };
|
||||
}
|
||||
|
||||
function applySidecarEntangleToSafetensors(
|
||||
safetensorsBytes: Buffer,
|
||||
options: ProtectionOptions,
|
||||
@@ -799,6 +857,7 @@ function protectTargetInner(options: ProtectionOptions): ProtectionResult {
|
||||
let sidecarEntangleApplied = false;
|
||||
let wasmSidecarLoaderHookApplied = false;
|
||||
let pycSidecarLoaderHookApplied = false;
|
||||
let ggufSidecarLoaderHookApplied = false;
|
||||
const counteredVectors = new Set<ReaAttackVectorId>();
|
||||
|
||||
if (
|
||||
@@ -1446,10 +1505,17 @@ function protectTargetInner(options: ProtectionOptions): ProtectionResult {
|
||||
finalGguf = entangled.buffer;
|
||||
if (entangled.applied) {
|
||||
sidecarEntangleApplied = true;
|
||||
// Host-verify STRING KV only: GGUF loaders ignore unknown metadata
|
||||
// keys, so strip / sibling delete still load. Do not record
|
||||
// gguf-sidecar-entangle until a load-time hook lands (same honesty
|
||||
// bar as .pyc marshal trailer / ASAR archive-note-only).
|
||||
const hook = applySidecarGgufLoaderAutoWire({
|
||||
ggufBytes: finalGguf,
|
||||
ggufBasenames: [basename(absInput), basename(absOutput)],
|
||||
outputDir: dirname(absOutput),
|
||||
sourceDir: dirname(absInput),
|
||||
outputGgufPath: absOutput,
|
||||
});
|
||||
if (hook.applied) {
|
||||
ggufSidecarLoaderHookApplied = true;
|
||||
counteredVectors.add("gguf-sidecar-entangle");
|
||||
}
|
||||
}
|
||||
}
|
||||
writeProtectedOutput(absOutput, finalGguf);
|
||||
@@ -1720,6 +1786,7 @@ function protectTargetInner(options: ProtectionOptions): ProtectionResult {
|
||||
...(sidecarEntangleApplied ? { sidecarEntangle: true as const } : {}),
|
||||
...(wasmSidecarLoaderHookApplied ? { wasmSidecarLoaderHookApplied: true as const } : {}),
|
||||
...(pycSidecarLoaderHookApplied ? { pycSidecarLoaderHookApplied: true as const } : {}),
|
||||
...(ggufSidecarLoaderHookApplied ? { ggufSidecarLoaderHookApplied: true as const } : {}),
|
||||
...(wasmExportsRenamed > 0 ? { wasmExportRenameApplied: true as const } : {}),
|
||||
...(wasmExportRenames.length > 0 ? { wasmExportRenames } : {}),
|
||||
...(wasmExportOrderShuffled ? { wasmExportOrderShuffled: true as const } : {}),
|
||||
|
||||
+14
-1
@@ -15,7 +15,7 @@ import { inspectNativeBinary } from "../shields/formats/native.js";
|
||||
import { inspectSqliteBuffer } from "../shields/formats/sqlite.js";
|
||||
import { verifySelfSealSelfDigest } from "../shields/runtime/self-seal.js";
|
||||
import { jsModuleHasHoistedDependencyGraph } from "../shields/javascript/ast-shield.js";
|
||||
import { isWasmSidecarLoaderHookWired, isPycSidecarLoaderHookWired } from "../shields/runtime/sidecar-entangle.js";
|
||||
import { isWasmSidecarLoaderHookWired, isPycSidecarLoaderHookWired, isGgufSidecarLoaderHookWired } from "../shields/runtime/sidecar-entangle.js";
|
||||
|
||||
export interface VerifyTargetOptions {
|
||||
readonly selfSealEnabled?: boolean;
|
||||
@@ -194,6 +194,18 @@ export function verifyTarget(
|
||||
}
|
||||
}
|
||||
|
||||
let ggufSidecarLoaderHookApplied = false;
|
||||
if (afterAudit.detectedFormat === "model-gguf") {
|
||||
try {
|
||||
ggufSidecarLoaderHookApplied = isGgufSidecarLoaderHookWired({
|
||||
ggufBytes: readFileSync(absTarget),
|
||||
ggufPath: absTarget,
|
||||
});
|
||||
} catch {
|
||||
ggufSidecarLoaderHookApplied = false;
|
||||
}
|
||||
}
|
||||
|
||||
let esmImportHoistBeforeHooks = false;
|
||||
if (
|
||||
afterAudit.detectedFormat === "javascript" ||
|
||||
@@ -231,6 +243,7 @@ export function verifyTarget(
|
||||
...(esmImportHoistBeforeHooks ? { esmImportHoistBeforeHooks: true } : {}),
|
||||
...(wasmSidecarLoaderHookApplied ? { wasmSidecarLoaderHookApplied: true } : {}),
|
||||
...(pycSidecarLoaderHookApplied ? { pycSidecarLoaderHookApplied: true } : {}),
|
||||
...(ggufSidecarLoaderHookApplied ? { ggufSidecarLoaderHookApplied: true } : {}),
|
||||
});
|
||||
|
||||
const agentActions: VerificationAgentAction[] = [];
|
||||
|
||||
@@ -354,4 +354,33 @@ export const REA_INTERNAL_LIMITS = Object.freeze({
|
||||
*/
|
||||
ANDROID_PACKED_TABLE_BYTES: 16_777_216,
|
||||
MAX_PE_OPTIONAL_HEADER_BYTES: 4_096,
|
||||
/**
|
||||
* New REA ceilings on tip c84fde05e (345 commits after 31aaec6b4; 230 fix /
|
||||
* 41 refactor / 22 test / 19 perf / 14 docs / 9 ci / 10 feat). The feat
|
||||
* commits add inspection capability (FDO packaging/dlopen notes #7237,
|
||||
* Ghidra prototype conflicts #8161, MIPS32 PIC shared images, RISC-V
|
||||
* initializers / PIC, cargo-auditable dependencies from WASM #5069,
|
||||
* DOS COM import, Ghidra 12.2, EM_PPC64 classification) — not plantable
|
||||
* densify trip targets. Honesty catalog only:
|
||||
*
|
||||
* - `FDO_ALLOWANCE` (64 KiB) is a shared per-report JSON decode cap for
|
||||
* ELF FDO packaging / dlopen notes (`ElfFdoNote.ts`, #7237). Planting
|
||||
* dense 64 KiB notes would bloat the ELF; overflow is left undecoded.
|
||||
* - `BUNDLE_INFO_PLIST_INPUT_BYTES` (64 MiB) and `INFO_PLIST_DECODE_MIB`
|
||||
* (16) cap Info.plist admission / decode (`AppBundleExecutable.ts`).
|
||||
* A 64 MiB property list would bloat or break the app bundle.
|
||||
* - `MAX_WASM_SECTIONS` (65,536) caps cargo-auditable WASM walks
|
||||
* (`RustWasmImage.ts`, #5069). Planting 65,536 sections would bloat
|
||||
* or break the module.
|
||||
* - Also noted, not catalogued as new trip targets: `MIPS_GNU_ATTRIBUTES_MAX_BYTES`
|
||||
* (4096, format validation), PE resource `METADATA_BYTES` (8 MiB),
|
||||
* `ASN1_DEPTH_BUDGET` (64, Authenticode nesting — planting would
|
||||
* invalidate the certificate), `GUARDIAN_READINESS_BYTES` (4096, JADX
|
||||
* workspace, not plantable). `MAX_VENDOR_HEADER_SIZE` (1 MiB) was
|
||||
* already catalogued as `ANDROID_BOOT_MAX_HEADER_SIZE`.
|
||||
*/
|
||||
FDO_ALLOWANCE: 65_536,
|
||||
BUNDLE_INFO_PLIST_INPUT_BYTES: 64 * 1024 * 1024,
|
||||
INFO_PLIST_DECODE_MIB: 16,
|
||||
MAX_WASM_SECTIONS: 65_536,
|
||||
});
|
||||
|
||||
@@ -557,6 +557,19 @@ export {
|
||||
pycModuleStemFromBasename,
|
||||
relativePycPathFromLoader,
|
||||
PYC_SIDECAR_LOADER_MARKER,
|
||||
generateSidecarGgufLoaderSnippet,
|
||||
prependGgufSidecarLoaderHook,
|
||||
entangleGgufLoaderWithSidecars,
|
||||
computeGgufSidecarCodeIdentity,
|
||||
sourceMentionsGgufFile,
|
||||
isSafePythonGgufLoader,
|
||||
selectPythonGgufLoader,
|
||||
listSameDirPythonGgufLoaderCandidates,
|
||||
listGgufLoaderSearchDirectories,
|
||||
discoverPythonGgufLoader,
|
||||
isGgufSidecarLoaderHookWired,
|
||||
relativeGgufPathFromLoader,
|
||||
GGUF_SIDECAR_LOADER_MARKER,
|
||||
generateSidecarGgufMetadataNote,
|
||||
asarMemberPathFromMain,
|
||||
selectAsarMainJsMember,
|
||||
|
||||
@@ -3419,7 +3419,7 @@ const MAX_PYC_LOADER_CANDIDATE_BYTES = 2 * 1024 * 1024;
|
||||
const MAX_PYC_LOADER_AUTOWIRE_CHARS = 64 * 1024;
|
||||
const PYC_LOADER_EXTENSIONS = [".py"] as const;
|
||||
const ALREADY_ENTANGLED_PY_RE =
|
||||
/bla-self-seal\/v1|_verify_sibling_digest_|integrity binding mismatch|_installPycSidecarLoaderHook|_installWasmSidecarLoaderHook/;
|
||||
/bla-self-seal\/v1|_verify_sibling_digest_|integrity binding mismatch|_installPycSidecarLoaderHook|_installWasmSidecarLoaderHook|_installGgufSidecarLoaderHook/;
|
||||
|
||||
/** Module stem of a CPython cache filename (`sample.cpython-312.opt-1.pyc` → `sample`). */
|
||||
export function pycModuleStemFromBasename(filename: string): string {
|
||||
@@ -3794,7 +3794,7 @@ export function bindGgufSidecarEntanglement(
|
||||
...plan,
|
||||
entryHookKind: "gguf-metadata-note",
|
||||
status: "bound",
|
||||
note: "Bound — GGUF bla.sidecar.v1 STRING metadata note planted; ship sidecars alongside and verify via verifyGgufSidecarMetadataNote. Host verify fails closed on strip/alter; GGUF loaders ignore unknown STRING KVs (no process-start fail until a load-time hook lands). Tensor bytes stay bit-exact.",
|
||||
note: "Bound — GGUF bla.sidecar.v1 STRING metadata note planted; ship sidecars alongside and verify via verifyGgufSidecarMetadataNote. Host verify fails closed on strip/alter; GGUF loaders ignore unknown STRING KVs unless a same-directory Python loader was auto-wired with the load-time hook. Tensor bytes stay bit-exact.",
|
||||
};
|
||||
return {
|
||||
plan: boundPlan,
|
||||
@@ -3814,8 +3814,9 @@ export function entangleGgufEntryWithSidecars(options: {
|
||||
readonly sidecars: readonly SidecarBindingPart[];
|
||||
readonly seedTag?: string;
|
||||
readonly softFold?: boolean;
|
||||
readonly allowEmptySidecars?: boolean;
|
||||
}): SidecarGgufBindResult | null {
|
||||
if (options.sidecars.length === 0) return null;
|
||||
if (options.sidecars.length === 0 && options.allowEmptySidecars !== true) return null;
|
||||
return bindGgufSidecarEntanglement({
|
||||
entryPath: options.entryPath,
|
||||
sidecarPaths: options.sidecars.map((s) => s.pathLabel),
|
||||
@@ -3912,8 +3913,9 @@ export function stripGgufSidecarMetadataNote(entryBytes: Uint8Array): Buffer {
|
||||
* digest. Missing note / missing sibling / altered sibling ⇒ ok:false.
|
||||
*
|
||||
* DEFENSIVE ONLY — for tests, loaders, and harden post-pass verification.
|
||||
* GGUF loaders ignore unknown STRING KVs, so strip does not break load until
|
||||
* a load-time hook lands (same honesty bar as .pyc marshal trailer).
|
||||
* GGUF loaders ignore unknown STRING KVs, so strip does not break load unless
|
||||
* a same-directory Python loader was auto-wired with the load-time hook
|
||||
* (same honesty bar as the .pyc import-time hook).
|
||||
*/
|
||||
export function verifyGgufSidecarMetadataNote(
|
||||
options: GgufSidecarVerifyOptions,
|
||||
@@ -3971,6 +3973,552 @@ export function verifyGgufSidecarMetadataNote(
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* SHA-256 of a GGUF container with the `bla.sidecar.v1` STRING KV stripped
|
||||
* (alignment-safe rebuild). Files that do not carry the note hash as-is, so
|
||||
* the identity is stable across planting for canonically padded inputs.
|
||||
*
|
||||
* DEFENSIVE ONLY.
|
||||
*/
|
||||
export function computeGgufSidecarCodeIdentity(bytes: Uint8Array): string {
|
||||
let stripped: Buffer;
|
||||
try {
|
||||
stripped = stripGgufSidecarMetadataNote(bytes);
|
||||
} catch {
|
||||
stripped = Buffer.from(bytes);
|
||||
}
|
||||
return createHash("sha256").update(stripped).digest("hex");
|
||||
}
|
||||
|
||||
/** Idempotence marker inserted with the optional GGUF load-time hook. */
|
||||
export const GGUF_SIDECAR_LOADER_MARKER = "_installGgufSidecarLoaderHook" as const;
|
||||
|
||||
export interface SidecarGgufLoaderInjectOptions {
|
||||
/**
|
||||
* SHA-256 hex (or list) of the protected GGUF with the sidecar metadata
|
||||
* note stripped. Files whose identity matches MUST present a valid
|
||||
* `bla.sidecar.v1` STRING KV; other GGUF files pass through unchanged.
|
||||
*/
|
||||
readonly codeIdentityHex: string | readonly string[];
|
||||
/**
|
||||
* Paths to the protected `.gguf`, relative to the loader file's directory
|
||||
* (`Path(__file__).resolve().parent`). Sidecar paths inside the note are
|
||||
* resolved against the `.gguf` file's own directory.
|
||||
*/
|
||||
readonly ggufRelPaths: readonly string[];
|
||||
readonly bindingName?: string;
|
||||
}
|
||||
|
||||
export interface SidecarGgufLoaderInjectResult {
|
||||
readonly schema: typeof SIDECAR_ENTANGLE_SCHEMA;
|
||||
readonly preamble: string;
|
||||
readonly bindingName: string;
|
||||
readonly codeIdentityHexes: readonly string[];
|
||||
readonly ggufRelPaths: readonly string[];
|
||||
}
|
||||
|
||||
function normalizeGgufCodeIdentity(hex: string): string {
|
||||
const value = hex.toLowerCase();
|
||||
if (!/^[0-9a-f]{64}$/.test(value)) {
|
||||
throw new Error("codeIdentityHex must be a 64-char lowercase hex SHA-256 digest");
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
/**
|
||||
* Emit a self-contained host Python snippet that, at import time of the
|
||||
* loader it is inserted into, reads the sibling `.gguf`, and when the
|
||||
* stripped-body identity matches, verifies the `bla.sidecar.v1` STRING
|
||||
* metadata note and sibling sidecar digest. Strip or sibling delete fails
|
||||
* closed. Unrelated GGUF files (identity mismatch) and a missing file pass
|
||||
* through so a later open can fail naturally.
|
||||
*
|
||||
* The hook does not rewrite the `.gguf` (tensor infos, relative tensor
|
||||
* offsets, and tensor bytes stay untouched; the existing alignment-safe
|
||||
* metadata note is the only GGUF mutation, and it is planted separately).
|
||||
* It is host-opt-in: `protectTarget` / `hardenProject` auto-wire it only
|
||||
* into a known same-directory Python loader (`loader.py`, `__init__.py`,
|
||||
* `__main__.py`) that already mentions the `.gguf` basename — never a
|
||||
* test / already-entangled / self-seal file. Only then is
|
||||
* `gguf-sidecar-entangle` recorded and `gguf-sidecar-host-verify-only`
|
||||
* cleared.
|
||||
*
|
||||
* DEFENSIVE ONLY. Opt-in. No crack / RE advice.
|
||||
*/
|
||||
export function generateSidecarGgufLoaderSnippet(
|
||||
options: SidecarGgufLoaderInjectOptions,
|
||||
): SidecarGgufLoaderInjectResult {
|
||||
const raw = options.codeIdentityHex;
|
||||
const list = (Array.isArray(raw) ? raw : [raw]).map((hex) =>
|
||||
normalizeGgufCodeIdentity(hex),
|
||||
);
|
||||
if (list.length === 0) {
|
||||
throw new Error("codeIdentityHex must contain at least one digest");
|
||||
}
|
||||
const relPaths = (options.ggufRelPaths ?? []).filter(
|
||||
(p) => typeof p === "string" && p.length > 0,
|
||||
);
|
||||
if (relPaths.length === 0) {
|
||||
throw new Error("ggufRelPaths must contain at least one path");
|
||||
}
|
||||
const tag = list[0]!.slice(0, 8);
|
||||
const bindingName = options.bindingName ?? `${GGUF_SIDECAR_LOADER_MARKER}_${tag}`;
|
||||
const idsLiteral = JSON.stringify(list);
|
||||
const pathsLiteral = JSON.stringify(relPaths);
|
||||
const keyLiteral = JSON.stringify(GGUF_SIDECAR_METADATA_KEY);
|
||||
const schemaLiteral = JSON.stringify(SIDECAR_ENTANGLE_SCHEMA);
|
||||
const kindLiteral = JSON.stringify(GGUF_SIDECAR_METADATA_KIND);
|
||||
|
||||
const lines: string[] = [];
|
||||
lines.push(`# ${GGUF_SIDECAR_LOADER_MARKER}`);
|
||||
lines.push(`def ${bindingName}():`);
|
||||
lines.push(` import hashlib as _sc_hashlib_${tag}`);
|
||||
lines.push(` import json as _sc_json_${tag}`);
|
||||
lines.push(` import pathlib as _sc_pathlib_${tag}`);
|
||||
lines.push(` _sc_ids_${tag} = ${idsLiteral}`);
|
||||
lines.push(` _sc_idset_${tag} = set(_sc_ids_${tag})`);
|
||||
lines.push(` _sc_paths_${tag} = ${pathsLiteral}`);
|
||||
lines.push(` _sc_key_${tag} = ${keyLiteral}`);
|
||||
lines.push(` _sc_schema_${tag} = ${schemaLiteral}`);
|
||||
lines.push(` _sc_kind_${tag} = ${kindLiteral}`);
|
||||
lines.push(` _sc_base_${tag} = _sc_pathlib_${tag}.Path(__file__).resolve().parent`);
|
||||
lines.push(` def _sc_skip_${tag}(_sc_buf_${tag}, _sc_off_${tag}, _sc_ty_${tag}, _sc_depth_${tag}):`);
|
||||
lines.push(` if _sc_depth_${tag} > 64 or _sc_off_${tag} > len(_sc_buf_${tag}):`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` if _sc_ty_${tag} in (0, 1, 7):`);
|
||||
lines.push(` return _sc_off_${tag} + 1 if _sc_off_${tag} + 1 <= len(_sc_buf_${tag}) else None`);
|
||||
lines.push(` if _sc_ty_${tag} in (2, 3):`);
|
||||
lines.push(` return _sc_off_${tag} + 2 if _sc_off_${tag} + 2 <= len(_sc_buf_${tag}) else None`);
|
||||
lines.push(` if _sc_ty_${tag} in (4, 5, 6):`);
|
||||
lines.push(` return _sc_off_${tag} + 4 if _sc_off_${tag} + 4 <= len(_sc_buf_${tag}) else None`);
|
||||
lines.push(` if _sc_ty_${tag} in (10, 11, 12):`);
|
||||
lines.push(` return _sc_off_${tag} + 8 if _sc_off_${tag} + 8 <= len(_sc_buf_${tag}) else None`);
|
||||
lines.push(` if _sc_ty_${tag} == 8:`);
|
||||
lines.push(` if _sc_off_${tag} + 8 > len(_sc_buf_${tag}):`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_ln_${tag} = int.from_bytes(_sc_buf_${tag}[_sc_off_${tag}:_sc_off_${tag} + 8], "little")`);
|
||||
lines.push(` _sc_end_${tag} = _sc_off_${tag} + 8 + _sc_ln_${tag}`);
|
||||
lines.push(` return _sc_end_${tag} if _sc_end_${tag} <= len(_sc_buf_${tag}) else None`);
|
||||
lines.push(` if _sc_ty_${tag} == 9:`);
|
||||
lines.push(` if _sc_off_${tag} + 12 > len(_sc_buf_${tag}):`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_elem_${tag} = int.from_bytes(_sc_buf_${tag}[_sc_off_${tag}:_sc_off_${tag} + 4], "little")`);
|
||||
lines.push(` _sc_cnt_${tag} = int.from_bytes(_sc_buf_${tag}[_sc_off_${tag} + 4:_sc_off_${tag} + 12], "little")`);
|
||||
lines.push(` if _sc_cnt_${tag} > len(_sc_buf_${tag}) - (_sc_off_${tag} + 12):`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_cur_${tag} = _sc_off_${tag} + 12`);
|
||||
lines.push(` for _sc_i_${tag} in range(_sc_cnt_${tag}):`);
|
||||
lines.push(` _sc_nxt_${tag} = _sc_skip_${tag}(_sc_buf_${tag}, _sc_cur_${tag}, _sc_elem_${tag}, _sc_depth_${tag} + 1)`);
|
||||
lines.push(` if _sc_nxt_${tag} is None:`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_cur_${tag} = _sc_nxt_${tag}`);
|
||||
lines.push(` return _sc_cur_${tag}`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` def _sc_parse_${tag}(_sc_buf_${tag}):`);
|
||||
lines.push(` if len(_sc_buf_${tag}) < 24 or _sc_buf_${tag}[0:4] != b"GGUF":`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_ver_${tag} = int.from_bytes(_sc_buf_${tag}[4:8], "little")`);
|
||||
lines.push(` if _sc_ver_${tag} not in (2, 3):`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_tc_${tag} = int.from_bytes(_sc_buf_${tag}[8:16], "little")`);
|
||||
lines.push(` _sc_kc_${tag} = int.from_bytes(_sc_buf_${tag}[16:24], "little")`);
|
||||
lines.push(` if _sc_kc_${tag} > 100000 or _sc_tc_${tag} > 1000000:`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_off_${tag} = 24`);
|
||||
lines.push(` _sc_kvs_${tag} = []`);
|
||||
lines.push(` _sc_align_${tag} = 32`);
|
||||
lines.push(` for _sc_i_${tag} in range(_sc_kc_${tag}):`);
|
||||
lines.push(` _sc_start_${tag} = _sc_off_${tag}`);
|
||||
lines.push(` if _sc_off_${tag} + 8 > len(_sc_buf_${tag}):`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_klen_${tag} = int.from_bytes(_sc_buf_${tag}[_sc_off_${tag}:_sc_off_${tag} + 8], "little")`);
|
||||
lines.push(` _sc_kend_${tag} = _sc_off_${tag} + 8 + _sc_klen_${tag}`);
|
||||
lines.push(` if _sc_kend_${tag} + 4 > len(_sc_buf_${tag}):`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_key_${tag}2 = _sc_buf_${tag}[_sc_off_${tag} + 8:_sc_kend_${tag}].decode("utf-8", "replace")`);
|
||||
lines.push(` _sc_vty_${tag} = int.from_bytes(_sc_buf_${tag}[_sc_kend_${tag}:_sc_kend_${tag} + 4], "little")`);
|
||||
lines.push(` _sc_nxt_${tag} = _sc_skip_${tag}(_sc_buf_${tag}, _sc_kend_${tag} + 4, _sc_vty_${tag}, 0)`);
|
||||
lines.push(` if _sc_nxt_${tag} is None:`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` if _sc_key_${tag}2 == "general.alignment" and _sc_vty_${tag} == 4:`);
|
||||
lines.push(` _sc_a_${tag} = int.from_bytes(_sc_buf_${tag}[_sc_kend_${tag} + 4:_sc_kend_${tag} + 8], "little")`);
|
||||
lines.push(` if _sc_a_${tag} > 0 and _sc_a_${tag} % 8 == 0:`);
|
||||
lines.push(` if _sc_a_${tag} > 1048576:`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_align_${tag} = _sc_a_${tag}`);
|
||||
lines.push(` _sc_kvs_${tag}.append((_sc_key_${tag}2, _sc_start_${tag}, _sc_nxt_${tag}, _sc_vty_${tag}))`);
|
||||
lines.push(` _sc_off_${tag} = _sc_nxt_${tag}`);
|
||||
lines.push(` _sc_kvend_${tag} = _sc_off_${tag}`);
|
||||
lines.push(` for _sc_t_${tag} in range(_sc_tc_${tag}):`);
|
||||
lines.push(` if _sc_off_${tag} + 8 > len(_sc_buf_${tag}):`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_nlen_${tag} = int.from_bytes(_sc_buf_${tag}[_sc_off_${tag}:_sc_off_${tag} + 8], "little")`);
|
||||
lines.push(` _sc_nend_${tag} = _sc_off_${tag} + 8 + _sc_nlen_${tag}`);
|
||||
lines.push(` if _sc_nend_${tag} + 4 > len(_sc_buf_${tag}):`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_off_${tag} = _sc_nend_${tag}`);
|
||||
lines.push(` _sc_ndims_${tag} = int.from_bytes(_sc_buf_${tag}[_sc_off_${tag}:_sc_off_${tag} + 4], "little")`);
|
||||
lines.push(` if _sc_ndims_${tag} > 8:`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_off_${tag} += 4`);
|
||||
lines.push(` _sc_tail_${tag} = _sc_ndims_${tag} * 8 + 4 + 8`);
|
||||
lines.push(` if _sc_off_${tag} + _sc_tail_${tag} > len(_sc_buf_${tag}):`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_off_${tag} += _sc_tail_${tag}`);
|
||||
lines.push(` _sc_tiend_${tag} = _sc_off_${tag}`);
|
||||
lines.push(` _sc_has_${tag} = len(_sc_buf_${tag}) > _sc_tiend_${tag}`);
|
||||
lines.push(` _sc_rem_${tag} = _sc_tiend_${tag} % _sc_align_${tag}`);
|
||||
lines.push(` _sc_dstart_${tag} = _sc_tiend_${tag} if _sc_rem_${tag} == 0 else _sc_tiend_${tag} + (_sc_align_${tag} - _sc_rem_${tag})`);
|
||||
lines.push(` return (_sc_ver_${tag}, _sc_tc_${tag}, _sc_kvs_${tag}, _sc_kvend_${tag}, _sc_tiend_${tag}, _sc_align_${tag}, _sc_dstart_${tag}, _sc_has_${tag})`);
|
||||
lines.push(` def _sc_strip_${tag}(_sc_buf_${tag}):`);
|
||||
lines.push(` _sc_lay_${tag} = _sc_parse_${tag}(_sc_buf_${tag})`);
|
||||
lines.push(` if _sc_lay_${tag} is None:`);
|
||||
lines.push(` return _sc_buf_${tag}`);
|
||||
lines.push(` _sc_ver_${tag}, _sc_tc_${tag}, _sc_kvs_${tag}, _sc_kvend_${tag}, _sc_tiend_${tag}, _sc_align_${tag}, _sc_dstart_${tag}, _sc_has_${tag} = _sc_lay_${tag}`);
|
||||
lines.push(` if not any(_sc_kv_${tag}[0] == _sc_key_${tag} for _sc_kv_${tag} in _sc_kvs_${tag}):`);
|
||||
lines.push(` return _sc_buf_${tag}`);
|
||||
lines.push(` _sc_kept_${tag} = [_sc_kv_${tag} for _sc_kv_${tag} in _sc_kvs_${tag} if _sc_kv_${tag}[0] != _sc_key_${tag}]`);
|
||||
lines.push(` _sc_hdr_${tag} = bytearray(24)`);
|
||||
lines.push(` _sc_hdr_${tag}[0:4] = b"GGUF"`);
|
||||
lines.push(` _sc_hdr_${tag}[4:8] = int(_sc_ver_${tag}).to_bytes(4, "little")`);
|
||||
lines.push(` _sc_hdr_${tag}[8:16] = int(_sc_tc_${tag}).to_bytes(8, "little")`);
|
||||
lines.push(` _sc_hdr_${tag}[16:24] = int(len(_sc_kept_${tag})).to_bytes(8, "little")`);
|
||||
lines.push(` _sc_parts_${tag} = [bytes(_sc_hdr_${tag})]`);
|
||||
lines.push(` for _sc_kv_${tag} in _sc_kept_${tag}:`);
|
||||
lines.push(` _sc_parts_${tag}.append(_sc_buf_${tag}[_sc_kv_${tag}[1]:_sc_kv_${tag}[2]])`);
|
||||
lines.push(` _sc_parts_${tag}.append(_sc_buf_${tag}[_sc_kvend_${tag}:_sc_tiend_${tag}])`);
|
||||
lines.push(` _sc_meta_${tag} = b"".join(_sc_parts_${tag})`);
|
||||
lines.push(` if not _sc_has_${tag}:`);
|
||||
lines.push(` return _sc_meta_${tag}`);
|
||||
lines.push(` if _sc_dstart_${tag} > len(_sc_buf_${tag}):`);
|
||||
lines.push(` return _sc_buf_${tag}`);
|
||||
lines.push(` _sc_prem_${tag} = len(_sc_meta_${tag}) % _sc_align_${tag}`);
|
||||
lines.push(` _sc_pad_${tag} = 0 if _sc_prem_${tag} == 0 else _sc_align_${tag} - _sc_prem_${tag}`);
|
||||
lines.push(` return _sc_meta_${tag} + (b"\\x00" * _sc_pad_${tag}) + _sc_buf_${tag}[_sc_dstart_${tag}:]`);
|
||||
lines.push(` def _sc_extract_${tag}(_sc_buf_${tag}):`);
|
||||
lines.push(` _sc_lay_${tag} = _sc_parse_${tag}(_sc_buf_${tag})`);
|
||||
lines.push(` if _sc_lay_${tag} is None:`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_kvs_${tag} = _sc_lay_${tag}[2]`);
|
||||
lines.push(` _sc_hit_${tag} = None`);
|
||||
lines.push(` for _sc_kv_${tag} in _sc_kvs_${tag}:`);
|
||||
lines.push(` if _sc_kv_${tag}[0] == _sc_key_${tag} and _sc_kv_${tag}[3] == 8:`);
|
||||
lines.push(` _sc_hit_${tag} = _sc_kv_${tag}`);
|
||||
lines.push(` break`);
|
||||
lines.push(` if _sc_hit_${tag} is None:`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_start_${tag} = _sc_hit_${tag}[1]`);
|
||||
lines.push(` _sc_klen_${tag} = int.from_bytes(_sc_buf_${tag}[_sc_start_${tag}:_sc_start_${tag} + 8], "little")`);
|
||||
lines.push(` _sc_vstart_${tag} = _sc_start_${tag} + 8 + _sc_klen_${tag} + 4`);
|
||||
lines.push(` if _sc_vstart_${tag} + 8 > len(_sc_buf_${tag}):`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_vlen_${tag} = int.from_bytes(_sc_buf_${tag}[_sc_vstart_${tag}:_sc_vstart_${tag} + 8], "little")`);
|
||||
lines.push(` if _sc_vstart_${tag} + 8 + _sc_vlen_${tag} > len(_sc_buf_${tag}):`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` try:`);
|
||||
lines.push(` _sc_obj_${tag} = _sc_json_${tag}.loads(_sc_buf_${tag}[_sc_vstart_${tag} + 8:_sc_vstart_${tag} + 8 + _sc_vlen_${tag}].decode("utf-8"))`);
|
||||
lines.push(` except Exception:`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` if _sc_obj_${tag}.get("schema") != _sc_schema_${tag} or _sc_obj_${tag}.get("kind") != _sc_kind_${tag}:`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` _sc_dig_${tag} = _sc_obj_${tag}.get("digest")`);
|
||||
lines.push(` if not isinstance(_sc_dig_${tag}, str) or len(_sc_dig_${tag}) != 64:`);
|
||||
lines.push(` return None`);
|
||||
lines.push(` return _sc_obj_${tag}`);
|
||||
lines.push(` def _sc_digest_${tag}(_sc_entry_${tag}, _sc_parts_${tag}, _sc_seed_${tag}):`);
|
||||
lines.push(` _sc_h_${tag} = _sc_hashlib_${tag}.sha256()`);
|
||||
lines.push(` _sc_h_${tag}.update(_sc_entry_${tag}.encode("utf-8"))`);
|
||||
lines.push(` _sc_h_${tag}.update(b"\\x00")`);
|
||||
lines.push(` for _sc_sp_${tag}, _sc_data_${tag} in _sc_parts_${tag}:`);
|
||||
lines.push(` _sc_h_${tag}.update(_sc_sp_${tag}.encode("utf-8"))`);
|
||||
lines.push(` _sc_h_${tag}.update(b"\\x00")`);
|
||||
lines.push(` _sc_h_${tag}.update(_sc_data_${tag})`);
|
||||
lines.push(` _sc_h_${tag}.update(b"\\x00")`);
|
||||
lines.push(` _sc_h_${tag}.update(_sc_seed_${tag}.encode("utf-8"))`);
|
||||
lines.push(` return _sc_h_${tag}.hexdigest()`);
|
||||
lines.push(` for _sc_rel_${tag} in _sc_paths_${tag}:`);
|
||||
lines.push(` _sc_path_${tag} = _sc_pathlib_${tag}.Path(_sc_rel_${tag})`);
|
||||
lines.push(` if not _sc_path_${tag}.is_absolute():`);
|
||||
lines.push(` _sc_path_${tag} = _sc_base_${tag} / _sc_rel_${tag}`);
|
||||
lines.push(` try:`);
|
||||
lines.push(` _sc_raw_${tag} = _sc_path_${tag}.read_bytes()`);
|
||||
lines.push(` except Exception:`);
|
||||
lines.push(` continue`);
|
||||
lines.push(` _sc_ident_${tag} = _sc_hashlib_${tag}.sha256(_sc_strip_${tag}(_sc_raw_${tag})).hexdigest()`);
|
||||
lines.push(` if _sc_ident_${tag} not in _sc_idset_${tag}:`);
|
||||
lines.push(` continue`);
|
||||
lines.push(` _sc_note_${tag} = _sc_extract_${tag}(_sc_raw_${tag})`);
|
||||
lines.push(` if _sc_note_${tag} is None:`);
|
||||
lines.push(` raise RuntimeError("integrity binding mismatch")`);
|
||||
lines.push(` _sc_parts_${tag} = []`);
|
||||
lines.push(` _sc_dir_${tag} = _sc_path_${tag}.parent`);
|
||||
lines.push(` _sc_sp_list_${tag} = _sc_note_${tag}.get("sidecarPaths") or []`);
|
||||
lines.push(` if not isinstance(_sc_sp_list_${tag}, list):`);
|
||||
lines.push(` raise RuntimeError("integrity binding mismatch")`);
|
||||
lines.push(` for _sc_sp_${tag} in _sc_sp_list_${tag}:`);
|
||||
lines.push(` if not isinstance(_sc_sp_${tag}, str) or len(_sc_sp_${tag}) == 0:`);
|
||||
lines.push(` continue`);
|
||||
lines.push(` _sc_sib_${tag} = _sc_pathlib_${tag}.Path(_sc_sp_${tag})`);
|
||||
lines.push(` if not _sc_sib_${tag}.is_absolute():`);
|
||||
lines.push(` _sc_sib_${tag} = _sc_dir_${tag} / _sc_sp_${tag}`);
|
||||
lines.push(` try:`);
|
||||
lines.push(` _sc_parts_${tag}.append((_sc_sp_${tag}, _sc_sib_${tag}.read_bytes()))`);
|
||||
lines.push(` except Exception:`);
|
||||
lines.push(` raise RuntimeError("integrity binding mismatch") from None`);
|
||||
lines.push(` _sc_seed_${tag} = _sc_note_${tag}.get("seedTag")`);
|
||||
lines.push(` if not isinstance(_sc_seed_${tag}, str) or len(_sc_seed_${tag}) == 0:`);
|
||||
lines.push(` _sc_seed_${tag} = "bla-sidecar-v1"`);
|
||||
lines.push(` _sc_entry_${tag} = _sc_note_${tag}.get("entryPath")`);
|
||||
lines.push(` if not isinstance(_sc_entry_${tag}, str):`);
|
||||
lines.push(` _sc_entry_${tag} = ""`);
|
||||
lines.push(` if _sc_digest_${tag}(_sc_entry_${tag}, _sc_parts_${tag}, _sc_seed_${tag}) != str(_sc_note_${tag}.get("digest")).lower():`);
|
||||
lines.push(` raise RuntimeError("integrity binding mismatch")`);
|
||||
lines.push(` return`);
|
||||
lines.push(`try:`);
|
||||
lines.push(` ${bindingName}()`);
|
||||
lines.push(`finally:`);
|
||||
lines.push(` del ${bindingName}`);
|
||||
|
||||
return {
|
||||
schema: SIDECAR_ENTANGLE_SCHEMA,
|
||||
preamble: lines.join("\n"),
|
||||
bindingName,
|
||||
codeIdentityHexes: list,
|
||||
ggufRelPaths: relPaths,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Insert a GGUF load-time hook after shebang / encoding / docstring /
|
||||
* `from __future__`, idempotent when {@link GGUF_SIDECAR_LOADER_MARKER} is
|
||||
* already present. Preserves surrounding newlines the same way
|
||||
* `bindPythonSidecarEntanglement` does.
|
||||
*/
|
||||
export function prependGgufSidecarLoaderHook(
|
||||
entrySource: string,
|
||||
preamble: string,
|
||||
): string {
|
||||
if (entrySource.includes(GGUF_SIDECAR_LOADER_MARKER)) return entrySource;
|
||||
const eol = entrySource.includes("\r\n") ? "\r\n" : "\n";
|
||||
const analysis = findPythonPrologueInsertionOffset(entrySource);
|
||||
const headRaw = entrySource.slice(0, analysis.insertOffset);
|
||||
const tailRaw = entrySource.slice(analysis.insertOffset);
|
||||
const head = headRaw.replace(/(?:\r?\n)+$/, "");
|
||||
const tail = tailRaw.replace(/^(?:\r?\n)+/, "");
|
||||
const body = preamble.replace(/\r?\n/g, eol).replace(/(?:\r?\n)+$/, "");
|
||||
const parts: string[] = [];
|
||||
if (head.length > 0) parts.push(head);
|
||||
parts.push(body);
|
||||
if (tail.length > 0) parts.push(tail);
|
||||
return (
|
||||
parts.join(eol) +
|
||||
(entrySource.endsWith("\n") || entrySource.endsWith("\r\n") ? eol : "")
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Convenience: prepend a loader hook bound to this GGUF file's stripped
|
||||
* identity. Idempotent when the marker is already present.
|
||||
*
|
||||
* DEFENSIVE ONLY. Opt-in. No crack / RE advice.
|
||||
*/
|
||||
export function entangleGgufLoaderWithSidecars(options: {
|
||||
readonly loaderSource: string;
|
||||
readonly ggufBytes: Uint8Array;
|
||||
readonly ggufRelPath: string | readonly string[];
|
||||
}): { readonly code: string; readonly identity: string } {
|
||||
const identity = computeGgufSidecarCodeIdentity(options.ggufBytes);
|
||||
const paths = (
|
||||
Array.isArray(options.ggufRelPath) ? options.ggufRelPath : [options.ggufRelPath]
|
||||
).filter((p) => p.length > 0);
|
||||
const inject = generateSidecarGgufLoaderSnippet({
|
||||
codeIdentityHex: identity,
|
||||
ggufRelPaths: paths,
|
||||
});
|
||||
return {
|
||||
code: prependGgufSidecarLoaderHook(options.loaderSource, inject.preamble),
|
||||
identity,
|
||||
};
|
||||
}
|
||||
|
||||
const MAX_GGUF_LOADER_CANDIDATE_BYTES = 2 * 1024 * 1024;
|
||||
const MAX_GGUF_LOADER_AUTOWIRE_CHARS = 64 * 1024;
|
||||
|
||||
/**
|
||||
* True when `source` mentions `ggufBasename` in a quoted string / template.
|
||||
*/
|
||||
export function sourceMentionsGgufFile(source: string, ggufBasename: string): boolean {
|
||||
const name = (ggufBasename.replaceAll("\\", "/").split("/").pop() ?? "");
|
||||
if (!name || !name.toLowerCase().endsWith(".gguf")) return false;
|
||||
const quoted = new RegExp(
|
||||
`['"\`](?:[^'"\`\\\\]|\\\\.)*${escapeRegExpLiteral(name)}['"\`]`,
|
||||
);
|
||||
return quoted.test(source);
|
||||
}
|
||||
|
||||
/**
|
||||
* Conservative auto-wire predicate: a Python file with a known loader name
|
||||
* (`loader.py`, `__init__.py`, `__main__.py`) that already mentions the
|
||||
* target `.gguf` basename. Test, already-entangled, and oversized files
|
||||
* return false.
|
||||
*
|
||||
* DEFENSIVE ONLY. Opt-in. No crack / RE advice.
|
||||
*/
|
||||
export function isSafePythonGgufLoader(options: {
|
||||
readonly source: string;
|
||||
readonly filename: string;
|
||||
readonly ggufBasename: string;
|
||||
}): boolean {
|
||||
if (!isPythonPycLoaderFilename(options.filename)) return false;
|
||||
if (options.source.length > MAX_GGUF_LOADER_AUTOWIRE_CHARS) return false;
|
||||
if (ALREADY_ENTANGLED_PY_RE.test(options.source)) return false;
|
||||
return sourceMentionsGgufFile(options.source, options.ggufBasename);
|
||||
}
|
||||
|
||||
function ggufLoaderPreference(loaderName: string): number {
|
||||
const base = basename(loaderName);
|
||||
if (/^loader\.py$/i.test(base)) return 0;
|
||||
if (/^__init__\.py$/i.test(base)) return 1;
|
||||
if (/^__main__\.py$/i.test(base)) return 2;
|
||||
return 3;
|
||||
}
|
||||
|
||||
export interface PythonGgufLoaderCandidate {
|
||||
readonly name: string;
|
||||
readonly source: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Pick a single same-directory Python loader, or `null` when none qualify
|
||||
* or several unranked files all qualify (do not guess). Preferred names:
|
||||
* `loader.py`, then `__init__.py`, then `__main__.py`.
|
||||
*
|
||||
* DEFENSIVE ONLY. Opt-in. No crack / RE advice.
|
||||
*/
|
||||
export function selectPythonGgufLoader(
|
||||
ggufBasename: string,
|
||||
candidates: readonly PythonGgufLoaderCandidate[],
|
||||
): string | null {
|
||||
const qualifying = candidates.filter((c) =>
|
||||
isSafePythonGgufLoader({
|
||||
source: c.source,
|
||||
filename: c.name,
|
||||
ggufBasename,
|
||||
}),
|
||||
);
|
||||
if (qualifying.length === 0) return null;
|
||||
const scored = qualifying.map((c) => ({
|
||||
c,
|
||||
score: ggufLoaderPreference(c.name),
|
||||
}));
|
||||
scored.sort((a, b) => a.score - b.score || a.c.name.localeCompare(b.c.name));
|
||||
const best = scored[0]!;
|
||||
if (best.score >= 3) return null;
|
||||
const ties = scored.filter((s) => s.score === best.score);
|
||||
if (ties.length > 1) return null;
|
||||
return best.c.name;
|
||||
}
|
||||
|
||||
export interface DiscoveredPythonGgufLoader {
|
||||
readonly absPath: string;
|
||||
readonly name: string;
|
||||
readonly source: string;
|
||||
}
|
||||
|
||||
function readGgufLoaderCandidateSource(absPath: string): string | null {
|
||||
try {
|
||||
const st = statSync(absPath);
|
||||
if (!st.isFile() || st.size <= 0 || st.size > MAX_GGUF_LOADER_CANDIDATE_BYTES) {
|
||||
return null;
|
||||
}
|
||||
return readFileSync(absPath, "utf8");
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Directories to search for a Python loader next to a `.gguf`. */
|
||||
export function listGgufLoaderSearchDirectories(ggufPath: string): string[] {
|
||||
return [dirname(ggufPath)];
|
||||
}
|
||||
|
||||
/**
|
||||
* List same-directory `.py` candidates (no recursion). Unreadable or
|
||||
* oversized files are skipped.
|
||||
*/
|
||||
export function listSameDirPythonGgufLoaderCandidates(
|
||||
directory: string,
|
||||
): DiscoveredPythonGgufLoader[] {
|
||||
let names: string[] = [];
|
||||
try {
|
||||
names = readdirSync(directory);
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
const out: DiscoveredPythonGgufLoader[] = [];
|
||||
for (const name of names) {
|
||||
if (!isPythonPycLoaderFilename(name)) continue;
|
||||
const absPath = join(directory, name);
|
||||
const source = readGgufLoaderCandidateSource(absPath);
|
||||
if (source === null) continue;
|
||||
out.push({ absPath, name, source });
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Discover the unique safe Python loader for `ggufBasename` in the first
|
||||
* search directory that yields a selection. Later directories are not
|
||||
* consulted once one directory produces a hit (output dir wins over source).
|
||||
*
|
||||
* DEFENSIVE ONLY. Opt-in. No crack / RE advice.
|
||||
*/
|
||||
export function discoverPythonGgufLoader(options: {
|
||||
readonly ggufBasename: string;
|
||||
readonly searchDirectories: readonly string[];
|
||||
}): DiscoveredPythonGgufLoader | null {
|
||||
const seen = new Set<string>();
|
||||
for (const directory of options.searchDirectories) {
|
||||
if (typeof directory !== "string" || directory.length === 0) continue;
|
||||
if (seen.has(directory)) continue;
|
||||
seen.add(directory);
|
||||
const candidates = listSameDirPythonGgufLoaderCandidates(directory);
|
||||
const selected = selectPythonGgufLoader(options.ggufBasename, candidates);
|
||||
if (!selected) continue;
|
||||
const hit = candidates.find((c) => c.name === selected || c.absPath === selected);
|
||||
if (hit) return hit;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* True when a same-directory Python file already carries the GGUF load-time
|
||||
* hook bound to this file's code identity (used by defenseScore honesty).
|
||||
*
|
||||
* DEFENSIVE ONLY.
|
||||
*/
|
||||
export function isGgufSidecarLoaderHookWired(options: {
|
||||
readonly ggufBytes: Uint8Array;
|
||||
readonly ggufPath: string;
|
||||
}): boolean {
|
||||
const identity = computeGgufSidecarCodeIdentity(options.ggufBytes);
|
||||
const directories = listGgufLoaderSearchDirectories(options.ggufPath);
|
||||
return directories.some((directory) =>
|
||||
listSameDirPythonGgufLoaderCandidates(directory).some(
|
||||
(c) =>
|
||||
c.source.includes(GGUF_SIDECAR_LOADER_MARKER) && c.source.includes(identity),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
/** Relative path from a loader file's directory to the protected `.gguf`. */
|
||||
export function relativeGgufPathFromLoader(loaderPath: string, ggufPath: string): string {
|
||||
const rel = relative(dirname(loaderPath), ggufPath).split("\\").join("/");
|
||||
return rel.length === 0 ? basename(ggufPath) : rel;
|
||||
}
|
||||
|
||||
|
||||
/** Test/helper: strip the sidecar note section from a WASM binary. */
|
||||
export function stripWasmSidecarCustomSection(bytes: Uint8Array): Buffer {
|
||||
return stripWasmCustomSectionByName(Buffer.from(bytes), WASM_SIDECAR_SECTION_NAME);
|
||||
|
||||
@@ -105,6 +105,14 @@ describe("Issue #40.6: defenseScore honesty", () => {
|
||||
expect(hostOnly.gaps.map((g) => g.id)).toContain("gguf-sidecar-host-verify-only");
|
||||
expect(hostOnly.gaps.map((g) => g.id)).not.toContain("sidecar-entangle-not-enabled");
|
||||
expect(hostOnly.cappedScore).toBe(93);
|
||||
|
||||
const hooked = applyDefenseScoreHonesty(99, {
|
||||
detectedFormat: "model-gguf",
|
||||
sidecarEntangleEnabled: true,
|
||||
ggufSidecarLoaderHookApplied: true,
|
||||
});
|
||||
expect(hooked.gaps.map((g) => g.id)).not.toContain("gguf-sidecar-host-verify-only");
|
||||
expect(hooked.cappedScore).toBe(99);
|
||||
});
|
||||
|
||||
it("keeps a SQLite host-only schema-comment note from closing the strip⇒fail gap", () => {
|
||||
|
||||
@@ -0,0 +1,437 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
mkdtempSync,
|
||||
mkdirSync,
|
||||
writeFileSync,
|
||||
rmSync,
|
||||
readFileSync,
|
||||
unlinkSync,
|
||||
} from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import {
|
||||
bindGgufSidecarEntanglement,
|
||||
computeGgufSidecarCodeIdentity,
|
||||
entangleGgufLoaderWithSidecars,
|
||||
extractGgufSidecarMetadataNote,
|
||||
generateSidecarGgufLoaderSnippet,
|
||||
isGgufSidecarLoaderHookWired,
|
||||
isSafePythonGgufLoader,
|
||||
prependGgufSidecarLoaderHook,
|
||||
GGUF_SIDECAR_LOADER_MARKER,
|
||||
GGUF_SIDECAR_METADATA_KEY,
|
||||
selectPythonGgufLoader,
|
||||
SIDECAR_ENTANGLE_SCHEMA,
|
||||
sourceMentionsGgufFile,
|
||||
stripGgufSidecarMetadataNote,
|
||||
} from "../src/shields/runtime/sidecar-entangle.js";
|
||||
import {
|
||||
encodeGgufString,
|
||||
encodeGgufStringKv,
|
||||
parseGgufLayout,
|
||||
} from "../src/shields/formats/gguf-layout.js";
|
||||
import { collectDefenseScoreGaps } from "../src/engine/defense-score-honesty.js";
|
||||
import { protectTarget } from "../src/engine/protector.js";
|
||||
import { hardenProject } from "../src/engine/harden.js";
|
||||
import { verifyTarget } from "../src/engine/verifier.js";
|
||||
|
||||
const PY3 = (() => {
|
||||
const r = spawnSync("python3", ["--version"], { encoding: "utf8" });
|
||||
return r.status === 0;
|
||||
})();
|
||||
|
||||
function buildMinimalGguf(): Buffer {
|
||||
const kv = encodeGgufStringKv("general.architecture", "llama");
|
||||
const name = encodeGgufString("w");
|
||||
const rest = Buffer.alloc(4 + 8 + 4 + 8);
|
||||
rest.writeUInt32LE(1, 0);
|
||||
rest.writeBigUInt64LE(1n, 4);
|
||||
rest.writeUInt32LE(0, 12);
|
||||
rest.writeBigUInt64LE(0n, 16);
|
||||
const header = Buffer.alloc(24);
|
||||
header.write("GGUF", 0, "ascii");
|
||||
header.writeUInt32LE(3, 4);
|
||||
header.writeBigUInt64LE(1n, 8);
|
||||
header.writeBigUInt64LE(1n, 16);
|
||||
const metaAndInfos = Buffer.concat([header, kv, name, rest]);
|
||||
const align = 32;
|
||||
const rem = metaAndInfos.length % align;
|
||||
const pad = rem === 0 ? Buffer.alloc(0) : Buffer.alloc(align - rem, 0);
|
||||
const data = Buffer.from([0xde, 0xad, 0xbe, 0xef]);
|
||||
return Buffer.concat([metaAndInfos, pad, data]);
|
||||
}
|
||||
|
||||
function tensorDataDigest(bytes: Buffer): string {
|
||||
const layout = parseGgufLayout(bytes)!;
|
||||
return createHash("sha256")
|
||||
.update(bytes.subarray(layout.dataStart))
|
||||
.digest("hex");
|
||||
}
|
||||
|
||||
const LOADER_BODY = `import pathlib
|
||||
_gguf = pathlib.Path(__file__).resolve().parent / "model.gguf"
|
||||
_raw = _gguf.read_bytes()
|
||||
assert _raw[0:4] == b"GGUF"
|
||||
print("ok", len(_raw))
|
||||
`;
|
||||
|
||||
const FUTURE_LOADER_BODY = `#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Loader docstring."""
|
||||
from __future__ import annotations
|
||||
import pathlib
|
||||
_gguf = pathlib.Path(__file__).resolve().parent / "model.gguf"
|
||||
_raw = _gguf.read_bytes()
|
||||
assert _raw[0:4] == b"GGUF"
|
||||
print("ok", len(_raw))
|
||||
`;
|
||||
|
||||
function runLoader(
|
||||
dir: string,
|
||||
source: string,
|
||||
filename = "loader.py",
|
||||
): { status: number; stdout: string; stderr: string } {
|
||||
writeFileSync(join(dir, filename), source, "utf8");
|
||||
const r = spawnSync("python3", [filename], {
|
||||
cwd: dir,
|
||||
encoding: "utf8",
|
||||
timeout: 15_000,
|
||||
});
|
||||
return {
|
||||
status: r.status ?? 1,
|
||||
stdout: r.stdout ?? "",
|
||||
stderr: r.stderr ?? "",
|
||||
};
|
||||
}
|
||||
|
||||
describe("sidecar GGUF load-time hook", () => {
|
||||
it("generateSidecarGgufLoaderSnippet rejects non-hex identities", () => {
|
||||
expect(() =>
|
||||
generateSidecarGgufLoaderSnippet({
|
||||
codeIdentityHex: "deadbeef",
|
||||
ggufRelPaths: ["model.gguf"],
|
||||
}),
|
||||
).toThrow(/64-char/);
|
||||
});
|
||||
|
||||
it("computeGgufSidecarCodeIdentity is stable across planting the note", () => {
|
||||
const entry = buildMinimalGguf();
|
||||
const unbound = computeGgufSidecarCodeIdentity(entry);
|
||||
const bound = bindGgufSidecarEntanglement({
|
||||
entryPath: "model.gguf",
|
||||
sidecarPaths: ["sib.bin"],
|
||||
seedTag: "loader-id-v1",
|
||||
entryBytes: entry,
|
||||
sidecarBytes: [new TextEncoder().encode("sidecar-payload\n")],
|
||||
});
|
||||
expect(computeGgufSidecarCodeIdentity(bound.bytes)).toBe(unbound);
|
||||
expect(unbound).toMatch(/^[0-9a-f]{64}$/);
|
||||
const stripped = stripGgufSidecarMetadataNote(bound.bytes);
|
||||
expect(computeGgufSidecarCodeIdentity(stripped)).toBe(unbound);
|
||||
expect(Buffer.compare(stripped, entry)).toBe(0);
|
||||
expect(tensorDataDigest(bound.bytes)).toBe(tensorDataDigest(entry));
|
||||
});
|
||||
|
||||
it("prepend is idempotent and preserves shebang / encoding / docstring / futures", () => {
|
||||
const inject = generateSidecarGgufLoaderSnippet({
|
||||
codeIdentityHex: "a".repeat(64),
|
||||
ggufRelPaths: ["model.gguf"],
|
||||
});
|
||||
expect(inject.schema).toBe(SIDECAR_ENTANGLE_SCHEMA);
|
||||
expect(inject.preamble).toContain(GGUF_SIDECAR_LOADER_MARKER);
|
||||
const once = prependGgufSidecarLoaderHook(FUTURE_LOADER_BODY, inject.preamble);
|
||||
expect(once.startsWith("#!/usr/bin/env python3\n")).toBe(true);
|
||||
expect(once).toContain("# -*- coding: utf-8 -*-");
|
||||
expect(once).toContain('"""Loader docstring."""');
|
||||
expect(once).toContain("from __future__ import annotations");
|
||||
const hookAt = once.indexOf(GGUF_SIDECAR_LOADER_MARKER);
|
||||
const futureAt = once.indexOf("from __future__ import annotations");
|
||||
expect(hookAt).toBeGreaterThan(futureAt);
|
||||
const twice = prependGgufSidecarLoaderHook(once, inject.preamble);
|
||||
expect(twice).toBe(once);
|
||||
});
|
||||
|
||||
it("isSafe / select only admit known loaders that mention the .gguf", () => {
|
||||
const loader = LOADER_BODY;
|
||||
const noMention = "print('hello')\n";
|
||||
const helper =
|
||||
'import pathlib\n_gguf = pathlib.Path("model.gguf")\n';
|
||||
const testLoader =
|
||||
'import pathlib\n_gguf = pathlib.Path(__file__).parent / "model.gguf"\n';
|
||||
const already = `${loader}\n# _verify_sibling_digest_deadbeef\n`;
|
||||
|
||||
expect(sourceMentionsGgufFile(loader, "model.gguf")).toBe(true);
|
||||
expect(sourceMentionsGgufFile(noMention, "model.gguf")).toBe(false);
|
||||
|
||||
expect(
|
||||
isSafePythonGgufLoader({
|
||||
source: loader,
|
||||
filename: "loader.py",
|
||||
ggufBasename: "model.gguf",
|
||||
}),
|
||||
).toBe(true);
|
||||
expect(
|
||||
isSafePythonGgufLoader({
|
||||
source: testLoader,
|
||||
filename: "test_loader.py",
|
||||
ggufBasename: "model.gguf",
|
||||
}),
|
||||
).toBe(false);
|
||||
expect(
|
||||
isSafePythonGgufLoader({
|
||||
source: already,
|
||||
filename: "loader.py",
|
||||
ggufBasename: "model.gguf",
|
||||
}),
|
||||
).toBe(false);
|
||||
expect(
|
||||
isSafePythonGgufLoader({
|
||||
source: helper,
|
||||
filename: "helper.py",
|
||||
ggufBasename: "model.gguf",
|
||||
}),
|
||||
).toBe(true);
|
||||
expect(
|
||||
isSafePythonGgufLoader({
|
||||
source: noMention,
|
||||
filename: "loader.py",
|
||||
ggufBasename: "model.gguf",
|
||||
}),
|
||||
).toBe(false);
|
||||
|
||||
expect(
|
||||
selectPythonGgufLoader("model.gguf", [
|
||||
{ name: "helper.py", source: helper },
|
||||
{ name: "loader.py", source: loader },
|
||||
{ name: "__init__.py", source: loader },
|
||||
]),
|
||||
).toBe("loader.py");
|
||||
expect(
|
||||
selectPythonGgufLoader("model.gguf", [
|
||||
{ name: "__init__.py", source: loader },
|
||||
{ name: "__main__.py", source: loader },
|
||||
]),
|
||||
).toBe("__init__.py");
|
||||
expect(
|
||||
selectPythonGgufLoader("model.gguf", [
|
||||
{ name: "helper.py", source: helper },
|
||||
]),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it.skipIf(!PY3)("load-time hook: intact model opens and keeps tensor bytes", () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "bla-gguf-loader-ok-"));
|
||||
try {
|
||||
const entry = buildMinimalGguf();
|
||||
const data0 = tensorDataDigest(entry);
|
||||
const snap = Buffer.from("sidecar-payload\n");
|
||||
const bound = bindGgufSidecarEntanglement({
|
||||
entryPath: "model.gguf",
|
||||
sidecarPaths: ["sib.bin"],
|
||||
seedTag: "gguf-loader-ok",
|
||||
entryBytes: entry,
|
||||
sidecarBytes: [snap],
|
||||
});
|
||||
writeFileSync(join(dir, "model.gguf"), bound.bytes);
|
||||
writeFileSync(join(dir, "sib.bin"), snap);
|
||||
const hooked = entangleGgufLoaderWithSidecars({
|
||||
loaderSource: LOADER_BODY,
|
||||
ggufBytes: bound.bytes,
|
||||
ggufRelPath: "model.gguf",
|
||||
});
|
||||
expect(hooked.code).toContain(GGUF_SIDECAR_LOADER_MARKER);
|
||||
expect(tensorDataDigest(bound.bytes)).toBe(data0);
|
||||
const ran = runLoader(dir, hooked.code);
|
||||
expect(ran.status, ran.stderr || ran.stdout).toBe(0);
|
||||
expect(ran.stdout).toMatch(/ok /);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it.skipIf(!PY3)("load-time hook fails closed on metadata strip or sibling delete", () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "bla-gguf-loader-fail-"));
|
||||
try {
|
||||
const entry = buildMinimalGguf();
|
||||
const snap = Buffer.from("sidecar-payload\n");
|
||||
const bound = bindGgufSidecarEntanglement({
|
||||
entryPath: "model.gguf",
|
||||
sidecarPaths: ["sib.bin"],
|
||||
seedTag: "gguf-loader-fail",
|
||||
entryBytes: entry,
|
||||
sidecarBytes: [snap],
|
||||
});
|
||||
writeFileSync(join(dir, "sib.bin"), snap);
|
||||
const hooked = entangleGgufLoaderWithSidecars({
|
||||
loaderSource: LOADER_BODY,
|
||||
ggufBytes: bound.bytes,
|
||||
ggufRelPath: "model.gguf",
|
||||
});
|
||||
|
||||
writeFileSync(join(dir, "model.gguf"), bound.bytes);
|
||||
const ok = runLoader(dir, hooked.code);
|
||||
expect(ok.status, ok.stderr || ok.stdout).toBe(0);
|
||||
|
||||
writeFileSync(join(dir, "model.gguf"), stripGgufSidecarMetadataNote(bound.bytes));
|
||||
const stripped = runLoader(dir, hooked.code);
|
||||
expect(stripped.status).not.toBe(0);
|
||||
expect(stripped.stderr + stripped.stdout).toMatch(/integrity binding mismatch/);
|
||||
|
||||
writeFileSync(join(dir, "model.gguf"), bound.bytes);
|
||||
unlinkSync(join(dir, "sib.bin"));
|
||||
const deleted = runLoader(dir, hooked.code);
|
||||
expect(deleted.status).not.toBe(0);
|
||||
expect(deleted.stderr + deleted.stdout).toMatch(/integrity binding mismatch/);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it.skipIf(!PY3)("protectTarget auto-wires a same-directory loader and records gguf-sidecar-entangle", () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "bla-gguf-loader-protect-"));
|
||||
try {
|
||||
const entry = join(dir, "model.gguf");
|
||||
writeFileSync(entry, buildMinimalGguf());
|
||||
writeFileSync(join(dir, "sib.bin"), "sidecar-payload\n");
|
||||
writeFileSync(join(dir, "loader.py"), LOADER_BODY, "utf8");
|
||||
|
||||
const res = protectTarget({
|
||||
inputPath: entry,
|
||||
outputPath: entry,
|
||||
intensity: "balanced",
|
||||
seed: "pt-gguf-loader-hook",
|
||||
sidecarEntangle: true,
|
||||
sidecarEntanglePaths: ["sib.bin"],
|
||||
inPlace: true,
|
||||
});
|
||||
expect(res.sidecarEntangle).toBe(true);
|
||||
expect(res.ggufSidecarLoaderHookApplied).toBe(true);
|
||||
expect(res.counteredReaVectors).toContain("gguf-sidecar-entangle");
|
||||
|
||||
const hooked = readFileSync(join(dir, "loader.py"), "utf8");
|
||||
expect(hooked).toContain(GGUF_SIDECAR_LOADER_MARKER);
|
||||
const ggufBytes = readFileSync(entry);
|
||||
expect(
|
||||
isGgufSidecarLoaderHookWired({
|
||||
ggufBytes,
|
||||
ggufPath: entry,
|
||||
}),
|
||||
).toBe(true);
|
||||
expect(extractGgufSidecarMetadataNote(ggufBytes)).not.toBeNull();
|
||||
expect(ggufBytes.includes(Buffer.from(GGUF_SIDECAR_METADATA_KEY))).toBe(true);
|
||||
|
||||
const ran = runLoader(dir, hooked);
|
||||
expect(ran.status, ran.stderr || ran.stdout).toBe(0);
|
||||
expect(ran.stdout).toMatch(/ok /);
|
||||
|
||||
const verification = verifyTarget(entry, undefined, {
|
||||
sidecarEntangleEnabled: true,
|
||||
});
|
||||
const gapIds = (verification.defenseScoreGaps ?? []).map((g) => g.id);
|
||||
expect(gapIds).not.toContain("gguf-sidecar-host-verify-only");
|
||||
|
||||
const gaps = collectDefenseScoreGaps({
|
||||
detectedFormat: "model-gguf",
|
||||
sidecarEntangleEnabled: true,
|
||||
ggufSidecarLoaderHookApplied: true,
|
||||
});
|
||||
expect(gaps.map((g) => g.id)).not.toContain("gguf-sidecar-host-verify-only");
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it.skipIf(!PY3)("protectTarget does not rewrite when sidecarEntangle is off", () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "bla-gguf-loader-off-"));
|
||||
try {
|
||||
const entry = join(dir, "model.gguf");
|
||||
writeFileSync(entry, buildMinimalGguf());
|
||||
writeFileSync(join(dir, "sib.bin"), "sidecar-payload\n");
|
||||
writeFileSync(join(dir, "loader.py"), LOADER_BODY, "utf8");
|
||||
|
||||
const res = protectTarget({
|
||||
inputPath: entry,
|
||||
outputPath: entry,
|
||||
intensity: "balanced",
|
||||
seed: "pt-gguf-loader-off",
|
||||
inPlace: true,
|
||||
});
|
||||
expect(res.sidecarEntangle).not.toBe(true);
|
||||
expect(res.ggufSidecarLoaderHookApplied).toBeUndefined();
|
||||
expect(res.counteredReaVectors).not.toContain("gguf-sidecar-entangle");
|
||||
expect(readFileSync(join(dir, "loader.py"), "utf8")).toBe(LOADER_BODY);
|
||||
expect(extractGgufSidecarMetadataNote(readFileSync(entry))).toBeNull();
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it.skipIf(!PY3)("protectTarget does not rewrite an unknown helper file", () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "bla-gguf-loader-unsafe-"));
|
||||
try {
|
||||
const entry = join(dir, "model.gguf");
|
||||
writeFileSync(entry, buildMinimalGguf());
|
||||
writeFileSync(join(dir, "sib.bin"), "sidecar-payload\n");
|
||||
const helper = LOADER_BODY;
|
||||
writeFileSync(join(dir, "helper.py"), helper, "utf8");
|
||||
|
||||
const res = protectTarget({
|
||||
inputPath: entry,
|
||||
outputPath: entry,
|
||||
intensity: "balanced",
|
||||
seed: "pt-gguf-loader-unsafe",
|
||||
sidecarEntangle: true,
|
||||
sidecarEntanglePaths: ["sib.bin"],
|
||||
inPlace: true,
|
||||
});
|
||||
expect(res.sidecarEntangle).toBe(true);
|
||||
expect(res.ggufSidecarLoaderHookApplied).toBeUndefined();
|
||||
expect(res.counteredReaVectors).not.toContain("gguf-sidecar-entangle");
|
||||
expect(readFileSync(join(dir, "helper.py"), "utf8")).toBe(helper);
|
||||
expect(extractGgufSidecarMetadataNote(readFileSync(entry))).not.toBeNull();
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it.skipIf(!PY3)("hardenProject auto-wires dist/loader.py beside model.gguf", () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "bla-gguf-loader-harden-"));
|
||||
try {
|
||||
const dist = join(dir, "dist");
|
||||
mkdirSync(dist, { recursive: true });
|
||||
writeFileSync(join(dist, "model.gguf"), buildMinimalGguf());
|
||||
writeFileSync(join(dist, "sib.bin"), "sidecar-payload\n");
|
||||
writeFileSync(join(dist, "loader.py"), LOADER_BODY, "utf8");
|
||||
|
||||
const report = hardenProject({
|
||||
directory: dir,
|
||||
intensity: "balanced",
|
||||
seed: "harden-gguf-loader-hook",
|
||||
sidecarEntangle: true,
|
||||
});
|
||||
expect(report.overallSuccess).toBe(true);
|
||||
expect(report.sidecarEntangleError).toBeUndefined();
|
||||
|
||||
const outDist = join(report.outputDirectory, "dist");
|
||||
const hooked = readFileSync(join(outDist, "loader.py"), "utf8");
|
||||
expect(hooked).toContain(GGUF_SIDECAR_LOADER_MARKER);
|
||||
const ggufBytes = readFileSync(join(outDist, "model.gguf"));
|
||||
expect(extractGgufSidecarMetadataNote(ggufBytes)).not.toBeNull();
|
||||
expect(ggufBytes.includes(Buffer.from(GGUF_SIDECAR_METADATA_KEY))).toBe(true);
|
||||
expect(
|
||||
isGgufSidecarLoaderHookWired({
|
||||
ggufBytes,
|
||||
ggufPath: join(outDist, "model.gguf"),
|
||||
}),
|
||||
).toBe(true);
|
||||
|
||||
const ran = runLoader(outDist, hooked);
|
||||
expect(ran.status, ran.stderr || ran.stdout).toBe(0);
|
||||
expect(ran.stdout).toMatch(/ok /);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user