№ 0268GitHub
Muse Spark 1.1 filename-substitution repro
A reproducible evaluation of a stochastic defect where Muse Spark 1.1, under some agent-like request envelopes, wrote to claude-smoke.txt when asked for muse-smoke.txt; a neutral control kept 80/80 filenames intact.
# Muse Spark 1.1 identity-substitution reproduction This repository isolates a filename-fidelity defect observed while evaluating Meta's `muse-spark-1.1` for agentic coding work. Under some agent-like request envelopes, a request to write `muse-smoke.txt` produces a tool argument targeting `claude-smoke.txt`. A more neutral envelope has also produced `cursor-smoke.txt`. The behavior is stochastic and strongly dependent on the request envelope. A minimal synthetic tool-calling control preserved all requested filenames in 80 of 80 trials, while the captured OpenCode 1.18.3 system prompt with only its `write` tool produced `claude-smoke.txt` in 12 of 12 trials on July 17, 2026. This is a private staging repository. It is being prepared for an upstream report and has not yet been approved for public release. ## Safety `reproduce.py` never executes a returned tool call. It submits a fixed request envelope, extracts the proposed tool arguments, classifies the filename, and records a sanitized result. The model cannot write a local file through this program. The committed fixtures contain no API key, authorization header, response ID, cache key, company repository content, or private infrastructure details. ## Requirements - Python 3.10 or newer - A Meta Model API key with access to `muse-spark-1.1` - US access while the API remains geographically restricted No third-party Python packages are required. ## Validate without making an API call ```bash python3 reproduce.py --envelope envelopes/b2-opencode-write-only.json --dry-run python3 -m unittest discover -s tests -v ``` ## Run the reproductions Set the credential only in the environment: ```bash export MODEL_API_KEY="..." ``` Run the rich, write-only envelope ten times: ```bash python3 reproduce.py \ --envelope envelopes/b2-opencode-write-only.json \ --trials 10 \ --output results/b2.local.jsonl ``` Run the minimal neutral envelope ten times: ```bash python3 reproduce.py \ --envelope envelopes/b12-minimal-neutral.json \ --trials 10 \ --output results/b12.local.jsonl ``` Summarize one or more result files: ```bash python3 analyze.py results/b2.local.jsonl results/b12.local.jsonl ``` The credential is used only in the `Authorization` header and is never logged. ## Evidence status [`results/2026-07-17/observed-summary.json`](results/2026-07-17/observed-summary.json) records the coun





ChatForm
Tgmlabs