shipwithmuse

№ 0268GitHub

Muse Spark 1.1 filename-substitution repro

A reproducible evaluation of a stochastic defect where Muse Spark 1.1, under some agent-like request envelopes, wrote to claude-smoke.txt when asked for muse-smoke.txt; a neutral control kept 80/80 filenames intact.

Muse Spark 1.1 filename-substitution repro on github.com
jacobstephens2/muse-spark-identity-substitution-reproREADME ↗
# Muse Spark 1.1 identity-substitution reproduction

This repository isolates a filename-fidelity defect observed while evaluating Meta's `muse-spark-1.1` for agentic coding work. Under some agent-like request envelopes, a request to write `muse-smoke.txt` produces a tool argument targeting `claude-smoke.txt`. A more neutral envelope has also produced `cursor-smoke.txt`.

The behavior is stochastic and strongly dependent on the request envelope. A minimal synthetic tool-calling control preserved all requested filenames in 80 of 80 trials, while the captured OpenCode 1.18.3 system prompt with only its `write` tool produced `claude-smoke.txt` in 12 of 12 trials on July 17, 2026.

This is a private staging repository. It is being prepared for an upstream report and has not yet been approved for public release.

## Safety

`reproduce.py` never executes a returned tool call. It submits a fixed request envelope, extracts the proposed tool arguments, classifies the filename, and records a sanitized result. The model cannot write a local file through this program.

The committed fixtures contain no API key, authorization header, response ID, cache key, company repository content, or private infrastructure details.

## Requirements

- Python 3.10 or newer
- A Meta Model API key with access to `muse-spark-1.1`
- US access while the API remains geographically restricted

No third-party Python packages are required.

## Validate without making an API call

```bash
python3 reproduce.py --envelope envelopes/b2-opencode-write-only.json --dry-run
python3 -m unittest discover -s tests -v
```

## Run the reproductions

Set the credential only in the environment:

```bash
export MODEL_API_KEY="..."
```

Run the rich, write-only envelope ten times:

```bash
python3 reproduce.py \
  --envelope envelopes/b2-opencode-write-only.json \
  --trials 10 \
  --output results/b2.local.jsonl
```

Run the minimal neutral envelope ten times:

```bash
python3 reproduce.py \
  --envelope envelopes/b12-minimal-neutral.json \
  --trials 10 \
  --output results/b12.local.jsonl
```

Summarize one or more result files:

```bash
python3 analyze.py results/b2.local.jsonl results/b12.local.jsonl
```

The credential is used only in the `Authorization` header and is never logged.

## Evidence status

[`results/2026-07-17/observed-summary.json`](results/2026-07-17/observed-summary.json) records the coun

Also filed under Benchmarks & research

See all →
  1. 0590

    Meta's OpenCode token share hits 45%★

    $META just went from 3.5% to 45.4% token share on OpenCode in just over two weeks Muse Spark 1.3 being good + free is enough to become the default for most users Default gets you usage → usage gets you data → data makes the next model better Anthropic and OpenAI can’t afford

    @thetomcollins

    X post

    Benchmarks & research

  2. 0107

    How Meta built safety into Muse★

    Meta's engineering write-up on Muse security: isolated VMs, a separate Sentinel permission authority, credential surrogation and layered prompt-injection defenses, with bug bounties up to $300,000.

    research.meta.ai

    Resource

    Benchmarks & research

  3. 0101

    Muse Spark 1.3 tutorial: testing Meta's efficiency claims★

    DataCamp's Josep Ferrer ran Muse Spark 1.3 on three real coding tasks. Two used 23–32% fewer completion tokens, but a refactor used 70% more, for a net 12% cost increase.

    datacamp.com

    Resource

    Benchmarks & research

  4. 1067

    Muse Voice Transcribe tested on clinical diarization

    Compared diarization models on 15 mock doctor-patient consultations (~2.4 h): Meta Muse Voice Transcribe scored 13.04% DER at ~92 s per request via API, behind Pyannote (2.89%) and Nemotron 3 (4.80%).

    u/MajesticAd2862

    Reddit post

    Benchmarks & research

More GitHub

See all →
  1. 1074

    Ziggy, a 24/7 voice assistant on Muse Spark

    An always-on laptop voice companion: local Porcupine wake word, local Whisper speech-to-text in Hindi and English, Muse Spark via the Meta Model API as the brain, offline TTS and local conversation memory.

    @TanayYadavDev

    GitHub

    Agents & automation

  2. 1066

    Muse remote-control bridge for Intel Macs

    Since Muse for Mac ships only for Apple silicon, this small service lets Muse's cloud VM screenshot, click and type on an Intel Mac, compressing each 5K frame to about 150 KB and handling Retina coordinate scaling.

    @LilMuh

    GitHub

    Errands & personal agent

  3. 0995

    BitNet Gateway: cheap CPU triage, Muse Spark decides

    A hybrid FastAPI gateway where a 1-bit BitNet layer on CPU filters routine traffic and Muse Spark 1.3 is called only for sales, urgency, money or low-confidence cases; the author estimates the local filter handles about 60-70% of volume.

    @rafaelnovaes22

    GitHub

    Agents & automation

  4. 0992

    Uber for Muse connector

    An Express-based Muse connector that books Uber rides from natural language, with routes for fare estimates and ride requests, connector auth middleware and a privacy page.

    @AshutoshKD

    GitHub

    Connectors & MCP

Curator picks

  1. 1046

    Medical bills audited line by line, $4,000 saved★

    Got Muse logged in to my medical provider’s portal, he pulled the itemized bills, and questioned every line. So far he’s found several times I’d been double billed, asked for some discounts and has saved me over $4,000. If your moat is bureaucracy, you’re cooked.

    @Ryan_Holdaway

    X post

    Errands & personal agent

  2. 1013

    Shop Pay agentic checkout on every Shopify store★

    We are excited to announce we are partnering deeply with Muse to enable agentic checkout with Shop Pay on all Shopify stores, offering people an easy and delightful way to shop and check out with Muse.

    @tobi

    X post

    Business & commerce

  3. 1009

    Private e-book library app from Google Drive★

    Muse built me a private library for the e-books and articles in my Google Drive. Everything is organized by topic, and each section opens onto its own subcategorized shelves. Each book opens like a real book and is readable in-app

    @chiasmus_cap

    X post

    Apps & websites

  4. 1007

    Plumbing company run by a Muse agent★

    I still can’t believe I can run my plumbing company with an agent so easily. I send this message to my Muse agent while in bed at 6am. And it: updates my job board, texts customer, updates office manager who arrives at 8am in slack Notifies technician

    @HouseHackerJon

    X post

    Agents & automation