<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://mumen.musa.ai/feed.xml" rel="self" type="application/atom+xml" /><link href="https://mumen.musa.ai/" rel="alternate" type="text/html" /><updated>2026-06-04T10:13:09-07:00</updated><id>https://mumen.musa.ai/feed.xml</id><title type="html">mumen musa</title><subtitle>Research, builds, things I&apos;m thinking through. Mostly for future-me.</subtitle><author><name>Mumen Musa</name></author><entry><title type="html">How I built a self-running pipeline for our piano lessons</title><link href="https://mumen.musa.ai/2026/06/03/piano-lesson-processing-pipeline/" rel="alternate" type="text/html" title="How I built a self-running pipeline for our piano lessons" /><published>2026-06-03T13:00:00-07:00</published><updated>2026-06-03T13:00:00-07:00</updated><id>https://mumen.musa.ai/2026/06/03/piano-lesson-processing-pipeline</id><content type="html" xml:base="https://mumen.musa.ai/2026/06/03/piano-lesson-processing-pipeline/"><![CDATA[<p><em>From a Microsoft Teams recording to a dated folder of transcripts, subtitles, and summaries — with no manual steps after the file lands.</em></p>

<p>Every week, the boys in our house take piano lessons over Microsoft Teams. Every week, I want a permanent searchable record of what was taught. The free tier of Teams doesn’t record, and the paid tier is a few dollars a month I didn’t want to add to the household budget. So I started recording the screen with OBS. That was the workaround. Then I started keeping the files. Then I started running them through a transcription pipeline. Then I started diarizing them so the transcript knew who was speaking. Then I started generating a structured summary at the top of each one. The pipeline that resulted is self-running: I drop a new recording into a watched folder, the next hour the pipeline processes it, and a few hours later I have a transcript, subtitles, a speaker map, and a lesson summary, all in a dated subfolder, all searchable.</p>

<p>This post is about how I built it, what the pieces are, and what it has already surfaced from the lessons I record.</p>

<h2 id="architecture">Architecture</h2>

<p>The full production flow:</p>

<pre><code class="language-mermaid">flowchart TB
    A[New .mp4/.mkv/.mov&lt;br/&gt;lands in NAS&lt;br/&gt;Piano-Recordings/_inbox/] --&gt; B[Hourly cron:&lt;br/&gt;watch-piano-inbox.sh]
    B --&gt; C{File size&lt;br/&gt;unchanged for 60s?}
    C -- no --&gt; B
    C -- yes --&gt; D[Move to&lt;br/&gt;_processing/]
    D --&gt; E[ffmpeg:&lt;br/&gt;16kHz mono WAV]
    E --&gt; F[whisperx:&lt;br/&gt;transcribe + align]
    F --&gt; G[pyannote&lt;br/&gt;speaker-diarization-3.1]
    G --&gt; H[rename_speakers.py:&lt;br/&gt;vocabulary heuristic&lt;br/&gt;SPEAKER_xx → role]
    H --&gt; I[claude --print:&lt;br/&gt;structured summary&lt;br/&gt;HOME=/Users/mini-1]
    I --&gt; J[Output:&lt;br/&gt;dated folder&lt;br/&gt;YYYY-MM-DD_HHMMSS/]
    J --&gt; K[.mp4 .wav .txt .srt&lt;br/&gt;.vtt .tsv .json&lt;br/&gt;.speakers.json&lt;br/&gt;.summary.md]
    J --&gt; L[Watcher reports&lt;br/&gt;to Slack on activity,&lt;br/&gt;silent when idle]
    
    style A fill:#e1f5e1
    style K fill:#e1e1f5
    style G fill:#fff4e1
    style H fill:#fff4e1
    style I fill:#fff4e1
</code></pre>

<p>Six stages. The orchestrator script chains them. The watcher at the front is the only thing continuously running; the rest fire only when there’s a new file.</p>

<h2 id="stage-1-ffmpeg--extract-audio">Stage 1: ffmpeg — extract audio</h2>

<p>The recordings are screen captures of Teams calls: <code class="language-plaintext highlighter-rouge">.mp4</code> or <code class="language-plaintext highlighter-rouge">.mkv</code> files with both audio and video. WhisperX wants audio. The first stage extracts a 16 kHz mono WAV:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffmpeg <span class="nt">-i</span> <span class="s2">"</span><span class="nv">$INPUT</span><span class="s2">"</span> <span class="se">\</span>
  <span class="nt">-ac</span> 1 <span class="nt">-ar</span> 16000 <span class="se">\</span>
  <span class="nt">-c</span>:a pcm_s16le <span class="se">\</span>
  <span class="s2">"</span><span class="nv">$WORKDIR</span><span class="s2">/</span><span class="k">${</span><span class="nv">BASENAME</span><span class="k">}</span><span class="s2">.wav"</span>
</code></pre></div></div>

<p>Why 16 kHz mono: WhisperX’s required input format. Why we keep the WAV after the run: re-runs are much faster when audio extraction is already done, and re-running happens often during prompt tuning.</p>

<h2 id="stage-2-whisperx--transcribe-and-align">Stage 2: whisperx — transcribe and align</h2>

<p>WhisperX is Whisper with forced alignment. The <code class="language-plaintext highlighter-rouge">small</code> model handles initial transcription. A separate alignment pass fixes word-level timestamps. The output is word-level segments with start and end times, which the next stage needs.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>whisperx <span class="s2">"</span><span class="nv">$WORKDIR</span><span class="s2">/</span><span class="k">${</span><span class="nv">BASENAME</span><span class="k">}</span><span class="s2">.wav"</span> <span class="se">\</span>
  <span class="nt">--model</span> small <span class="se">\</span>
  <span class="nt">--language</span> en <span class="se">\</span>
  <span class="nt">--output_dir</span> <span class="s2">"</span><span class="nv">$WORKDIR</span><span class="s2">"</span> <span class="se">\</span>
  <span class="nt">--initial_prompt</span> <span class="s2">"</span><span class="nv">$INITIAL_PROMPT</span><span class="s2">"</span> <span class="se">\</span>
  <span class="nt">--compute_type</span> int8 <span class="se">\</span>
  <span class="nt">--batch_size</span> 8
</code></pre></div></div>

<p>Two knobs worth flagging:</p>

<ul>
  <li><strong><code class="language-plaintext highlighter-rouge">--compute_type int8</code></strong> is the right choice on Apple Silicon CPU. <code class="language-plaintext highlighter-rouge">float16</code> is rejected on CPU. <code class="language-plaintext highlighter-rouge">int8</code> runs at about 5x real-time on the M-series chip this pipeline runs on.</li>
  <li><strong><code class="language-plaintext highlighter-rouge">--initial_prompt</code></strong> is the single biggest accuracy lever after model size. The default prompt is short and vocabulary-only: a list of piano terms (<code class="language-plaintext highlighter-rouge">scale</code>, <code class="language-plaintext highlighter-rouge">chord</code>, <code class="language-plaintext highlighter-rouge">treble clef</code>, <code class="language-plaintext highlighter-rouge">bass clef</code>, <code class="language-plaintext highlighter-rouge">sharp</code>, <code class="language-plaintext highlighter-rouge">flat</code>, <code class="language-plaintext highlighter-rouge">natural</code>, <code class="language-plaintext highlighter-rouge">slur</code>, <code class="language-plaintext highlighter-rouge">tie</code>, <code class="language-plaintext highlighter-rouge">measure</code>, <code class="language-plaintext highlighter-rouge">harmonic interval</code>, <code class="language-plaintext highlighter-rouge">melodic interval</code>, <code class="language-plaintext highlighter-rouge">whole note</code>, <code class="language-plaintext highlighter-rouge">half note</code>, <code class="language-plaintext highlighter-rouge">quarter note</code>, <code class="language-plaintext highlighter-rouge">eighth note</code>, <code class="language-plaintext highlighter-rouge">middle C</code>, <code class="language-plaintext highlighter-rouge">finger one</code> through <code class="language-plaintext highlighter-rouge">finger five</code>, <code class="language-plaintext highlighter-rouge">left hand</code>, <code class="language-plaintext highlighter-rouge">right hand</code>, <code class="language-plaintext highlighter-rouge">forte</code>, <code class="language-plaintext highlighter-rouge">piano</code>, <code class="language-plaintext highlighter-rouge">mezzo</code>, <code class="language-plaintext highlighter-rouge">crescendo</code>, <code class="language-plaintext highlighter-rouge">staccato</code>, <code class="language-plaintext highlighter-rouge">legato</code>). Long sentence-form prompts leak into transcripts — Whisper echoes them as if they were spoken, especially on low-confidence segments. Keep the prompt a vocabulary list. Override per-run when a specific piece is being practiced.</li>
</ul>

<h2 id="stage-3-pyannote--speaker-diarization">Stage 3: pyannote — speaker diarization</h2>

<p>Transcription without speaker labels is mostly useless. A transcript that says “good job” with no idea who said it is a dead end. Diarization is the step that figures out who said what.</p>

<p>The pipeline uses <code class="language-plaintext highlighter-rouge">pyannote/speaker-diarization-3.1</code> from HuggingFace. The model clusters audio segments by voice, and the output is segments tagged with <code class="language-plaintext highlighter-rouge">SPEAKER_00</code>, <code class="language-plaintext highlighter-rouge">SPEAKER_01</code>, etc. The model downloads on first run (~50 MB) and is cached. It requires a HuggingFace token and explicit acceptance of the pyannote license.</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">pipeline</span><span class="p">({</span>
    <span class="s">"audio"</span><span class="p">:</span> <span class="n">wav_path</span><span class="p">,</span>
    <span class="s">"min_speakers"</span><span class="p">:</span> <span class="mi">2</span><span class="p">,</span>
    <span class="s">"max_speakers"</span><span class="p">:</span> <span class="mi">3</span><span class="p">,</span>
<span class="p">})</span>
</code></pre></div></div>

<p><code class="language-plaintext highlighter-rouge">min_speakers=2 max_speakers=3</code> is the right setting for a 1-on-1 lesson with occasional drop-in from a parent. Auto-detect (no constraint) over-segments the teacher’s voice into 4+ clusters because the model treats slight pitch variations as different speakers. The constraint forces clean turn-by-turn labels. It is a hint, not a hard cap — pyannote can still emit a <code class="language-plaintext highlighter-rouge">SPEAKER_UNKNOWN</code> bucket for unassigned segments when the actual count exceeds the cap. The rename step handles those.</p>

<h2 id="stage-4-rename_speakerspy--vocabulary-heuristic">Stage 4: rename_speakers.py — vocabulary heuristic</h2>

<p>Raw diarization output is <code class="language-plaintext highlighter-rouge">SPEAKER_00</code>, <code class="language-plaintext highlighter-rouge">SPEAKER_01</code>. The pipeline needs role labels: <code class="language-plaintext highlighter-rouge">Teacher</code>, <code class="language-plaintext highlighter-rouge">Parent</code>, <code class="language-plaintext highlighter-rouge">Student</code>. The rename step walks each speaker’s text and scores the speaker against a small vocabulary table:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">TEACHER_TELLS</span> <span class="o">=</span> <span class="p">[</span>
    <span class="s">"good job"</span><span class="p">,</span> <span class="s">"let's try"</span><span class="p">,</span> <span class="s">"play it again"</span><span class="p">,</span> <span class="s">"one more time"</span><span class="p">,</span>
    <span class="s">"slower"</span><span class="p">,</span> <span class="s">"faster"</span><span class="p">,</span> <span class="s">"louder"</span><span class="p">,</span> <span class="s">"softer"</span><span class="p">,</span> <span class="s">"count out loud"</span><span class="p">,</span>
    <span class="s">"look at the page"</span><span class="p">,</span> <span class="s">"next piece"</span><span class="p">,</span> <span class="s">"let's start with"</span><span class="p">,</span>
<span class="p">]</span>

<span class="n">PARENT_TELLS</span> <span class="o">=</span> <span class="p">[</span>
    <span class="s">"say thank you"</span><span class="p">,</span> <span class="s">"sit up"</span><span class="p">,</span> <span class="s">"are you listening"</span><span class="p">,</span>
    <span class="s">"say hi"</span><span class="p">,</span> <span class="s">"wave"</span><span class="p">,</span> <span class="s">"be nice"</span><span class="p">,</span> <span class="s">"share"</span><span class="p">,</span>
<span class="p">]</span>

<span class="k">def</span> <span class="nf">classify</span><span class="p">(</span><span class="n">text</span><span class="p">,</span> <span class="n">scores</span><span class="p">):</span>
    <span class="n">text_lower</span> <span class="o">=</span> <span class="n">text</span><span class="p">.</span><span class="n">lower</span><span class="p">()</span>
    <span class="k">for</span> <span class="n">phrase</span> <span class="ow">in</span> <span class="n">TEACHER_TELLS</span><span class="p">:</span>
        <span class="k">if</span> <span class="n">phrase</span> <span class="ow">in</span> <span class="n">text_lower</span><span class="p">:</span>
            <span class="n">scores</span><span class="p">[</span><span class="s">"teacher"</span><span class="p">]</span> <span class="o">+=</span> <span class="mi">1</span>
    <span class="k">for</span> <span class="n">phrase</span> <span class="ow">in</span> <span class="n">PARENT_TELLS</span><span class="p">:</span>
        <span class="k">if</span> <span class="n">phrase</span> <span class="ow">in</span> <span class="n">text_lower</span><span class="p">:</span>
            <span class="n">scores</span><span class="p">[</span><span class="s">"parent"</span><span class="p">]</span> <span class="o">+=</span> <span class="mi">1</span>
    <span class="k">return</span> <span class="n">scores</span>
</code></pre></div></div>

<p>The speaker with the highest teacher score is labeled <code class="language-plaintext highlighter-rouge">Teacher</code>. If multiple speakers tie, the tiebreaker is total speech time. Speakers that score below a threshold default to <code class="language-plaintext highlighter-rouge">Student</code>. The mapping and the raw scores are written to a <code class="language-plaintext highlighter-rouge">.speakers.json</code> file next to the transcript, so the labels can be reviewed and the heuristic retuned without re-running diarization.</p>

<p>It is not perfect. About 10% of segments end up with the wrong label, usually on the boundary between “instructional encouragement” (teacher) and “encouragement” (parent). The summary step is forgiving about this. The human reader is forgiving about this. The search index is forgiving about this.</p>

<h2 id="stage-5-claude-summary">Stage 5: Claude summary</h2>

<p>The transcript is 30-50 minutes of dense audio. The summary is a short Markdown file with five sections: Topics, Pieces, Books/Pages, Issues, Homework. It tells a reader what the lesson covered, what book pages were worked from, what was flagged as a problem, and what to practice before the next class.</p>

<p>The summarizer calls <code class="language-plaintext highlighter-rouge">claude --print</code> with a structured system prompt. The transcript goes in as the user message. The output goes to <code class="language-plaintext highlighter-rouge">&lt;stem&gt;.summary.md</code>.</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">HOME</span><span class="o">=</span>/Users/mini-1 claude <span class="nt">--print</span> <span class="se">\</span>
  <span class="nt">--system-prompt-file</span> <span class="s2">"</span><span class="nv">$PROMPT_PATH</span><span class="s2">"</span> <span class="se">\</span>
  &lt; <span class="s2">"</span><span class="nv">$TRANSCRIPT_PATH</span><span class="s2">"</span> <span class="se">\</span>
  <span class="o">&gt;</span> <span class="s2">"</span><span class="nv">$SUMMARY_PATH</span><span class="s2">"</span>
</code></pre></div></div>

<p>The <code class="language-plaintext highlighter-rouge">HOME=/Users/mini-1</code> override is required. The <code class="language-plaintext highlighter-rouge">claude</code> CLI reads its OAuth credentials from the real user home, and the agent that runs the script has a sandboxed home. Without the override, the call fails with an auth error. The orchestrator sets this explicitly.</p>

<h2 id="stage-6-the-watcher-cron">Stage 6: the watcher cron</h2>

<p>The watcher is a small bash script that runs once an hour. It checks the inbox, waits 60 seconds to confirm the file is no longer being written, then moves it to <code class="language-plaintext highlighter-rouge">_processing/</code>, runs the pipeline, and parks the output in a dated folder. If the inbox is empty, the script stays silent — no Slack noise.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Piano-Recordings/
├── _inbox/        # drop new .mp4/.mkv/.mov here
├── _processing/   # watcher moves file here while pipeline runs
├── _failed/       # watcher moves file here on pipeline error (with .error.log)
└── YYYY-MM-DD_HHMMSS/   # completed lessons
</code></pre></div></div>

<p>The watcher is a real cron job, not a long-running daemon. Empty stdout means silent — the hourly tick is designed to produce no Slack output unless there’s actual work. On activity, the watcher emits a structured summary: how many files processed, how many failed, first 10 lines of each new lesson summary.</p>

<p>A stability check is the most important detail. macOS file copy is not atomic — a file can appear in the inbox before the bytes are flushed. The watcher waits 60 seconds of unchanged file size before processing. Skipping this step causes intermittent “file not fully written” errors on slow connections.</p>

<p>A stale rescue handles the failure case: any file stuck in <code class="language-plaintext highlighter-rouge">_processing/</code> for more than 2 hours gets moved to <code class="language-plaintext highlighter-rouge">_failed/</code> with an <code class="language-plaintext highlighter-rouge">.error.log</code> sidecar. Without it, a transient pipeline failure leaves a file in limbo forever.</p>

<h2 id="output-layout">Output layout</h2>

<p>Each completed lesson gets its own dated folder:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Piano-Recordings/2026-05-30_163000/
├── Video-20260530_163000-Meeting Recording.mp4
├── Video-20260530_163000-Meeting Recording.wav
├── Video-20260530_163000-Meeting Recording.txt
├── Video-20260530_163000-Meeting Recording.srt
├── Video-20260530_163000-Meeting Recording.vtt
├── Video-20260530_163000-Meeting Recording.tsv
├── Video-20260530_163000-Meeting Recording.json
├── Video-20260530_163000-Meeting Recording.speakers.json
└── Video-20260530_163000-Meeting Recording.summary.md
</code></pre></div></div>

<p>The <code class="language-plaintext highlighter-rouge">.txt</code> is the speaker-labeled transcript. The <code class="language-plaintext highlighter-rouge">.srt</code> and <code class="language-plaintext highlighter-rouge">.vtt</code> are the same content as subtitles. The <code class="language-plaintext highlighter-rouge">.tsv</code> is the machine-readable word-level output. The <code class="language-plaintext highlighter-rouge">.json</code> is WhisperX’s full output including word-level timestamps. The <code class="language-plaintext highlighter-rouge">.speakers.json</code> is the diarization debug file (mapping, scores, threshold). The <code class="language-plaintext highlighter-rouge">.summary.md</code> is the human-readable summary. The <code class="language-plaintext highlighter-rouge">.wav</code> is kept for fast re-runs.</p>

<h2 id="performance">Performance</h2>

<p>On Apple Silicon, the <code class="language-plaintext highlighter-rouge">small</code> Whisper model:</p>

<ul>
  <li>Transcription alone: ~5x real-time (7.6 minutes of audio → ~1.5 minutes of compute)</li>
  <li>With diarization added: ~1x real-time (the pyannote model and the alignment pass roughly double the time)</li>
  <li>A 30-minute lesson completes in about 30 minutes of wall clock on the M-series chip</li>
  <li>A <code class="language-plaintext highlighter-rouge">medium</code> model is 2-3x slower; I haven’t needed it</li>
</ul>

<h2 id="pitfalls-the-things-that-broke-and-how-i-fixed-them">Pitfalls (the things that broke and how I fixed them)</h2>

<p><strong>whisperx 3.7.5 hard-pins torch ~=2.8.0.</strong> I couldn’t downgrade torch to bypass a <code class="language-plaintext highlighter-rouge">weights_only</code> issue introduced in PyTorch 2.6+. PyTorch 2.6+ defaults to <code class="language-plaintext highlighter-rouge">weights_only=True</code> for <code class="language-plaintext highlighter-rouge">torch.load</code>, which rejects the <code class="language-plaintext highlighter-rouge">omegaconf</code> classes in pyannote’s model files. The fix is a small Python wrapper that monkey-patches <code class="language-plaintext highlighter-rouge">torch.load(weights_only=False)</code> unconditionally — <code class="language-plaintext highlighter-rouge">lightning_fabric</code> passes <code class="language-plaintext highlighter-rouge">weights_only=True</code> explicitly, so a conditional patch doesn’t help. The wrapper is <code class="language-plaintext highlighter-rouge">whisperx_safe.py</code>. The pipeline invokes whisperx through this wrapper, never through the bare <code class="language-plaintext highlighter-rouge">whisperx</code> CLI.</p>

<p><strong>Long initial prompts leak into transcripts.</strong> A prompt containing book titles and series names appeared verbatim as a transcript line for a low-confidence audio segment. Whisper echoed the prompt as if it were spoken. The fix is a strict rule: the prompt is a vocabulary list, not a sentence. Per-run override is allowed for specific piece titles, kept short.</p>

<p><strong>Auto-detect over-segments the teacher’s voice.</strong> Without <code class="language-plaintext highlighter-rouge">min_speakers=2 max_speakers=3</code>, pyannote splits the teacher’s voice into 4+ clusters based on slight pitch variations. The constraint forces a clean turn-by-turn label set.</p>

<p><strong><code class="language-plaintext highlighter-rouge">~</code> in agent context expands to a sandbox home.</strong> The orchestrator script always uses absolute paths to files on the NAS and to the workspace. Shell <code class="language-plaintext highlighter-rouge">~</code> in this context points to the agent sandbox home, not the real user home, and the NAS is mounted under <code class="language-plaintext highlighter-rouge">/private/tmp/nas_movies</code>, not under a <code class="language-plaintext highlighter-rouge">/Volumes/</code> path.</p>

<p><strong>The NAS mount is at <code class="language-plaintext highlighter-rouge">/private/tmp/nas_movies</code>, not <code class="language-plaintext highlighter-rouge">/Volumes/video</code>.</strong> macOS’s auto-mount puts SMB shares under <code class="language-plaintext highlighter-rouge">/Volumes/</code>, but the SMB share on this NAS is mounted at the temporary path for a reason I no longer remember. The script uses the absolute path everywhere; nothing in the pipeline assumes <code class="language-plaintext highlighter-rouge">/Volumes/</code>.</p>

<h2 id="what-the-system-has-already-surfaced">What the system has already surfaced</h2>

<p>I have ~17 hours of audio in the archive now. Reading back through the summaries, the transcripts surface things the videos alone never would have. A few worth flagging:</p>

<p><strong>The 38-minute lesson.</strong> Z.M. had a lesson that ran almost 40 minutes — twice the normal length — because he got frustrated with a piece and refused to play. The teacher didn’t push. She sat with him. She let him try, let him fail, let him try again. The summary captured, in its own words, that the student experienced frustration, the teacher used patience, and the piece was eventually completed. That is what happened. The transcript is the only reason I know the texture of how that lesson actually went.</p>

<p><strong>A.M. coaching Z.M.</strong> A.M. has been playing longer. In one of Z.M.’s lessons, after the teacher explained the sharp-within-measure rule, A.M. was at the piano before the next call and walked Z.M. through it himself. The summary noted that the older sibling provided peer instruction during an inter-lesson period and demonstrated an internalized grasp of the rule. I would not have written that sentence in a thousand years. The model surfaced it because diarization caught a third voice during what was supposed to be a one-on-one lesson. I had no idea it was happening. The model did.</p>

<p><strong>“Rockin’ Rabbits.”</strong> One of the first pieces in the book is called “Rockin’ Rabbits.” The model — at least in early runs — kept hearing it as “Rockin’ Rabies.” A five-year-old saying the word “rabbits” produces audio the model is not confident about, and its first guess was a different small animal entirely. I have at least four transcripts in a row that all say “Rockin’ Rabies.” It became a family joke. The model got it wrong consistently enough that the wrongness became a record in its own right — a small data point on what a five-year-old’s voice sounds like to a model trained mostly on adult speech.</p>

<h2 id="what-the-transcripts-make-possible-that-the-videos-dont">What the transcripts make possible that the videos don’t</h2>

<p>The pipeline was supposed to be an archive. It became a study tool. The reason is that text is searchable, indexable, and reviewable in a way video isn’t. A few of the things the transcripts let us do that the videos alone never could:</p>

<p><strong>Vocabulary growth across time.</strong> A simple <code class="language-plaintext highlighter-rouge">grep "staccato" *.txt</code> across all 17 weeks of lessons returns every mention, in order, with the date. You can see when the teacher first introduced the term (lesson 4), how many times she repeated it across the first month, when one of the boys started using it correctly, and when she stopped having to explain it. The vocabulary of piano has a curriculum — techniques like staccato, legato, curved fingers; rhythms like eighth-note pairs, syncopation, dotted-quarter + eighth; tempo terms like andante, allegro. The transcripts are the curriculum rendered in a searchable form. As a parent, I can see which concepts are landing and which the teacher has to keep re-teaching. The list is the teaching plan. The transcripts are its audit log.</p>

<p><strong>The “Issues” list is a weekly diagnosis.</strong> Every summary has a section called “Issues” — what the teacher flagged as problems that week. Reading 17 weeks of Issues in chronological order is like reading a doctor’s chart: “curved fingers, slowing down on the upbeat, missing the repeat sign.” Each issue is a target. At practice time, we know which of the boys’ instincts the teacher is still correcting, and we listen for them. The video doesn’t give you that. The video gives you 30 minutes of footage and the lesson is over.</p>

<p><strong>Piece progression, measured.</strong> You can see when a piece was introduced (the teacher says “let’s start with X”), when it was reviewed (“let’s play X again this week”), and when it was retired (“you know this one now”). Across 17 weeks the boys worked through 11 pieces, and the transcripts are the only record of how long each one took to land. “Hot Cross Buns” took two weeks. “Alpine Song” took six. The data is in the text. The videos would require a manual review we never would have done.</p>

<p><strong>Look-back without rewinding.</strong> When the teacher says, mid-lesson, “remember what I told you about X last week,” the transcripts let you look up last week’s lesson and find X. With a video, you’d have to scrub through 30 minutes of footage and hope you remembered the timestamp. With a transcript, it’s a 5-second search. This changed practice at home. We started being able to answer our own questions about what the teacher had said, instead of waiting until the next lesson to ask.</p>

<p><strong>The 1% better pattern.</strong> Reading the summaries in order, a quiet pattern emerges: each boy gets 1% better at one thing every week. The first month is mostly counting out loud. The second month is mostly fingering. The third month is mostly dynamics. The summaries surface the change without me having to find it. The model is the one doing the diff. Reading 17 weeks of “what changed this week” answers the question I would have asked, if I’d thought to ask it: how do you get better at piano? Slowly, one thing at a time, with a teacher who notices.</p>

<p>The pipeline was supposed to be a record. The pipeline is now a teacher aid, a study tool, and a parent cheat sheet. None of that was the original plan. All of it was enabled by the fact that the lessons became text.</p>

<h2 id="what-the-model-still-gets-wrong">What the model still gets wrong</h2>

<p>The archive is not the lessons. It is an approximation of the lessons. The model still mishears proper names on the first run about a third of the time. When two students play the same phrase in unison, diarization merges their voices and assigns the line to one speaker. The summaries smooth over small moments — a teacher telling one of the boys to sit up, a parent gently correcting which one actually needed to sit up — because the model averages across many small choices, and the right one is in the average, but not the one it picks.</p>

<p>I have to keep remembering that. The temptation to treat the transcript as authoritative is the new version of believing the camera saw the truth. The camera didn’t see the truth. The transcript isn’t the lesson. Both are flattened versions of something that happened between a few people in a room with a piano. The archive is a record that the lesson happened. It is not a record of what the lesson was.</p>

<p>But a record that the lesson happened is enough. It is more than I had before. It is more than the average family has when the boys are grown.</p>

<h2 id="whats-next">What’s next</h2>

<p>The current pipeline is enough. It runs itself, the lessons are searchable in a way the videos alone never were, and the archive is a permanent record. The next moves are nice-to-haves:</p>

<ul>
  <li><strong>Voice fingerprinting.</strong> Replace the vocabulary heuristic with voiceprint ID. Once the teacher, parent, and each student have a stable voiceprint, diarization labels speakers directly without scoring their text. This would clean up the 10% mislabel rate.</li>
  <li><strong>Cross-lesson search.</strong> The transcripts are <code class="language-plaintext highlighter-rouge">.txt</code> files. A simple ripgrep across all dated folders answers questions like “when did harmonic intervals come up.” A proper search index (SQLite FTS or Meilisearch) would make this faster and more flexible.</li>
  <li><strong>Per-lesson practice notes.</strong> A second pass that produces a short note aimed at the student, summarizing what was learned and what to work on, in language a young student can read.</li>
</ul>

<p>The pipeline is the goal. Everything after that is polish.</p>]]></content><author><name>Mumen Musa</name></author><summary type="html"><![CDATA[From a Microsoft Teams recording to a dated folder of transcripts, subtitles, and summaries — with no manual steps after the file lands.]]></summary></entry><entry><title type="html">Researching a village the internet forgot</title><link href="https://mumen.musa.ai/2026/06/02/researching-a-village-the-internet-forgot/" rel="alternate" type="text/html" title="Researching a village the internet forgot" /><published>2026-06-02T09:00:00-07:00</published><updated>2026-06-02T09:00:00-07:00</updated><id>https://mumen.musa.ai/2026/06/02/researching-a-village-the-internet-forgot</id><content type="html" xml:base="https://mumen.musa.ai/2026/06/02/researching-a-village-the-internet-forgot/"><![CDATA[<p>There is a village north of Ramallah called Turmus Ayya. My family is from there. Wikipedia gives it a handful of paragraphs and the usual census stub. I wanted more — not for a paper, not for a project, just for myself. So I sat down one evening with a language model and asked it to help me dig.</p>

<p>What follows is the process, and a few of the things it turned up. The full report runs <a href="/assets/reports/turmus-ayya-comprehensive-report.pdf">twenty-four pages</a>. This is the part I think is worth writing down.</p>

<h2 id="the-first-prompt">The first prompt</h2>

<p>I told the model what village I was looking for, gave it three spelling variants, and added one clause that turned out to matter more than anything else in the rest of the conversation:</p>

<blockquote>
  <p>Even if it has information about other villages around that might reference it, or at least relatively or very closely mentions it.</p>
</blockquote>

<p>That single instruction is what turned up <em>Turbasaim</em> — a Crusader-Latin form of the name that I had never seen in any modern source. <em>Turbasaim</em> doesn’t appear in Wikipedia. It doesn’t appear in the village’s own profile pages. It shows up in a nineteenth-century Palestine Exploration Fund nomenclature study that itself was reading twelfth-century church cartularies.</p>

<p>I asked the model where, exactly. It pointed me at Eugène de Rozière’s <em>Cartulaire de l’église du Saint-Sépulcre de Jérusalem</em>, published in 1849 from Vatican manuscripts, and at Reinhold Röhricht’s <em>Regesta Regni Hierosolymitani</em>, the master registry of Crusader-era charters. The entry is dated <strong>14 August 1145</strong>. Patriarch William I of Jerusalem is settling a dispute between Pontius, abbot of Mount Tabor, and Peter, prior of the Holy Sepulchre. The patriarch awards Mount Tabor the church at <em>casale Sancti Egidii</em> — the village of St. Gilles, which is the next village over from mine, today called Sinjil — together with half the tithes of <em>Turbasaim et Dere</em>.</p>

<p>Half the tithes. That’s the thing that lodged in my head. The Crusader Latin doesn’t grant anyone the village. It grants <em>half the income on agricultural produce</em> from the village to the church next door. Eight hundred and eighty years ago, my grandparents’ village was an income stream in a dispute between two French monastic orders, written down in a manuscript that eventually ended up in the Vatican.</p>

<p>The name had been sitting in a footnote for a hundred and forty years.</p>

<h2 id="what-the-research-actually-turned-up">What the research actually turned up</h2>

<p>The model produced a <a href="/assets/reports/turmus-ayya-comprehensive-report.pdf">24-page comprehensive report</a>. Here are the findings that surprised me most.</p>

<p><strong>The 1145 grant has a sequel.</strong> Thirty years later, on <strong>17 October 1175</strong>, the same two institutions met again. This time the Holy Sepulchre got the church back — along with the half-tithe rights to St. Gilles, <em>Turbasaym</em>, and <em>Dere</em>. The reciprocal act from Mount Tabor also mentions <em>vineas vero et domos</em>, “vineyards and houses,” sold for a sum in bezants. So the ecclesiastical presence was not just a fiscal abstraction. There were physical houses and vineyards owned by the Latin Church in or near my village in 1175.</p>

<p><strong>The story doesn’t end with the Crusades.</strong> When Frankish power collapsed, the village did not vanish from the documentary record. It got picked up by the next regime. In <strong>1268–69 CE</strong> (668 AH), the Mamluk sultan <strong>Baybars</strong> founded a waqf — an Islamic charitable endowment — for the shrine of <strong>Nabi Musa</strong>, near Jericho. The deed, edited by Kamil al-Asali, lists the villages whose revenues would support the shrine. One of them is written <strong>ترمسعيا</strong> — <em>Turmusayya</em>. Already, by 1268, the name is in recognizably modern Arabic form. So the documentary line for my village runs: 1145 Crusader Latin → 1175 Crusader Latin → 1268 Mamluk Arabic → 1596 Ottoman tax register → 1838 American missionary survey → 1882 British survey → 1922, 1931, 1945 British censuses → today. Across eight centuries, three empires, and three languages, the name moved from <em>Turbasaim</em> to <em>Turmusayya</em> to <em>Turmus Ayya</em> without ever breaking.</p>

<p><strong>The “Dere” question.</strong> The 1145 and 1175 acts always pair Turbasaim with another place called <em>Dere</em>. Nobody knows for certain where Dere was. Conder, in the nineteenth century, proposed <em>Deir es-Sudan</em>, west of Sinjil. Twentieth-century scholarship moved the identification east, to <strong>Khirbet Deir el-Fikia / Ras ad-Deir</strong>, where Palestine Exploration Fund surveyors found “the ruins of a monastery and chapel” with masonry they identified as Crusader work. If that identification is right — and the consensus now leans that way — then the medieval landscape around my village included a small monastery whose stones are still on the ground. I had never heard of it.</p>

<p><strong>The negative result on Tur Shimon.</strong> Some twentieth-century scholars proposed that Turmus Ayya was the ancient <em>Tur Shimon</em> mentioned in rabbinic sources, partly on the resemblance of the names. The Israeli archaeologist Boaz Zissu argued against that identification — not on philological grounds but on dirt. Surface survey at Turmus Ayya produced sherds from several periods <em>but none from the Hellenistic period</em>. If the village were Tur Shimon, you’d expect Hellenistic-era pottery. There isn’t any. That’s the kind of finding a casual search will never give you, because nobody writes a Wikipedia paragraph about an identification that didn’t hold up.</p>

<p><strong>The sarcophagus.</strong> In <strong>1912</strong>, a richly carved marble sarcophagus was discovered near the village. Jacob E. Spafford of the American Colony in Jerusalem described it the next year. German art-historical cataloging dates it to <strong>240–250 CE</strong> and identifies the carving program as <em>Bacchus and the genii of the seasons</em>. So there was, in the Roman period, somebody important enough in the area to be buried in an imperial-quality sarcophagus carved with Dionysus. The model surfaced the German catalog entry, which an English-only search would never have found.</p>

<p><strong>The diaspora is real but the number is folklore.</strong> Multiple journalistic sources describe Turmus Ayya as a town where more than 80 percent of residents hold U.S. citizenship, with secondary migration to Latin America — Panama in particular. The model initially repeated the 80 percent figure as fact. When I asked where the data came from, the honest answer was: no official source. It is a journalistic estimate that has self-replicated across dozens of news articles. The underlying <em>pattern</em> is well-documented: large diaspora, remittance-built villas, seasonal occupancy. The specific number is folklore that sounds like data.</p>

<p><strong>The town is still being made and unmade.</strong> The model also surfaced something I knew but had not seen catalogued: the B’Tselem documentation of repeated settler attacks on Turmus Ayya in 2023, including arson against vehicles belonging to named members of the Abu ‘Awwad family — one of the village’s older lineages, attested in local material and continuous enough that the same family name shows up in modern human-rights litigation. The historical record for this village is not closed. It is being added to in real time, in a register very different from twelfth-century cartularies, but no less archival.</p>

<h2 id="the-matching-problem">The matching problem</h2>

<p>The reason a clause like “even something close” mattered: this is a place whose name has been written in Arabic, Ottoman Turkish, Latinized Crusader French, nineteenth-century English, nineteenth-century German, and modern transliterations that don’t agree with each other. The British Mandate’s own census officials wrote in 1931 that the transliteration system they were using was already being abandoned, <em>while the census was still going to press</em>.</p>

<p>If you search for “Turmus Ayya” you find the modern village. If you search for “Turmus ‘Aiya” you find the 1931 census. If you search for “Tourmous’aya” you find a French art-history article about a Roman sarcophagus. If you search for “Turbasaim” you find a 1145 land grant. If you search for ترمسعيا you find a 1268 Islamic waqf.</p>

<p>These are all the same place. None of them link to each other.</p>

<h2 id="what-i-learned-about-prompting">What I learned about prompting</h2>

<p>A few things crystallized over the course of the research that I want to remember.</p>

<p><strong>Ask for the negative result.</strong> The single most valuable thing the model surfaced wasn’t a confirmation, it was a contradiction — the Zissu argument that the Tur Shimon identification doesn’t hold up. Negative findings clarify what the evidence actually supports. You have to ask for them directly.</p>

<p><strong>Source classes matter more than source counts.</strong> I told the model to prioritize in roughly this order: official censuses, primary historical publications, explorer/survey literature, archaeology, and only then secondary compilations or local profiles. The reason is that local profiles tend to silently compress together primary facts, antiquarian conjectures, and oral etymologies. Without explicit ranking, the model treats them all as equally citeable. With ranking, it tells you when it’s reaching for the bottom of the stack.</p>

<p><strong>Filter your false positives early.</strong> Two categories of garbage results plagued every search: there is another place called <em>Tall al-Turmus</em> in Gaza, which is not the same village; and <em>turmus</em> in Arabic also means <em>lupini beans</em>, which dominate the search results for anything resembling the name. Telling the model these two patterns existed before it started cut the noise in half.</p>

<p><strong>Push deeper on the thread that surprises you.</strong> When <em>Turbasaim</em> came back, I didn’t move on to the next question. I spent the next several turns just on Crusader cartularies — which is how the 1175 follow-up, the Dere identification debate, and the Baybars waqf all surfaced. The diminishing returns on a single thread come much later than you expect.</p>

<p><strong>Treat the variant table as a precondition.</strong> Every script, every spelling, every transliteration convention, before issuing a single search. <em>Turbasaim, Turbasaym, Turbasym, Tourmous’aya, Turmus ‘Aiya, Turmus ‘Aya, ترمسعيا, Thormasia, Tur Shimon</em> — list them all up front, decide which are evidentiary and which are conjectural, then go.</p>

<h2 id="what-it-got-wrong">What it got wrong</h2>

<p>It is not a perfect collaborator.</p>

<p>It conflated dates more than once. An English-language summary called the sarcophagus “2nd century B.C.”, and a separate German art-history catalog dated the same object to 240–250 CE. The model initially reported both as if they were consistent. I had to push it to flag the discrepancy. When I did, it correctly noted that one source was a popular retelling and the other was specialist cataloging — but it didn’t volunteer that distinction on its own.</p>

<p>It over-anchored on the most-cited claim. The 80 percent diaspora figure appears in dozens of journalistic sources. The model repeated it confidently several times before I asked where the underlying data came from. There is no underlying data. It is an estimate that has self-replicated.</p>

<p>It is bad at adjudicating what <em>can’t</em> be known. Some questions — the exact pre-Roman name of the site, the full hamula genealogy of the village’s families — have answers that simply don’t exist in any accessible archive. The model wants to give you an answer. You have to actively push back and ask it to enumerate the limits of the evidence, not just summarize the evidence.</p>

<p>It cannot, on its own, distinguish <em>Turbasaim is securely attested in 1145</em> from <em>Turbasaim is identified with modern Turmus Ayya by later scholarly inference</em>. Those are two different epistemic claims. Holding them apart was one of the hardest parts of the conversation, and it required me — not the model — to keep reasserting the distinction.</p>

<h2 id="what-id-do-differently-next-time">What I’d do differently next time</h2>

<p>Three things.</p>

<ol>
  <li>
    <p><strong>Start with the negative-result prompt earlier.</strong> Asking “what identifications for this place have <em>failed</em>, and why?” should be one of the first questions, not one of the last. It calibrates everything that follows.</p>
  </li>
  <li>
    <p><strong>Build the variant table before doing any searches.</strong> I had to back-fill it after I’d already done a lot of single-spelling searches. Future-me should treat the variant table as a precondition, not a deliverable.</p>
  </li>
  <li>
    <p><strong>Ask for the source class of every claim, every time.</strong> Not “is this true,” but “what kind of source is this.” Charter versus regesta versus survey versus secondary compilation versus folk etymology. The five categories require five different levels of trust, and the model will assign all five the same confidence if you don’t make it differentiate.</p>
  </li>
</ol>

<hr />

<p>This is the first post on this blog. I want to write more like it — research notes, builds, things I’m working through. Mostly for future-me. The next one is probably about Obsidian, six years in.</p>

<p>If you have your own deep-research project — a hometown, a family name, an ancestor, a forgotten place — I’d encourage you to try this. The internet hasn’t forgotten the things you think it has. It has just filed them under names you don’t know yet. Mine was filed under <em>Turbasaim</em> in a Vatican manuscript and <em>ترمسعيا</em> in a Mamluk waqf, and it took the right kind of prompt to put them in the same place.</p>]]></content><author><name>Mumen Musa</name></author><summary type="html"><![CDATA[There is a village north of Ramallah called Turmus Ayya. My family is from there. Wikipedia gives it a handful of paragraphs and the usual census stub. I wanted more — not for a paper, not for a project, just for myself. So I sat down one evening with a language model and asked it to help me dig.]]></summary></entry></feed>