Six Years Later: Rebuilding MusicMixerSix Years Later: Rebuilding MusicMixer

MusicMixer was not a startup. It did not change the music industry. It was a Discord bot made by a middle schooler who wanted to play distorted songs in a voice channel, followed by a Flask website that spent more time fighting YouTube than mixing music.

Then it died.

Six years later, I rebuilt it as a private audio studio that runs FFmpeg inside the browser. That sentence makes the project sound tidier than it was. The surviving code tells a much better story: two Discord implementations totaling roughly 2,700 lines, a 166-line Flask backend, several incompatible ideas about queues and files, and comments such as “Only Works on Pycharm, not Vultr.”

This is a technical archaeology of all three versions: what each request actually did, why the original designs were reasonable to me at the time, where they failed, and how the same product promise eventually produced a completely different architecture.

Before MusicMixer, there was Taco Bot

In 2020 I built Taco Bot, a Discord bot for taco pictures, trivia, virtual taco coins, and whatever else sounded entertaining during lockdown. It reached more than 150 servers and became my first software with users I did not know.

That changed the way I built. People asked for features, noticed outages, and found states I had never tested. Taco Bot taught me APIs, asynchronous Python, deployment, and the memorable lesson that an in-memory currency disappears when a free Heroku dyno restarts.

MusicMixer was the natural next idea. Discord already had voice channels; I had just learned how commands and bots worked; FFmpeg could perform almost any audio operation I could imagine. The pitch was wonderfully direct:

Play. Mix. Download.

The bot would accept a YouTube URL or search, join the requester's voice channel, play the result, and optionally return a modified audio file. It could change speed and pitch, add echo, reverse a track, and produce the sort of bass boost that was much funnier through cheap headphones.

The interesting part is how much machinery those three verbs required.

Architecture one: a Discord command became a media job

The surviving bot files contain two overlapping approaches. One creates a direct FFmpeg stream from the media URL returned by youtube-dl. For paths that needed a file, I first tried downloading an MP3 directly, but that became finicky and incredibly slow. I switched to downloading a temporary WebM file, then used FFmpeg to convert it into the MP3 that Discord could play or attach to a message. The extra conversion step was substantially faster and more reliable than asking the downloader for an MP3 from the start.

Together, those paths were trying to satisfy three different needs:

• normal playback should begin quickly;
• file-backed jobs should avoid the slow, unreliable direct-MP3 download path; and
• playback, transformations, and downloadable results need a finished MP3 that Discord can consume.

1. 01A user types a mix command with a YouTube search, URL, or Discord attachment.
2. 02The bot resolves the request with youtube-dl or yt-dlp and reads its title, duration, thumbnail, and stream URL.
3. 03For direct playback, FFmpeg reads the remote stream. When the bot needs a file, it downloads a temporary WebM instead of requesting an MP3 directly.
4. 04FFmpeg converts that WebM into an MP3, applies any requested filters, and exposes audio for playback or writes an attachable result to disk.
5. 05The bot stores playback state in per-guild dictionaries, sends status messages or an attached result, and tries to delete every temporary file.

Command dispatch

The bot used discord.py's command decorators. Commands such as mix play, mix pause, mix queue, and mix bassboost were ordinary asynchronous Python functions receiving a Discord context object.

One early helper generated every uppercase and lowercase combination of the prefix “mix ” so users did not have to remember its casing. That means four letters became sixteen registered prefix variants. A later bot configuration also enabled case-insensitive commands, making the combinatorial helper unnecessary. It is a perfect artifact of how I learned: solve the behavior first, discover the abstraction later.

Before doing media work, a command checked whether the requester was in a voice channel. It attempted to connect, reused the guild's existing voice client when that failed, and reported an error if Discord still showed no connection. A voice-state event watched for the last human leaving a channel, waited two seconds so the bot's exit looked less abrupt, found the matching voice client, and disconnected it.

That is product behavior hiding inside event handling. The bot did not merely play bytes; it had to behave like a considerate participant in every server.

Resolving media without freezing the bot

youtube-dl extraction is blocking work. Running it directly inside an async command would stop the Discord event loop from responding to other servers. The better of the two implementations used loop.run_in_executor to move extraction to a thread:

partial = functools.partial(
    ytdl.extract_info,
    search,
    download=False,
    process=False,
)
data = await loop.run_in_executor(None, partial)

It first resolved a search into a webpage, then performed a second extraction to obtain a usable media URL. For normal playback, that URL went directly into FFmpegPCMAudio with reconnect options. Discord received decoded PCM without requiring MusicMixer to retain the whole song.

That streaming path was the cleanest part of the old architecture. It separated metadata lookup from media transport, kept disk use low, and respected the event loop. It was also dependent on a short-lived remote stream URL and on youtube-dl continuing to understand YouTube.

The transformation path could not be quite as ephemeral. It asked the extractor for a temporary WebM, chose a filename, then invoked FFmpeg against that local file to produce the final MP3. WebM was not the desired user-facing format. It was the faster, more dependable intermediate format.

The queue was five dictionaries and a callback

Playback state lived in module-level dictionaries:

queues = {}
nowplay = {}
playin = {}
songs = {}
loop = {}

Each key was a Discord guild ID. queues held playable audio sources; songs held parallel human-readable labels; nowplay held the current label; and playin acted as a coarse “a transformation is already running” flag.

When Discord finished one source, its voice callback called playqueue. The function popped the next audio object and the next label, updated “now playing,” and registered itself again as the completion callback. Skip worked almost accidentally elegantly: stopping the current voice source triggered the same callback, which advanced the queue.

It worked, but the state had no single owner. Removing index 3 required removing index 3 from two lists and trusting that they remained aligned. A restart erased every queue. A failure between the two mutations could give the audio source the wrong title. Creating a real Song class in one version improved the presentation, but most queue operations still depended on parallel global collections.

Today I would model one queue item containing its source, metadata, status, and requester. In 2020, multiple dictionaries felt like a database.

FFmpeg was the real product

The bot's commands were mostly different wrappers around FFmpeg filters:

Legacy MusicMixer FFmpeg filters

Feature	Legacy filter
Faster	atempo=2.0
Slower	atempo=0.5
Higher pitch	resample, raise the effective sample rate, then compensate tempo
Lower pitch	resample, lower the effective sample rate, then compensate tempo
Echo	aecho
Reverse	reverse for video and areverse for audio
Bass distortion	acrusher with an extremely aggressive preset

Pitch deserves a closer look. Changing a track's sample rate changes both its pitch and its speed. The legacy command changed the effective rate, resampled back to 44.1 kHz, and then used atempo to partially compensate for the speed change. The idea was sound even though the constants were hand-tuned and varied between files.

The implementation around those filters was much less reliable:

subprocess.Popen(
    'ffmpeg -i ' + input_name + filter_text + output_name,
    shell=True,
)
await asyncio.sleep(4)
if not os.path.isfile(output_name):
    await asyncio.sleep(6)

A four-second sleep is not process coordination. A ten-minute track and a ten-second clip do not finish on the same schedule. The code sometimes checked whether the output existed and waited another six seconds, but a file can exist before FFmpeg has finished writing it.

Another helper was better: asyncio.create_subprocess_shell followed by proc.communicate() actually waited for completion and captured stdout, stderr, and the exit code. The codebase contains both approaches, which shows the system halfway through learning how subprocesses work.

NoteThe shell boundary was also a security boundary+

The old bot and Flask site built commands by concatenating URLs, filenames, and output names into a string passed to a shell. Because some of those values came from users, a specially constructed name could be interpreted as shell syntax rather than an FFmpeg argument.

Random filenames reduced accidental collisions; they did not make the shell safe. The correct fix is to avoid the shell and pass a validated argument array directly to the process. The current compiler does exactly that.

Temporary files were the hidden system

Every modified request created at least two files: a downloaded source and a generated output. The bot then had to decide when both were safe to delete.

That sounds easy until playback, queues, and Discord attachments overlap:

• deleting an input before FFmpeg finishes breaks conversion;
• deleting an output immediately after queueing can break later playback;
• retaining every output eventually fills the host's disk;
• scanning the working directory for filenames containing “youtube” can capture another request's file; and
• absolute paths that work on my Mac do not exist on a Linux host.

The code contains paths for my Desktop, /root/mixer, and a Vultr deployment. It also contains several sequences that create an FFmpegPCMAudio object, add it to a queue, and remove the backing file immediately. That might appear to work when FFmpeg has already opened the file on a Unix machine, then fail for a later queue item or another operating system.

Duration checks, including one hour for expensive transformations and sometimes two or three hours elsewhere, were an early attempt at resource governance. Per-guild playin flags were another: do not let the same server start a second transformation while one is running. But those flags were manually set and reset across many duplicated command branches. An exception could leave a guild permanently “busy” until the process restarted.

The lesson was not merely “write cleaner Python.” Media lifetime and job state were part of the product domain. Treating them as cleanup after the interesting FFmpeg call made the whole system fragile.

Architecture two: the bot escaped into Flask

Running a Discord music bot was becoming unpleasant. Voice connections changed, downloads failed, and the application needed an always-on machine containing FFmpeg, a downloader, and enough temporary storage. Moving the workflow to a website removed the voice-channel lifecycle and made the product easier to understand: provide media, choose an operation, download the result.

The Flask backend had two primary endpoints:

• /convert accepted a media link, output format, and output name;
• /modify accepted either a link or uploaded file plus an audio effect.

1. 01A synchronous Flask route reads form fields and, for uploads, a multipart file.
2. 02The route creates a new asyncio event loop for its request thread.
3. 03yt-dlp downloads or converts the linked media, or Werkzeug saves the uploaded file under a sanitized input filename.
4. 04For modifications, the backend maps a named effect to an FFmpeg filter and starts a shell subprocess.
5. 05Flask returns the generated file with send_file and immediately attempts to remove it.

The website was a genuine product improvement. It separated “modify a file” from “play something in Discord,” supported more output formats, and made the workflow accessible without adding a bot to a server.

Architecturally, however, it moved the same filesystem pipeline behind HTTP. The async functions still performed blocking extractor work. Each synchronous route created a new event loop and called run_until_complete, which added async ceremony without creating background processing. The request stayed open until downloading and FFmpeg finished.

There was still one shared upload directory, no durable job record, and no per-request workspace. The backend tried to delete a generated file in the same return expression as send_file. Depending on the server and operating system, that can race the response's consumption of the file. Failure handling usually returned the homepage with a generic message, leaving it unclear which stage had failed or which intermediate files remained.

The backend also accepted user-controlled names and constructed shell commands from them. secure_filename protected an uploaded input filename, but link inputs, output names, and the combined command still crossed an unsafe shell boundary.

None of this is a criticism of Flask. Flask did exactly what I asked it to do. The problem was that I had converted an interactive media job into a long HTTP request without designing job isolation, lifecycle, or trust boundaries.

Why MusicMixer disappeared

The old system depended on three things I did not control:

1. YouTube's current delivery behavior;
2. youtube-dl or yt-dlp adapting to that behavior; and
3. a free host allowing media downloading, FFmpeg execution, and enough disk activity.

When a download failed, it was difficult to know which layer had changed. PythonAnywhere eventually restricted the downloader calls MusicMixer relied on unless I moved to a paid plan. That was a reasonable hosting policy and an unreasonable expense for a student maintaining an old side project.

The Discord implementation already admitted the deployment split in its own comments: an operation worked in PyCharm but not on Vultr. The Flask rewrite changed the interface without removing the infrastructure dependency. As school became busier and hosting became less free, MusicMixer quietly stopped being worth the operational effort.

The project was archived because its most important feature depended on the least stable part of its architecture.

Rebuilding the promise instead of the code

When I returned to MusicMixer in 2026, I did not begin by porting Python to TypeScript. I began by deciding what the product was allowed to assume.

The old promise was still good:

Give MusicMixer some media, make understandable choices, and receive the version you wanted.

Almost everything underneath that promise could change. The new constraints were:

• media should never be uploaded to an application server;
• the hosted app should have no account system or analytics;
• every job should own its inputs, outputs, state, and cleanup;
• FFmpeg commands should be compiled from validated data, not interpolated shell text;
• expensive work should never block the interface;
• interrupted work should become restartable rather than disappearing;
• browser memory and storage limits should be visible before a job starts; and
• YouTube import should not return merely because the old product had it.

Those choices turned MusicMixer into a static Next.js application. The host serves code and version-pinned FFmpeg WebAssembly assets. The user's device does the media processing.

Architecture three: a local audio engine in the browser

The current application separates product state, command compilation, media storage, and execution.

1. 01The browser receives local File objects and stages them in a job-specific OPFS directory.
2. 02A versioned, validated recipe describes the operation, output format, timing, and sound transformations.
3. 03A deterministic compiler turns the recipe into FFmpeg argument arrays and one or more output plans.
4. 04A dedicated Web Worker copies inputs into FFmpeg’s temporary WebAssembly filesystem and runs each plan.
5. 05Completed outputs return to job-specific OPFS storage; IndexedDB records the job state and references.
6. 06The interface exposes progress, cancellation, retry, deletion, individual downloads, and ZIP export.

Recipes replaced command branches

The old bot had a large function for each effect, repeating download, duration checks, FFmpeg startup, queue insertion, messaging, and cleanup. The new site represents the user's intent as a versioned RecipeV1:

type RecipeV1 = {
  version: 1
  operation: 'convert' | 'extract' | 'trim' | 'split' | 'merge'
  output: {
    format: OutputFormat
    bitrateKbps: number
    sampleRate: number
    channels: number
  }
  trim: { start: number; end: number | null }
  split: SplitSettings
  transforms: TransformSettings
}

Validation constrains speed to 0.5–2×, pitch to one octave in either direction, gain and tone controls to explicit ranges, fades to sixty seconds, split counts to fifty outputs, and incompatible combinations such as normalization plus manual gain.

The UI offers friendly presets: Original, Voice, Music, and Small file. Each preset produces the same recipe structure as the advanced controls. There is one execution path whether someone clicks “Voice” or manually chooses mono, 44.1 kHz, voice cleanup, compression, and −16 LUFS normalization.

A command compiler replaced the shell

The compiler takes a recipe plus trusted internal input paths and produces argument arrays. User-visible filenames are normalized into bounded output names; nothing invokes a shell.

It also owns the transformation order. A job can trim, change speed, shift pitch, resample, choose channels, clean voice noise, adjust bass and treble, compress dynamics, add echo, normalize loudness, reverse, and fade. Filter order matters: fading before reversing or normalizing before gain would produce a different result.

Pitch is now expressed in semitones. The compiler calculates 2 ** (semitones / 12), adjusts the effective sample rate, resamples to 48 kHz, and chains tempo filters to restore duration. The tempo helper decomposes values outside FFmpeg's 0.5–2 range into multiple legal filters, even though the current UI remains within that range.

Split operations compile into several output plans. Merge compiles an ordered concat graph. Compatible video can be copied; cross-container or speed-changing video is marked as an expensive transcode so the interface can warn before starting it.

The worker owns execution

FFmpeg WebAssembly runs inside a dedicated Web Worker. The interface and worker communicate through typed messages: load, probe, run, cancel, progress, completion, cleanup, and structured errors.

For each job, the worker:

1. loads either the multithread or compatibility FFmpeg core;
2. copies OPFS inputs into FFmpeg's in-memory filesystem;
3. validates that every input contains a supported audio stream;
4. compiles the recipe;
5. executes each output plan while forwarding aggregate progress;
6. writes completed bytes back to OPFS; and
7. removes the temporary WebAssembly files in a finally block.

Cancellation terminates FFmpeg, creates a fresh engine instance, and leaves the job in an explicit cancelled state. This is more expensive than pretending an arbitrary FFmpeg operation can always stop cleanly, but it gives cancellation a defined result.

Errors are translated into user-facing categories such as invalid input, storage exhaustion, engine failure, and cancellation. Recent FFmpeg logs remain available as diagnostic detail without becoming the only thing the user sees.

OPFS and IndexedDB have different jobs

The Origin Private File System holds large binary inputs and outputs. IndexedDB holds small structured job records: recipe, status, progress, phase, errors, and paths to the binary media.

That separation matters. Serializing large media blobs into application state would make queue updates expensive. Storing only files would leave no durable record of what operation they belonged to.

On reload, jobs recorded as probing or running are converted to “ready” with an “Interrupted, ready to restart” phase. The browser cannot promise that work survives a closed tab, but MusicMixer can preserve enough state to make the interruption recoverable.

The queue runs sequentially. That is intentional resource control, not a missing optimization. Multiple concurrent FFmpeg WebAssembly instances would multiply memory pressure on the same device. Each non-merge input becomes its own job; merge inputs become one ordered job.

Threads, isolation, and offline behavior

When crossOriginIsolated and SharedArrayBuffer are available, the worker loads the multithread FFmpeg core. Otherwise it loads a single-thread compatibility build. Production sends COOP and COEP headers to enable isolation, along with tighter resource, framing, and permissions policies.

A service worker caches the application shell and the selected FFmpeg engine. After those assets are present, the interface can load offline and active work does not depend on a network connection. “Local-first” here includes both the media and the tool required to process it.

The browser is not magic

Moving compute to the client removes an upload server; it does not remove constraints.

WebAssembly FFmpeg is slower than native FFmpeg. Inputs must be copied from OPFS into the engine's working filesystem, which temporarily increases storage and memory use. Browsers can evict storage, suspend a background tab, or terminate a large process.

MusicMixer rejects inputs at the documented 2 GB WebAssembly boundary. Before a job starts, it estimates a working set from input bytes, output bitrate or uncompressed PCM size, and a 256 MB engine allowance. It asks for confirmation when the estimate is above 1 GB, consumes more than half the reported free storage, or requires an expensive video transcode.

The storage panel shows usage and quota, requests persistent storage when possible, and offers explicit removal of individual jobs or all local media. This is the lifecycle model the old random-filename cleanup was reaching for.

NoteHow I verify the privacy claim+

The test suite does more than look for “No uploads” in the interface. A Chromium test generates WAV files in memory, runs real FFmpeg WebAssembly processing, downloads the output, records every network request, and asserts that no non-GET or non-HEAD media request occurred.

Other tests compile a deliberately unsafe filename and verify that shell syntax does not reach the argument plan, exercise deterministic filter ordering, validate split and merge graphs, reject corrupted media, and reload the production app offline.

What the rebuild intentionally lost

The new MusicMixer does not join Discord voice channels. It does not play a YouTube search. It does not run yt-dlp on the server.

That last omission is a product decision, not an unfinished route. URL importing requires a native downloader, stays coupled to changing platform behavior, and raises real permission and terms questions. It does not fit the hosted site's promise that the application server never receives media.

If URL import returns, it belongs in an optional desktop edition where yt-dlp and native FFmpeg run on the user's machine and the user confirms they are authorized to download the source. The web application should not quietly regain its least reliable dependency merely for feature parity with its younger self.

The architecture, side by side

MusicMixer architecture across three generations

Concern	Discord bot	Flask website	Current browser studio
Input	YouTube search, URL, Discord attachment	URL or uploaded file	Local browser file
Compute	Bot host	Flask host	User's device
Media storage	Shared working directory	Shared upload/output directories	Per-job OPFS directories
Job state	Module-level dictionaries	Lifetime of one HTTP request	Typed IndexedDB records
FFmpeg invocation	Interpolated shell strings	Interpolated shell strings	Validated argument arrays
Concurrency	Per-guild flags and sleeps	One blocking request per conversion	Persisted sequential queue
Recovery	Process restart clears everything	Retry the form	Interrupted jobs become restartable
Delivery	Discord voice or attachment	send_file	Local Blob download or ZIP
Cleanup	Scattered os.remove calls	Delete around response delivery	Job-owned cleanup and explicit storage controls
Trust boundary	Media crosses Discord, YouTube, and the host	Media uploads to the app server	Media stays in the browser

The table makes the rewrite look inevitable. It was not. Each earlier version solved the problem I could see at the time. Discord made distribution and playback easy. Flask made the interaction legible. The browser version became possible because FFmpeg WebAssembly, OPFS, workers, and my own understanding of the system had matured.

What six years changed

Looking back at code written when I was young is uncomfortable in a useful way. There are broad exception handlers, repeated functions, absolute paths, deleted files that may still be in use, and secrets where secrets should never be. There are also correct instincts:

• move blocking extraction off the event loop;
• key state by guild so servers do not share a queue;
• cap long-running requests;
• generate unique filenames;
• clean up expensive media;
• preserve metadata for the person waiting in Discord; and
• make a terminal tool understandable through a small set of verbs.

The current version is not evidence that those instincts were wrong. It is what they look like after learning to model them explicitly.

The biggest lessons are not specific to audio:

1. Async syntax is not a job system. Work needs ownership, status, cancellation, recovery, and resource policy.
2. A random filename is not isolation. Give each job a directory and a lifecycle.
3. Never let user input become shell syntax. Validate structured intent and pass argument arrays.
4. Cleanup is part of delivery. “The response was returned” does not necessarily mean the consumer finished reading the file.
5. External dependencies define product reliability. If the main feature depends on an adversarially changing platform, that is architecture, not a footnote.
6. Preserve the promise, not the implementation. A rewrite earns its cost when it removes assumptions rather than translating them.

MusicMixer remains a silly project. The bass control still ends at “Oh my god.” That is part of why I wanted to keep it.

But it is also one of the clearest records I have of learning to build. Taco Bot gave me an audience. The Discord bot taught me media pipelines. Flask taught me that a new interface does not automatically create a new architecture. The browser rewrite taught me to begin with trust and lifecycle instead of the FFmpeg command in the middle.

Six years later, the idea finally has the architecture it needed.

Try the private studio at musicmixer.shauryav.com or read the current source. The shorter product overview is available on the MusicMixer project page.