HarmonAI

What is HarmonAI?

HarmonAI, styled Harmonai on its own site, is a Stability AI lab that releases open-source generative audio tools for musicians and producers. Its stated aim is to let artists generate their own custom sound libraries, under the tagline “AI by musicians, for musicians”.

The lab’s main public release is Dance Diffusion, a family of diffusion models that create new audio clips, variations and blends from training data such as piano recordings or glitch samples. You run it in a Google Colab notebook or from the code on GitHub. TechCrunch reported in 2023 that HarmonAI trained the model behind Stability’s Stable Audio and now acts as Stability’s music research arm. The website was still online in October 2026, but its GitHub repositories were last updated between June 2023 and April 2024.

Key Features

• Dance Diffusion models: Dance Diffusion generates new audio from diffusion models trained on specific sound collections. AudioCipher lists checkpoints such as Maestro-150k for piano, Glitch-440k for noise and Honk-140k for Canadian geese recordings.
• Audio variations: The notebook regenerates an uploaded WAV file into new variations of the same sound. Producers can feed in a loop or one-shot and get related but different samples back within minutes.
• Track interpolation: Dance Diffusion can blend two audio files by interpolating between them. The result mixes the character of both sources, which suits sound design experiments more than finished songs.
• Custom model training: The sample-generator repository provides tools to train a generative model on any set of audio samples. That lets you build a model on your own drum hits or field recordings instead of the stock checkpoints.
• Google Colab notebook: A ready-made Colab notebook runs the models in a browser on Google’s GPUs. You still step through code cells, but you avoid installing Python libraries or buying a graphics card.
• Open-source codebase: HarmonAI publishes its work on GitHub under the MIT licence, including sample-generator, an audio diffusion fork and the oobleck audio codec. Developers can inspect, modify and reuse the code in their own projects.

Pros

• Completely free: HarmonAI charges nothing for its models, notebooks or code. Directories including Toolradar and Toolmage confirm there are no paid tiers, so your only possible cost is extra Colab or GPU time.
• Permissive licence: The GitHub repositories use the MIT licence, which allows commercial reuse of the code. That gives developers and studios more freedom than closed music generators with restrictive terms.
• Train on your own sounds: Sample-generator lets you fine-tune a model on your own library rather than someone else’s catalogue. The output then reflects your sonic identity, which matches the lab’s goal of putting power back with artists.
• Stability AI pedigree: HarmonAI sits inside Stability AI and, according to TechCrunch, trained the model behind Stable Audio. Its research fed directly into one of the better-known commercial AI audio products.
• Active community roots: The project runs a public Discord server and lists its X account and GitHub on the homepage. Musicians and researchers can share notebooks, checkpoints and results with each other there.

Cons

• Technical setup: HarmonAI has no polished app, so you work through Colab cells or a Python environment. Beginners who expect a text box and a play button will find the learning curve steep.
• Rough audio quality: AudioCipher describes Dance Diffusion output as grainy, warbly and often strange. The clips work as raw material for sound design, but rarely as finished, release-ready music.
• Slowing development: The public repositories were last updated between June 2023 and April 2024. Stability’s newer audio work, such as Stable Audio Open, ships under the main Stability-AI account instead.
• WAV-only input: The Dance Diffusion notebook accepts WAV files hosted at a public URL ending in .wav. Producers working in MP3 or inside a DAW must convert and upload files before each run.
• Colab limits: Free Google Colab sessions have limited GPU time, and AudioCipher recommends a paid upgrade for regular use. Long training runs can also time out before a custom model finishes.

Who is Using HarmonAI?

• Electronic producers: Generating unusual samples and textures.
• Sound designers: Building custom effects for games and film.
• AI audio researchers: Studying and extending diffusion models.
• Developers: Training models on their own sample libraries.

Pricing

HarmonAI is free, with all code and models released as open source (official site and GitHub, checked October 2026).

The software costs nothing, but heavy use may need paid Colab compute or your own GPU. Google sets those compute prices, so check its current plans before starting a long training run.

What Makes HarmonAI Unique?

Training on your own sounds is the core idea behind HarmonAI. Instead of offering a closed generator built on an unknown catalogue, the lab released the code and notebooks so a producer can build a model from personal samples and keep full control of the result. Its research also shaped Stable Audio, giving this small open project an outsized influence on AI music tools.

Compatibilities and Integrations

• Platform: Google Colab in a browser, or local Python with a GPU.
• Audio format: WAV files in, WAV files out.
• Code: Open-source repositories on GitHub under MIT.
• Community: Discord server and X account for support.

HarmonAI Tutorials

A first Dance Diffusion session takes about ten minutes in Colab.

1. Open the Dance Diffusion notebook linked from the HarmonAI community and sign in with Google.
2. Run the setup cells to install dependencies and check for a GPU.
3. Choose a pre-trained model and the v-iplms sampler.
4. Paste a public URL to a WAV file, or generate without one.
5. Run the generation cell and download the clips you like.

How We Rated It

Summary

HarmonAI suits producers, sound designers and researchers who are comfortable with notebooks and want free, open models they can train on their own audio. Its main limitation is usability, since there is no app and the output sounds rough beside commercial generators. Development has also slowed as Stability’s audio work moved to Stable Audio. Our tip is to start with the stock Dance Diffusion models in Colab before committing GPU time to custom training.