How can you use GANs to generate music?

💡 In my opinion, GANs are one of the most powerful tools for music generation. 🎶 Generator Music GANs use two neural networks, a generator, and a discriminator. The generator creates music mimicking real tunes, while the discriminator tells real from generated. 🎵 Discriminator This network learns from real music data and rates generated tracks. The generator adjusts to trick the discriminator effectively. 🎧 Adversarial training Both networks train concurrently in a competition. The generator aims to deceive, and the discriminator strives to spot the fake. This continues until the generator makes music indistinguishable from genuine tracks.

A more recent (2014) subset of Machine Learning (ML) where at its core it employs indirect training through the discriminator where essentially the system is teaching itself. The generator tries to generate new content based on historic data to emulate the historical data while the discriminator is essentially a check that tries to find faults to identify fake data, these faults are refed into the system the generator tries to produce a new output based on the "faults". This loop goes on until the fake output is indistinguishable from the real one

Experience the power of Generative Adversarial Networks - a groundbreaking technology that revolutionizes synthetic data generation. With two key components - the generator and discriminator - GANs play a vital role for synthetic data generation.

Here are some examples of recent GAN-generated music I am aware of: - BachGAN: BachGAN is a GAN that was trained on a dataset of Bach's chorales. The generator can generate new pieces of music that sound like Bach wrote them. - MuseGAN: MuseGAN is a GAN that can generate music in various styles, including pop, rock, and classical. - Jukebox: Jukebox is a GAN that can generate music in various genres, including hip-hop, pop, and classical. Jukebox is able to generate music that is indistinguishable from human-made music.

Even when using human feedback, evaluating sound generated by GANs is challenging, aside from the subjectivity itself, but also due to the time required to listen to and assess each audio sample, unlike images which can be quickly and visually appraised. This can be seen as a minor detail and I also underestimated this time factor until I had to work on a project using GANs and sound and the bottleneck became the evaluation process.

The best way to explore GANs for tasks like music generation is to try them out - and for that you don't need to be a technical expert behind their architecture! There are tons of very convenient freely and openly available Colab notebooks, HuggingFace models and so on that leverage different types of GANs - e.g.MuseGAN, GANSynth, MusicGen, AudioCraft and so on that you can try out with your own prompts, and even fine-tune them on your own sub-samples of audio. Some of these models even come with their own demos - wherein, they have a simple UI wherein you could literally just give it a text prompt and have it generate a beautiful melody conditioned on that prompt.

Before jumping into GANs for Music generation. Consider checking out already freely available tools. Great tools to get started with: Suno.ai (on discord) -> fantastic quality and language understanding, especially with Vocals! Stable Audio -> from stability AI, great for sounds, effects and instrumental. Rifffusion -> 12 second songs, large variety and customization If you decide to go further... then go into the examples shared by the contributors above for the google collab notebooks, which are a treasure trove by themselves :)