Behind the Technology of Mix Assistant

This article references a previous version of Neutron. Learn about Neutron 4 and its powerful features including Assistant View, Target Library, Unmask, and more by clicking here.

What is Mix Assistant?

A common first step in mixing a multitrack recording is to set the levels for each track in your session. This can be difficult and labor-intensive, especially for complex sessions. The sheer number of tracks you might have may be intimidating enough to close your project file and begin exploring a new musical idea before completing the one in front of you.

This is where the new Mix Assistant feature in Neutron 3 Advanced comes into the fold. Mix Assistant gives you a starting point for your mix by listening to it and automatically suggesting gain levels for all the tracks in your session. While analyzing your session, Neutron 3 Advanced bundles your tracks into logical groups (voice, bass, percussion, musical, and a dedicated user-selected focus group), allowing you to quickly dial in broad-stroke adjustments to Mix Assistant’s initial suggestions. In just a few clicks, you can achieve a roughly balanced mix for even the most complex sessions. Further granular refinements can be adjusted on a per-track basis within the Visual Mixer or using the faders in your DAW.    

Mix Assistant marks a paradigm shift for assistive technology at iZotope. Prior to the release of Nectar 3, our assistive technology operated on single tracks (either a track in a multitrack recording or a master track). The Unmask feature in Nectar 3 was our first attempt at performing cross-track analysis, where we listen for masking issues between pairs of tracks, and automatically set up digital signal processing to resolve them. Pushing this concept further, we wanted to devise an algorithm that could look at your entire mix, and push it in the right direction from the get-go.

Sounds complicated, no?

Developing an algorithm like the one used in Mix Assistant is by no means an easy task, especially when you consider that it needs to apply to different genres, track counts, and overall arrangements—and we wanted it to require minimal upfront user input. When you consider all the nuances that are potentially at play when leveling a session, it seems almost impossible to devise an automated solution that could apply across the board. Nonetheless, when approached with big and complex problems, a common engineering practice is to make broad, general assumptions, see how far they get you, and incrementally refine your approach accordingly.

Mix Assistant relies on the assumption that the track arrangements in most sessions can be lumped into the same groups, even if a few of them are empty. It uses our machine learning technology to automatically assign each track to a group. Through some tinkering, we’ve figured out roughly how each of these groups should sit in a mix. We’ve found that while there will always be outlier examples, the resulting AI that we’ve developed gets your levels most of the way there. As is the case with all of our assistive technology, Mix Assistant’s Balance feature is designed to give you a starting point, and allows (even implores!) you to make changes to its suggestions to get results more to your liking. Specifically, Mix Assistant allows you to quickly make modifications to the relative balance of the different groups (we all know how important tonal balance is to a mix), as well as re-assign any of your tracks to a different group in case we didn’t hit out of the park for you from the start.

Mix Assistant developer pro tip:

Try to pick just a few (1–3) focus tracks so as not to drown out their impact to the overall mix. Also, while Mix Assistant performs best when it has heard your entire session, it might not be totally necessary to get a good result depending on the nature of your session. As long as Mix Assistant has heard a representative amount of each track actively doing what it’s doing, it’s probably good enough to get you something decent, and you might just need to make more adjustments using the sliders.

—Shahan Nercissian, Senior Research Engineer, iZotope

How to use assistants in your workflow

Mix Assistant is the fifth assistant to join the iZotope family (alongside Track Assistant in Neutron 3, Master Assistant in Ozone 9, Repair Assistant in RX 8, and Vocal Assistant in Nectar 3 Plus). All of these assistants are meant to lend a helping hand, suggesting starting points that take stabs at getting a bulk of the utilitarian stuff out of the way so you can focus on being creative at different steps in the process. Most music production workflows involve these five steps after tracking is complete.

1. Repair:

This includes doing things like trimming, comping, pitch correction, and noise reduction. For the latter case, RX 8 is an invaluable tool, and its Repair Assistant can help to salvage less than ideal recordings.

2. Rough mix:

Once we have usable tracks, the mixing process begins, usually with a rough setting of level and pan for each track in your session. Mix Assistant in Neutron 3 will help you achieve a rough balance for your mix through the Balance feature.

3. Processing tracks in isolation:

Now that levels and stereo image are in a good place, we process tracks in isolation with tools like EQ and compression. Track Enhance in Neutron 3 (or Track Assistant as it’s called in Neutron Elements and Neutron Standard) and Vocal Enhance in Nectar 3 can help to both apply surgical processing and bring life to any track in your mix.  

4. Processing across tracks:  

Our tracks sound good on their own, but it’s time to make additional mixing decisions to get rid of muddiness and get all of the parts to gel together. Here, the Unmask feature in Nectar 3 can help to resolve masking issues between pairs of tracks or buses.

5. Mastering:

Your session has come a long way, and you might be getting excited about putting your work out into the world (if you can even stand to listen to it at this point), but it still needs some final touches to get it to be release-ready. Master Assistant in Ozone 9 can help achieve the proper tonal balance, dynamics, and loudness for any aesthetic or music platform.

Throughout the process, it’s natural to occasionally jump back to a previous step in the process.  For example, if you make many EQ cuts to a track, you might find yourself re-adjusting its gain.

Should we be worried about the implications of technologies like Mix Assistant?

While we’re excited about Mix Assistant, we can understand how a feature such as this one can be met with some skepticism. In light of the impending AI takeover around us, it is only natural that these technologies be polarizing and received with some hesitation, and perhaps even anger. Ironically, we have always surrounded ourselves with audio processors that exhibit some form of intelligence, but because their level of intelligence was relatively low, their utility was so obvious, and their adoption has become so commonplace, nobody thinks twice about whether or not to use them. For example, compressors are ubiquitous in audio production. One could argue that a compressor is also a form of AI, which intelligently lowers the gain of an audio signal based on its momentary signal strength. As such, we can give our tracks consistency or body without having to spend hours manually writing gain automation.

We should consider it to be a healthy practice to question the line where utility stops and creativity begins, and be open to adopting new workflows that capitalize on our findings. After we developed Mix Assistant, I think many of us were in shock that a lot of what we thought to be craft could be distilled into objective theory. This is actually a good thing because it enlightens us on where to focus our creative energy and time so that we forge our musical endeavors into new, untapped territories.