For the unfamiliar, sugar sk*-*lls is the moniker of Ben Marcantel, an electronic musician that has been creating a slew of music since 2010’s release Another Michael Mann Film. He’s collaborated with Matt Glassmeyer (Meadownoise) and Lindsey John (NUDITY) on his track “Tethers“, as well as Coupler and Lylas on “Right Hand Hand“, not to mention the slew of original compositions along the way. Stylistically, he’s likely most known for making work that falls somewhere in the chiptune arena but this classification has become woefully inaccurate over the past few years as he explores new instrumentation, implementation and experimentation.
The latest release, explicit ethical agent, not only pushes the envelope of styles Sugar Sk*-*lls participates in, it poses much larger questions about the creative process and the intrinsic nature of the decision making process.
That all sounds very heady, I know, but knowing a little background on the album helps to simplify and clarify the inquiry. In mid 2018, Marcantel starting playing around with Google’s Magenta AI for Musicians, a set of tools that generates a composition based a combination of rules derived from 10,000 classical recordings and an input from the user. Sugar Skulls input a single chord of his choosing, received a generated composition from the AI and ran that through his usual synthesizer. In his own words, “The program was responsible for the content of the relationships between the notes; pitch, duration, rhythm, frequency, etc.” That being the case, was this Sugar Sk*-*lls music at all?
From a purely subjective point of view, the album sounds fine. It doesn’t feel particularly tied to the decade of work that came before it, nor does it feel completely alien within that context. It’s interesting at times and needlessly meandering at others. That being said, had I not known of the specific creation process for this music, maybe I would feel differently, there is no way to know now.
In the liner notes for the album (preserved in full below) there is more background about the creation process and the questions the output puts forth. What would the result be if the AI was seeded with all the previous works of Sugar Sk*-*lls? Would the output be more aligned with expectations and, if so, would that make it anymore “true” to his body of work? If the output required no additional seed of input (i.e. the chord provided), would that make the generated composition any less original?
Marcantel summarizes the situation quite eloquently in final writeup for the album; giving a much broader viewer of the situation at hand than just one musician tinkering around:
The creation of music is not just bound to the musician’s ability to perform an idea on an instrument or knowledge of music theory, but also the distillation of a musician’s stylistic influences. A tool that can manipulate content via styles like swatches on a palate already exists for image creation. However, just as deceased musicians can be made into holograms that tour previous material and generate revenue without the musician’s explicit consent, the stylistic essence of a musician can potentially be quantified and made to output new works without the musician present. The technology will present new methods to exploit musicians and create new modes of expression. Ultimately it will further blur the lines of listening to and creating music, which is a kind of subversion I fully support.
Just as Netflix and YouTube suggest content that you may like based on previous viewings, it’s entirely plausible that services like Spotify or Apple Music will begin to deploy original, generative, music that is seeded from your previous listens. In fact, that’s already happening. Sugar Sk*-*lls foray into this world may not have created music that can be definitively attributed to Marcantel but it certainly poses some interesting questions.
explicit ethical agent liner notes in full.
On April 1st of 2018, I spent a few hours using a browser-based software program that can create potentially endless compositions. The program uses machine learning and a database of classical scores to write its own musical rules deciding how and what notes should follow previous notes to complete/continue a musical phrase. During the process, I had a few questions: Is this music? Is this Sugar Skulls music? What are the implications if the listener answers yes to either of those questions?
Traditional computer programs are given a set of rules to execute a designated end. The ability of a program to execute its goal successfully is dependent upon the ability of the programmer to abstract the goal into a set of repeatable rules and to define the conditions in which those rules exist. The program should accomplish the goal explicitly and only that goal, programs bugs are unexpected behavior from actions or unaccounted conditions. Simple repeatable actions occurring within a limited set of conditions are ideal for traditional programming methods.
Western music theory is one attempt at formalizing musical creativity/expression and could be thought of as a series of rules derived from the analysis of compositions. Compositional rules like scales, modes, chord resolutions, rhythmic patterns, etc. describe a method of organization for an expected repeatable musical output. However, a large portion of music theory explores how and when to subvert its own rules. The creative act for the musician is not the execution of musical rules but the combination, mutation, organization, and disregard of musical rules according to stylistic taste.
The mathematical nature of musical harmony would seem to suggest the composition of music could be an ideal candidate for automation. However, the problem of how to automate the stylistic decisions a musician might make in regard to composition into a set of rules is exponential. This reducibility problem means automation in music has been applied to limited aspects of composition. Machine learning works with an opposite approach. Rather than a programmer define every action for every condition, a program is given a large number of examples of what its output should be and it devises its own rules to create the expected output. Simplistically, the larger and more varied the data set of examples is the more likely the desired output will be achieved.
The program I used was built using Google’s Magenta AI for Artists tools. The programmer’s data set was approximately 10,000 classical musical scores. The interface is a visual representation of a piano keyboard. The user selects a set of notes and the program composes music based on those notes. The output can be either sound or MIDI note information, so I sent the MIDI notes into the synthesizer that I typically compose with. Because Sugar Skulls compositions are modal, with each composition limited to a range of notes, I input a single chord for the program to use to generate the compositions of each track based on my own stylistic taste. The result is an album of the compositions generated by the AI program, played through a synthesizer patch.
I was curious how the result would mirror my own output as Sugar Skulls. If the result did not meet my expectations, how and what should be altered until the music felt like it met the abstract intuitive rules of my own stylistic taste. As mentioned, I chose the kinds of chords the compositions were derived from, as well as the sounds the compositions were performed with. The program was responsible for the content of the relationships between the notes; pitch, duration, rhythm, frequency, etc. I resisted the impulse to edit out perceived mistakes or arrange the output via sampling. The listener can make their own judgment as to whether the result is Sugar Skulls music or an interesting experiment.
Since I used the beta version of this program, many tools are now available that use similar technology to aid musicians in quickly generating many of the surface-level steps in music composition. It isn’t difficult to envision any number of dystopian scenarios where this kind of technology is exploitative. Given the model the program is built on, a logical leap would be to replace the data set with my own (or any musician’s scores) as the expected output. Another leap would include analyzing sound wave information to find what types of timbres or instrument sounds a musician chooses based on compositional conditions. Finding the threshold at which a musician’s style can be recreated is a loaded possibility.
Sampling in music has been a contentious topic since the introduction of the technology that makes it possible. One side considers sampling to be theft while the other considers it transformative recontextualization that is another tool of expression. There is no clear way to unpack who gets to decide who owns a work and what constitutes an original work. Machine learning presents the problem of a different kind of sampling. Rather than the appropriation and manipulation of sound artifacts, the stylistic approach of a musician can be appropriated to generate new content.
As the technology matures and improves, it will invariably alter the nature of musical creativity. The introduction of photography and the technology’s ability to render an image of an individual didn’t make portrait drawing and painting redundant. Photography changed and opened up the possibilities, meaning, and purpose of human rendered forms. In the case of music, AI driven compositional technology could have a democratizing effect on the creation of music. The creation of music is not just bound to the musician’s ability to perform an idea on an instrument or knowledge of music theory, but also the distillation of a musician’s stylistic influences. A tool that can manipulate content via styles like swatches on a palate already exists for image creation. However, just as deceased musicians can be made into holograms that tour previous material and generate revenue without the musician’s explicit consent, the stylistic essence of a musician can potentially be quantified and made to output new works without the musician present. The technology will present new methods to exploit musicians and create new modes of expression. Ultimately it will further blur the lines of listening to and creating music, which is a kind of subversion I fully support.