Frame Drummer Pro

About

Since the release of ‘Jitter’ (a visual extension of the Max software development environment) in 2003, it was much easier to develop home-made software. I already loved Max, so Jitter was a dream come true. Realizing a cool software concept was now just a few clicks away.

I decided to make my future audiovisual work with my own, self developed  software only. No more Adobe After Effects, Final Cut Pro, Sample Cell, EXS24. I was not happy with these tools. They worked in non-real-time, with complex interfaces and an exhausting workflow with little room for improvisation and ‘live jamming’. Besides that I believe that you develop better tools when you really depend on them. I wanted to start from scratch and re-build my working process with instruments optimized for my artform: video sampling and visual music.

The first AV instruments I created in 2003 were the DVJ mixer, the Frame Drummer Pro and the skrtZz pen (later combined into SenS I). The Frame Drummer Pro is live video-editing/video-sampling software that turns Logic Pro (music software) into an audio-visual instrument. All features work in real time and are optimized for a very fast, musical workflow, for live jamming and improvisation.

Because hardware in 2003 was not powerful enough to smoothly manipulate video in real-time and run Logic Pro at the same time, the Frame Drummer Pro ran on a laptop, controlled through MIDI by a separate computer running Logic Pro.

Contrary to my home made video sampler, X-Pose and the audio samplers I worked with, the Frame Drummer Pro didn’t work with individual videos mapped to MIDI notes, but worked with a big chunk of video (a movie scene of music video). This big chunk of video was automatically, sliced up and mapped to MIDI notes by ’the EboLyzer’ with live adjustable controllers.

This made it much easier to use a piece of video and opened up an interesting new way of editing video in real time. Instead of creating a video edit manually, the video edit is created automatically by feeding the Frame Drummer Pro a MIDI pattern and playing with the controllers live until a nice edit is found. An intuitive way to create audio-visual grooves, beats, skrtZzes and melodies. Surprising edits that otherwise never would have been found.

Later versions of the Frame Drummer Pro added live controls for adjusting the video start and loop points, changing EboLyzer settings (sensitivity etc.) and manipulating the MIDI pattern (MIDI filters). All paramaters were manually controllable or by Logic Pro with ‘automation lines’.

In the Generative Tracks project I focussed on this new way of creating tracks, and tried to minimize manual editing while making a cool sounding/looking track. But  this technique is used in all Frame Drummer Pro tracks, as you can see in the examples below.

Later versions of the Frame Drummer Pro also supported skrtZz-ing, granular synthesis and live video recording. The Frame Drummer Pro was also a great tool for live performing and jamming. For this purpose an internal sequencer was added. You can read about the different versions of the Frame Drummer Pro below.

Frame Drummer Pro 0.5

The first version of the Frame Drummer Pro didn’t have many adjustable parameters. Only for pitch, loop length and video start point offset. It also had the option to slice videos automatically according to a musical grid (1/8, 1/4 1/2 or full notes). Very convenient to slice music videos. This feature I used for the DVJ 3.0 project.

Frame Drummer Pro 1.0

Version 1.0 added options to control the EboLyzer live and more options to change the video start point. It also added ‘Melody Mode’ and ‘Drum Mode’. Melody Mode made it easy to change the MIDI note mapping. Instead of mapping different points in the video (samples) to MIDI notes, one point in the video was mapped to all notes and pitched. Drum Mode was used for a MIDI drumkit. Version 1.0 also added MIDI FX to manipulate incoming MIDI patterns, to create new grooves and melodies live.

Frame Drummer Pro 2.0

Version 2.0 added the option to render video. Before version 2.0, I used screen capture software to record the composition. But these recordings were of low quality, because the computer was not powerful enough to do live video triggering and recording at the same time (see the Garbage man composition above). Rendering solved this issue. For version 2.0 the EboLyzer was updated as well, to analyze the character of the audio track and filter out points that sound like a snare-drum, percussion/hi-hat or bass-drum.. This made it easier to use the same MIDI pattern on different sets of samples and create tracks faster. I named this technology the AGT. In the ‘AGT – generated tracks‘ project I focussed on this concept, ‘SenS II‘ improved this technology. This concept also plays an important role in the sequencer of ‘Senna‘.

Frame Drummer Pro 3.0

Version 3.0 added some basic visual effects, like rota-zoom, zoom in/out, fade in/out, mosaic and wave. It also added more MIDI effects and improved video playback and rendering.

Frame Drummer Pro 4.0

Version 4.0 was an experimental version that supported playback of multiple videos at the same time. Playing more than one video slowed down playback too much, so fast edits and jumps in the video didn’t look smooth anymore.

Frame Drummer Pro 5.0

Version 5 was a major update, because it added a sequence player and support for the skrtZz pen. This made the Frame Drummer Pro a very fun live performance instrument. The sequence player was used to alternate and mix between different MIDI loops on the fly, live (like Ableton Live’s Session View). Combined with the other controls (MIDI FX, start point offset, EboLyzer controls) I was able to create new beats, grooves and tracks live. When the skrtZz pen was used, the Frame Drummer Pro switched to ‘skrtZz mode’ instantly and interrupted the sequencer. This made it easy to improvise rhythmically and skrtZzally at the same time. Using two laptops and a video mixer, I was able to mix beats and skrtZzes and do an improvised audiovisual performance live. Super fun. I miss those performances.

Frame Drummer Pro 6.0

Version 6.0 added granular mode, for real-time time-stretching and glitching. Also, the interface was cleaned up. The Frame Drummer Pro had become a complex system at this point and that made it crash sensitive. We tried to improve that in the SenS I project.

Frame Drummer Pro 7.0

Version 7 was an experimental version that added the option to record video live. A very cool feature. Recorded videos were automatically analyzed by the EboLyzer and mapped to MIDI notes, so they could be used immediately in a track. Great for live performances and improvised studio productions. In the SenS I project we tried to add this feature to the Frame Drummer Pro properly. In SenS II it was succesfully added. The feature was optimized in SenS IV and used in many performances. Read more about the Frame Drummer Pro 7 on the project page.

Team

Produced and developed by:

• Jeroen Hofs
• Thijs Koerselman
• Aart Muis