SenS III

About

Seamless integration into music software is very important to create the optimal audio-visual instrument. So, it was crucial to integrate SenS III as seamless as possible into the workflow of Logic Pro. Fortunately, in 2006 hardware was powerful enough to run Logic Pro and SenS III on one computer, while playing and mixing four videos with basic audio-visual effects. Enough power to make complex composions in real-time and perform them live!

SenS III consists of the SenS III application (written in Max), many editors to edit different aspects of a SenS project and two plug-ins for Logic Pro. The feature set of SenS III is similar to that of SenS I, SenS II and the Frame Drummer Pro 7, but much more user friendly, more advanced and optimized for Logic Pro. New was the Decor Builder and the Decor Frame concept. A great way of making real-time motion graphic compositions and doing video/live camera mixing. Another great innovation was the video meta-data editor.

Main features

Tight integration in Logic Pro. To work with audio and visuals at the same time, in one, unified creative process, it is crucial that SenS III fits seamlessly into the workflow of Logic Pro. All features of SenS III should be easily accessible inside Logic Pro.

Two ‘VST plug-ins’ (made with Max/Pluggo) transfer MIDI, automation and controller data to SenS III with very little latency. This data controls advanced ‘Functions/mappings’ for advanced video triggering (including videobeats and video skrtZz-ing), file selecting (with meta-data), audio-visual effects and video mixing.

Without leaving Logic Pro (or the MIDI editor even) all creative features of SenS III are easily accessible this way. Logic Pro is turned into an audio-visual instrument! When Max for Live was introduced we did this even better.

3D video mix and real-time motion graphics. SenS III introduced the Decor Frame concept. This makes it easy to create visual compositions of videos, graphics and 3D models in a virtual 3D space live. It is a like a 3D video mixer that works in real time! I invested a lot in this concept for SenS IV and SenSei. Eventually I liked this way of real-time motion graphic design better than professional software like Adobe After Effects.

Audio-visual effects. Audio-visual effects play an important role in my art. Audio-visual effects are used to manipulate visuals and sounds at the same time and to generate completely new visuals and sounds. It is a great source of artistic expression. Keeping the link between audio and visual intact, while they are distorted, makes the effect much more impactful. Audio-visual effects are also important to make any AV sample fit into a composition.

Meta-data + EboLyzer + AGT. Videos used in SenS III are enhanced with meta-data that describes the character and theme of the sound and visual. This makes it easier to organise and use them in Logic Pro. The AV analyser creates meta-data automatically, filtering out points that sound like a snaredrum, hi-hat or bassdrum or are harmonious. This makes it easier to use a MIDI pattern on different sets of samples and create tracks faster. I named this technology the AGT. Very useful for creating video-beats or melodies live.

Live video sampling. I started using live recordings in my performances in 2004. This made performing live much more interesting. SenS III is equipped with a live video recorder and an audio-visual analyser to select the best parts of the recordings automatically and filter them to make them fit in the show quickly.

Flexible video playback. SenS III is a creative instrument, so expressiveness is important. Therefore all creative features work in real-time and are very flexible. For video playback we developed a very flexible video player to manipulate the timing and pitch of videos. Cutting, slicing, triggering, skrtZz-ing, scrubbing, stretching, pitching, speeding up/down can be done with MIDI controls and MIDI notes. An advanced Function/mapping system makes is very easy to control all these features live from Logic Pro.

SenSorSuit. I have been using a motion tracking suit (‘SenSorSuit‘) for my performances since 1999. This suit can also be used to control SenS III.

Rendering. In 2006, computer hardware was not powerful enough to play, mix and distort videos and record this composition at the same time. Therefore SenS III has the ability to render the visual composition in non-real-time (like the Frame Drummer Pro).

Projects

SenS III is used for many projects in 2006 and early 2007. SenS III was used to mix visuals and live cameras during the Games in Concert event with the famous Metropole Orchestra. It is used for the opening of the Amsterdam Museum Night, the Science and Tech Summit, the NanoPodium conference and a brand event for the Postbank (ING bank). Find more compositions here.

Overview

SenS III is controlled by Logic Pro with two different VST plug-ins. The Global plug-in is loaded on a single MIDI track to control global parameters, like selecting a ‘Decor Frame’, controlling the virtual camera, starting a video recording and enabling the SenSorSuit.

The Advanced plug-in can be used on up to four different MIDI tracks to control the four available video players. Both plug-ins have a monitor window to check the data sent to the SenS III application.

The SenS III application is the heart of SenS III. This application processes the incoming data from Logic Pro and processes the video playback, mixing and effects. It runs in a separate instance from Logic Pro to maximize reliability and performance. In the interface of the application are buttons to open different editors and monitors. Initially SenS III had only two video players, because of hardware limitations. After a more powerful Apple computer was released in 2006, SenS III was capable of handling four video players.

SenS III introduces the ‘Decor Frame’ concept. A great way to make and control real-time motion graphic compositions. A Decor Frame is a video mix preset. The Decor Builder is used to make Decor Frames. Its interface was constantly changing and expanding, because I invested a lot in this concept. With a Decor Frame four video players and 15 graphics can be mixed in a virtual 3D space with blend modes and effects. Also rotation, color and scale is set in a Decor Frame. A Decor Frame also contains settings for the virtual 3D camera. Decor Frames automatically crossfade/morph from one to another when triggered from Logic Pro. A great way to control a video mix and motion graphics live!

Graphics are managed in the Locations Editor. Here graphics are loaded and made available to the Decor Frame editor.

SenS III was equipped with twelve build-in audio-visual 3D effects. In the Editors window these effects are mapped to the video players. Initially effects on only two video players was supported, later four video players. Since audio-visual effects play an important role in my art, this concept was too rigid and limited and soon replaced by the AVBlock concept in SenS IV.

The AV Meta-data Editor was used to add meta-data to videos. This meta-data describes the theme and character of the video. Meta-data can be added manually or automatically with the AV Analyzer. This meta-data controls different aspects of the audio-visual effects and is used to organize videos automatically by the AGT. This makes it easier to use large video files and work with many video samples (or trigger points in a video) at the same time.

With the Frame Drummer Pro I already experimented a lot with automated video editing concepts. Compositions like LLib LLik, the DVJ 3.0 tracks and the live sampling scenes in the Verknipt performance, showed that automated video editing is a very interesting concept that creates a new style of video and is very useful for fast, live, improvised composing.

The EboLyzer analyzes video files to filter out parts which character is similar to a snaredrum, bassdrum, hi-hat or is harmonious. This data is used by the AGT to organise videos (or trigger points in a video) so they can easily be used in a MIDI sequence (a MIDI template). This data can be altered manually with the AV Meta-data Editor.

The Function Monitors show how the incoming data from Logic Pro is used to control the video players, AGT, audio-visual effects and the video mix. This data can not be edited in the monitor windows. All editing happens in Logic Pro.

In 2006, computer hardware was not powerful enough to play, mix and distort videos and record this composition at the same time. Therefore SenS III has the ability to render the visual composition in non-real-time. First, the incoming data from Logic Pro was recorded live with the Recorder, then this data was slowly, step-by-step, processed and rendered into a new video by the Renderer. This concept was introduced with the Frame Drummer Pro and is very useful.

My SenSorSuit can be used to control SenS III. The PakCheck monitor shows the incoming data from the suit.

In the Options window different global settings are stored, like the size and position of the output window.

All SenS III project settings are stored in a so called TrackBank.

History of SenS III

SenS III has a long history, starting in the 1980’s and is the result of many experiments and prototypes, like the 1995 video sampler, EboNaTor, SenSorSuit, Frame Drummer Pro, skrtZz pen, DVJ mixer, SenS I, and SenS II.

SenS III was followed up by SenS IV, Senna, SenSei, Interactive Tracks, DVJ Mixer 3.0 and EboSuite.

The main purpose of starting EboStudio and the SenS project was to create the ideal audio-visual instrument, and to make that instrument available for everybody. This eventually led to the release of EboSuite in 2017.

Team

Produced and developed by Eboman and SmadSteck

• Bas van der Graaff
• Jeroen Hofs
• Mattijs Kneppers