Isadora as Statemachine for the Performance Engine

Author

Machiel Veltkamp , MAPLAB, Stichting z25.org

Date

05-03-2014

Abstract

Currently the Media and Performance Lab (MAPLAB) uses a custom developed state machine build in Max/MSP. This state machine acts as a central interface to connect different digital inputs to outputs. The state machine was built as a prototype and while it has proved its usability it also showed its limitations. Therefore a new version of this state machine is researched. This research explores the possibilties of using custom developed Isadora plugins as a new state machine for use in the MAPLAB.

Introduction

In 2013, an area of research was initiated by the MAPLAB to research the use of existing ( commodity ) software suitable for artistic and rapid prototyping. This was initiated to provide reliable solutions for end users in addition to the research and development environment of the Performance Engine (PE) [1] system. The main focus was a replacement of the central part of the PE: the “State Machine”. This component is the central interface for the Performance Engine and provides the user with an accessible overview of all available inputs and outputs of the entire system and the possibility of connecting these in a easy way.

As software used for the development of a new “State Machine” we chose to research the possibilities of Isadora. This is a software application developed by Troika Ranch to be able to work with projected video in an easy and comprehensive way for their theatre performances. Isadora is an application to be used by theatre makers and not necessarily computer technically skilled persons. The software does not have a steep learning curve and uses visual programming and different metaphors to make it easier to work with for artists. The software enables you to quickly combine different blocks of functionality enabling users to create interactive playback of digital media.

Two of our main reasons for choosing Isadora as platform to develop a new “State Machine” where:

  1. The MAPLAB already works with Isadora as one of the tools used in workshops. So it would be ideal to further expand the use of the software within MAPLAB.
  2. For Isadora there is an SDK2 available which makes it possible to develop your own custom plugins for Isadora in C++ for Windows and OS X and thereby adding new functionality to the application.

We wanted to research the possibilities of using of Isadora as “State Machine” for the Performance Engine.

Methods

Before starting a full fledged development project cycle we first wanted to do a small preliminary research to be able get to know the (im)possibilities of the Isadora SDK, it’s ease of use and to check whether we could use it to to develop the functionality that we needed for a new “State Machine”.

So the questions for our preliminary research where:

  1. Can we successfully compile a (empty) plugin using the Isadora SDK for windows and OS X
  2. Is it possible to receive OSC messages on a specific port within a custom Isadora Plugin
  3. Is it possible to send OSC messages to an address and port within a custom Isadora Plugin
  4. Can we dynamically create inputs within a custom Isadora Plugin
  5. Can we dynamically create outputs within a custom Isadora Plugin

If this preliminary research is completed successfully we then could explore the next steps in developing the needed custom Isadora Plugins. Any other outcome would conclude that Isadora was not suited for our use case.

After successfully doing the preliminary research we started planning the next steps in creating the “State Machine” in Isadora. We decided to do the development in different iterations and make a distinction in what needed to be done in three different areas:

  1. Development of plugins receiving messages through OSC over the network
  2. Development of plugins sending messages through OSC over the network
  3. Other requirements

For the first two areas we decided to start with the most basic and simple plugin we could think of to facilitate (part of) the needed functionality and expand from there. Using this approach we also had the chance to get further acquainted with the Isadora SDK, the code design and patterns to realize our specific goals.

Results

Expanding on our iterative development approach as outlined above our result is the development of the following five custom Isadora Plugins doing OSC communication and using the “State Machine” functionality.

StatemachineReceive plugin

A custom Isadora plugin that receives OSC messages on a specified port and sends them to the “inputs” or “outputs” output depending on what kind of “State Machine” compatible data it receives.


The “State Machine” receive plugin

Create Inputs

A custom Isadora plugin that work in cooperation with the StatemachineReceive plugin to create a series of outputs based on the initialisation of a Performance Engine input module.


The “State Machine” receive plugin connected with the CreateInputs plugin, shown with the PE inputtester module initiated

Create Outputs

A custom Isadora plugin that work in cooperation with the StatemachineRecieve plugin to create a series of inputs based on the initalisation of a Performance Engine output module.


The “State Machine” receive plugin connected with the CreateOutputs plugin, shown with the PE outputtester module initiated

Performance Engine Input Client

A custom Isadora plugin that allows Isadora to act as a “State Machine” input module


The Performance Engine input module plugin

Performance Engine Output Client

A custom Isadora plugin that allows Isadora to act as a “State Machine” output module


The Performance Engine output module plugin

With these plugins Isadora can be used as a “State Machine” or a Isadora patch can be used to behave as a Performance Engine input or output module. Creating “State Machine” functionality within Isadora gives the opportunity to re-use the different modules build for the Performance Engine. This enables more flexible and easy to use communication between different input and output modules while also adding logic to the processing of the signals using different Isadora actors. The developed plugins are functional however the usability of the functionality needs to be evaluated by doing user tests. Code and binaries are available through z25.org and the MAPLAB. The code is available under a LGPL license.

Future work

The future work on this research will consist of two directions.

  1. User testing the produced plugins during different events at the MAPLAB to see how they perform.
  2. Acquiring missing functionality in Isadora we could possibly overcome by developing custom plugins for it.

Acknowledgements

This research has been made possible by the support and help of the Isadora Community, especially Mark Cognolio. Financial support by the MAPLAB.

References

1The Performance Engine is a system developed by the MAPLAB to create a system of modules to enable quick prototyping with digital media using a variety of inputs and outputs. http://maplab.talltreelabs.org/r-d/tools/ Retrieved 08-09-2014

2Isadora Software Development Kit enabling other programmers to develop additions to the software. http://troikatronix.com/download/software-developer-kit/ Retrieved 08-09-2014