Our current focus in the studio is on developing a tool to facilitate audio visual jamming. This is something we started to explore with the Lochgelly project last year. That project was our first time using Isadora software to make a space to enable interactive exploration of sound and video. We had some MIDI controllers and webcams as controls and live video/sound inputs into an Isadora setup with the output projected onto a wall. It made for an exciting and chaotic space. We only managed a six week run of open studio afternoon sessions before everything closed down. And not being able to work alongside people since then has meant everything’s been on hold with this project for a year. A year when we’ve been working with care homes to develop resources to support front line care staff to get co-creative stuff happening themselves with our Jam Today project (but that’s another story).
Moving Forward
With spring 2021 upon us and the hope of things opening up once more later this year, now seems like a great time to get back into figuring out how to use Isadora to make a functional and accessible A-V jamming tool to use in sessions. Our initial sessions in Lochgelly highlighted for me the importance of creating some sort of clear order for the jamming to happen inside of. For these sessions, I’d created minimal order, focusing instead on maximum possibilities – all that lovely chaos and fluidity, but it is possible to have too much of a good thing. When you want to make an activity accessible and friendly, order does help. The people who joined us for a jam did join in and have a good time, but I do think that I can improve things – both in terms of the co-creative experience, and the audio visual stuff we all make together – by making something more structured. Also, it was all a bit exciting (stressful) for me never quite knowing how everything with the hardware devices and software would interact together as we delivered the sessions. Time to embrace a little order.
The Back Story
In my practice, I create video and sound work using other people’s software and I’ve long been curious about trying some creative coding myself. I do plan to explore some coding – I’ve learnt HTML and CSS to build this site as a first step and I want to learn about Processing and explore the possibilities of building interactive input devices using Arduino. This is a longer term plan and I’m not prepared to wait until I’ve learnt to code to be able to dig in with creating our jamming tool. So, this is where Isadora comes in. The idea behind the software – created by musician Mark Coniglio – is to offer creative practitioners a tool to put together creative software spaces without the need to code things themselves. Ideal for me, right now.
The Plan
I’m going to write (at least) weekly here on the blog as I develop the tool, as a way to think things through as I learn and experiment. Isadora works by giving you building blocks (actors) that interact with each other and with inputs from devices (like MIDI controllers and drawing tablets) and sources (like webcams and mics). It’s all about interaction. It offers a massive range of possibilities and I know that a big challenge for me will be to reign-in everything I’d like to build – and so avoid a big pile of controllers and wires and the possibility of creating stuff that looks and sounds just messy. So, the plan is to focus first on building individual A-V jamming instruments that are stable and accessible. What I’m looking to do is create instruments that will impact the projected video output in real time. Some will also have an audio element. The video imagery will consist of layers of recorded video, still images, live video input and generated elements – you see the potential for chaos! Pre-recorded video, still image and audio resource slots will be available to load participants material. The musical input into the jam will mainly happen live in the space with acoustic instruments.
The Potential Instruments
Here’s the wish list for now (I’ll write here about them as I put them together):
- A Soundscape/Texture Keyboard – using a small 25 key MIDI keyboard to trigger individual (looped and single-shot) sounds that can be played together in real time. The keys will also trigger still images to create a visual texture layer.
- A Coloured Shapes Keyboard – using a small 25 key MIDI keyboard to trigger and manipulate generated coloured squares, altering size, colour, position and opacity.
- A Live Drawing Tablet – using a tablet with pen input and programmable buttons to draw onto the video output.
- A Picture Mixer – using a Korg NanoKontrol2 MIDI device to mix 8 still images – altering size, position and opacity of each, output to a picture layer.
- A Video Mixer – using an Akai LPD8 MIDI controller to select and mix 8 videos.
- A Sharing Stuff Webcam – using a webcam to offer the ability to share things visually in real time, output to the screen
I also want to explore how to use webcams to capture sound and movement in the space and how this may impact on the visual output. And how to use mics to allow specific directional input of sounds in the space and how these may impact on the visuals. All quite vague with these as of now. I do have a Kinnect movement sensor that I’ll be having a play with too – apparently these work well with Isadora.
Early days, but keen to get cracking.
Find out more about Isadora on the Troikatronix website here: https://troikatronix.com/
Emma