We seek to engender co-creative spaces. Spaces where folk can come together and create together. We take a deliberately passive role. We embrace inaction.

• Being rather than doing
• Sitting together, being together
• Making and allowing the time and space to just be together
• Ensuring everyone has the time and space to be
• Feeling okay with silence, with nothing happening, with the spaces
• Lovely big spaces of possibility

It’s out of the spaces

that stuff will emerge

An idea, a thought, a word, a sound

A movement

Something to take somewhere

Or let go . . .

Let’s just get together and see what happens. Let’s see if we can find the flow. The flow for us, for this group, for this gathering, for this moment.

Let’s just be together

Making space for each other

Imagery from AinsleyPhillips Projects Culture Collective project at Kalisgarth Care Centre, Westray and the Glaitness Day Centre, Kirkwall, delivered 2022 – 2023

As things are opening up and people are working out ways of adapting the way they are going to deliver projects, Emma and I are planning our approach to our jamming with audio visual projects. 

I am thinking of how best to facilitate the music side of the jam. Along with all the pre-prepared instruments we will be inviting people to come along with any musical instrument. This is intentional as the jamming approach is all about people coming along and making something together in the moment with very little need to control things. My role as musician is to use my knowledge and facilitating skills to avoid a cacophony of clashing notes, and sounds, and to enable everyone to have a pleasant and harmonious experience. To this end, I am revising and expanding my knowledge on how different instruments interact with each other and the language used. There will be people in the session ranging from those who have never played an instrument before to people who have focused on little else since a young age, so I need to make the session accessible to all. 

Helping everything to fit together 

I intend to use the same scale or set of usable notes, such as the C pentatonic scale, on all instruments as this is a great way of bringing everyone together to make a harmonious sound. So, I need to consider how different instruments could become part of this pentatonic space.  

If a person came along with a mandolin, cello, violin or viola, the open strings are the notes that belong to the C pentatonic scale. As long as the strings were plucked or bowed, on open strings, the sound would fit nicely into what else was going on. There is a lot to explore with long notes and rhythmical playing and setting parameters of usable notes can still be an enjoyable challenge for the most accomplished musician. 

Wind and Brass instruments would have the same notes to play with but as some are at different pitches they would need to transpose to come together. A skilled guitarist would know the chords that fit with C pentatonic and would be able to contribute wonderfully in E, A, D, G, B, E (low to high) tuning. Alternatively the open strings can be retuned to D,A,D,G,A,D (low to high) for example and create some great sounds just playing with the open strings. 

This is a great opportunity for everyone to explore improvising. Many musicians have perfected the art of reading music but not had the inclination to improvise. So this is a way for all the participants to get together on a more level playing field.  

I wonder what instruments will turn up? I would love to hear your thoughts and any ideas you may have that will help people with such a wide range of skills and abilities to make a beautiful sound together and have a great jamming experience. 

A Colour Keyboard

A keyboard that allows you to play coloured squares and full screen colour filters. The premise is to have a keyboard that responds to the different frequencies of the keys by changing colours on the stage

The MIDI controller is an Akai LPK25 mini 25 key keyboard. I’ve used colour dot stickers to give a visual for the colours assigned to each key and stickers with a raised profile on the first octave white keys to give a visual and tactile prompt (as the two octaves do different things).

In Isadora, a single colour is assigned to each note of the octave. The first octave alters the colour (fixed), size (fixed) and position (random) of the square. Squares fade in/out (from 0 – 70% opacity) with a 5 sec sustain. The second octave alters just the colour of the square/full-screen filter with a fixed opacity of 20% – the extra C on the right toggles between full-screen and a centred fixed-size square for the second octave keys. A single button on left toggles the second octave visuals on and off (I do need to figure-out a way of covering the other left hand buttons as you could stop things from working by shifting the key up and down).

Playing possibilities

The instrument can be used to play laid over the base layers of video/images or over a plain colour base layer

• Use just the first octave to play the squares
• Use the second octave set as a fixed background square that you have colour control of, to add the first octave squares to
• Use the second octave set as a full-screen filter and play the first octave squares over this
• Use the full screen filter alone to play a colour filter over the layers below

A Drawing Tablet

An instrument to give a pen and palette of colours to allow freehand drawing onto the projected image.

The hardware is a Huion Tablet, with wireless pen and 12 single press buttons. This makes a serial connection with Isadora. The software treats the pen movements as a mouse and the buttons as computer keyboard buttons.

In Isadora, drawing happens when pen touches on the tablet only and the mark opacity is set at 30 % so the marks sit on top of the base layers, the mark width is affected by the speed of your drawing the mark (faster = thicker) The buttons give a ten colour palette plus two reset buttons – one to erase the last mark and one to erase all marks. There’s a little white dot to show where you’re about to draw a mark on the projected output – without this it’s hard to imagine where you’re actually about to draw on the screen. The patch does run quite resource intensive once you have lots of marks, so it works best when you make fewer continuous marks rather than loads of little marks.

I do think this instrument brings another way to add visual elements to the jam and I’m pleased with that, I’m not a big drawer myself so it will be interesting to see how others chose to use it during the jam sessions.

I want to focus on getting each instrument to be as simple as possible in terms of the Isadora set-up and also what the physical interface looks like. In Isadora they need to run using as little computer resources as possible, so everything runs smoothly together. As physical interfaces they need to be easy to understand and access – I feel like it’s important to make them friendly looking, like something anyone can have a go with.

I’ll include images of the Isadora setup, the physical interface and an example of the output, as I talk about the instruments. At this point, I’m looking to see how I can put things together and identify any potential problems. The output of the jams is to video which will be projected in the jamming space. All thoughts and comments (including ‘What on earth do you think you’re doing?’) warmly welcomed. Here’s the first two instruments:

The Video Mixer

Mixing layers of video is core to my own practice, so first up I wanted to make an instrument to mix videos. This is an instrument to mix 8 videos. The videos are short loops – so the imagery is focused on symbolism, colour, form and texture, rather than narrative, although short spurts of narrative action would be possible. In terms of participatory projects, the idea is to be able to load participants video into the instrument – after I’ve re-encoded it to play nicely in Isadora.

The hardware is an Akai LPD8 with 8 pads and 8 swivel knobs (this controller has useful software to alter the functions of it’s pads and knobs). I’ve added number stickers to the pads and knobs, as I found myself counting to ID controllers when I was trying the interface out. I do think this helps and I also plan to make a printed layout card with space for a thumbnail still printout from each video so the player has a real world visual identifier for each video.

In Isadora I’ve set-up the knobs to control the opacity of each video and the pads to switch a Desaturate filter on and off (fully saturated / 50% desaturated). Once I start to explore using the instruments together, I’ll need to consider where the videos sit layer wise with the other still image and generative elements. Building this has also highlighted that running eight videos at once is resources intensive, as I go forward I’ll need to consider options around reducing the resolution of the videos or reducing the number of videos in play at any one time.

The Image Mixer

An instrument to mix 8 still images. Here I wanted to make something that can enable collage creation, I wanted the images to be able to sit on top of each other, but also alongside each other. Like the videos, participants will be invited to get their images into the instrument for jamming. I make the images 1920 x 1080 px at 150 dpi (to allow for zooming)

The hardware is a Korg NanoKontrol2 with 8 sliders, 8 knobs and three press buttons for each slider/knob. This controller also has some buttons on the left hand panel which I may bring into play as I experiment but I haven’t as yet – I may decide to cover these buttons somehow if they’re not going to be used, to simplify the interface. I’ve numbered each track (or image) for clarity.

In Isadora I’ve set-up the sliders to alter the opacity and the knobs to alter the size of each image (from 30% to 200%). To discourage too much intense focus on exact positioning, I’ve decided to make the position selector a button (‘R’) that selects a random position. The ‘S’ button switches blur on and off and the ‘M’ button switches a 50% Desaturation filter on and off. I’ve made image 1 a base layer image that can’t be zoomed out beyond 100%, so that, as long as this instrument is being used, this image will form the canvas.

Our current focus in the studio is on developing a tool to facilitate audio visual jamming. This is something we started to explore with the Lochgelly project last year. That project was our first time using Isadora software to make a space to enable interactive exploration of sound and video. We had some MIDI controllers and webcams as controls and live video/sound inputs into an Isadora setup with the output projected onto a wall. It made for an exciting and chaotic space. We only managed a six week run of open studio afternoon sessions before everything closed down. And not being able to work alongside people since then has meant everything’s been on hold with this project for a year. A year when we’ve been working with care homes to develop resources to support front line care staff to get co-creative stuff happening themselves with our Jam Today project (but that’s another story).

Moving Forward

With spring 2021 upon us and the hope of things opening up once more later this year, now seems like a great time to get back into figuring out how to use Isadora to make a functional and accessible A-V jamming tool to use in sessions. Our initial sessions in Lochgelly highlighted for me the importance of creating some sort of clear order for the jamming to happen inside of. For these sessions, I’d created minimal order, focusing instead on maximum possibilities – all that lovely chaos and fluidity, but it is possible to have too much of a good thing. When you want to make an activity accessible and friendly, order does help. The people who joined us for a jam did join in and have a good time, but I do think that I can improve things – both in terms of the co-creative experience, and the audio visual stuff we all make together – by making something more structured. Also, it was all a bit exciting (stressful) for me never quite knowing how everything with the hardware devices and software would interact together as we delivered the sessions. Time to embrace a little order.

The Back Story

In my practice, I create video and sound work using other people’s software and I’ve long been curious about trying some creative coding myself. I do plan to explore some coding – I’ve learnt HTML and CSS to build this site as a first step and I want to learn about Processing and explore the possibilities of building interactive input devices using Arduino. This is a longer term plan and I’m not prepared to wait until I’ve learnt to code to be able to dig in with creating our jamming tool. So, this is where Isadora comes in. The idea behind the software – created by musician Mark Coniglio – is to offer creative practitioners a tool to put together creative software spaces without the need to code things themselves. Ideal for me, right now.  

The Plan

I’m going to write (at least) weekly here on the blog as I develop the tool, as a way to think things through as I learn and experiment. Isadora works by giving you building blocks (actors) that interact with each other and with inputs from devices (like MIDI controllers and drawing tablets) and sources (like webcams and mics). It’s all about interaction. It offers a massive range of possibilities and I know that a big challenge for me will be to reign-in everything I’d like to build – and so avoid a big pile of controllers and wires and the possibility of creating stuff that looks and sounds just messy. So, the plan is to focus first on building individual A-V jamming instruments that are stable and accessible. What I’m looking to do is create instruments that will impact the projected video output in real time. Some will also have an audio element. The video imagery will consist of layers of recorded video, still images, live video input and generated elements – you see the potential for chaos! Pre-recorded video, still image and audio resource slots will be available to load participants material.  The musical input into the jam will mainly happen live in the space with acoustic instruments.

The Potential Instruments

Here’s the wish list for now (I’ll write here about them as I put them together):

• A Soundscape/Texture Keyboard – using a small 25 key MIDI keyboard to trigger individual (looped and single-shot) sounds that can be played together in real time. The keys will also trigger still images to create a visual texture layer.
• A Coloured Shapes Keyboard – using a small 25 key MIDI keyboard to trigger and manipulate generated coloured squares, altering size, colour, position and opacity.
• A Live Drawing Tablet – using a tablet with pen input and programmable buttons to draw onto the video output.
• A Picture Mixer – using a Korg NanoKontrol2 MIDI device to mix 8 still images – altering size, position and opacity of each, output to a picture layer.
• A Video Mixer – using an Akai LPD8 MIDI controller to select and mix 8 videos.
• A Sharing Stuff Webcam – using a webcam to offer the ability to share things visually in real time, output to the screen

I also want to explore how to use webcams to capture sound and movement in the space and how this may impact on the visual output. And how to use mics to allow specific directional input of sounds in the space and how these may impact on the visuals. All quite vague with these as of now. I do have a Kinnect movement sensor that I’ll be having a play with too – apparently these work well with Isadora.

Early days, but keen to get cracking.

Find out more about Isadora on the Troikatronix website here:  https://troikatronix.com/

Emma

We’ll be using this area of the site to post updates on our current projects and stuff we’re developing in the studio. The plan is to post weekly. As an artist and musician collaboration, our onging focus is on exploring ways to bring our two practices together – embracing and enjoying the ways that this working together challenges us both and helps us to grow creatively. We live and work together, so the collaborative process is immersive at the best of times, and this last year of disruption has intensified things even more. We are both immensely grateful to have had another creative to live this brutal time alongside and the enforced studio time has gifted us a space to step back and ‘woodshed’ (to use some musician slang). Since March last year, our co-creative projects alongside people have been on hold and we’re starting this blog as a way of documenting the re-emergence of our project-based participatory practice as things and possibilities start to open this year.

Emma