Notes from a meeting With Sally Linkenauger Lancaster University Psychology Dept. After discovering Sally's research at the BRnet conference I wanted to experience her experiments, so I asked if I could visit her lab. The VR lab looks like a normal office space, PCs scattered around the outside, and loads of VR tech hanging around. On closer inspection, a network of cameras is installed around the space to enable complex motion capture and augmented reality experiments to be undertaken. There is a treadmill in the corner. In the experiment I tried, a leap motion sensor captures the movements of your real hand, allowing you to articulate a virtual hand. The experiment requires some time to become used to the VR environment. At first,
Author: antonyhall
Artist, educator, and researcher working between the fields of science and art.
Seeing with the tongue – Paul Bach-y-Rita –
"You don't see with the eyes. You see with the brain" Paul Bach-y-Rita, Science News Online 1 Sept, 2001; vol. 169, no. 9 Paul Bach-y-Rita was one of the first neuroscientists to study the idea of Neuroplasticity. He did a number of experiments on sensory substitution, developed the idea of "Brain Port" in 1998. This interface uses a camera to feed an image to an electrode array placed on the tongue. Blind patients were able to see using the tongue. And a chair, the back of which is packed with solenoids, the camera feeds an image to these. After time and with training the user is able to sense images, at first basic shapes, and with more time can sense more detail such
Aperture navigation
From the field research trip to the Middlewood Trust, I developed a set of goggles with a small hole and slot apertures. Participants report different perceptions of time and space, and of sound and distances. I thought I had walked for a great distance, only to find I had only moved a few meters. One participant spoke of having 'microscope eyes' and how 'the closer you get to things, the less you can see of them' focussing and depth of field also become more apparent. Here is the extract from my experience: "Seeing through the small aperture also had the effect of making things seem like an old movie [grainy and soft-edged]. After wearing these for a while, I could hear others
Navigations
The Ganzfeld [ 'total field' ] experiment is a form of perceptual deprivation, giving an experience of a uniform field of light [More information here] . has become a staple activity in my workshops. Rather than sitting still and listening to sounds I have been opening this up as a mobile activity, or 'navigation' as I have started to call it. It serves as an icebreaker, often requiring people to work in pairs to move through spaces, following ropes or sonic stimuli. The purpose is to heighten the participant's awareness through altering their perceptual experience of space. Participants become aware of new structures of light or start using the body in a different way in order to move, becoming more
Lenticular goggles
Speaking with an artist while making plans for a future Action Lab, I came up with the idea of making these lenticular glasses [Above], which use a lenticular lens [textured with lines or ridges] which have the effect of dividing the visual field into lines. In an urban or interior space, vertical and horizontal lines are common. These become enhance or reduced depending on the orientation of the lens. Using the lens in a vertical alignment, steps become invisible, but when the lens is rotated they become enhanced. Using a combination of both things get even stranger. Point light sources create strong bright lines. I have yet these in a natural environment.
The Multi-faceted body
A few notes from the BRnet Conference the Multi-faceted body. June 2018, Heriot-Watt University, Edinburgh, UK I felt strangely at home among so many research projects that use the hand as a model for the wider issues of body representation and ownership. Keynote speaker Mel Slater's talked about his playful and engaging research. Slater’s work centres around the idea of using immersive virtual reality as a medium to transform the self, not just the reality of a space. His research projects include self-help therapy. The project Personified Self Interaction (Osimo et al., 2015) allowed ‘you’ [embodied within an avatar of Freud] are able to talk to ‘yourself’ [an automated replay of you, embodied within an avatar of yourself] and vice-versa. This enables the
Phantom Presence
This experiment induces the sensation of a phantom presence in the room. The participant blindfolded is asked to use a stylus to prod an empty space in front of them. Using a tactile feedback system and a robot arm, the participant feels as if they are prodding themselves in the back. As the experiment progresses the system adds a delay to this prodding. At this stage, the participants become freaked out believing that someone else if prodding them... https://youtu.be/GnusbO8QjbE Link to the paper... Neurological and Robot-Controlled Induction of an Apparition https://www.cell.com/current-biology/abstract/S0960-9822(14)01212-3
Fieldwork / Night walk
I teamed up with artist Annie Carpenter to pull together a small group of artists and friends for a night of ‘fieldwork’. We organised an overnighter to do some experiments and have discussions together in the relaxed atmosphere of Middlewood Trust study centre; an off-grid permaculture farm. I had worked here before with [Annie and Sam Illingworth] doing some workshops with their students on a previous 'field research' style project. The concept captured my interest. We wanted to create a situation where we could work as well as have time and space to chat about ideas with others. My equipment consisted of my laptop, Arduino, [with a relay shield for experiments] and some electronics, GoPro, 360-degree camera. Also a heavy rechargeable
Tactile Anchoring Device Prototype 1
Here is my prototype device intended to help autonomously generate the 'invisible hand illusion'. For this experiment, I created a series of brushes which rotate at different speeds stroking empty space. The idea was that the participant watches this device, while the brushing motion is replicated one their real hand hidden nearby. This is building towards a piece of work called 'On the embodiment of a discrete Volume of Empty Space' [ See http://antonyhall.net/blogtactile-anchoring-device/ ] https://www.youtube.com/watch?v=DI-f6KgRD2w&w=700&h=400
Augmented senses
In this experiment, they created a simple device [Hearspace App] incorporating a compass and headphones. It "allows users to reliably hear the direction of magnetic North as a stable sound object in external space on a headphone. They found that "long-lasting integration into the perception of self-rotation. Short training with amplified or reduced rotation gain in the magnetic signal can expand or compress the perceived extent of vestibular self-rotation, even with the magnetic signal absent in the test" I was struck by this statement "sensory substitution and augmentation research has aimed to restore sensory functionality from non-invasive afferent signals of artificial sensors...there has been little concrete evidence that truly perceptual experiences have ever been obtained via this approach" Sensory augmentation: integration of an auditory
“On the embodiment of a discrete volume of empty space”
As I mentioned before [Illusions of Invisible, alien hands, 3 arms, and shrinking bodies…] The Invisible Hand Illusion is a version of the Rubber hand illusion which uses no fake rubber hand at all. Instead, the participant focuses on an empty space in place of their hand. Sometimes a handless stump, and as a control, a wooden plank. they found that participants could embody a ‘Discrete Volume of Empty Space’. This interests me as it fits with a number of works which challenge the physicality of the art object, instead, I have presented amorphous forms of bubbling materials or even a plinth which automatically concealed the object. The idea of making an automated system to generate the illusion of an invisible hand seems the next step for my research.
Simulation of touch
https://www.youtube.com/watch?v=LKMndCEr4Mw
Poster – BRnet Conference 2018
Electronic Taste Perception Workshop – Radiona
[Here is some documentation from one of the activities for the 're-mapping the senses workshop earlier this year...] Our sense of taste is directly affected by the colour and smell of the food. Experiments prove that the colour of a drink affects our perception of its sweetness for example. Altering the sound of the food, say adjusting the high-end frequencies while eating crisps can also affect our perception of the crunchiness of those crisps [see the paper here. Playing with these assumptions and expectations can create heightened food experiences. There have been a number of studies that suggest it is possible to simulate Sweetness, bitterness, sourness. Specifically, we test these settings as claimed by the Vocktail project [see below] to simulate the following sensations... Sour: magnitude of current: 180
Making Immersive Experiences
“ What do new immersive technologies, such as augmented reality, virtual reality and artificial intelligence, offer storytellers and makers? And do they change the stories we choose to tell audiences? Two internationally renowned digital storytellers and makers present their latest creative projects and help us to find answers to these questions.” Notes from 'Making Immersive Experiences 1: Lance Weiler, Storytelling Lab, Columbia University + Chris Mullany, Marshmallow Laser Feast' Organised by Immersion Research Group, Manchester Metropolitan University Immersive storytelling is not something I have really ever thought about in relation to my work, however, I find the ideas discussed resonated strongly with my own practice and collaborations I have been part of. While listening to the speakers I realised I have taken part in
VR technology
Some notes and links, reflecting on the VR mask. The absurdity of the interface is both clumsy and undignified technology orientated experience.... https://www.youtube.com/watch?v=NtwZXGprxag The Sword of Damocles 1968 was the first HMD unit, invented by Ivan Sutherland. https://www.youtube.com/watch?v=gWLHIusLWOc "The intention of Birdly® is to fulfil people’s ancient dream of flying. With virtual reality (VR) and robotics technology, SOMNIACS creates an extremely vivid full-body experience that makes you instantly forget the mechanics and computer codes behind this spectacular apparatus. The immersive and interactive nature of Birdly® serves one goal: to enjoy the ultimate freedom of a bird and intuitively explore the skies." http://www.somniacs.co/ https://www.youtube.com/watch?v=qh2UdRKNqH4
Experimental Psychology Lab
I have been considering what an Experimental Psychology lab looks like. What objects might these contain? Typically these consist of office like rooms, chairs and tables, perhaps an observation room. These days a computer, sensors for eye tracking and biometric measurement would be commonplace. Headphones, and VR equipment. EEG and MRI image may also be used in more advanced studies. Ultimately Experimental Psychology is an interdisciplinary subject and experiments could cross over to neurosciences, AI and robotics. But a google search of Experimental Psychology laboratory yields some interesting results. I'd be interested to know what other instruments and devices I should look at when thinking about this field of research. Please let me know if you know what any of these items might
Clay Hand Experiment – Whitworth Art Gallery
This was the first of the public workshops performed specifically for my research. My experiments so far have taken place as 1-1 sessions in artists studios or at the university. In a more open drop-in format, the participants were invited to experiment on themselves. Opening up the environment to one of play rather than pure participation. Breaking the experimenter/participant loop, creating a flexible relationship between myself and the participants. This was more in keeping with the idea of a phenomenological investigation allowing the experience to ‘unfold’. This had a dual effect of actively involving and empowering the participants while being able to observe the process from the outside. This meant notes and make recordings could be taken more effectively. Further reading:Read more
Drawing the unfeasible objects
I chose to draw each clay object created for the unfeasible objects experiments. Why? The photographs don't seem to do justice to the significance of the object, its purpose and the process its undergone. Both with my self and the participant. They say drawing itself is a form of embodiment. "In the gesture of a drawing, there abides the question of how human beings hold memory. A trace of the body, the projection of an emotion, a record of the experience of seeing are woven into the gestured mark, a kinetically vitalized inscription that can serve as a site of empathy and invitation as much as a line of mimetic description" Marking Time, Figuring Space: Gesture and the Embodied Moment. Sara Schneckloth The
Clay hand experiments at Radiona
Some documentation from the workshop at Radiona in Zagreb [See Re-mapping the senses workshop] [For more information about the clay hand experiment see here] [For more information about the clay hand experiment see here]
Proprioceptive drift visualisation
[wpvideo VartTqKO ] My experiments show a strong ‘drift’ in the perceived location of the hand before and after the experiment. This is known as proprioceptive drift. I asked people to locate the position of their index finger under a platform before and after embodying a clay object. And also draw their hand without looking before and afterwards. This animation shows these superimposed, illustrating the drift away from the real location of the hand.
Fish-brain-machine / Radiona workshop
After constructing the Fish-brain-machine PCB circuits we spent some time experimenting and describing the hallucinogenic visuals created by the stroboscopic light. The ping pong balls over the eyes diffuse the LED light, making for a more intense effect - and enabling use with eyes open. Here they describe some of the effects including seeing colours and 'a strange experience' of seeing with only one eye - I get this exact same feeling when using it. It is also hard to know if your eyes are open or closed. [See also Re-mapping the senses workshop ] https://www.youtube.com/watch?v=Lwne5xICLkI
Spontaneous sensations
Possibly one of the simplest experiments possible to conduct, requiring no special equipment. Simply stare at your hand for 5 mins. The experiment is described in the paper “the tickly homunculus and the origins of spontaneous sensations arising on the hands" in which you focus on your hand while staring at it (convergent focusing) or divergent focusing (staring at red marker next to the hand you are focussing on) for just 10 seconds and report the sensations. Here are the outcomes superimposed from 8 participants. In this experiment, many of the bemused participants described a tingling where the hand made contact with the table. They were bemused because I did this experiment in the context of our methods and methodologies discussion group - where I was
Radiona workshop, Zagreb
I'm doing a workshop related to my research, perceptual illusions and altering perception through experiments, Clay hand illusion and other activities based on sound and light and taste, at Radiona Zagreb, 21-22/04/2018 https://radiona.org/ check out the web Re-mapping the senses workshop page and resources here...
Fish-brain-machine
As part of my Enki exhibition at Kapellica Gallery in Ljubljana 2012, I developed a related perceptual illusions and brain hacks workshop with Marc Dusseiller [Hackteria], as part of the gallery’s Biotech program. We came up with the idea to make a special issue circuit for the workshop and we set to work designing a circuit the encapsulated the Enki project in miniature. After a couple late nights, we came up with this super cool PCB design. Marc worked hard to create a fully functional efficient design, which was also aesthetically pleasing. The outline of the fish is also the ground in the circuit. This has to be the most ultra minimal brain-machine available to build. 6 components. We spent further
Strange face illusion with two-way Mirror feedback
“… a quiet room dimly lit by a 25 W incandescent light. The lamp was placed on the floor behind the observer so that it was not visible either directly or in the mirror. A relatively large mirror (0.5 m60.5 m) was placed about 0.4 m in front of the observer. The luminance of the reflected face image within the mirror was about 0.2 cd mÿ2 and this level allowed detailed perception of fine face traits but attenuated colour perception…The task of the observer was to gaze at his/her reflected face within the mirror. Usually, after less than a minute, the observer began to perceive the strange-face illusion…” Giovanni B Caputo, Perception, 2010, volume 39, pages 1007 – 1008 2010
Mechanically Augmented Reality
When I first learned about the Rubber hand illusion [RHI] I immediately began to think about ways in which the process could be automated to create an artwork, in which the experimenter's presence was removed. This could be done using sensors and a microcontroller to articulate solenoids to tap fingers for example. That thought train was on a back burner until discovered the work of fellow MMU PGR Lin Charlston. As part of her PhD research [See her profile here 'The Multimodal book as organism, artefact and assemblage: non-human agency in processes of growing and making'] She invented a beautifully simple, entirely mechanical device just to do exactly this. Your hand is placed inside a box, while the other hand turns a crank. On top of the box, another rubber hand
On the embodiment of unfeasible objects
Further to my last post detailing experiments relating to the embodiment of invisible and even third hands, here are some notes on my first 'Clay hand Illusion' experiments... The ‘Rubber hand illusion’ shows it is possible to convince participants that a rubber hand is their own by placing it in front of them while stroking it in the same way as their hidden real hand. The use of self-made clay hands, or objects [see below] in place of the rubber hand raises several interesting possibilities for exploration, which move away from the embodiment of replica body parts, and towards the possible embodiment of modified body parts, or completely 'unfeasible' objects.The clay allows for the gradual and immediate morphing of forms and for the
Illusions of Invisible, alien hands, 3 arms, and shrinking bodies…
I have previously posted about the original 'rubber hand illusion' in which participants are convinced a fake rubber hand is their own. A classic low tech experiment that can help us "understand how sight, touch and “proprioception” the sense of body position, combine to create a convincing feeling of body ownership, one of the foundations of self-consciousness (Nature 1998, vol 391, p 756 )” Further to this research recent studies have gone on to experimentally induct of out of body experiences or create body swap illusions. [ Henrik Ehrsson and Olaf Blanke http://www.eurekalert.org/pub_releases/2007-08/ucl-foe081407.php ] Between these two experiments, several interesting versions of hand-related experiments also exist which demonstrate the plasticity of our body perception... The Alien Hand Experiment first created in the 1960s predates the Rubber hand experiment, seems more 'trick'
Blind drawing of hand
Blind drawing of hand starting from the left and using a continuous line, before and after experiments. Could this exercise be used as a measure of "proprioceptive drift" before and after 'Rubber hand illusion' type experiments? I will be exploring this idea soon with the research group at BEAM lab...