Interview: The Makers of a Robotic Hand That Can ‘Feel’Monday, November 23rd, 2015
Human engineering is often seen as the pinnacle for technological achievement, be it the wheel, the steam engine or the space rocket. Human-engineering has also come a long way, especially in recent times. While the reality of fiction such as RoboCop is still beyond this present time, significant strides are currently being made in the field of understanding biomechanics and developing robotic biomechanics.
Research and innovation are key ingredients in innovative breakthroughs with key life-enriching applications that science and technology can bring to our lives. For instance, it was due to the endeavors of Gavriil Ilizarov, the Soviet doctor who developed the Illizarov fixator apparatus in the 1950s, that this writer’s leg could be saved from a destructive road accident. Such innovation can help improve or even save lives.
Hacked spoke to Vikram Pandit, a prosthetics research engineer and operations manager at Los Angeles-based SynTouch, a company at the forefront of sensory technology that empowers robots to literally replicate our sense of touch. Vikram is also a congenital amputee, born with a single hand and is deemed “instrumental in informing SynTouch’s prosthetics research strategy,” with the unique insight and perspective he offers.
The company was recently awarded two federal research grants that total over $2.5 million over the next few years to develop advanced tactile sensing applications, or, as SynTouch calls it – Machine Touch®.
The company notes:
SynTouch’s products make it possible to do everything you’ve come to expect from your own sense of touch: they enable perception, improve dexterity, prevent damage, and provide awareness.
How does it feel to be your company’s prototype, Vikram?
Vikram: I enjoy being at the cutting edge of prosthetic research. Being patient zero where all of the technology is being tested is awesome and definitely makes me feel like I am using my amputation to help others in my place.
Do you see such innovation generally pushing forward at a faster rate for robotic applications, more-so than for humans?
Vikram: Absolutely. Humans are far more complex than robots. We are barely scratching the service of the human neurological system while we are making leaps in robotic advancement. While we like to think humans can do two things or more at once, we really are only capable of doing one thing and monitoring our second activity passively. A robot, with its onboard computer can be programmed to recognize and accomplish many tasks at once.
Could you explain how the fingertips detect what they feel?
Vikram: The BioTac has three sensing modalities: force, vibration, and temperature. The sensor is a flexible circuit molded in epoxy with 19 sensing electrodes surrounded by a silicone skin and inflated with a fluid. Force is calculated from the electrodes, which use impedance in the conductive fluid to measure the distance between the skin and the electrode – as the BioTac is pushed on a table, the distance between the skin and the electrodes is smaller.
A pressure sensor embedded in the epoxy core picks up vibrations in the skin transmitted through the fluid. The fingerprints on the outside the skin actually amplify these vibration signals enormously. Temperature is sensed through a thermistor placed at the tip of the BioTac.
Are there sensors externally located on a residual limb that will help control the prosthesis? If so, does that relay then tell you that you’re holding a super hot cup of coffee?
Vikram: Myoelectric hands universally function off of sensing electrodes located on the residual limb. Their location is based on what muscles on the residual limb produce the strongest signals. The sensors are located on top of the skin, and pick up the electrical activity in a muscle when it flexes. Almost all myoelectric users have two of these sensors, one for opening the terminal device, and the other closing.
At SynTouch we have performed experiments where the sensations from the BioTac were relayed to me through a number of devices we call tactors. The three tactors were placed on my upper arm or bicep and each was correlated with a specific modality of the BioTac. An air pressure cuff similar to a sphygmomanometer was used to squeeze my arm to mirror a force applied to the BioTac. A small cellphone vibrator buzzed on my skin to emulate the sensor picking up vibration, while a small peltier chip relayed temperatures to me. I was able to distinguish hot soup vs cold soup among other things but I didn’t find these tactors useful.
As a unilateral amputee, my sound hand is better and faster than anything available with current technology in sending sensations to my brain, and the foreign signals were more distracting than helpful. Please review the following paper to find out more. Link [PDF].
Is it overwhelming and distracting when with the feedback from the prosthesis? How would you compare it with your other arm?
Vikram: See above. The current methods of relaying sensation as crude and I can only describe them as being annoying.
Is there a means to use the prosthesis on its own, i.e. let it compute and operate on its own without having to let you know what it feels?
Vikram: After coming to the conclusion that the tactors worked, but were distracting and actually took away from the experience of using a prosthetic hand, we moved in a new direction. I found through the experiments that I liked contact detection for fragile grasping. We then developed a new sensor that distilled down the features of the BioTac, called the NumaTac. An air filled open celled foam, the sensor can detect changes in pressure signaling a contact event.
Three of these sensors have been customized into the fingers of a prosthetic hand and can signal the hand that an object has been grabbed. This allows the hand to move quickly and responsively while empty, but slow down the fingers once an object has been grasped to enable low contact forces. This allows a prosthetic user to consistently pick delicate objects like eggs, consistently and reliably.
(It is at this point that Matthew Borzage P.hd., a founding partner SynTouch and operations manager weighed in to add to the answer.)
Matt: The idea that the hand has some local intelligence guiding its interactions may seem odd, but keep in mind that your hands do thousands of adjustments every second you perform tasks with them, all without you consciously thinking about each movement. It takes humans years to learn to do tasks seamlessly, but once we do the mechanics of pouring a glass of water, using a fork and knife, playing an instrument, typing, and, picking up eggs, all become reflexive, and possible to do using the sense of touch alone.
The information the sense of touch acquires is used in reflexive and automatic ways, that the brain expects to be present. Most prosthetic hands remove these reflexes, and require that amputees use their full attention to perform these simple tasks, which is frustrating! The simplified sensor and the reflex Vikram mentions allows users to do these tasks without requiring as much attention, just like you expect to be able to do.
If BioTac was presumably developed for robotic applications, how far away is BioTac from being integrated with an amputee’s prosthesis?
Vikram: Our sensors are 4 years away from becoming commercially available. We are working with the Department of Defense and the Veterans Administration to integrate our technology into prosthetic devices.
In order to truly feel and sense touch, are we decades away from having the technology and tools to have it merge with the human body’s nervous system?
Vikram: Yes. The nervous system is massively complex and a considerable amount of research must occur before “cyborg” technology is viable. We are decades away from a prosthesis being seamlessly integrated into a human’s nervous system.
How frequently do you use the prosthesis and when do you put away the hook-arm for good?
Vikram: I only have one myoelectric arm right now. Unfortunately at the office and on a hot day I do not wear my prosthesis. Hot days make the socket uncomfortable and sweaty. At the office, I type on a computer and using my prosthesis would be clunky and slow.
What other innovations are SynTouch working on? Are there ideas and concepts in the drawing board that you can reveal?
Vikram: Yes definitely. SynTouch’s most interesting technology up and coming technology is our texture characterization technology. The BioTac was designed with the human finger in mind and can feel everything it can sense (except pain for obvious reason.) SynTouch has built a device integrating the BioTac that can quantify texture. Since the BioTac is designed to feel like a human, it is perfect for this application. Take a look at our client list as an example of who we have worked with.You can liken it to color matching at Home Depot to a degree. Consumer Goods companies are very interested in this technology, as the feel of a product is important to a consumer.
An example is in R&D. Company X wants cloth of a certain texture. Company X produces a number of cloth samples with different formulations in an attempt to achieve the desired texture. Our machine can actually quantify each sample’s texture and the company can use this information to make an informed decision on their cloth production.
Editor’s Note: Hacked thanks Vikram and Matthew for their time and answers.
Images from SynTouch.