Thinking about a new hat. How about Haptic Assisted Technology! It’s the new go-to tech in town, though no stranger to the driving experience. Older cars had a diverse tactile landscape with levers, knobs, and buttons. This landscape consisted of “anticipatory haptics”, recognized with the sense of touch, that reduces the need to look away from the road while driving. One study referenced in a recent New York Times article found that the use of manual transmission in teenagers with ADHD driver attention and safety, indicating a benefit that occurs with more engagement with driving not less.

Fast forward to today, and many vehicles are missing many traditional haptic controls. Instead, untextured buttons and smart-screens dominate.  The problem is that smart screens distract the driver’s gaze from the road and require study for content and precise finger selection. In all its forms and sizes, these screens are seductive and addictive eye candy that cause distraction and endanger the driver.

Driver assistance technologies and education can improve safety, such as lane-departure warning and blind-spot detection systems. Haptic feedback enhances these systems, such as by applying resistance to a dangerous turn in the steering wheel. A University of South Florida study found that one way to enhance blind spot detection was by sending a vibration to the hands of the driver during an unsafe lane change. Many vehicle brands depend solely on an audio or visual alert for this purpose, which may go unnoticed because of traffic complexity, road noise, and other distractions. Multi-sensory feedback that includes haptics may be more effective.

SofTrek has previously presented another type of Haptic Assist Technology that in conjunction with other systems may mitigate the danger of online distraction by keeping eyes on the road and hands on a steering wheel with haptic controls.

Consider this scenario:

You are driving to work in the middle highway lane on a sunny day. Your car senses the vehicles in every direction by sound waves, pattern recognition, and vehicle to vehicle and vehicle to road communication and keeps you at a full distance from other objects on the road.  You are looking at the windshield and a superimposed projection of a weather website. The format of the website includes menu items that match the position and feel of steering wheel button. You can make reading a news story online projected onto the windshield and can make selections on the web page because steering wheel buttons have tactile features that match icons on the steering wheel.  When you want to select an item, such as the weekend forecast, you press a steering wheel button that corresponds to the position and feel of the menu item. This keeps your hands on the steering wheel and eyes forward on the superimposed view of the road. Like a roadside billboard, the projection may be off to the side or not fully opaque, so that you can still see the road. Other driver assistance technologies kick in to cause abrupt disappearance of the projected website if driving conditions become complex or if an unexpected traffic event occurs. For example, a car or an object cutting into your lane would trigger automated collision warning and autonomous emergency braking.   When safety prevails, the driver can multitask and use haptic steering wheel buttons and projected menus to make phone calls, manage infotainment, navigate the internet and operate the vehicle controls.

Sound far fetched? Consider air flight and the evolution from flying kites to blasting rockets to Mars and beyond. When we consider the past and let our imagination soar, autonomous vehicles become a reasonable part of the future. Assistance technologies are part of the evolution in that direction and include a HAT worth wearing!

Footnotes: Online search terms like “Haptic Assistance Technology” or “Haptic Assisted Technology”  generate a list for “Haptic Assistive Technology.” These refer to sensory substitution aids for individuals with visual. Braille, designed to help the blind, is a prime example but may have related application for the sighted, such as situations where the sense of touch is needed because of darkness or the out of the line of sight position of one’s hands.

Images from iStock photo

Sometimes when I think about what the future will be like when autonomous vehicles take over the roads, the tale of the Sleepy Hollow comes to mind. It is the story of a man making his way home on a dark country road, when he hears distant galloping and is frozen with fear by the approach of a horse saddled with a headless man. Though dead and lacking the usual above shoulder attachment, the figure races forward with purpose and precision. Though there are differences, the autonomous car can also be scary. A recent article in the New York Times describes explicit hostility where this technology is being tested because of safety and job loss concerns. Deadly accidents are one factor that put the kibosh on consumer acceptance which is in flux.  A AAA survey reveals that most drivers are not comfortable with sharing the road with driverless cars.

Like the evolution of living beings, technology is a work in progress that depends on endless experimentation and adaption to challenges and change.  Who knows what the driverless car will be like in a hundred years? Past experience may guide our expectations. Pedestrians had to get used to walking where horses and more recently driven cars tread. Now the challenge is getting used to driverless vehicles. As with Sleepy Hollow, dealing with fear is key. Trust needs to be built. Errors need to be understood. The public needs to be educated on how to use and interact with this technology safely, just like we needed to learn how to operate stoves. What do you do when the headless horseman races towards you? How do you interact and make sure that you are being recognized?    

SofTrek has a solution that keeps a head on the horseman by keeping the driver’s “eyes on the road hands on the wheel”. To do this, SofTrek adds tactile features to computer buttons so that they can be located without the need to look away from a display. If these buttons are on a steering wheel and matched with the format of projected menus, the driver does not need to look down to locate a desired button or away from the road to reach for a dashboard or cell phone screen. Safety is maintained by enabling a sudden attention switch to the road by what is sensed by the driver and the vehicle’s computer.

By combining different solutions, there are happy trails to Sleepy Hollow. Men and women, their horses, and little children too, depend on them.  

When I had my office in the old building, I really enjoyed having a window to the outside. The flow of the day shifted with the sunlight and clouds, the rustle of the trees, and the movement of people and animals outside while I was tied to a monitor with reading echocardiograms.

That all changed with the construction of a shiny new office tower and my relocation to a new, spacious office, which was very nice except for the absence of windows. I missed the “brain breaks” that glancing outside gave me. Reading echos depends on constant data processing, actively forgetting and making room for new incoming so managing mental fatigue is very important.

I looked to my hobbies to put breathers into my workflow.  This included an aquarium with live plants, sandstone, and peaceful fish, like guppies and mollies. The chaotic activity of my fish helps release the knots in my mind. A hydroponic system called AeroGarden ®  was placed beside the aquarium. The system generates a mood-lifting natural light and grows stunning flowers that prompt happy comments from people walking by my open door. I teach the beauty of life can be found in the structure of plants and hearts.

Fatigue management also consists of a stunning 4K monitor on the wall in front of my desk that was generously provided to me by my employer. The large monitor mirrors my desktop so that cardiology fellows can follow and learn from my interpretations as I point to difficult structures with my mouse. Sometimes, there will be a group of fellows, residents, and medical students learning from me. At times, I raise my sit-stand desk so we can stretch and re-frame.  

I can play nature videos on the 4K monitor like the ones from Nature Relaxation™  or Youtube and revel from vistas of Norwegian fjords, Patagonians landscapes, and the underwater dances of whales. The monitor is my virtual window. It helps me focus and lighten my mood. When I use the 4K monitor for teaching, I position my laptop beside my desktop monitor and glance at nature images on a smaller screen. My students enjoy this very much because it inspires pleasant banter and intermingles thoughts of nature with the learning of cardiology concepts. The restorative images give me the stamina to analyze massive amounts of medical data.

My experience is supported by what has been published. Research pioneered by Rachel and Stephen Kaplan shows that seeing nature manages mental fatigue and stress and improves memory and attention. Renowned writer and neurologist Oliver Sacks MD notes in his essay collection “Everything In Its Place” that nature “exerts its calming and organizing effects on our brains” and describes how exposure to gardens benefited his patients. He asserts that this is also “critical for people working long days in windowless offices”. These are powerful observations that concur with my experience. Furthermore, projected images also do the trickfor me; they help me stay focused and in flow. It addresses a nature deficit disorder, described by Richard Louv in children, but can affect adults too. His book, “The Nature Principle”, describes how connecting with nature promotes health, creativity and mental acuity. For education, video projections allow a simulation of the outdoor classroom inside.

Wherever I go, I see people glued to their screens, together and separated at the same time.  But these screens can also be beneficial. Many people work in rooms and cubicles without windows and have similar needs. The good news is that we can benefit from screens to access nature and work in a room with a virtual view.

Disclaimers and credits: No endorsement of this blog or website is implied by references to individuals, institutions, companies or their products.

A Room with a View is a book by E.M. Forster that was published in 1908.
Oliver Sacks MD quote from  New York Times article and his essay collection  “Everything in Its Place”  

Have you ever eaten trail mix while driving?  No need to look into the bag to make selections: M&Ms, cashews, or raisins are easy picking thanks to the sense of touch or haptics, which is how fingers “see”.  I extrapolated the trail mix experience to haptic stickers which I put on different car controls. I wanted to see whether doing this improved my driving experience when placed on the A) steering wheel cancel cruise control button, B)  drive button and C)  heads-up display.

Here’s what I found:

1. For steering wheel buttons that adjust cruise control (left photo), my right thumb moved back and forth between the toggle that adjusts speed to the cancel button that has a haptic sticker on its surface.  If the haptic sticker was not there, I would either have to look down to find it or use the brake pedal. Either of these options would require more effort or be more distracting. Because the steering wheel button application was helpful, I have applied more haptic stickers to other buttons and continue to rely on them.
2. The orange haptic sticker on the drive button (middle photo) was helpful to a lesser degree. I need to reach for the button to start driving and watch the movement of my finger.  Having a distinct tactile target may still be helpful by providing tactile feedback when the button has been touched and possibly by shortening the time that I am visually guiding my reach.
3. The haptic stickers on the heads-up display (right photo) took more getting used to but also may have also be advantageous like with the drive button. Having a tactile target seems more natural, satisfying and confirmatory than reaching for a flat screen without distinct tactile features. Flat screens take one’s eyes off the road.  It is distracting to need to watch one’s finger while reaching, study the display, and make sure that the desired option has been selected.  (Consumer reports provides a nice review on this topic.)

Conclusion. I found it helpful to add tactile features to the controls in my car, especially on the steering wheel but to a lesser degree when reaching. Doing this helped keep my eyes on the road and hands on the wheel. In other words, keep haptics close at hand.                You would not want to eat your trail mix any other way!

 

WARNING: THIS PURPOSE OF THIS POST IS TO ILLUSTRATE HOW ADDING TACTILE FEATURES TO BUTTONS MAY BE USEFUL. BECAUSE SAFETY STUDIES HAVE NOT BEEN DONE ON PERSONALIZED TACTILE MODIFICATION OF CONTROLS, SOFTREK DOES NOT RECOMMEND THAT READERS FOLLOW THESE EXAMPLES.  SOFTREK INC ASSUMES NO RESPONSIBILITY IN THIS REGARD FOR ANY INJURY, DEATH, OR LOSS OF ANY KIND. THE PERSONALIZED MODIFICATIONS DESCRIBED ABOVE HAVE NOT BE ENDORSED BY ANY MANUFACTURER OR REGULATORY INSTITUTION. 

PHOTOGRAPHS ARE DERIVED FROM MAXI_AIDS, INC “BUMP-DOTS” AND ACURA VEHICLE AND iSTOCK PHOTO. NO ENDORSEMENT BY THESE COMPANIES OF THIS POST IS IMPLIED AND iSTOCK PHOTO.

There was a time long ago when there were no smartphones or internet and virtual reality was imagination.  In the 60s, when I was 13, reading Superman and Batman comic books was a favorite activity. The best part was the page on the back with ads for things like whoopee cushions, stink bombs, and x-ray glasses, that could see through clothes! The makers of these glasses were onto something big!!  I appreciate this much more now because of a great TED talk by neuro-scientist David Eagleman that discusses umwelt or one’s experience of reality, which  is determined by what we can sense and think. For example, on my morning walk,  my dog constantly needs to stop to smell street posts. I  used to find this annoying, but now I understand her umwelt which allows her to read the daily dog news coded in scent.

Another thing I learned from the talk was that our brains are like computer processors that discern patterns from the inputs of our senses, which are like interchangeable computer peripherals. When one sense fails, our brains can adapt to discern patterns from other senses. This allows the blind to use hearing to “see” sound. Technology can also compensate for blindness by integrating with the optic or auditory nerves so that electronic peripherals can send vision or sound to our brains. Technology augments our senses. One may wear glasses  to see infrared, like snakes do, or ultraviolet light, like bees, or wear hearing aids to hear sounds waves beyond our natural range. I would like to find a nasal device to enjoy the daily dog news with my pooch – what a bonding experience that will be!

Augmentation, miniaturization, and biologic integration – this seems to be where we are going. As a physician, I see new frontiers, which allowed me to understand why a certain tumor makes a “plop” sound as it moves between the chambers of the heart. As an inventor focused on automotive innovation, I see  ways our senses can connect drivers with their vehicle. I proposed putting buttons on the steering wheel that are recognized by the sense of touch so that drivers can control their vehicles without the need to look away from the road. The toolbox of all our senses will plug’n play our brains and devices together in the realm of robotic avatars. New superpowers are enabling us to do tasks distant from our physical location – like perform surgery or explore Mars. While this all seems fantastic, I need some new comic books to find out what happens next.

Additional comments and hyperlink reference notes:

• X-ray glasses in the 1960s seems pretty lame  compared to the internet of thought that pervades today’s adolescent mind. How unfortunate!
• The Argus® II Retinal Prosthesis System wirelessly transmits data from a video camera in a blind patient’s glasses to an eye implant that electrically stimulates remaining retinal cells. This enables perception of visual patterns.  A brief presentation can be found here.
• Cochlear implants consist of a microphone, processor, transmitter and electrode array that transforms sound into electrical stimulation of the auditory nerve. This technology can help a deaf person perceive speech and other sounds.
• I published a paper called “The Etiology of Atrial Myxoma Tumor Plop” in the Journal of the American College of Cardiology, which was based on frame by frame analysis of ultrasound images. I showed that when an atrial myxoma moves from the left atrium to the left ventricle, it obstructs the mitral valve and causes a high velocity blood jet that may correspond to a tumor plop sound.
• Our hands represent the ultimate approach for machine control because of their fine motor and sensory abilities.  Because touch is how hands  “see”, adding textures, shapes, and temperature differences to interfaces will enhance our ability to manually control devices.
• James Cameron reached the deepest depths of almost 7 miles with his Deepsea Challenger submersible. There are many advantages to explore biologically perilous environments on Earth (and other planets) using robots that remotely integrate with our body and mind, with our being safely situated eating popcorn. Robotic avatars can extend human  senses and abilities vastly in all fields of endeavor, especially medicine.
• Photograph of digital eye is from iStock Getty Images by Brian A Jackson.  Dog photos are copyrighted by GeriCoh LLC.