Webcam Heart Rate: Quickens the pulse or flatlines?

Posted by Matthew Rosazza on May 29, 2015         0 comments         Bookmark and Share

Just by common intuition, we know that the speed at which our heart beats has a lot to do with what goes on inside our heads.  Exciting or scary movies are often described in their trailers as “heart-racing.”  Our hearts go “pitter-patter” or they may “skip a beat” when we are flustered by the objects of our affection.  So it would seem that heart rate has something to do with arousal in exciting situations.  This is partially true.

If you remember back to high school health class, heart rate is controlled unconsciously by the autonomic nervous system (ANS).  Within the autonomic nervous system there are the sympathetic (SN) and parasympathetic nervous systems (PN).  When an exciting situation is presented, the SN kicks in with the famous “fight or flight response,” which prepares the body for quick mobilization.  Afterwards, the PN sets in and acts as a dampener to allow the body to resume activities it would conduct during rest1.

An example of this in action would be if you are hiking and you come across a bear in the woods.  First, your heart rate would actually lower as you become focused on the problem right in front of you (PN activation).  Heart rate would then increase as you get ready to make your move (SN activation).  Once safe, it lowers once again to its resting rate (PN).  Let’s hope you choose to back away from the bear instead of fighting it.

Instead of seeing heart rate as a one way street (excitation or no excitation), it is more of a two way street of excitation and calming in order to prepare for the situations at hand.  With that being said, we can learn quite a lot psychologically from the lowering of heart rate.  Lower heart rate coincides with higher attention levels, as just seen in the bear example where the hiker was initially more focused2,3.

In the lab, we observe this by means of an EKG (electrocardiogram).  Electrodes placed on the skin can pick up the changes in electrical pulses from the heart. This signal is run through an amplifier and a digital converter to the recording software.  From there it is filtered and analyzed for beats per minute.  Other than counting beats while staring at a watch, this is the most basic and time-tested way of finding heart rate, the gold standard method.  One common alternative is PPG (photoplethysmogram), which involves illuminating the skin and measuring light absorption, which changes as blood pumps through capillaries.

*****

The two methods described above require special electronic equipment and physically contact with the subject.  But what if you want to measure heart rate without any of that?  Can it be done remotely?  What if you wanted to find heart rate by just looking at the person (or rather, looking at a video of them)?  A team at MIT’s Computer Science and Artificial Intelligence Laboratory believes this is possible.  Using a process they developed called Eulerian Video Magnification [http://people.csail.mit.edu/mrub/vidmag/], they created a means to detect seemingly unnoticeable movement in digital video.

First, a person sits in front of a video camera (any camera, which may be a low-quality webcam), and through the software they select a region on their forehead that they want to analyze.  Next, the software breaks down the region into three hues:  red, green, and blue.  It keeps only the green hues, because they show the greatest change in color value as blood circulates through the forehead.  Finally, it measures the change in color value across all the pixels of the region and their relationship to each other.  Through this and a lot of complicated calculation, it finds a pattern:  heart beats.

*****

In my last post, I discussed automated facial coding (http://www.hcdi.net/blog/view.cfm?bID=41) and listed several companies that boast their ability to detect emotions through the use of a simple webcam.  One in particular, Affectiva (http://www.huffingtonpost.com/2012/12/24/affectiva-emotion-recognition-technology_n_2360136.html), also uses this heart rate detection technology in their analysis.  This software has been around for over two years, so it makes sense that it had made the movement from experimentation to application.  Nonetheless, I was curious:  just how accurate is this webcam heart rate, and should the data obtained be considered on par with data gained by traditional means (EKG)?

I was able to obtain the webcam heart rate software called “Pulse Capture” for free from a file sharing site [http://sourceforge.net/projects/pulsecapture/].  There seem to be multiple versions of the software because the code is open source, but I picked one of the more popular options.  For my basic, informal experiment, I decided to record 5 minutes of sitting still, absolutely calm.  I hooked myself up to the EKG and fired up Pulse Capture.  Here’s what I got:


[Orange: EKG, Blue: Webcam HR, Y-axis: BPM, X-axis: time(s)]

As you can see, it’s not very pretty.  I ran some numbers, and the webcam heart rate is off by an average of 2.5 beats per minute throughout the entire session.  It’s off by 23 BPM at the point where it strays the most from EKG.  That’s 4% off on average and 32% off at its max.  I don’t know about you, but personally I’m not a big fan of those numbers.
Looking at this data, I see places where errors in acquisition occurred, namely at seconds 1, 65, 145, and 185.  The issue with this software is that it needs to “lock on” to the region of interest first before it calculates heart rate, and any movement, no matter how slight, disrupts the process.  The amount of stillness I had to maintain to keep it locked on was very unrealistic given all the fidgeting a person is prone to do during the course of looking at media on a screen.

*****

THE VERDICT:

My verdict on this webcam heart rate technology is that it is not properly adapted quite yet to be used for real-time, phasic responses to time-dependent variables.  In non-scientist jargon, it’s not reliable enough to see if heart rate changes in reaction to anything specific.  Though I believe it could be of some use for more general, tonic applications, i.e. before and after a workout, or anything with more global and obvious differences.  It just isn’t reliable enough to generate a steady flow of accurate numbers throughout the entire course of a session.
With that being said, I would be skeptical of any companies that may offer any information based off of this means of data acquisition.  I am not going to outright dismiss their work though—there may be good uses for this technology.  For all I know, they may have tweaked the open source software to be more reliable, or perhaps the version I got was not the best to test with.  But it also seems there may be some fundamental problems with using this methodology for measuring accurate changes in heart rate.  After all, whenever anyone comes around saying they’ve come across a quick and easy way to get around an old problem, it’s always good to take their pitch with a grain of salt. 
Keep in mind this bit of wisdom:  everyone ideally wants something fast, cheap, and good.  Realistically, you can only pick two.  Good and fast isn’t cheap, good and cheap isn’t fast, and fast and cheap isn’t good.  In my opinion, heart rate data acquired by webcam falls in that latter category.

References:

1http://en.wikipedia.org/wiki/Autonomic_nervous_system

2Porges, S.W. and Raskin, D.C.  “Respiratory
and heart rate components of attention.”  Journal of Experimental Psychology  81.3 (1969):  497-503.  Print.

3Coles, M.G.  “Cardiac and respiratory activity during visual search.”  Journal of Experimental Psychology 
96.2 (1972):  371-379.  Print.

Featured Bloggers

Glenn Kessler
President and CEO

Arthur Kover, Ph.D.
Consulting Director, Advertising Research Services and Emeritus Professor of Marketing, Fordham University Graduate Business School


RSS Feed


Archive

May 8, 2015
Face Value: Theories of Emotion and their Application to Neuromarketing

May 1, 2015
Frequently Asked Questions about Applied Consumer Neuroscience

April 8, 2015
Apples and Oranges: The uses and abuses of EEG in market research

March 16, 2015
Thicke as Thieves

March 1, 2015
Our Lying Eyes?

February 23, 2015
Applied Consumer Neuroscience Can Help Optimize Political Communications

February 3, 2015
Is facial coding a valid means of collecting emotional state?

January 29, 2015
HCD Research Unveils New Neuro-Cognitive Division for Financial Services

January 12, 2015
The End Of Market Research As We Know It

November 17, 2014
Neuroscience is more than emotion

September 11, 2014
We used to stuff envelopes... Now we do brain science

August 20, 2014
Sensory Marketing and Sensory Branding

May 8, 2014
The Strange Fear of Truth in the Marketing Profession

March 7, 2014
Marketing Research Supplier or Technology Company?

February 10, 2014
Don’t employ Neuromarketing Research... Unless it is part of an integrated approach

January 3, 2014
The science of interpreting advertising messages:
The parable of the elephant and the optimistic prediction that we will now be able to describe the whole animal

October 3, 2013
Observations from 53rd Annual Meeting of the Society for Psychophysiological Research in Florence, Italy

September 25, 2013
How the Body Informs Design

September 3, 2013
Brains and Brands - Introducing Gayle Marks, Media Psychophysiologist at HCD Research

March 4, 2013
Wrapping Your Brain around Neuromarketing and Biometric Research Measures

December 5, 2012
Time to Measure Up: Brand Communication Research Measures

November 15, 2012
Politics on the Brain

October 16, 2012
Conducting Neuromarketing Research the Old-Fashioned Way

September 27, 2012
Media Psy Ops: Developing Psychological Profiles of Media Audiences

August 30, 2012
Reflections on a "Biometric" Summer

July 3, 2012
Neuromarketing Research: A Tale of Two Approaches

June 11, 2012
Using Emotion in Brand Communication: Lessons Learned from Political TV Programming

May 22, 2012
Neuromarketing Research Suffers from a Brand Image Problem

May 8, 2012
Icebergs and Brand Communication: Taking a Closer Look at What Lies Below the Surface

July 1, 2011
Social Media Research: A Powerful Supplement to Traditional Marketing Research

May 20, 2010
Does Traditional Media Advertising Impact Consumer Sentiment in New Media?

April 30, 2010
To Refresh or Not Refresh: The Value of Monitoring Campaign Wear Out

April 9, 2010
Tiger Woods and Dad in New Nike Ad...Doesn't Help Nike or Tiger

March 23, 2010
Delivering Effective and Ineffective Apologies

February 24, 2010
Super Bowl Ads 2010

November 13, 2009
Measuring the ROI of In-Market Advertising Campaigns

October 22, 2009
Designing Websites: Research Objectives are Similar to Print and Broadcast Communications

January 30, 2009
Relaxing the Death-Grip of Norms in Communications Research