Webcam Heart Rate: Quickens the pulse or flatlines?
Posted by Matthew Rosazza on May 29, 2015 0 comments
Just by common intuition, we know that the speed at which our heart beats has a lot to do with what goes on inside our heads. Exciting or scary movies are often described in their trailers as “heart-racing.” Our hearts go “pitter-patter” or they may “skip a beat” when we are flustered by the objects of our affection. So it would seem that heart rate has something to do with arousal in exciting situations. This is partially true.
If you remember back to high school health class, heart rate is controlled unconsciously by the autonomic nervous system (ANS). Within the autonomic nervous system there are the sympathetic (SN) and parasympathetic nervous systems (PN). When an exciting situation is presented, the SN kicks in with the famous “fight or flight response,” which prepares the body for quick mobilization. Afterwards, the PN sets in and acts as a dampener to allow the body to resume activities it would conduct during rest1.
An example of this in action would be if you are hiking and you come across a bear in the woods. First, your heart rate would actually lower as you become focused on the problem right in front of you (PN activation). Heart rate would then increase as you get ready to make your move (SN activation). Once safe, it lowers once again to its resting rate (PN). Let’s hope you choose to back away from the bear instead of fighting it.
Instead of seeing heart rate as a one way street (excitation or no excitation), it is more of a two way street of excitation and calming in order to prepare for the situations at hand. With that being said, we can learn quite a lot psychologically from the lowering of heart rate. Lower heart rate coincides with higher attention levels, as just seen in the bear example where the hiker was initially more focused2,3.
In the lab, we observe this by means of an EKG (electrocardiogram). Electrodes placed on the skin can pick up the changes in electrical pulses from the heart. This signal is run through an amplifier and a digital converter to the recording software. From there it is filtered and analyzed for beats per minute. Other than counting beats while staring at a watch, this is the most basic and time-tested way of finding heart rate, the gold standard method. One common alternative is PPG (photoplethysmogram), which involves illuminating the skin and measuring light absorption, which changes as blood pumps through capillaries.
The two methods described above require special electronic equipment and physically contact with the subject. But what if you want to measure heart rate without any of that? Can it be done remotely? What if you wanted to find heart rate by just looking at the person (or rather, looking at a video of them)? A team at MIT’s Computer Science and Artificial Intelligence Laboratory believes this is possible. Using a process they developed called Eulerian Video Magnification [http://people.csail.mit.edu/mrub/vidmag/], they created a means to detect seemingly unnoticeable movement in digital video.
First, a person sits in front of a video camera (any camera, which may be a low-quality webcam), and through the software they select a region on their forehead that they want to analyze. Next, the software breaks down the region into three hues: red, green, and blue. It keeps only the green hues, because they show the greatest change in color value as blood circulates through the forehead. Finally, it measures the change in color value across all the pixels of the region and their relationship to each other. Through this and a lot of complicated calculation, it finds a pattern: heart beats.
In my last post, I discussed automated facial coding (http://www.hcdi.net/blog/view.cfm?bID=41) and listed several companies that boast their ability to detect emotions through the use of a simple webcam. One in particular, Affectiva (http://www.huffingtonpost.com/2012/12/24/affectiva-emotion-recognition-technology_n_2360136.html), also uses this heart rate detection technology in their analysis. This software has been around for over two years, so it makes sense that it had made the movement from experimentation to application. Nonetheless, I was curious: just how accurate is this webcam heart rate, and should the data obtained be considered on par with data gained by traditional means (EKG)?
I was able to obtain the webcam heart rate software called “Pulse Capture” for free from a file sharing site [http://sourceforge.net/projects/pulsecapture/]. There seem to be multiple versions of the software because the code is open source, but I picked one of the more popular options. For my basic, informal experiment, I decided to record 5 minutes of sitting still, absolutely calm. I hooked myself up to the EKG and fired up Pulse Capture. Here’s what I got:
[Orange: EKG, Blue: Webcam HR, Y-axis: BPM, X-axis: time(s)]
As you can see, it’s not very pretty. I ran some numbers, and the webcam heart rate is off by an average of 2.5 beats per minute throughout the entire session. It’s off by 23 BPM at the point where it strays the most from EKG. That’s 4% off on average and 32% off at its max. I don’t know about you, but personally I’m not a big fan of those numbers.
Looking at this data, I see places where errors in acquisition occurred, namely at seconds 1, 65, 145, and 185. The issue with this software is that it needs to “lock on” to the region of interest first before it calculates heart rate, and any movement, no matter how slight, disrupts the process. The amount of stillness I had to maintain to keep it locked on was very unrealistic given all the fidgeting a person is prone to do during the course of looking at media on a screen.
My verdict on this webcam heart rate technology is that it is not properly adapted quite yet to be used for real-time, phasic responses to time-dependent variables. In non-scientist jargon, it’s not reliable enough to see if heart rate changes in reaction to anything specific. Though I believe it could be of some use for more general, tonic applications, i.e. before and after a workout, or anything with more global and obvious differences. It just isn’t reliable enough to generate a steady flow of accurate numbers throughout the entire course of a session.
With that being said, I would be skeptical of any companies that may offer any information based off of this means of data acquisition. I am not going to outright dismiss their work though—there may be good uses for this technology. For all I know, they may have tweaked the open source software to be more reliable, or perhaps the version I got was not the best to test with. But it also seems there may be some fundamental problems with using this methodology for measuring accurate changes in heart rate. After all, whenever anyone comes around saying they’ve come across a quick and easy way to get around an old problem, it’s always good to take their pitch with a grain of salt.
Keep in mind this bit of wisdom: everyone ideally wants something fast, cheap, and good. Realistically, you can only pick two. Good and fast isn’t cheap, good and cheap isn’t fast, and fast and cheap isn’t good. In my opinion, heart rate data acquired by webcam falls in that latter category.
2Porges, S.W. and Raskin, D.C. “Respiratory
and heart rate components of attention.” Journal of Experimental Psychology 81.3 (1969): 497-503. Print.
3Coles, M.G. “Cardiac and respiratory activity during visual search.” Journal of Experimental Psychology
96.2 (1972): 371-379. Print.
President and CEO
Arthur Kover, Ph.D.
Consulting Director, Advertising Research Services and Emeritus Professor of Marketing, Fordham University Graduate Business School
March 16, 2015
Thicke as Thieves
March 1, 2015
Our Lying Eyes?
February 23, 2015
Applied Consumer Neuroscience Can Help Optimize Political Communications
February 3, 2015
Is facial coding a valid means of collecting emotional state?
January 12, 2015
The End Of Market Research As We Know It
November 17, 2014
Neuroscience is more than emotion
September 11, 2014
We used to stuff envelopes... Now we do brain science
August 20, 2014
Sensory Marketing and Sensory Branding
March 7, 2014
Marketing Research Supplier or Technology Company?
September 25, 2013
How the Body Informs Design
December 5, 2012
Time to Measure Up: Brand Communication Research Measures
November 15, 2012
Politics on the Brain
October 16, 2012
Conducting Neuromarketing Research the Old-Fashioned Way
September 27, 2012
Media Psy Ops: Developing Psychological Profiles of Media Audiences
August 30, 2012
Reflections on a "Biometric" Summer
July 3, 2012
Neuromarketing Research: A Tale of Two Approaches
March 23, 2010
Delivering Effective and Ineffective Apologies
February 24, 2010
Super Bowl Ads 2010
November 13, 2009
Measuring the ROI of In-Market Advertising Campaigns
January 30, 2009
Relaxing the Death-Grip of Norms in Communications Research