Small steps. Big impact.

THE FITBIT BLOG

Fitbit Healthy Habits: Week 3

| Leave a comment

Creating a healthy lifestyle is always worth it, but never easy. It can be tempting to try and tackle every unhealthy thing in your life all at once. But studies have shown that our willpower is finite, so trying to change too many of your habits at once will at the very least make you exhausted, and at most cause you to burn out on staying healthy earlier. So we’re challenging each and every reader with bite-sized daily tips in the Fitbit #HealthyHabits challenge!

Fitbit Healthy Habits: Wek 3We’re already half-way through our Healthy Habits challenge! If you’re just starting now, you can go back to Week 1: Stress or Week 2: Nutrition, or feel free to start this week and return to previous weeks later. You can tackle this in whatever order, and as often as, you’d like!

read more

You talkin’ to me?

| 4 Comments

In our last blog post, we showed that you could read President Obama’s and Mr. Romney’s heart rate right off of their faces during their first presidential debate.  We got a lot of questions asking us to explain the technology behind it a little more, and we’re eager to show other ways it can be used.  It turns out that this works with all sorts of video, including our favorite Hollywood movies.  To demonstrate this technology to you a little further, today we’ll be looking at Robert De Niro during his seminal “You talkin’ to me?” scene in Taxi Driver (1976).

First, how does this work?  The basic idea is that the color of your face fluctuates slightly as blood perfusion under the skin changes from your heart’s pumping. If you measure the color change, you can measure heart rate. It sounds simple, but it’s not. One thing that makes this difficult is user motion: if the person being measured moves, it becomes difficult to disentangle cardiac-induced color changes from changes in lighting conditions on the face, rotations of the face, etc. But in some cases, we’re actually able to filter out some of this motion. Pretty cool, right?

Now let’s walk through the scene as a way to show how this technique works.  Fig. 1 (below) depicts the motion of Mr. De Niro during the performance. Big spikes indicate lots of motion and flat, low regions indicate periods of relative stillness. We see that the sequence where Mr. De Niro says,You make the move,” is the most still period. It’s about 15 seconds prior to “You talkin’ to me?”, but it’s a good starting point to try this.

Figure 1. Mr. De Niro undergoes significant motion in the “You talkin’ to me?” scene. The most still period is when he says “You make the move”, which precedes “You talkin’ to me?” by about 15 seconds

There’s still motion during this sequence, so to get rid of it, we’ll do a simple feature tracking on Mr. De Niro’s face, as shown in the movie below.

Now taking that patch of skin and analyzing its color changes, we can derive the cardiac signal for Mr. De Niro and get his heart rate.  Fig. 2 shows a nice sequence (below). In this chart, you can see the color intensity swing up and down in a visible pattern that corresponds to specific heart beats. The rest is just arithmetic to figure out the rate at which the heart is beating. Mr. De Niro’s heart rate for this scene: 95 bpm!  The average heart rate for adults falls more around 72 bpm.  Since Mr. De Niro was in his early 30’s at the time of the movie and seems to be in good shape, his heart rate was probably slightly elevated, which isn’t suprising since it was a pretty exciting moment in the movie where he pulls out a gun with a crazy mechanical contraption.  Or maybe Mr. De Niro did jumping jacks right before the scene?

Figure 2. Mr. De Niro's cardiac signal during the "You make the move" sequence. The overlaid hearts denote heart beats.

Now that we’ve cut our teeth on the easy part, let’s see if we can find Mr. De Niro’s heart rate during the epic, zeitgeist-defining moment of truth.  This movie shows the “You talkin’ to me?”  sequence with the feature tracker going to filter out motion.

Notice that there’s significant motion throughout, but there is one sweet spot when things are still.  Fig. 3 below shows the corresponding cardiac signal during that interval with beats overlaid.  The answer: 79 bpm!

Figure 3. Mr. De Niro's cardiac signal during the "You talkin' to me?" scene.

Figure 3. Mr. De Niro's cardiac signal during the "You talkin' to me?" scene. The overlaid hearts denote heart beats.

Did Robert De Niro know that he would change popular culture permanently with a single line?  …Ultimately, we don’t know.  What we do know is that his heart rate was a little elevated, so he was at least probably excited about his work.  Check out the scene in its entirety below.  Warning: there is some profanity in this clip!

There’s a lot that could be done with this technology in the long run, from tracking the heart rates of our nation’s leaders to tracking your friends’ or your own heart rate during a video chat.  It could be really interesting to see how heart rate changes when you’re interrupted by your boss or coworkers at the office.  Of course, some care still needs to be taken with video compression, optical and digital filtering, motion, and the like – all of which interferes with the cardiac signal. Nevertheless, it’s still a lot of fun to think about.

Finally, some caveats: this should not be taken as a rigorous scientific work, but more as a fun show-and-tell prototyping project.  There may be confounding factors in this analysis that have not been investigated in detail.  If you can think of them or have any additional thoughts or comments, we’d enjoy hearing them all.

Fitbit Asks: Who “Won” Last Night’s Debate?

| 8 Comments

After last night’s presidential debate, the nagging question seems to be “who won: President Obama or Governor Romney?”  It’s a subjective, squishy question that can only be made somewhat quantitative by polling a lot of people on how they felt about the debate.  But what if, instead, you could get some objective measurements on the candidates themselves?  What if you could get insight into their mental state while they debated?  That would be cool… and it’s the object of today’s Fitbit blog post.

You can tell a lot from a person’s face.  A person smiles, they’re happy (usually); a person frowns, they’re not.  That’s the easy stuff.  Believe it or not, you can go much deeper.  Did you know that you can actually measure a person’s heart rate from his or her face?  Really. And from someone’s heart rate, there are various inferences that we can make: such as if they’re stressed, or even lying.

Once we figured out that this was possible, our first thought was, “Wouldn’t it be cool if we could use this to measure the stress levels of our nation’s leaders? Say, for example, during the next presidential debate?” Last night, under the bright lights of the presidential debate stage, that’s exactly what Fitbit Labs did. Just by looking at their faces.

In the above images, a faint blue box shows the area we’re focusing on for our results. It’s pretty cool: using a regular video camera and some advanced computer algorithms, it’s possible to observe small color changes on someone’s skin as his or her heart pumps.  It’s very much like checking for someone’s pulse by touching his wrist, except in this case the “pulses” are color fluctuations in the face coming from blood volume changes under the skin. In order for this technique to work, the person shouldn’t move too much or the motion ends up interfering with the very subtle colors that you’re trying to measure.  And forget about fancy camera panning like you see used in movies for dramatic effect.

Of course, “normal” heart rates vary from person to person a lot as well. Luckily, in the US we actually publish our leaders’ physicals for the public to see, so we know what each candidate’s heart rate was starting off.

Table 1: Resting heart rate for Pres. Obama and former Gov. Romney from their most recent medical reports

Table 1: Resting heart rate for Pres. Obama and former Gov. Romney from their most recent medical reports

During Gov. Romney’s opening remarks, he stayed still for a few seconds and we saw that his heart rate was 73 bpm, or a 83% elevation above his resting heart rate of 40 bpm.  Not bad.  President Obama’s rebuttal a few minutes later came up at 96 bpm, or a 43% elevation above his resting heart rate of 67 bpm. Even though Romney’s heart rate was lower in the beginning, compared to his resting heart rate he was more worked up right from the start.

Now here’s a breakdown of the results across the debate:

  • Romney’s opening argument: 73 bpm (+83%)
  • Obama’s rebuttal: 96 bpm (+43%)
  • Obama’s closing speech: 59 bpm (-12%)
  • Romney’s closing speech: 75 bpm (+88%)
Or, maybe it’s easier to see as a timeline across the debate from opening to close:
Figure 2: Heart rate elevation for Pres. Obama and former Gov. Romney through the debate

Figure 2: Heart rate elevation for Pres. Obama and former Gov. Romney through the debate

What does this mean? Let’s walk through the debate.

There’s actually a number of different inferences you can make from the data we found. A person’s heart rate naturally accelerates during times of stress — such as a crucial debate being televised to millions of people that decide who will run the entire country. We could look at an elevated heart rate as a measure of nervousness. In that case, we would infer that President Obama was more nervous towards the beginning of the debate, but proved to be one cool cat as the pressure mounted towards the end. In this line of reasoning, Romney, was steadily nervous throughout – and to a higher degree than Obama was. But in this case, we’re making the assumption that the elevated heart rate is related to stress, when that’s not necessarily the case. What if, instead, Romney’s heart rate is a reflection of how amped up he is about his ideas? This could just mean that Romney was really “in the zone” for the debate, while Obama failed to bring a certain excitement for the task at hand. This was, in fact, a popular criticism of Obama’s performance.

So who DID win the debate? Well, if you want the candidate who kept his cool, it was Obama. If you want the one who was most excited, it was Romney. Maybe before the next debate Obama should drink some caffeine to get that energy up. It will be interesting to see how the next debate compares, now that the pressure is really on to be more energetic.

This also makes us here at Fitbit wonder if this sort of technology might have a place in debates in the future. Would a candidate ever agree to be hooked up to an EKG during the debate? Or at the very least, could such data be used to train for debates? We can’t wait to find out.

Disclaimers:

This is just our best guess at the candidates’ heart rates during the debate.  There are a lot of reasons to take these with a grain of salt, but we’d be thrilled to see if others could reproduce or refute our analysis.  Of the things to consider:
1. The heart rate measurements were taken during small clips when there’s no movement. No movement is necessary for accuracy, but ideally we’d have used longer clips (~15 seconds, at least).
2. There’s some natural variation in heart rates as well, especially when a person is calm, and “heart rate” is usually defined as the average heart rate over some time. Since we’re using short clips, we might not have seen enough heart beats to make an accurate measurement.
3. Lighting can change on stage and that would affect the results.
4. In all likelihood, both candidates are wearing makeup for the debate. Since we’re measuring changes in skin coloring, makeup can mask some of this.
5. The videos we used were compressed, and of lower quality. A higher definition video would likely be more accurate.
6. Heart rate alone is not the best measure of stress.  Measures of heart rate variability, skin conductance, etc. used together are much more reliable.