In our last blog post, we showed that you could read President Obama’s and Mr. Romney’s heart rate right off of their faces during their first presidential debate. We got a lot of questions asking us to explain the technology behind it a little more, and we’re eager to show other ways it can be used. It turns out that this works with all sorts of video, including our favorite Hollywood movies. To demonstrate this technology to you a little further, today we’ll be looking at Robert De Niro during his seminal “You talkin’ to me?” scene in Taxi Driver (1976).
First, how does this work? The basic idea is that the color of your face fluctuates slightly as blood perfusion under the skin changes from your heart’s pumping. If you measure the color change, you can measure heart rate. It sounds simple, but it’s not. One thing that makes this difficult is user motion: if the person being measured moves, it becomes difficult to disentangle cardiac-induced color changes from changes in lighting conditions on the face, rotations of the face, etc. But in some cases, we’re actually able to filter out some of this motion. Pretty cool, right?
Now let’s walk through the scene as a way to show how this technique works. Fig. 1 (below) depicts the motion of Mr. De Niro during the performance. Big spikes indicate lots of motion and flat, low regions indicate periods of relative stillness. We see that the sequence where Mr. De Niro says,“You make the move,” is the most still period. It’s about 15 seconds prior to “You talkin’ to me?”, but it’s a good starting point to try this.
There’s still motion during this sequence, so to get rid of it, we’ll do a simple feature tracking on Mr. De Niro’s face, as shown in the movie below.
Now taking that patch of skin and analyzing its color changes, we can derive the cardiac signal for Mr. De Niro and get his heart rate. Fig. 2 shows a nice sequence (below). In this chart, you can see the color intensity swing up and down in a visible pattern that corresponds to specific heart beats. The rest is just arithmetic to figure out the rate at which the heart is beating. Mr. De Niro’s heart rate for this scene: 95 bpm! The average heart rate for adults falls more around 72 bpm. Since Mr. De Niro was in his early 30′s at the time of the movie and seems to be in good shape, his heart rate was probably slightly elevated, which isn’t suprising since it was a pretty exciting moment in the movie where he pulls out a gun with a crazy mechanical contraption. Or maybe Mr. De Niro did jumping jacks right before the scene?
Now that we’ve cut our teeth on the easy part, let’s see if we can find Mr. De Niro’s heart rate during the epic, zeitgeist-defining moment of truth. This movie shows the “You talkin’ to me?” sequence with the feature tracker going to filter out motion.
Notice that there’s significant motion throughout, but there is one sweet spot when things are still. Fig. 3 below shows the corresponding cardiac signal during that interval with beats overlaid. The answer: 79 bpm!
Did Robert De Niro know that he would change popular culture permanently with a single line? …Ultimately, we don’t know. What we do know is that his heart rate was a little elevated, so he was at least probably excited about his work. Check out the scene in its entirety below. Warning: there is some profanity in this clip!
There’s a lot that could be done with this technology in the long run, from tracking the heart rates of our nation’s leaders to tracking your friends’ or your own heart rate during a video chat. It could be really interesting to see how heart rate changes when you’re interrupted by your boss or coworkers at the office. Of course, some care still needs to be taken with video compression, optical and digital filtering, motion, and the like – all of which interferes with the cardiac signal. Nevertheless, it’s still a lot of fun to think about.
Finally, some caveats: this should not be taken as a rigorous scientific work, but more as a fun show-and-tell prototyping project. There may be confounding factors in this analysis that have not been investigated in detail. If you can think of them or have any additional thoughts or comments, we’d enjoy hearing them all.
After last night’s presidential debate, the nagging question seems to be “who won: President Obama or Governor Romney?” It’s a subjective, squishy question that can only be made somewhat quantitative by polling a lot of people on how they felt about the debate. But what if, instead, you could get some objective measurements on the candidates themselves? What if you could get insight into their mental state while they debated? That would be cool… and it’s the object of today’s Fitbit blog post.
You can tell a lot from a person’s face. A person smiles, they’re happy (usually); a person frowns, they’re not. That’s the easy stuff. Believe it or not, you can go much deeper. Did you know that you can actually measure a person’s heart rate from his or her face? Really. And from someone’s heart rate, there are various inferences that we can make: such as if they’re stressed, or even lying.
Once we figured out that this was possible, our first thought was, “Wouldn’t it be cool if we could use this to measure the stress levels of our nation’s leaders? Say, for example, during the next presidential debate?” Last night, under the bright lights of the presidential debate stage, that’s exactly what Fitbit Labs did. Just by looking at their faces.
In the above images, a faint blue box shows the area we’re focusing on for our results. It’s pretty cool: using a regular video camera and some advanced computer algorithms, it’s possible to observe small color changes on someone’s skin as his or her heart pumps. It’s very much like checking for someone’s pulse by touching his wrist, except in this case the “pulses” are color fluctuations in the face coming from blood volume changes under the skin. In order for this technique to work, the person shouldn’t move too much or the motion ends up interfering with the very subtle colors that you’re trying to measure. And forget about fancy camera panning like you see used in movies for dramatic effect.
Of course, “normal” heart rates vary from person to person a lot as well. Luckily, in the US we actually publish our leaders’ physicals for the public to see, so we know what each candidate’s heart rate was starting off.
During Gov. Romney’s opening remarks, he stayed still for a few seconds and we saw that his heart rate was 73 bpm, or a 83% elevation above his resting heart rate of 40 bpm. Not bad. President Obama’s rebuttal a few minutes later came up at 96 bpm, or a 43% elevation above his resting heart rate of 67 bpm. Even though Romney’s heart rate was lower in the beginning, compared to his resting heart rate he was more worked up right from the start.
Now here’s a breakdown of the results across the debate:
- Romney’s opening argument: 73 bpm (+83%)
- Obama’s rebuttal: 96 bpm (+43%)
- Obama’s closing speech: 59 bpm (-12%)
- Romney’s closing speech: 75 bpm (+88%)
What does this mean? Let’s walk through the debate.
There’s actually a number of different inferences you can make from the data we found. A person’s heart rate naturally accelerates during times of stress — such as a crucial debate being televised to millions of people that decide who will run the entire country. We could look at an elevated heart rate as a measure of nervousness. In that case, we would infer that President Obama was more nervous towards the beginning of the debate, but proved to be one cool cat as the pressure mounted towards the end. In this line of reasoning, Romney, was steadily nervous throughout – and to a higher degree than Obama was. But in this case, we’re making the assumption that the elevated heart rate is related to stress, when that’s not necessarily the case. What if, instead, Romney’s heart rate is a reflection of how amped up he is about his ideas? This could just mean that Romney was really “in the zone” for the debate, while Obama failed to bring a certain excitement for the task at hand. This was, in fact, a popular criticism of Obama’s performance.
So who DID win the debate? Well, if you want the candidate who kept his cool, it was Obama. If you want the one who was most excited, it was Romney. Maybe before the next debate Obama should drink some caffeine to get that energy up. It will be interesting to see how the next debate compares, now that the pressure is really on to be more energetic.
This also makes us here at Fitbit wonder if this sort of technology might have a place in debates in the future. Would a candidate ever agree to be hooked up to an EKG during the debate? Or at the very least, could such data be used to train for debates? We can’t wait to find out.
This is just our best guess at the candidates’ heart rates during the debate. There are a lot of reasons to take these with a grain of salt, but we’d be thrilled to see if others could reproduce or refute our analysis. Of the things to consider:
1. The heart rate measurements were taken during small clips when there’s no movement. No movement is necessary for accuracy, but ideally we’d have used longer clips (~15 seconds, at least).
2. There’s some natural variation in heart rates as well, especially when a person is calm, and “heart rate” is usually defined as the average heart rate over some time. Since we’re using short clips, we might not have seen enough heart beats to make an accurate measurement.
3. Lighting can change on stage and that would affect the results.
4. In all likelihood, both candidates are wearing makeup for the debate. Since we’re measuring changes in skin coloring, makeup can mask some of this.
5. The videos we used were compressed, and of lower quality. A higher definition video would likely be more accurate.
6. Heart rate alone is not the best measure of stress. Measures of heart rate variability, skin conductance, etc. used together are much more reliable.
Meet Zip: the latest addition to the Fitbit Tracker line! In developing the Zip, we reexamined every aspect of what makes our trackers great - from how the clip would work, to the shape of the device — but in the whole design process, the one feature that we kept coming back to was: how do we make a tracker that doesn’t have a button so that it’s more water resistant, but still has a display that you can interact with and is fun to use?
This inspired several lines of investigation on motion-based device interaction. We tried different gestures, from flicks of the wrist to turning the device upside down. “It was interesting to see that men and women naturally prefer different motion interactions,” observed Dr. Shelten Yuen, our Chief Scientist. “Guys flick one way – towards their bodies – but women flick in the opposite way. And most of the gestures were either too hard to learn or not very fun to do. The one universal, easy, and enjoyable motion was the tap,” a simple motion that most of us are already used to from using smartphones or tablets.
Making it even trickier, though, we wanted to leverage the motion sensors inside the tracker to detect taps but NOT add any false steps or false calories at the same time (because they use the same motion sensors!). We did a lot of research on how people tap the Zip: how hard, how soft, where, tapping while walking, tapping while running, tapping standing still… everyone at Fitbit could be found testing out the tap feature at one time or another. We even went to the more extreme motions of dropping the Zip, or throwing it against a table. This allowed us to build some very sophisticated pattern recognition algorithms to accurately detect taps and steps without fear of them being mistaken one for the other.
It took some work, but it was worth it so that you can enjoy your Zip, and have fun with the new tap interface while also resting well in the knowledge that your Zip is sweat, rain, and water-resistant. You can learn more, or buy a Zip online now. Let us know what you think by heading to our Facebook page!
I had never consistently used a scale before getting my Fitbit Aria Wi-Fi Smart Scale. I’ve always been one who measures my weight based on how well my pants fit. Although, I have occasionally told myself that my pants just shrunk. So I was surprised, after using my Aria Scale for a few weeks, how much I love this scale — and I think I like it so much especially because I’m not a weight watcher.
With other scales, I never really knew what my weight number meant. I could never remember what my weight was last week, yet alone the month before. And more importantly, I never knew what was driving my weight. And if I saw any weight decrease (or increase), was it from fat? With Aria, I can see everything. It’s easy, and dare I say it… it’s even fun.
Maybe calling it fun is going a little too far, but I can’t deny that I get a little excited about checking out my weight and % body fat charts online, or on the Fitbit mobile apps. And since I log food and use a Fitbit Tracker, I can see how my active (or inactive) day and my food consumption affects my weight. We all understand the basics of how food consumption and lack of activity really packs on the weight. With Aria, you can actually see the effect of food and activity on your weight, through the data it tracks.
So I wasn’t much of a scale person, but being able to view my health as simple, measurable data is priceless. Let us know what you think about the Aria Scale by going on Facebook’s Fitbit Fan page or by emailing us at email@example.com. Or, you can learn more about the Aria Scale by going to http://www.fitbit.com/product/aria.
We at Fitbit love when we find unique, thoughtful blog posts about our products. The below is one such blog about the power of Fitbit badges, originally posted on Louis Gray’s personal blog. Louis Gray is a Google employee, technology blogger, and dad. We came across his blog by accident, but loved it so much that we wanted to share it with everyone. You can read more of Louis Gray’s blog at louisgray.com.
I am not going on a diet – and I have no interest in going to the gym, even if Google makes it incredibly easy to eat healthy on campus, and gym membership is free with equipment abundant. It’s just not me. But despite this clear disinterest in my making any kind of physical life change, I have been wearing a Fitbit the last week, obsessively counting my steps, climbing the stairs and tracking how many miles I make on foot. I’ve even been wearing the lightweight tracker at night to see how long and how well I sleep – working to optimize that as well.
So why would I resort to such silliness? It’s the stinkin’ badges – helped along by casual competition with friends, and now, despite my best attempts to not make any actual alterations to how I behave, I am sure I am doing things that are actually better for me, in the same way that Foursquare recommendations have pushed me to new venues and trying new things, based on badges and recommendations from friends.
Our free Fitbit app for Android has arrived! Of all the digital fitness device companies out there, we’re happy to announce that we’re among the first to offer both an iPhone and Android app. You can download the Fitbit app for Android for free from Google Play.
When we released our iPhone app last October and saw the warm reception it received, we started working on our Android app. We made a point to design our app specifically for Android phones, rather than just try to make the existing app work on Android. We hope you’ll be happy with the results.
With this app, you can log your food, activities, weight and water while on the go, as long as you have a wireless connection. Everything syncs to (and from) your Fitbit.com account, and you can view up to two weeks of data for easy access when you’re away from your computer.
The updated Food Plan is also on the Android app, so that you have all the information you may need to make smart decisions about your calorie intake throughout the day; to learn more, check our earlier blog entry. For our iPhone users, the updated Food Plan will be available with our next iPhone app update at the end of the month. We’ll be adding new features and updates to both apps, so keep your feedback coming and make sure you keep your Fitbit app up to date.
And a special thank you to all our Android users for your patience! The app is only available in the US, Canada, and the UK right now, but will be expanded to further countries as we expand our direct international sales. Happy tracking and logging — wherever you go!
After our release of Food Goal, we got a lot of great feedback and suggestions from our Fitbit fans about it. We’re excited to announce our answer to those suggestions with a large update to Food Goal. In fact, we’ve changed so much that we completely renamed it to Food Plan.
What’s changed? Quite a lot, actually, though the Food Plan has kept the same general look and feel as it had before. Everything that was in Food Goal is still available, like the gauge to show whether or not you’re in the “calorie zone”. But now, we’re providing a lot more information right at your fingertips, and the Food Plan has become even better at helping you manage your calorie intake each day.
Smarter calorie estimates: Your Food Plan sets each day based on your historic activity levels. It will calculate how many calories you burn each day on average, and base your suggested calorie intake on this. As you wear and sync your Tracker throughout the day, your Food Plan adjusts dynamically based on whether you’re exercising more or less than that average, so that your goal remains as accurate as possible. And as your activity trends change over time, your Food Plan’s starting estimate will adjust to your new activity levels as well.
Calories in vs calories out: Additionally, many of you wanted a way to judge how well you were doing at any one point of the day. You can now view your total calories burned compared to your total ca lories eaten, and see whether or not you are currently within your plan’s deficit goal zone. So if you’ve just logged your lunch and wonder how you’re doing for the day so far, you can see how your total calories burned so far for the day measure up against the total calories you’ve eaten. There’s no more guessing if you’re behind or on track so far for the day. With this meter, it’s always clear where you stand.
Tips and Tricks: To get the most out of your Food Plan, make sure to log your food throughout the day. You can do this on the go through your smartphone on our apps, or by visiting our mobile website.
Overall, we’ve tried to make changes that provide you with the information you need to make smart, healthy decisions and be better prepared to achieve your goals. We hope you enjoy the Food Plan, and welcome any feedback you might have. “Like” us on Facebook to join the conversation there.
You can find all of the great apps that work with Fitbit in one easy location at our App Gallery! There’s currently eight different apps or websites we integrate with that span a variety of functions, from organizing your health information to finding even more motivation and community for your fitness goals. And they’re all completely free!
Ever since we first launched Fitbit there’s been a lot of interest in app developers wanting to create more uses for the Fitbit data, and we’ve been excited to see what they’re coming up with. We’ve also heard all of your feedback about wanting more sites to port information into Fitbit, or vice versa.
With the App Gallery, you can take your Fitbit data with you to sites and apps we’re partnering with. If you’ve been logging food on LoseIt or MyFitnessPal, you can now sync that information to your Fitbit.com account – and vice versa! Join a group challenge at Endomondo for some extra motivation, or track your heart rate with Digifit.
The App Gallery is something we’ll be continuously adding to. Any developer can make a Fitbit app with our API, so keep an eye out for many more apps to come!
We’re excited to let you be among the first to learn about our newest product — the Fitbit Aria Wi-Fi Smart Scale. Aria will accurately track your weight, % body fat, and BMI over time and sync your weight data to your Fitbit.com account through your wireless network. Online charts and graphs will track your weight and body composition over time to help you understand exactly what’s happening to cause any bumps in the road.
The Aria was inspired by you, our customers. Many of you asked for an easier way to track weight. We did just that and more. Because the Aria syncs through your Wi-Fi, your data is automatically sync’d (no more numbers to input). You can invite up to 7 other household members to create their own accounts with the scale, but only you will be able to see your measurements, even with those in your household. And the scale is smart enough to recognize who you are just by stepping on it, so no need to worry about your information ending up in the wrong place.
Because the Aria is a Fitbit device, it’s integrated with all our online tools to create a complete weight management system. Use Fitbit’s Food Plan and set your calorie targets; by logging food, you can plan and manage your calorie consumption throughout the day. You can earn new badges as you work towards your weight goals, or share your information with friends and family to find the support you need.
The last holiday cookie has been munched. The feasting is almost done. Now that the holiday season is coming to an end (and 2012 is around the corner), everyone seems to be making fitness or getting into shape a New Year’s Resolution… mostly because we‘ve been binging over the holidays and need to get rid of those extra pounds.
Here at Fitbit we want you to put fitness on your Resolution list, but for a very different reason: because it leads to amazing health benefits — like mitigating diabetes and Alzheimers… and even living longer!! Just dedicating a small amount of time each day to light exercise can garner all these amazing health benefits. And it’s fun too.
In a very entertaining and motivating video, Dr. Mike Evans walks through the scientific research that shows how simple exercise — like walking just 30 minutes a day (every day) — can be the single most important thing you can do for your health. Check it out. It will get you to put moving on your resolution list, and keep it too!
It seems simple — 30 minutes a day — or for Fitbitters, we talk about hitting 10,000 steps a day. If fitness is on your 2012 New Year’s Resolution, let us know what you are aiming to do each and every day. We are aiming to do 10,000 steps 100% of the days in January. And we’ll report back our status at the end of January. Let us know at firstname.lastname@example.org if you want to join our resolution or “Like” us on Facebook to tell us there.