Belle Beth Cooper is the co-founder of Hello Code. This post originally appeared on the Exist blog and was republished with permission.
Sleep tracking is a big part of what we do at Exist. We currently have Jawbone UP and Fitbit integration, so we can import sleep data from trackers in both of these lines.
We’re also planning to let users choose to optimise either sleep or activity using Exist, so we want to know as much as we can about how these devices track sleep and how accurate they are.
I dived into some research to work out what’s going on when you set your Jawbone UP or Fitbit to sleep mode. Both devices use accelerometers to track your movements, including the speed and direction of your motion.
This is how they track your activity during the day, and how they tell when you’re asleep.
When you set your Fitbit or Jawbone UP device to “sleep mode,” it monitors your movements. When you sync your device the next morning, the software translates those movements into sleep data.
If you’ve ever edited your sleep times the next morning, you’ll have seen a glimpse into how this works—data about your movement during “awake” time gets re-analysed when you tell the software you were actually asleep then, and you then see it charted as sleep data.
This method of using a device to track movements in order to measure sleep is called actigraphy.
What is actigraphy?
Actigraphy is often done in sleep studies using an “actigraph” device—similar to a Fitbit or Jawbone UP, it’s usually a device worn on the wrist that tracks movement while you’re sleeping. Software then translates those movements into periods of sleep and wake.
For people with sleep disorders or general sleep disruptions, actigraphy is a convenient way to have their sleep patterns studied by a clinic without having to sleep in the lab. An actigraph device can be worn at home, and isn’t usually too uncomfortable—most of them look like a watch:
An actigraph can also be worn 24/7 and track sleepiness during the day, based on movement. Thus, actigraphy might be used for sleep studies where convenience is important.
Generally, sleep researchers use polysomnography, or PSG, to study sleep in a lab. This is known as the “gold standard” for measuring sleep.
This means that sleep researchers have agreed that PSG is the definitive way to measure sleep, and most studies on the accuracy of actigraphy compare it to PSG data.
A PSG test involves sleeping in a lab where your brain waves can be monitored by an EEG test, where electrodes on your scalp measure your brain waves. This can be inconvenient, but is more accurate.
Cool fact: PSG tracking is really inconvenient in space, so actigraphy is sometimes used to easily track the sleep of astronauts! Here’s a picture of an actigraph device used in a NASA-funded, Harvard University study of scientists living on Martian time, with a fake watch face painted on it:
Research into actigraphy
Most research into actigraphy has used a purpose-built actigraph device, rather than a Fitbit or Jawbone UP (although a Fitbit was included in this study) and has been focused on measuring the accuracy against a PSG test.
Study results have varied, but for the most part the consensus is that actigraphy is generally accurate enough to track sleep in healthy adults with “normal” sleep patterns.
Problems arise when your sleep is disrupted—for instance, if you have a sleep disorder or you just have disrupted sleep on a regular basis. The more disrupted your sleep, the less accurate actigraphy will be at tracking it.
Mostly this is because actigraphy just tracks one thing: movement. Sleep researchers know that there’s much more to accurate sleep data—most importantly, brain waves and eye movements are required to assess sleep phases.
In an article on sleep tracking accuracy, Michael Scullin, a post-doctoral fellow at Emory University School of Medicine’s Department of Neurology was quoted as saying this:
Consumers should not expect that these devices will be able to distinguish between sleep stages because these devices rely on movements, whereas sleep stages are defined primarily by brain activity.
Which means if you’re using a fitness tracker that claims to tell you how long you spend in each sleep stage, it’s probably not very accurate after all.
As far as time asleep and sleep quality, actigraphy devices generally have a high margin of error. They either under- or overestimate time asleep.
If you’ve tried one of these, you’ve probably noticed that either it thinks you’re asleep when you lie still in bed, or it fails to realise you’re awake when you start moving around.
My co-founder Josh has found this with his Fitbit One, which has both a “regular” sleep mode and a “sensitive” sleep mode. In regular mode, his sleep time is overestimated, while sensitive mode underestimates how much time he’s asleep for.
Wake time has been found to be the most tricky for actigraphs to get right, since periods of no movements will almost always register as sleep. So if you don’t move “enough” while you’re awake, you might have more sleep time recorded than is actually accurate.
In Josh’s case, when he lies still, trying to fall asleep, that time is often counted as if he is already sleeping.
Another issue with using actigraphy is that it’s open to user error. For instance:
- not setting (or turning off, if you use a Fitbit) sleep mode at the right time
- forgetting to wear your tracker at all
- sleeping in a position that prevents movement of the arm your tracker is on
- or even breathing movements that register with your tracker and count as time you were awake
Usually, wearing a tracker on your trunk (clipped to your waistband) or your ankle will pick up fewer movements. If you’re finding that your tracker overestimates your time asleep, you could try switching the position, especially if you have a clip-on style tracker like Josh’s Fitbit One.
Lastly, it’s important to know that the software used to analyse your movement data makes a big difference to how accurate your sleep stats are. There are a few different ways of analysing movement data, and some devices employ multiple methods for a more accurate reading.
In addition, different methods of analysis might be more accurate. For instance, having a lower threshold for movements (i.e. fewer movements required to count as awake time) is a more accurate way of predicting time asleep.
This could be how Fitbit’s sensitive sleep mode works—just a small movement counts as time awake, so the “time asleep” stats are usually spot on, but restless time asleep is often counted as “awake time.”
Having a higher threshold (i.e. you have to move a lot to register time awake) is better for predicting awake time, but leads to lower accuracy in tracking of sleep time. Fitbit’s regular sleep mode counts a lot more of Josh’s time in bed as him being asleep, since he has to move a lot to tell it he’s awake. This means the “awake time” data is right more often, but his sleep time is overestimated.
Should you trust it?
So what does all this mean? Should you trust your fitness tracker to know when you’re asleep or not?
I’m no sleep expert but from what sleep researchers are saying and the results of studies into actigraphy, it seems like it’s okay to trust it as a general guide of your sleep time and efficiency, so long as you generally sleep well.
If you have (or think you have) a sleep disorder, it’s not a good idea to trust actigraphy. In that case you really want to visit a sleep clinic and get a PSG test done.
In terms of different sleep phases like deep sleep and REM sleep, I haven’t come across a sleep researcher yet who is willing to vouch for accuracy in sleep phase prediction based on movement alone. So it seems like we should be using the overall sleep time as a guide and not giving much credence to supposed sleep phases our trackers are measuring.
