Let's dig a little deeper into that Apple Watch heart study

At the recent American College of Cardiology conference in New Orleans, the results of an eight-month study into Apple’s heart rate technology were published. The study, sponsored by Apple and carried out by Stanford University, analyzed 419,000 Apple Watch wearers to determine whether Apple's heart rate technology could accurately detect atrial fibrillation.

Although Apple made a big splash with its wearable last year by rolling an electrocardiogram into the newest Series 4 model, the latest Apple Watch wasn’t actually used in the study, which began in November 2017. Participants (including myself) were monitored using the Apple Watch Series 1, 2 and 3, none of which have an EKG, but for the study used optical heart rate technology to detect irregularities which can hint at AFib.

Explained: Why AFib is the next big thing for Apple and Fitbit

The Apple Heart Study app, which participants had to request to be a part of, intermittently checked the wearer for an irregular pulse using the Watch’s optical sensor. If it did detect one, the wearer got a notification telling them to schedule an appointment with a specially assigned doctor through the app. They would also be sent ECG patches to wear for a week, so that researchers could see if the Watch was correct in its predictions.

The findings presented some nice digestible headlines: 0.5% of participants received warnings of irregular rhythms; 84% of those who did were found to be accurately in AFib episodes at the time of the detection; and one third of participants who got an irregular pulse notification and went on to use the ECG patch did actually have AFib.

That third one seems quite low, but according to the researchers this is not unusual as AFib is an intermittent condition, so the symptoms come and go.

So on the face of it, it sounds like the Apple Watch is pretty good at picking up heart problems, right? Well, not necessarily. In fact, the numbers might be promising in some respects, but there were many limits on this study that mean it might not be all that helpful.

Control and observation

One problem, which some cardiologists that I’ve spoken to have with this study, is that it was observational rather than a randomised control trial i.e. researchers were looking at a very specific group of people (those who own an Apple Watch) and therefore can’t draw conclusions beyond that.

You have no way of knowing how many had AFib that the Watch did not detect

So you can't really say much about people who don't own an Apple Watch. Consider how the wealth discrepancy could skew the data; only people who can afford an Apple Watch can participate in the study.

The other red flag is that the study didn’t go on to monitor people who did not get an AFib notification. Because of that, it’s not possible to know the probability of not having AFib if the Watch says you don’t have it i.e. we don’t know how many false negatives there truly were.

“The idea of screening for obscure medical conditions using the watch is crazy, and will inevitably cause more harm than good,” Larry Husten, owner of CardioBrief and veteran cardiology journalist, tells me over the phone. “You have no way of knowing that, of those 400,000 people in the study, how many had AFib that the Watch did not detect. False negatives are potentially a big issue in this population. People might feel symptoms, not get an alert, and decide to not see a doctor.”

There’s a good tweet thread from cardiologist Venk Murthy on this here which breaks down why the number of false negatives (i.e. the number of people who didn’t get the notification but did have AFib) were so high.

However, not all of the people who got a notification followed up – just 57% actually sought medical attention, according to the results.

And then there's a third issue that's been raised: the treatments. It’s possible that a lot of people who do exhibit signs of AFib – particularly those who are younger – are better monitoring it than using treatments, which include blood-thinning drugs (known as anticoagulants) that come with their own health risks. There’s a cost-benefit analysis when it comes to deciding whether AFib patients should take these drugs, but as more people discover they have signs of AFib, there's the possibility that more will take measures that could cause more harm than good.

Apple itself acknowledges that much more work needs to be done here, as does Stanford. Yes, the Watch shows some promising signs as a screening device, but as this technology gets into more hands, the research must be robust enough to ensure it doesn’t open a can of worms for medical professionals who are going to be faced with patients who coming into their office and pointing at notifications on their Apple Watches.

It's also worth pointing out that only a tiny fraction of the population even own an Apple Watch. Brandon Ballinger, co-founder of Cardiogram, an app that uses heart rate data from wearables to predict heart rate diseases, says Apple’s health research announcements have generally had a positive knock-on effect for Cardiogram, but adds: “From a business perspective, each health insurer or employer who signs a contract with Apple faces an immediate question: what about the 80+% of their members who own an Android phone or another wearable?”