Tag: probability

Norway’s Death Reports in Frail Elderly; reading a medical news article

Posted on by Dr. Nykki

a negative BINAXNow Covid test

My brain is wired for panic

At one week out from my vaccine, I’ve had no new concerns. I have friends and family members reporting fever, fatigue, malaise, nausea, and vomiting – sometimes after the first dose – so I’m grateful to my immune system for all the favors it’s doing for me. Keep it up, immune system!

 

I’ve had a news article come up on several of my feeds lately, and so today’s diary is going to be about the Norway data and reading breaking medical news articles.

 

The article in question, depending on your news source of choice, begins with the headline “Norway Raises Concern Over Vaccine…” or “23 Die After Pfizer Vaccine…”; and that’s certainly a headline that will raise concern. I’m concerned! But we’ve been talking a lot about science lately, and the way that our brains tell us things that may or may not be strictly true, so I’m going to take you through a deep dive into the publicly available information and what it means from a scientific standpoint.

 

The article I’m using is from the British Medical Journal (a peer-reviewed publication with reasonable standing in the medical community, so a source I trust), but it reflects most of the same information you’ll find from any news source that is doing its research: Norwegian physicians had reported 23 deaths in patients who had recently received the Pfizer coronavirus vaccine. Those deaths were in care home patients (nursing home patients), many of whom were “very frail”. Some of those patients underwent autopsies and Norwegian health authorities felt that the side effects of the vaccine may have contributed to the deaths of the patients. 

 

I’m going to walk you through the way I responded to this information, and let’s see where it takes us, shall we? 

 

Brain (FULL PANIC MODE ON): That’s a lot of deaths! What’s going on?!! Why are all these people dying  in Norway?! I need to call my nursing homes and make sure nobody is dead!! What if I have been telling people bad information?! What if I’m a bad scientist and a worse doctor?! 

 

My brain is very excitable. It is also basically programmed for threat avoidance, because our brains are supposed to keep us alive, so it’s better in general if they’re a little paranoid (especially if you’re living in a world full of unpredictable things actively trying to kill you, as humans have done for much of our history). But that means that sometimes we have to take a moment, let the initial shock and awe wear off, and take another look at what we’re seeing. 

 

So let’s take a look, now that my brain has stopped running in panicked circles and screaming, shall we? 

 

Step 1: How reliable is the source of my information? 

 

I had the same general headline from multiple news sources, which is usually (but not always) a positive sign of reliability, and so I walked it back to a medical journal that I trust for confirmation. 

 

The data itself comes from Norway – which has a centralized government-funded health care system, in which enrollment is automatic. Citizens have the option to purchase private insurance, but only about 10% of the population has done so. The relationship between the Norwegian government and the citizens of Norway is generally good. This means that they have ready access to certain health data through national registries, and the information we are being given is likely to be reliable. 

 

Step 2: What are the actual facts being reported? 

 

BMJ reports upwards of 20,000 doses of Pfizer’s vaccine were administered in Norway in the last few weeks and 23 deaths were reported in frail elderly patients “shortly after” receiving the vaccine. 13 of those deaths have been investigated — and it was felt in those cases that vaccination responses may have contributed to the deaths. The article also notes that around 400 deaths a week occur in care homes in Norway on a normal basis.

 

We are not given a clear definition of “frail” or “elderly”, although both of those terms are kind of a “we know it when we see it” thing for doctors. We also are not given any information about whether the reported deaths are more than the usual number expected. Remember — Norway is monitoring its citizens’ health just as closely as the US is (maybe more closely) and so it’s expected that medical personnel should report EVERYTHING so that the data scientists can sort it out. 

 

Step 3: What is the relationship between the facts? 

 

There is a potential correlation between administration of the vaccine and death in frail elderly patients. That means that it’s possible that these two numbers are going to move together:  if more frail elderly patients are vaccinated, then more of them will die shortly after the vaccination at a rate of about 10 deaths per 10,000 doses administered. It’s also possible that these two numbers have nothing in common, and as more frail elderly patients are vaccinated, more of them do very well. 

 

There is also concern for causation: in the patients whose deaths have been investigated thus far, it’s possible that vaccine responses may have been a factor in their deaths. That’s certainly a point of concern, and one that I’m going to be following for more information on.. 

 

Step 4: What other information do I already have that is important to consider? 

 

Pfizer did present data on safety in patients who were age 80 and up — but as widespread vaccination programs take place, we have rapidly exceeded the number of people enrolled in clinical trials. That means that instead of 73,000 data points (Pfizer enrolled 43,000 people and Moderna enrolled 30,000 people in Phase 3) we have over 12 million in the US alone. 

 

The more data we have, the smaller our p value gets, and that means that we can be more confident when we say that a particular reaction is (or is not) likely to be related to vaccination.

 

I take care of frail elderly patients in care homes on a regular basis. I worry about things like fevers, not eating well, vomiting, and diarrhea in those patients. They don’t have a lot of extra reserve — so when they get dehydrated, low on electrolytes, or hypoglycemic (low on blood sugar), that can be a really serious event that they may not be able to recover from. So I can certainly understand from a medical standpoint where enthusiastic vaccine responses (the kind that sent my otherwise-healthy colleagues home from work for a day) might have a devastating effect on someone without that reserve. 

 

Step 5: What am I going to do with all this information?

 

I’m not going to let my brain tell me to panic. Panicking does not help me be a better doctor, and it does not help me provide better information to my patients. 

 

I AM going to flag this topic in my mind as something that I want to continue to learn more about — I expect more data to come out as scientists and doctors across the world continue to observe and report what they are seeing, and I want to know what the data says. Because this is important. 

 

I AM going to use the information I have to inform my discussions with frail elderly patients and their families. It’s going to be important to talk about the possibility of an enthusiastic vaccine response and what that means, as well as to talk about what their risk of catching COVID-19 is and what that means. I can’t make a decision about vaccination for my patients and their families; my job is to be the best resource that I can be for them, and to help them make the best decision they can for themselves — and that decision may be different from one patient to another. 

 

Step 6: Keep learning. 

 

Never stop learning. Never stop asking questions. 

And don’t be afraid to say “I don’t know, but I’ll find out.”

 

 

 

 

Vaccination and asymptomatic infection spreaders

Posted on by Dr. Nykki

 

I was vaccine hesitant, too

Today’s update: If I push on the spot where I got my shot it’s a little sore. Otherwise, I’ve been in excellent health.  I continue to hear reports of low-grade fevers, fatigue, and headaches from others, so do be aware your experience may not be quite as uneventful as mine. It all depends on how enthusiastically your immune system gets involved, and how sensitive you are to that activation.

 

I have a confession to make: As delighted as I am to have received my shots (and my delight and enthusiasm are not feigned), I spent the greater part of 2020 skeptical about the creation of, development timeframe, and efficacy of any theoretical coronavirus vaccine. I spent several months in the late summer and early autumn anxiously fretting about political pressure and the timeline of a vaccine release. I made up my mind that I was going to wait – 3 to 6 months, approximately – after the large-scale release of any purported vaccine before I got my shot. 

 

I worried about timing. I worried about who I could trust. I worried about what would happen if this shot didn’t actually work well enough to stop the pandemic (at one point, we were hoping that the vaccine would be 50% effective). 

 

So what changed? 

 

In September, a number of major vaccine manufacturers released their trial protocols. Part of being a responsible scientist is designing your experiments in advance, and releasing the trial protocols meant basically that the manufacturers told us what they were looking for, how they were measuring it, and what the criteria were for announcing success before the formal end of the trial. 

 

Understanding what we were testing gave me more confidence than any ongoing news scrutiny — and knowing the numbers that would indicate a tentative success (something that many trials, not just these, have baked into their protocols) – publicly – in advance – restored additional confidence. 

 

And so when the data was released, and the early results were tallied, my questions were no longer about fundamental issues of trust (although one should always continue to ask questions). They were about points of science – in a field where my training gave me enough knowledge to know what to ask. And that’s how I found myself getting my shot in December – and excited to do so – but still wearing my mask, even now. 

 

One question tonight.

It was going to be two – but I write really long posts.  So tomorrow: mRNA and vaccination while pregnant or breastfeeding.

Why are we talking about vaccinated people spreading coronavirus? Wasn’t that the point?

 

Short answer: About one in 20 people who are vaccinated are still going to get symptomatic COVID-19 (although it’s likely to be milder), and we don’t know how well the vaccine protects against infection that doesn’t cause symptoms. 

 

Long answer: Ready for some math? 

 

Remember when I said that the vaccine manufacturers told us what they were looking for? That’s a really important point in understanding what’s going on here. You see, when the vaccine trials were designed, they were designed to test for efficacy against symptomatic infection. Pfizer and Moderna both tested any trial participants who showed symptoms as part of their protocols, but they did not perform routine surveillance testing (testing everyone in the study, symptoms or not, at regular intervals) as part of their protocols. 

 

There’s a lot of math that happens when you design an experiment. Most of it revolves around eliminating the possibility that any results you have are actually due to random chance. I’m not going to delve deeply into statistics here, because honestly very few people find it super interesting (I do!) — but here’s an example of why it’s important to have a statistician involved when you plan your experiment. 

 

Let’s say I’m going to do a classic coin flip experiment. In order to decide whether the coin I have in my pocket is actually fair, I will flip it five times. My mind tells me that if I flip this coin and get five heads in a row, then the coin is obviously not fair, right? Probability tells me that about 3% of the time I’m going to get 5 heads in a row on 5 flips with a perfectly fair coin. This number – the probability that random chance has accounted for my results – is known as p.

 

And maybe for a coin flip, 3% (that’s a p of 0.03) is an acceptable chance to take. When people’s lives are at stake, researchers like their numbers to be a little more definite, so they involve a statistician to help design the test. In the coin flip above, if I expand my number of flips to 10, there is a 0.09% chance (for a p of 0.0009) that all 10 will be heads by random chance. That’s a much smaller number – and more on the scale of what we’re looking for. 

 

Pfizer and Moderna both released their interim trial data for symptomatic coronavirus infections (the thing the trial was designed to test) – and that data showed 94-95% effectiveness with a p <0.0001. That means that with regard to symptomatic infections, the numbers we are seeing are less than 0.01% likely to be due to random chance.

 

However.

Neither trial was designed (built specifically to eliminate outside factors) or powered (with the statistician involved up front) to look for asymptomatic infections. That means that any information coming out of those trials (and there is some) about asymptomatic infections is being done as a retrospective or “look back” analysis. There may be enough data to give us an answer (Moderna has submitted some numbers coming from the tests they administered at the time of second vaccination) but it’s never going to be quite as solid as the conclusions that the trial was actually designed to draw. 

 

That means – ultimately – we don’t know how well this vaccine protects against asymptomatic infections. And because up to 40% of infections in unvaccinated people are asymptomatic, that’s a pretty large “don’t know”. If you add that to the 5% of people (1 in 20) who are vaccinated who may still get infected, that’s enough wiggle room for a lot of viruses to slip through. 

 

I’ve been playing role-playing games since high school (if you hadn’t figured out by now what a huge nerd I am). 1 in 20 happens a lot more often than your brain tells you it should, even with the very best dice.

 

We still have a lot to learn. In order to save as many lives as possible while scientists take the time they need to learn it, I’m going to keep wearing my mask and keeping my distance. 

 

Just in case. 

 

Because every single one of you is that important to me.