How Testing Can Kill

Doctors are sometimes terrible at statistics, and our biases -- including the hugely prevalent interventionalist bias in American medicine -- inform the way we look at numbers and probabilities in really dramatic ways. Here's an article ("Bias in the ER" from Nautilus) I read recently that talks about where doctors fall short, and how they can improve, in interpreting the numbers.

One really egregious example of this is in the way we often look at screening tests. I've had a lot of med students in my time as a tutor in medical school and my time as an educator in residency ask questions that shed light on how rudimentary are understanding often is when it comes to the risks of running a test. And we need our patients to understand this too! I wrote an illustrative example -- let's pretend there exists a disease called blarg cancer.

Drawing (1).jpeg

Let’s do a math exercise with made-up diseases and numbers. Let’s say we had an organ called the blarg and sometimes people get blarg cancer. Blarg cancer is bad and results in significant morbidity and mortality so we want to screen for it. We find a molecule that shows up in the blood of almost every single person with blarg cancer – what a great screen! We are so pumped. But we soon start to realize that sometimes it shows up in the blood of people that DON’T have blarg cancer. So some people we’re screening are showing up as positives even though they don’t have blarg cancer. But no biggie, right? It’s worth it because we’re catching people with blarg cancer we wouldn’t otherwise catch.

Let’s pause the story to take a terminology break: sensitivity is a quality measure of a test that tells us how often the test turns POSITIVE when someone has the disease you’re looking for. If a test is highly sensitive, this means that if you have the disease, the test is going to turn positive almost every single time. This means you’re catching basically everyone with the disease. So the screen for blarg cancer I described above is highly sensitive, because I said the molecule “shows up in the blood of almost every single person blarg cancer.” Another way of thinking about it is that a test with very high sensitivity has a very low false negative rate. People who get a negative test result can rest assured they almost certainly do not have the disease.

Another quality measure of a test is the specificity, which measures how many people WITHOUT the disease will have a NEGATIVE test. We call it specificity because the question we’re asking is: is this test specific to the disease we’re looking at? Will it turn negative every time someone doesn’t have the disease, or are there some cases in which it turns positive for a reason other than the disease in question? A highly specific test means if you get a positive test, you can be pretty sure that this person has the disease. In other words, a test with very high specificity has a very low false positive rate. The screen for blarg cancer I described above doesn’t have great specificity, because I said, “sometimes it shows up in the blood of people that DON’T have blarg cancer.”

Sensitivity and specificity are measures that we calculate for EVERY TEST we do in medicine! We often think of our tests as arbiters of truth-in-diagnosis, but that is a really dangerous myth. Unfortunately we just don’t have magic diagnosis-revealer wands we can wave over patients to determine if they have diseases we’re looking for, and we need to be really careful not to think of the diagnostic tests we do as diagnosis-revealer wands, because it can cause a lot of problems, some of which I’m going to outline now.

Alright, back to our blarg cancer screening test. Let’s say that our test has a sensitivity of 95%. Wow! A+! Such a good sensitivity. That means that if 100 people have blarg cancer, 95 of them will test positive. You’re catching almost everyone.

Let’s say the test has a specificity of 80%. That’s not so bad, right? Still a B? But definitely not as good as the sensitivity. It means that of 100 people that don’t have blarg cancer, 80 of them will have a negative test. In other words, 20 people out of 100 without blarg cancer will test positive.

Let’s figure out what all these numbers mean. Blarg cancer has a prevalence of around 1%. This means that if you picked 100 people from a crowd, 1 of them would have blarg cancer. If your hospital has a patient population of 100,000 then 1000 have blarg cancer. The sensitivity of your test means that if you start screening you will catch 1000*0.95 = 950 of the people with blarg cancer. The specificity of your test means that of the 99,000 people that don’t have blarg cancer, 99,000*0.8 = 79,200 of them will have a negative test. But – uh oh – that means 99,000-79,200 = 19,800 will have a positive test even though they don’t have blarg cancer.

Now most people see that number and say to themselves, “Well, it’s not that big of a deal. There’s a psychological discomfort but at least we’re catching the people that do have cancer!” But it’s not so simple. What do you do after you have a positive screen for blarg cancer? You have to do something about it! So you remove their blargs. All the positive screens, or 19,800 + 950 = 20,750 people have surgery to get their blargs removed.

Surgery complication rates for blargectomy are about 20%. That’s any complication, including anything from a minor infection of the site to something more serious. Perioperative mortality rates are much lower, 2%. That’s not too many people (good job, surgeons!), so it makes you not worry too much. It includes people that died in surgery, died shortly after surgery, or died for a reason directly related to their surgery (e.g. sepsis from surgical infection). So let’s do the math. If 19,800 people got surgery that didn’t really need it (false positives), then 0.02*19,800 = 396 people are going to die from complications of a surgery that they didn’t need.

Well, you think, that really sucks. But we did surgery on 950 people that really needed it! But wait a second – surgery isn’t always effective. Cancer really sucks, and even when we do surgery for blarg cancer, it’s only effective in the long run 30% of the time. Surgery saves the lives of 30% of people with blarg cancer, so you saved 950*0.3 = 285 people.

So think about that. You saved 285 people, but you killed 396. Still want to do that test?

And that’s not taking into account a whole host of other factors, like lead-time bias, complications of surgery that aren’t fatal but are life-altering, and the mental health implications.

“But Monica,” you’re probably thinking, “Blarg cancer and all those numbers are made up and this is a totally hypothetical scenario.” Yes, I made this example up, but it’s a teaching example that reflects REAL LIFE EXAMPLES. We have learned this lesson time and time again. Read up on PSA screeningCA-125 screening, and dementia screening. This lesson is the reason we only screen smokers for lung cancer. This lesson is the reason mammography for breast cancer screening is undergoing so many changes in recommendations.

“Okay, Monica, I’m convinced. So should we stop screening then?” NO!! That’s not what I’m saying at all! There are plenty of screening tests that have held up under scrutiny and proven themselves to be effective and worth potential adverse effects. The medical community currently holds up the pap smear as an example of a good screening test (when it's administered appropriately and at appropriate intervals), with high sensitivity and high specificity, with a high enough prevalence of disease to make it worth it, with the potential to save lives if caught earlier, and with a lower rate of adverse effects.

Cancer sucks. It takes so many lives from us and results in so much suffering and tragedy. Please don’t think that I’m not taking cancer seriously, or trivializing it. But we can do harm too. And we do. We HAVE to be careful in what we do as a medical community. We have to take that seriously.

So what’s the message then? The message is that as physicians it's our job to understand that everything we do incurs risk and that we need to work with patients to use the information we have to determine when that risk is worth it and when it isn’t.

Basically, use your brain. Do the math. Be a critical thinker. IT’S OUR JOB. Our patients need us!

Big Dreamers

Last week I switched to a medicine wards month. Sometime in my first few days I dreamt I woke up in a hospital bed on a ward that was outside, a cool blue feeling permeating the corridor like a heavy morning dew. My team was rounding. It was my team in both senses – I was a patient on their team, and also I was one of the team’s interns. They rounded on me in the manner of the latter. I gathered I had been out of it for a while – I didn’t know the day or what was going on – but soon pieced together that I had ovarian cancer, and my status had been grave at first. They told me my hemoglobin was 2.0 on admission.

The team talked shop for a while about how I was doing and what the next steps were. The attending concluded by saying to the crowd, “I knew we should have imaged you when I felt that mass on exam a few months ago! I told the oncologist but he wasn’t impressed. And look at that – fucking ovarian cancer! I told him!” He hadn’t told me he’d felt a mass at my last visit.

Like many of my dreams, this one will be a fun one to talk about in psychoanalysis when I finally have time for 3 hours of training-focused therapy a week. I think even with the most simple modalities of reflection this dream says a lot about some of the worst places of turmoil we go as residents.

How does someone be a doctor and a person at the same time? How do you navigate your role as intern on a team without losing the perspective of the patient? What if you’re sick too?

I spend a lot of time on teams feeling like the one closest to patients, feeling like it’s my job to play arbiter in the ways I can, to try to pick through the politics of maintaining safe and congenial team dynamics without feeling like I’ve thrown patient respect out the window. It’s hard to remind doctors to take a minute to remember the person without tromping all over the ways they’ve developed to cope, and without layering on too much acerbic vibes between you and your co-workers. It’s especially hard when you’re having trouble remembering yourself.

And, of course, there’s the Resident as Patient theme. I was a patient and my own doctor in this scene. I think it indicates I see myself as sick in some ways. This isn’t new to me. Residency makes you sick. You work more hours than you ever have with more responsibility on your shoulders you could have imagined – you get worse before you get better. You become less human before you become more human. But how do you treat yourself? It’s the obvious question, especially given that as doctors I think many of us are reflexively trying to come up with the treatment plan.

The finale is the gaslit resident who goes to work every day absorbing the message that all of the hardship we witness and work through in hospitals is somehow okay. In this dream my attending sends me the message, “You were sick, very sick, for three months. I didn’t tell you. Now I’m talking about it as if it’s not a big deal.” Gaslighting in residency – the underlying message sent when we avoid talking about the subject that what we witness isn’t a big deal – is something I’ve been running up against a lot the last few months.

Doctors cope with the nature of medicine in a lot of different ways, and it’s hard to be too judgmental about how we make it through. I understand that talking about how messed up everything is all the time isn’t sustainable, and that some form of repression/suppression is necessary to make it to tomorrow. But medicine is hard. I know that’s never gonna change. I’m not asking for it to – then it wouldn’t be medicine, and it wouldn’t be the career for me. But I think the way to achieve wellness in residents necessarily involves having better ways to talk about the hard things, to share the ways we get through them, and to have mentors that help us see how to become doctors that are better at appreciating humanity and more human themselves. I know that nobody’s really figured that out. But we need to be brainstorming more. My examples shouldn’t just be ones of repression and jaded hardening. They should be ones of stoic triumph in the face of huge challenges. We should at least be talking about how to get there.

Honoring the Hype

Someone asked me what to do if your career in medicine feels boring, you feel like you're only doing it because it's the right thing to do, and working part-time or choosing a specialty with good hours feels like hedonism. This is what I wrote in response.

If you think your career as a physician is boring, you need to take a long hard look at what you’re doing. The longer I reflect on what it means to be a physician, the more I understand it to be a calling. It’s truly a vocation, and I apply that word because I don’t think that it should be done by people who don’t feel compelled to don that yoke.

I’m not saying you need to be hyped for your career 24/7, or even every year of your life. But you’ve gotta have a little hype – that hype allows us to pick ourselves up on days it’s really tough, it makes us honor the privilege and responsibility of having lives in our hands, and it, hopefully, compels us to do better for our patients with a measure of humility.

There isn’t a heavenly ledger that catalogues your gives and your takes from this world. We are all giving and taking – in fact each act of giving is an act of taking in some way, too, and vice versa. That’s what being a member of this human race is. You shared your bread with your sister and you gained generosity. You taught your brother and you gained a student. Your mother gave you life and you gave her motherhood.

To harp on a similar concept, there is no purely selfless motivation for pursuing medicine. Guilt will only take you so far before it corrodes.

I know that guilt makes up some of the fuel that powers my own motivation, but there are many other things too. Ambition, drive. And a deep, insatiable love and curiosity about humanity that means I care deeply about people, and also find a great joy in connecting with them. Do you see how that has some selfishness in it? I really want to help people, and also this way of helping people is one I find deeply satisfying and stimulating. It incites passion and drive in me. All of these things are roiled up together into the reason I’ve stayed on this medicine track.

(And you know what? I’ve checked in with myself every step of the way, to make sure I still wanted to be on that track. Before starting med school, I gave myself permission to quit if I started to hate it, if I didn’t think it was right for me. And so I took that seriously, and feel ever more confident on this path. I’ll continue checking in as my life moves forward.)

This is why I don’t think there’s a purely selfless reason to pursue medicine, and I am suspicious of people that claim they have one.

Now the reason I’m talking about this is because it matters for patients. I’ve been saying this for a long time now, but please believe that I really mean it: a doctor that doesn’t have some hype, that doesn’t feel connected to their career and therefore doesn’t connect to patients – in other words, a doctor that’s burnt-out – that doctor isn’t a good doctor. S/he might show up to work every day, might diagnose and prescribe as indicated by the clinical guidelines, might dot all the i’s and cross all the t’s. But the more and more studies we do, the more we realize how essential the relational component of medicine is. Having a doctor that’s well improves patient health. It’s just a fact.

So if working part-time is what it takes to maintain your hype, then please work part-time. If specializing instead of entering primary care is what it takes to maintain your hype, then please specialize. Etc! When you see a patient, ask yourself if you’re there in that room with that patient, fully mindfully present, human-to-human; if you’re not, figure out what you need to do to make it happen.

I’m not saying you need to quit if you’re not feeling the hype 24/7. But I am saying that you should take care of yourself, and not make a decision based on guilt, and really try to know yourself in this career. Nosce te ipsum; know thyself.

I sincerely hope that helps.

Best,
Monica