Risk Evaluation and Mitigation Strategies

We think we understand risk when we talk about chances flipping a coin, but are all outcomes equally good or bad?
We think we understand risk when we talk about chances flipping a coin, but are all outcomes equally good or bad?

Earlier this month, I was approach to come and present my view as a patient to the Food and Drug Administration’s conference on Risk Evaluation and Mitigation Strategies (REMS). Prior to 2007, the FDA was tasked solely with determining whether a drug was “safe.” It was under this directive, that they pulled Tysabri, the drug I take to slow the progression of multiple sclerosis. They pulled the drug from the market because some people who took the drug died, and in fairness, death is a pretty good indication something may be unsafe. However, many patients wanted to make the same decision my wife and I eventually did, to take the drug because the benefits outweighed the risks. Not many had died on the drug, and many had benefited. Tysabri and MS were not the only drugs caught in the predicament of patients wanting a drug they felt outweighed the risks, and eventually the Food and Drug Administration Amendments Act of 2007 allowed the FDA to declare the benefits of a drug worth the risks of taking it. No longer were potential downsides of taking a medication the only determining factor for approval.

Years later, they are looking specifically at the impact of the various REMS on the healthcare delivery system and patient access. I was thrilled to present my impressions at the two day conference. In the first morning of the conference, we received the history of the REMS which the FDA is authorized to require. The strategies range from special certification of doctors, nurses and treatment delivery locations to specialized pharmaceutical dispensers to patient registries and monitoring.

When they got to the part on Patient Access and asked for questions, I asked if they thought Patients generally acted in their own best interests. I then gave the example of Tysabri and all the questions patients are routinely asked before each infusion. If we answer yes, we get no infusion until the doctors can clear us. If we say, “no, nothing new to report…” we are god to proceed. Why would we answer yes? To make matters worse, Tysabri is not approved for progressive forms of MS. However, there have been trials of it for progressive MS, and as of yet, there is no treatment approved for progressive MS. So if you transitioned to progressive MS, would you volunteer that information? The choice is between “may help” and “no help.” It is not in our interest to be honest, so how predictive do you think our answers will be? What’s more, we are setting up a situation where it is not in a patient’s best interest to tell their medical provider all of the information.

At the end of the first morning, I got to present. I went through my spiel about how poorly we all understand large numbers. Then I went into ways we deal with that inability to grasp large numbers and how I used the most common remedy for this failure, compare one large number with another. For me it was the risk of using Tysabri versus the risk of treating the most treatable kind of cancer with the treatment giving the best odds for long life (chemo for breast cancer caught early). The mortality rate for chemo is roughly 1 in 200. When I started Tysabri, my odds were estimated at 1 in 750, or 3.5 times better than if I took the most effective treatment for the most treatable kind of cancer.

Then I asked if we are sure what we are measuring when we look to approve drugs is correct in the first place. With my MS, the FDA looks at numbers of new lesions in a relatively short period of time. However, these are just spots on an MRI until a patient experiences symptoms, and the correlation between lesions and symptoms is poor. I have roughly 18 lesions on my brain and a half dozen on my spine. Yet, I work full time and run 5 miles multiple times a week. Still, one lesion near my brain stem had me in the hospital due to swallowing problems. It is not the number of lesions that matter to the patient. It is the impact on their life.

Finally, I closed recognizing the FDA has the power to regulate the availability of a treatment, and doctors have the ability to proscribe or refuse a treatment. Ultimately, it is the patient who has to make the final call on whether to pursue a treatment. Should not we do this with the best data available so that we can best assess whether the benefits are worth the potential risks? As a project manager, I have to write risk assessments all the time. However, my list of risks includes risks things may go right. That is what is taught in modern project management decision making, and yet all of the papers given to the patients with every drug speak only of the potential negative outcomes. If we are to make informed choices, how about giving us all of the known information about possible outcomes and known likelihoods?

The amazing thing to me as I finished was to see and hear some people applauding. That made for a great start to my last week as a 30 something. I sometimes find it a great boon to my sometimes tired spirits to think people take my words as meaningful. I’ll grant the final document from the two day conference mentioned only that “patients may be willing to take more risks,” but at least I think many in the crowd understood. That’s enough to make me feel I represented our position as best I could.

For more information about the conference, including a link to the presentations, go to Risk Evaluation and Mitigation Strategies (REMS)

Numbers and Risk_all

Share

“One, Two, Many, Lots, and Whole Bunches!” – Life in a Base 100 World

I have always been told we use a base ten numbering system.  I maintain we are a base ten times ten when it comes to absorbing the meaning of numbers.
I have always been told we use a base ten numbering system. I maintain we are a base ten times ten when it comes to absorbing the meaning of numbers.

There seems to be a logical disconnect in our brains when it comes to very large numbers. We have ten fingers and ten toes. We are fine counting to ten. When it comes to counting to 100, we don’t have big problem either. However, I note that I can put myself to sleep counting down from 100 by “1’s” or “2.5’s.” One hundred seems a natural barrier, and because we are a tens based society, ten times our natural barrier is still comprehendible. However, as we go further from hundreds our understanding of scale diminishes. When we start counting in thousands, we may as well go back to the childhood counting, saying “One, two, many, lots, and whole bunches!”

We can intellectually go beyond a thousand, but I note that when we do, we group things so that we are counting the groups again, never going beyond the hundreds. For example, 530,253,063 is said “five hundred thiry million, two hundred fifty-three thousand, and sixty-three.” We have kept our counting to the hundreds of a group. That seems a natural cognitive limit of our intuitive understanding.

I think this inability to think beyond hundreds inhibits some of our intuitive understanding of scale. I see this all the time even amongst those of us dealing with numbers all the time. At my work, a group of us play the lottery when the winnings are big enough for all of us to retire. We call it the “stupid people’s tax” because we all know the expected return for our money is nothing and we pay anyway. The odds of one in hundreds of millions feels like one in hundreds with the millions only understood intellectually.

It is with this in mind that I read much of the news about the Syrian refugees. I see reports where countries take in thousands or even tens of thousands, and it feels impressive for some group to advocate increasing the number of refugees from one thousand to ten thousand. It feels like the group advocating for ten thousand is much more heroic. I submit this thinking is at least partially the result of our inability to comprehend the number of refugees is estimated at 10.8 million. Again, we focused on the wrong parts when thinking about the scale of the crisis. Like the examples above, we thought about the numbers I underlined instead of the description after them. It is very hard to get to 10.8 million (number of refugees) when we are dealing with them a thousand to maybe ten thousand at a time. When I think about the true scale of the problem, it feels like the responses are akin to trying to put out California forest fires with one spoon full of water at a time. Some may bring the teaspoon while others bring a ladle, but how effective are either?

Don’t take this wrong, our minds inability to grasp large numbers has advantages. I take a drug that has a chance to cause severe brain infections and possibly kill me. The published odds I get on that happening to me are changing all the time. My neurologist asks at every visit if I am concerned by the odds and want to switch medications. My most recent numbers were one in seven hundred, and I told him again I will be concerned when my odds worsen to below one in two hundred. Above that, my mind treats the risk like the odds of being struck by lightening or dying in a car crash on the way to work. These things happen all the time, but the odds are not worth worrying about because my mind puts them all in the remote risk category. My minds inability to internalize the risk helps me live my day to day life. I justify my thinking about taking Tysabri by noting my odds are still better than a Cancer patient taking Chemo which has a mortality rate of one in two hundred. I do not think about the large number that is my odds of getting the brain infection. Rather I think about it in comparison to something else.

The comparison method is the only way I think most of us truly attach meaning to large numbers. This is what I am doing when I compare the mortality rate taking Tysbari with the mortality rate of a cancer taking chemo. When we release data on the United States economy, most people care more about the direction of the change in numbers and how fast they are changing rather than how big the actual base number was. Most of us really cannot intuit the GDP reports talking about trillions of dollars.

When it comes to large numbers, we just need to be careful to be mindful of what the large numbers are for which we see differences and the differences in scale between different large numbers. If we can manage these two obstacles, we might avoid some of the common mistakes in our perceptions of the universe in which we live. Maybe then we can stop comparing “many” to “whole lots.”

Share

Our Family's Stories of Growing Up

%d bloggers like this: