FDA Rejection Likely for Biovest
Data on cancer drug just not strong enough.
At the risk of getting stuck in a rut, there's more to talk about from Biovest (BVTI). On Monday, the company held a conference call about their BiovaxID data. I read the transcript on my Blackberry while waiting to celebrate the birthday of a journalist friend of mine. I have to say, I got quite a chuckle when I saw that management took shots at me on the conference call. The conference call has some additional details on it about the statistical issues, so I don't mind writing about this situation again.
The conference call had critical details about the clinical trial design that bear directly upon the drug's approvability by the FDA. This provides a perfect example of why any serious biotech investor has to have access to management in order to get detailed questions answered about trial design and data (or have someone knowledgeable do it for you). Props, by the way, to Rodman & Renshaw's Reni Benjamin for asking some good questions on the call.
So let's dive into the issues.
Biovest cleared up one question I had on manufacturing, though they managed to do it while misrepresenting what I said. If what they say is true -- and I don't have any reason to doubt it -- they should be congratulated for being one of only two active immunotherapy companies I know of who are working with patient-derived tumor samples with a failure rate under 5% in multi-center trials.
To be clear, however, I never suggested the manufacturing failure rate for BiovaxID was 20%, as was attributed to me on the call. Here's precisely what I said (emphasis added):
"Manufacturing failures are common in products like BiovaxID, where portions of the patient's tumor are required to make the drug. The fact that this drug is targeted to a blood cancer should reduce the failure rate as compared with other companies who focus on solid tumors, where failure rates average around 20%"
Color me impressed that Biovest reported a sub-3% failure rate. That's helpful information. Unfortunately, I don't know whether those patients were included or excluded from the interim analysis the company has released. I think they were included, as the FDA will certainly require.
New Biostatistical Problems
Management claims there was no statistical penalty involved in taking an interim efficacy analysis. The rules of biostatistics always require a statistical penalty when the outcome of the analysis could stop the trial.
I have a couple of guesses as to how this could still be true, as management claims. The most obvious is that this was merely a safety review by the DSMB (Data Safety and Monitoring Board) that became something more given the statistical situation. This sort of "unplanned stopping analysis" wouldn't have been proactively assigned an alpha, so management is being truthful when they say there was no statistical penalty in their filed statistical analysis plan.
This is an interesting line of thinking. I'd written about the odd wording in the initial press release attributed to the DSMB. Given this new information, my theory is the analysis from the point of randomization had such a high p-value that the DSMB would normally have ruled the trial was futile.
In a subsequent ad-hoc analysis of the data as measured from time of vaccination, the DSMB noted a 'statistically significant' result. That led to the DSMB's call to have Biovest talk about this with the FDA, in the hopes that someone in that bureaucracy would finally listen to reason and cut the trial some slack.
That's a perfectly reasonable conclusion for the DSMB to make. The only problem is the FDA's cutting small companies with first-in-class drugs like BiovaxID no slack these days.
Given other information provided on the call, however, this is not the biggest biostatistical issue facing Biovest investors any longer.
On the call, we learned the company's statistical analysis plan filed with the FDA declared the threshold p-value necessary to achieve statistical significance was p=0.01 on the primary endpoint of progression-free survival. The company's cited disease-free survival endpoint had a p-value of p=0.047.
So by their own statistical analysis plan, this trial is a biostatistical failure.
Management spoke about how there were no "bright lines" for p-values. In the post-2004 oncology drug approval era, there certainly are. You have to be below p=0.05 (Dendreon's (DNDN) p=0.053 on their primary endpoint didn't cut it, remember). Plus, in every recent case, you need to match every statistical threshold you set up for yourself. Since 0.047 > 0.01, the trial is a statistical failure.
A target p-value of p=0.01 isn't normal. There would have to have been a significant biostatistical reason for lowering the bar to that level, or else the target p-value would have been the standard p=0.05. I was disappointed that nobody on the call asked why the p-value was set at 0.01. The answer would probably be illuminating.
Incidentally, a p-value of 0.024 means there are 2.4 chances in 100 that the trial outcome is due to chance, not 2.4 chances in 1000 as management indicated on the call (according to the transcript). Minor point, but p-values are confusing enough without errors like that going uncorrected.
I want also to point out my assumption that management is using "disease-free survival" and "progression-free survival" interchangeably - which can be dangerous. So if you're an investor in this company, you need to get the answer to this question: "Is the disease-free survival endpoint you reported in the press release the same measurement as the prespecified primary endpoint of progression-free survival in this trial?"
If the answer's "no," then the picture becomes even bleaker. I'll give management the benefit of the doubt on this one. If I had money invested, however, I'd be on the phone right away.
The transcript was confusing about the size of the trial arms. At one point, management says there were 177 patients randomized, 118 of whom received vaccine. Later in the call, they say 117 patients received vaccine - 72 on BiovaxID and 39 on placebo. If I take the transcript as accurate, I get the following…
- 177 patients randomized.
- 59 patients spread among both arms progressed while "resting" and before receiving vaccine or placebo.
- 1 manufacturing failure or other type of dropout.
- 72 received BiovaxID.
- 39 received placebo.
- 7 not yet included in the data set.
If I'm right on the math above, then there's absolutely no way the trial is statistically significant from the time of randomization. If I assume a 40/20 (2:1) split in the 60 patients who dropped before the first vaccine, then 36% of the study arm (40/112) will perform no differently than does the placebo arm. That spells doom when the p-value you've started at is p=0.047.
I should note the transcript I worked from was the "final" and not the draft. The only difference is this: The former has been checked by an editor. That doesn't mean it's free of mistakes (they butchered the spelling of "Minyanville," for example), but they're usually accurate.
At 117 patients, I'm also afraid the "it's too small of a trial" idiots at the FDA will be out in force. Even assuming the FDA sees reason and agrees that measuring from time of vaccination is correct, they'll "only" be looking at 117 patients.
This issue is all the more likely to come up because Biovest abandoned its original enrollment plan, which called for 375 randomized patients. I vehemently oppose this type of attack on the significance of a statistical result, but investors must understand how exceptionally pervasive it is in both the regulatory and medical communities. You won't see me making this argument, but others certainly will.
The FDA is also going to want to look very closely at why 34% of the randomized patients dropped out of the study (again, assuming my math from the transcript is right) before receiving vaccine or placebo.
It's About the FDA, Not the Drug
If I make the significant assumption the trial enrollment and post-study treatment is balanced between trial arms, I can agree BiovaxID works.
So why all this doom and gloom from me?
Simply, whether the drug works and benefits patients doesn't matter to this FDA. That's illogical, it harms cancer patients, and it's fundamentally wrong on a number of levels. But it's the absolute truth. I've written on this at length both here and in the pages of my firm's research, and I've been quoted on the topic in the media.
The FDA, unless it undergoes some miraculous overhaul and jettisons Richard Pazdur and much of the oncology division's staff, will not approve this drug on this data set - not even for accelerated approval. The statistics simply aren't clean enough, and Biovest didn't complete the enrollment as planned.
Even if the company can get these data to qualify for the EU's "named patient" program, the revenues from that won't pay for another clinical trial to prove to the FDA these results are real.
And while the EU is more reasonable on biostatistics, I wouldn't bet a significant sum of money that the EU will approve it either. The news from Epicept (EPCT) today might mean the EU's version of accelerated approval might be working better than the FDA's. Again, I see open marketing approval in the EU as a long shot - annd reimbursement over there's a completely different can of worms.
I understand why some investors and management are so excited about these data. I've been there, and I have the scars on my portfolio to prove it. Those who think this is a slam dunk should look very hard at Dendreon's example. They had a similarly logical endpoint and statistical argument, and even got an FDA Advisory Board to vote overwhelmingly that the drug was safe and effective. The FDA still turned them down.
The BiovaxID data set is weaker (biostatistically speaking) than the Provenge dataset which was rejected by the FDA. That doesn't bode well for Biovest, its investors - or leukemia patients.
The information on this website solely reflects the analysis of or opinion about the performance of securities and financial markets by the writers whose articles appear on the site. The views expressed by the writers are not necessarily the views of Minyanville Media, Inc. or members of its management. Nothing contained on the website is intended to constitute a recommendation or advice addressed to an individual investor or category of investors to purchase, sell or hold any security, or to take any action with respect to the prospective movement of the securities markets or to solicit the purchase or sale of any security. Any investment decisions must be made by the reader either individually or in consultation with his or her investment professional. Minyanville writers and staff may trade or hold positions in securities that are discussed in articles appearing on the website. Writers of articles are required to disclose whether they have a position in any stock or fund discussed in an article, but are not permitted to disclose the size or direction of the position. Nothing on this website is intended to solicit business of any kind for a writer's business or fund. Minyanville management and staff as well as contributing writers will not respond to emails or other communications requesting investment advice.
Copyright 2011 Minyanville Media, Inc. All Rights Reserved.
Daily Recap Newsletter