Sorry!! The article you are trying to read is not available now.
Thank you very much;
you're only a step away from
downloading your reports.

Getting the Most Out of a Biostatistical Analysis

By

If you plan to profit based upon these analyses, you have to do the work to understand them beyond just flipping to the last page and reading the ending.

PrintPRINT


My firm has been going through an interesting experience in the last month that I think is worthwhile to share.

Biotech investors employ a ton of different strategies to try and gain insight into the companies they are interested in. We talk to doctors, nurses, and other medical professionals to understand how drugs work in the clinic and how they will be perceived in the marketplace. We talk to lab scientists and pore over arcane journal articles to understand the science behind the products because the science is more than half the battle. We talk to hedge funds because good companies can have crappy stock prices if Wall Street isn't behind the product and management. We also talk to hedge funds to determine what the expectations are for a certain trial. This is part of the regular path we travel before we have anything substantive to say about a company or a clinical trial.

On occasion, a situation presents itself where a statistical projection or analysis is useful. You hire a biostatistician, present him/her with a set of questions you'd like "answered," and shovel as much statistical information his/her way as you can. Then you sit back and wait for the results. The goal is to allow you to predict the outcome of an ongoing clinical trial.

We did this recently with YM Bioscience's (YMI) pivotal trial for tesmilifene. The so-called "DEC" trial investigates whether the addition of tesmilifene to standard anthracycline chemotherapy will improve survival for fast-relapsing breast cancer patients. The DEC trial is based upon a clinically successful trial called MA.19 that demonstrated a clinically and statistically significant survival advantage due to the use of tesmilifene.

I could go on for pages and pages explaining about the troubles we had trying to decode the exceedingly complicated Triangle Test employed for the statistical analysis of the DEC trial (our note on the results was 19 pages). But that's not my point here…

If you are a biotech investor, you will come across research like my firm published earlier this week. You'll likely start reading and then get bored about 1/3 of the way in. You'll skip to the summary section, read what the author has to say, and conclude that's "The Answer."

You just blew it.

The YM statistical analysis is not the first we've done and certainly won't be the last. It was, however, a vivid example of what a statistical analysis is and is not. To explain this, you have to understand that a statistical analysis designed to predict some future event has two distinct components, quantitative and qualitative.

Quantitative has to do with the math and the application of broadly accepted principles of biostatistics. There are clear right and wrong answers in different quantitative approaches. We are very confident our analysis of YM's DEC trial, for example, is the definitive biostatistical analysis of that trial. It is "right" where others have been "wrong."

Qualitative has to do with the assumptions you plug into the quantitative foundation to get your "answers." If your qualitative assumptions are incorrect, the best math in the world will not save you from getting wrong "answers." Because of this, two people with different assumptions can use the same quantitative foundation and get wildly different answers. Some qualitative assumptions are better (more reasonable) than others, but there are no right or wrong answers in an exercise like this.

The most important thing to realize about any statistical analysis is that it is a tool. The "answers" derived from the use of the tool are only as good as the assumptions you chose to use. Just because they were generated with fancy math doesn't prevent the answer from being wrong if you use bad assumptions.

When you skip to the "answer" portion of any biostatistical analysis, you almost certainly miss important information about how the tool was constructed. Even more important, you miss the discussion about the assumptions employed that generated the "answers" you read on the last pages of the report.

If you make an investment decision without reading and understanding the qualitative assumptions, you might as well have saved yourself the time taken to read any of it and simply tossed a dart or flipped a coin.

Beyond that, you shouldn't automatically take the assumptions of the authors as the only valid set of assumptions. For example, a prior analysis of the DEC trial by a different firm concluded that because the first analysis of the trial did not occur in January, the trial would be negative. This report caused a fair amount of selling. It was based upon a bad assumption that, when run through an accurate quantitative model, generated a false answer. The right answer is "early" or "late" meaning "same" or "different," but they have no relationship to "good" and "bad."

Now selling might have been the right approach. We won't know until the trial turns out to be positive or negative. If the trial turns out positive, then not understanding and evaluating the quality of the assumptions that generated that "answer" will cost people money.

Even if you don't invest in biotech, your participation in the financial markets (and politics and economics and most every part of life) ensures you'll have to deal with statistical reports of one kind or another. The principles are the same. There is a quantitative framework that is either right or wrong. The "answers" are based, however, on qualitative assumptions that usually have no clear right or wrong to them. Plug in different assumptions and you get a different answer.

If you plan to profit based upon these analyses, you have to do the work to understand them beyond just flipping to the last page and reading the ending.

< Previous
  • 1
Next >
Position in YMI

The information on this website solely reflects the analysis of or opinion about the performance of securities and financial markets by the writers whose articles appear on the site. The views expressed by the writers are not necessarily the views of Minyanville Media, Inc. or members of its management. Nothing contained on the website is intended to constitute a recommendation or advice addressed to an individual investor or category of investors to purchase, sell or hold any security, or to take any action with respect to the prospective movement of the securities markets or to solicit the purchase or sale of any security. Any investment decisions must be made by the reader either individually or in consultation with his or her investment professional. Minyanville writers and staff may trade or hold positions in securities that are discussed in articles appearing on the website. Writers of articles are required to disclose whether they have a position in any stock or fund discussed in an article, but are not permitted to disclose the size or direction of the position. Nothing on this website is intended to solicit business of any kind for a writer's business or fund. Minyanville management and staff as well as contributing writers will not respond to emails or other communications requesting investment advice.

Copyright 2011 Minyanville Media, Inc. All Rights Reserved.

PrintPRINT
 
Featured Videos

WHAT'S POPULAR IN THE VILLE