Doctor Black KItty Speaks Up about PHARMA Statistics

Health Care Renewal: Annals of Internal Medicine Shows Appropriate Skepticism About a Commercially Sponsored Clinical Trial

is a great summary of a journal article on Byetta. I actually have a vested interest in this medication, which you shoot into yourself twice a day :) I got a real cool package of goodies from the Drug company to use with this medication. I get Newletters by email and paper mail offering me additional products and information. HOWEVER, they DID NOT send me this article. I wonder why?

Here is the comment that I posted:

This posting is near and dear to my heart. It is very brave for the Annals of Internal Medicine to come out in this way about PHARMA research. I am sure the article was reviewed by Glasgow in the context of his Refit Model for generalizability. Dr. Glasgow is such a good soul and a brilliant man, that I am sure he was behind the editorial. I have not seen the original source or the article or the editorial. I wanted to write this UNBIASED by the truth, just from my experience.

I actually tried Byetta myself! I found the side effects unbearable and the MD did not involve herself in my treatment using this drug in a best practices manner, so I stopped the medication at large monetary cost AFTER INSURANCE REIMBURSEMENT.

The Annals is a very good journal, but there are many other PHARMA advertising journals that would take the article without many changes. So these papers can be peer reviewed and accepted in low level journals. Thus they become "truth" without Editorials.

This study has many obvious flaws that were well stated. But there are several OTHER major statistical problems with this paper.

Due to the differential drop out rate (Byetta makes most people nauseous) statistics need to be performed that account for the fact that people who "survive" Byetta are fundamentally different in some way from those who DO NOT complete the trial for ANY reason. This is a very tricky issue, THAT COULD HAVE BEEN ANTICIPATED, and the study could have been designed to account for the nausea rule out.

HEY PHARMA ANALYSTS, WHEN ARE U GOING TO REALIZE THAT THE CHARACTERISTICS (covariates, risk factors, endogenous factors) OF THOSE WHO ARE COMPLETERS (patients who complete the trial) are FUNDAMENTALLY DIFFERENT FROM THOSE WHO DROP OUT FOR ANY REASON (e.g., those who refuse treatment, can not be contacted, can't tolerate medication). This is one of the dirty little secrets that they don't want to address. Everyone has some lifestyle changes or baseline level of diet, self-care, and feelings about the role of medication in their control of blood sugar. These variables need to be covariates in the analyses. Randomization doesn't necessarily control this.

I went to the FDA meeting for PHASE 1 - 3 trials, and they almost kicked me out.

The reason being, the FDA would only approve baseline severity as a covariate. I told them that was ludicrous in real life.

To do the RIGHT THING, non-response propensity weights needed to be­­ used. In addition, covariates also need to be accounted for in the model to evaluate the treatment effects. Did they use random coefficient regression techniques? In my opinion, and this could be debated, is that this type of modeling procedure (weighted and random slopes and intercepts) would be the only appropriate statistical analysis methodology for this design. GEE is not the optimum model, however, this is what the FDA, in their ADVANCED classes recommend. My methods (and those of many others – Hat Tip to Hedeker) would allow the researchers to look at people as having different baseline characteristics (e.g., HbA1c, BMI, age) or INTERCEPTS and different trajectories or slopes over the study period. It would also allow the use of all the data, even those with only baseline data to be used in the analyses (in this case weighting is not needed).

What percentage of data was collected (Number of questionnaires you actually have divided by the (# patients randomized * #assessments))? What is the coefficient of acceptability of treatment (how many folks were approached and how many agreed to be consented? How about the tolerability of the treatment? That seemed pretty bad.

I would be interested in knowing WHO are they attempting to generalize to? All races? All ages? All socioeconomic levels? All levels of baseline type 2 diabetes severities?

What are their coefficients of generalizability or their REFIT numbers?

The reason these types of PHARMA studies are designed and analyzed in a sub par method is because the people designing and conducting and analyzing these studies are UN-INFORMED. I don't think they actually do this on purpose. I believe it is due to EGO and IGNORANCE. Being part of MANY PHARMA studies like this one, I can honestly say that the crappy design usually is the brain child of the PI of content. That MD usually has had NO training WHAT-SO-EVER in experimental design or statistics, as these are courses that are NOT taught as part of medical school. The MD designs the study and it is PAID FOR by a drug company, who is very DEFERENTIAL to the MD design.

For the statistics, they usually hire an individual who is a strict biostatistician and not a PhD with content in the area WHO IS ALSO a statistician (like me).

The Stat person doesn't get to HAVE ANYTHING TO DO AT ALL with the design of the data, and only gets it when the study is completed (sometimes its all washed by then). Then this Biostatics expert does what is asked of him/her AND DOES NOT QUESTION AUTHORITY. This is because the VAST majority of Biostat people are not English as a first language, are usually from India or China or Korea and have little diabetes or psychiatry content knowledge, and to the analyst, the data could be numbers from ANY study.

I teach Biostat PhDs how to analyze data for psychiatric health services research. Many of my publications are in the field of diabetes and its intersection with psychiatry. Universally, not only do the PhDs know NOTHING about the terminology, theories or DSM coding for psychiatry but IN GENERAL they NEVER question the PIs. NOT AT ALL. This type of blind acceptance of authority is problematic.

Now add in the scientific REP from the DRUG Company. This individual's role is to collect a portfolio of research projects that positively promote their products. These individuals GRAB the forms or have them uploaded as soon as the patient completes them. THEY OFTEN DON'T LET THE PIs EVEN KEEP THEIR OWN DATA, let alone SEE the data from the other experimental sites (if there are ones).

I have argued often with these people and have been allowed to see the data. Then I have hours and hours of phone conferences while I try to tell them how to do the data correctly. Then it usually gets shot up the ladder in the DRUG Company to the top OLD MAN who is in charge of data. This MAN is likely to have NEVER read another stat paper in years, so is working about 20 years behind the time. SIGH!!

On the other hand, I give everyone a hard time, and submit their data to rigorous exploration and analysis. It is for that reason that RAND gets so much business at a very high price. Most researchers don't want a person like me reviewing their study. Because I tell it like it is, just like these Editors... bravo!!