8. Hypothesis Testing, Experiments, and Predictions

 

Knowledge is expanded when we can verify or falsify a hypothesis. That’s because the experimental tests are constructed in such a way that the hypothesis is likely to be a widely applicable explanation of certain facts, rather than an isolated case. This sort of experiment is controlled, which means that the experimental setups differ by only one variable (see Mill’s method of difference). The experimental group is the one that gets the variable, while the control group does not.

Causal claims that result from experiments should reflect five criteria:

1.    There should be a correlation between cause and effect.

2.    The cause should not precede the effect.

3.    The cause should be in the proximity of the effect.

4.    A set of necessary and sufficient conditions should exist.

5.    Alternative explanations should be ruled out.

 

Now read the The Fallacy of Assigning Null Hypothesis and the article “FDA vs. Hydroxychloroquine” below

 

 

The Fallacy of Assigning Null Hypothesis

 

Criminal-court example. A good example that demonstrates the importance and complexity of the choice of the null and alternative hypotheses is the criminal court cases: The defendant is assumed not guilty unless proven otherwise beyond a reasonable doubt. This is the well-known “presumed innocence.” In the language of hypothesis testing, the “not guilty” hypothesis should be the null and the “guilty” hypothesis should be the alternative. That is mainly because the mistake of convicting an innocent person (type I error) is usually considered more serious than setting a criminal free (type II error).2 “Beyond a reasonable doubt” can be formulated as “type I error probability cannot exceed a small threshold,” although this threshold may have been set so small that type II error (setting criminals free) probability is too large.

 

FDA vs. Hydroxychloroquine

If you use hydroxychloroquine to treat people with COVID-19 you put them at increased risk of “serious heart rhythm problems and other safety issues, including blood and lymph system disorders, kidney injuries, and liver problems and failure.”

Not only is this drug unlikely to kill or inhibit the virus, it will not decrease “the likelihood of death” nor will it will is speed up recovery.

How ever it should be noted that a recent study from the Journal of Infectious Diseases (I’ll call it the “hydroxychloroquine study”) reported

“…from admission (into the hospital) to receipt of hydroxychloroquine was 1 day (1–2). Overall crude mortality rates were 18.1% in the entire cohort, 13.5% in the hydroxychloroquine alone group, 20.1% among those receiving hydroxychloroquine + azithromycin (azithromycin was possibly used for ensuing pneumatic issues), 22.4% among the azithromycin alone group, and 26.4% for neither drug.”

 

Others studies have shown no statistically significant results. The authors or this study argue that

“In contrast, in our patient population, 82% received hydroxychloroquine within the first 24 h of admission, and 91% within 48 h of admission. Because treatment regimens likely varied substantially (including delayed initiation) across the 25 hospitals that contributed patients to the study, it is not surprising that the case-fatality rate among the New York patients was significantly higher than in our study.”

Having quoted this it should be stated that their are problems in this study. According to some researchers calling out possible bias in the previous quote;

“This statement neglects the extensive statistical adjustment for between-facility variation in our publication, the generalizability benefit of including 25 hospitals into the cohort with differing therapeutic protocols and approaches, and it misrepresents the fatality rate in our study. We reported 20.3% (95% CI: [18.2–22.4%]) fatality from 292 deaths in 1,438 patients, whereas Arshad et al. report 18.1% from 460 deaths in 2541 patients in a later era of the COVID-19 epidemic. We fail to find a difference between these studies’ fatality rates both practically and statistically….We underscore the concerns raised that bias may have been introduced into the study’s design by reserving the combination of hydroxychloroquine and azithromycin for patients with “minimal cardiac risk factors.””

The researchers from the hydroxychloroquine study intentionally gave drugs to those who they thought they could save to make their numbers look hire and the patients were also given a steroidal treatment. its questionable how much the hydroxychloroquine actually did.

Also remember that the hospitals, all 25 of them, probably had different treatments that the hydroxychloroquine study didn’t account for.

What’s interesting is that most of the people in this study are in the high risk group. (Of course they test us when we are at our most vulnerable. Having said that, people more likely to die in this study were older white people.). I’d caution anybody arguing for this as a cure. Remember that patients were given a dose that wasn’t supposed to explode their heart, and they were in a hospital setting and not the Walgreens flu drug aisle. They were given the drug early and I still wonder about the severity of those particular cases (If they would have recovered with hospital care alone.) Also, they were probably being watched more closely, which probably also increased their chances of recovery. This study was done under very specific circumstances, it was obviously biased and one cannot disregard the other studies that contradict it.

To put it bluntly ‘it’s a bad study.’

 

 

Consider the Criminal-court example. Then answer both questions

 

1.    The lynching of Jesse Washington^[1]. In 1916 Jesse Washington was lynched  mutilated and set a blaze. Jesse, a black man, plead guilty to the rape and murder of Lucy Fryer, a white woman. After the trial, in Waco Texas, he was lynched by a mob. Their were problems with Jesse’s testimony. Jesse suffered from possibile intellectual disabilities, plus he was illiterate, and its very likely that the murder weapon was planted. Because the evidence was weak it seems like the jury commited a type I error.

Considering the example above - is it better to let three guilty people go free or to kill, or imprison, one innocent person? - Don't think of it in terms of "we know this person did this crime" but it is likely that they could have. I think if you think about it in these terms it might make you more amenable to this concept. Again I don't like letting someone go for a possible crime, but it’s better than arresting someone else for a possible crime. Imagine if you are a police officer, you have some weak evidence that someone did a crime do you gather more evidence or do you arrest them? I think this may be a better question.

If you arrest them you risk locking up an innocent person, but there is a chance that they may be guilty. If you collect more evidence this could go either way, but let’s say that you don't have a lot of time, you don't have enough time to get more evidence. Do you arrest them?

2.    Give the mixed results from trials with Hydrochlorquine should we risk a type one error by advertising it as a cure?

 

---

[1] Bernstein, Patricia (2006). The First Waco Horror: The Lynching of Jesse Washington and the Rise of the NAACP. Texas A&M University Press. ISBN 978-1-58544-544-8.