With the gatslag of the announcement that the AstraZeneca vaccine does not work against the SA variant, the health department seems to favour the J&J vaccine since its SA trial leg encountered the SA variant which none of the others did. However, when you unpack the numbers, it is not hopeful as the effective sample size is exceedingly small. I posit, with good reason, that the effective sample size on which they base their unimpressive efficacy of 57% is 49!
I think that SA was correct to suspend the roll out of the AstraZeneca vaccine pending a better understanding of the problem with the SA variant. The authorities previously dropped the ball when they didn’t assiduously pursue vaccine alternatives from early on and they even managed to miss their Covax payment twice. The ‘failure’ of the AstraZeneca vaccine does not justify their torpitude and it is doubly imperative that we urgently pursue all options to guarantee supply and efficacy.
However, the fact that the latest crisis seems to have lit a fire under their bums is not cause for celebration as there is little out there that offers any guarantee that it will work against our variant.
I hear that the authorities have opened negotiations on the Sputnik V vaccine. Well, that is a non-starter. Although it is likely to get WHO certification, it has not been tested against the SA variant at all as far as I know.
The J&J solution is the only one that has been tested against our prevalent variant but it is actually quite a small study. Although I cannot get the detailed results so far, that which I can get are not very promising when you look at it properly.
It’s all to do with how the test results are interpreted. I am no statistician, but I did do and still remember enough stats to say that the primary efficacy quoted is simple algebra and the only time when sophisticated statistics are required are when you have to work out the confidence limits as you extrapolate the small sample to the whole population. That is beyond my ability, but we practically know how wrong statisticians can get it. After all who predicted Trump would have won in 2016.
Also, what is not realised by Joe Soap is just how small that efficacy sample actually is. Let me explain.
When testing a drug to cure a disease rather than a vaccine whose function is to prevent it, the placebo is used to take out biases and the placebo effect. However, when the trial is for a vaccine, the placebo has a further function. Since it is unethical, to put it mildly, to directly infect people with the vaccine to measure its efficacy, the placebo is used to measure how many people in a random sample have been exposed to the infecting agent. For example, 10,000 people receive the placebo and 10,000 receive the vaccine. If, say, 100 people are infected in the placebo group, then we would expect, with a fair degree of confidence, that 100 people would also have been exposed to the agent. The fact that only, say, 10 people actually got infected in the vaccine group leads us to surmise that the vaccine prevented 90 infections out of a potential 100 and is therefore 90% effective.
Therein lies the rub. We started off with what seemed like a large sample group of 20,000 but we actually ended up with a very small presumed sample group of 100 on which we based our efficacy calculations!
Furthermore, since that has to cover people of different ages, sexes, ethnicity, co-morbidities and pre-existing diseases like AIDS, detailed analysis becomes a bit threadbare. That is precisely what led to the claim by a German newspaper that the AstraZeneca vaccine was ineffective against 65+ year-olds. At first, I thought that this was just national pride talking, but, looking at the numbers, they were spot on and hence EU said non to use on 65+ year-olds.
In their 65+ cohort, they had 1 infection out of 319 placebos administered. On the vaccine leg, they had 340 patients and would have expected 1.069 infections but they only got 1! The efficacy is then calculated as 0,069/1.069 = 0.065 or 6.5% which is what the newspaper claimed.
Obviously, the sample size was way too small and it wasn’t 659 or 340 or even 319 but a presumed sample size of 1.069! To show just how ridiculous small sample sizes are, imagine if no one was infected on the placebo leg but still the 1 on the vaccine leg, one would be forced to conclude that the vaccine actually gave the patient the disease.
For the J&J trial in SA, we are not talking about ridiculously sample sizes like that when we look at it from an overall level, but nevertheless they are small and call into question whether you can draw enough conclusions from the trial.
Unfortunately, a lot of data is missing at the moment but what we do know is that 37207 were tested in the Americas and 6576 in SA of which 95% of the people infected had the SA variant. There were 468 infections overall at day 28 after vaccination. They further state that the SA leg demonstrated a 57% prevention of moderate to severe illness*.
If we were being infected at the same rate as for the Americas then we would expect to have had 70 infections in SA. This is not a terrible assumption to make as the trial was conducted before our rampant 2nd wave kicked in while both south and north America were having a pretty torrid time towards the end of last year.
If we not unreasonably presume that equal numbers of people received the placebo and the vaccination then, for an efficacy rate of 57%, we would have had only 49 infections in the placebo group and 21 infections in the vaccine group.
First of all, these are not great numbers to boast about – you’ve only roughly halved the problem at great expense and when you still have 43% of the vaccinated people getting moderately to severely ill, people will lose their belief in the vaccination.
Secondly, the presumed sample size for the efficacy measurement is a very small 49 and there is a lot of room for error particularly when you try to slice and dice that into all the sub-groupings.
Thirdly, the presumed sample size is actually even smaller than I presumed because the overall 468 infections included the mild infections and not just the moderate to severe infections*. When they are parsed out, the group that we are trying to compare to for efficacy calculations is smaller than 49.
As things stand at the moment, J&J is not the solution to our problems. But even less so, the Russian and Chinese vaccines whose trials included no cases of the SA Variant.
* Moderate roughly corresponds to hospitalisation and severe corresponds to ICU.