Bird flu vaccine strength could be under-or overestimated due to test variability

June 27, 2008, Toronto – A study comparing the tests being used by vaccine manufacturers to gauge the effectiveness of their H5N1 avian flu vaccines shows there is a lot of variation in the sensitivity of the tests, the British scientist leading the effort says.

Differences in the sensitivity of the tests mean companies could be underestimating or overestimating the power of their vaccines as they try to work out what is the smallest protective dose, experts admit.

As things stand now, there is no way to usefully compare the results of one company's clinical trials for their vaccine with a competitor's findings.

“If Company A's assay (test) happens to be 10 times more sensitive than Company B's, Company A and Company B could be evaluating the exact same thing but reach different answers about whether they worked or not,'' says Dr. John Treanor, a vaccine expert who knows of the study but is not involved in the work.

The effort, which included trying to develop an international standard against which H5N1 antibody tests could be measured, is being led by Dr. John Wood, a prominent influenza virologist with Britain's National Institute for Biological Standards and Control.

“The purpose of this is not to assess one lab against another.

It really is to develop a consensus on the values that are obtained,'' Wood explains.

“It allows WHO and governments and anyone, really, to make the comparisons.''

Treanor, who has conducted several trials of H5N1 vaccines, says Wood's work is critical to evaluating the various products in development.

“If you're going to set as the (regulatory) goal a specific criteria (for protection), then it is very important, I think, that when you're measuring that criteria you're using the same instrument for different vaccines,'' says Treanor, head of the infectious diseases division at the University of Rochester Medical Center in Rochester, N.Y.

Since H5N1 hit the global public health world's radar in a big way in early 2004, the pharmaceutical industry has been working to develop vaccines against it. Several companies have even pushed products through the regulatory process – Sanofi Pasteur has licensed an H5N1 vaccine in the United States and GlaxoSmithKline has licensed one in Europe.

But each manufacturer is using its own process. Some companies are making vaccines made from broken up or “split'' viruses – that's the way seasonal flu shots are made. Others are using whole viruses, which are thought to induce a stronger immune response.

Most are using proprietary adjuvants, chemicals that boost the power of a vaccine.

As they've released results of their clinical trials, other differences have emerged.

Some vaccines seem to induce what is thought to be a protective response with as little as two doses of 3.8 micrograms of vaccine.

Others have required two doses of 10 mcg or 15 mcg. The first vaccine to be licensed, an unadjuvanted vaccine made by Sanofi Pasteur, required two shots of 90 mcgs apiece – a massive and impractical amount of vaccine to use on each person.

Given that vaccine demand will far outstrip supply in the next pandemic, finding the smallest possible dose that could be used is a key goal of each vaccine maker.

But there is no standard test to gauge the efficacy of H5N1 vaccine. Each manufacturer has developed its own test; so too have the academic and public health laboratories that test for H5N1. And differences in the design of the tests – scientists call them assays – can lead to different findings.

“All of those assays have a million steps that all are subject to various things that could impact the result,'' Treanor says.

“They might incubate things a little longer, or shorter. They might add one thing first and another thing second. There are all sorts of things like that that can inherently change the answer, which would make it difficult to standardize.''

It's as if each lab has to weigh something in kilograms on an old-fashioned scale, but there is no such thing as a standard kilogram weight to place on the one side of the balance. So each company moulds its own, assuming that its kilogram actually weighs one kilogram.

Wood's project, which he undertook at the behest of the World Health Organization, is the first looking to see if the various tests are all roughly assigning the same value to the same thing.

They are not, he found.

“I think I can say, yes, there was a lot of variation,'' says Woods, who won't go into specifics. He notes, though, that the result isn't a surprise _ an earlier study he did comparing the various tests for a human strain of flu found a great deal of difference among those assays.

Wood is outlining his findings in a report for the WHO that will be presented at an expert meeting in October.

Treanor expects the results will lead to some fine-tuning of the understanding of dose size. As well, he believes they will provide clearer answers to questions about the benefits of different manufacturing processes, such as whether whole-virus vaccines actually can be given in smaller doses than split-virus ones.

“I think the broad strokes messages would still be the case. But the precise definition of where the cut-offs are and whether you are or are not achieving a specific hurdle or guideline, that might vary,'' he suggests.