How not to do science...

While perusing through the health tab on Google news this morning (why, oh why, do I inflict this on myself?), I came across a group of articles with headlines that caught my attention. They all dealt with a newly released study that showed First Nations babies in Quebec were suddenly having much higher birth weights than "normal" and that these bigger babies were more likely to die.  Eventually, I found the actual "study" these stories were based on.  Rather than link to a bunch of articles, I'll just link directly to the pdf.

What a horrifyingly bad example of what passes for "science" these days!

To read the news articles, First Nations babies in Quebec are suddenly being born much heavier than normal - the heaviest in the world, in fact (at least as for as the authors of the study know) - and that these "macrosomia" babies are at much greater risk of dying.

How was this determined?  Have medical professionals in First Nations communities noticed these higher incidents of vastly increased birth weights?  Have they expressed concern because so many of these kids are dieing?  Has some strange malady emerged that is causing babies to develop into abnormally large size?

The answer to all of the above would be "no."

It turns out these researchers didn't talk to a single First Nations person at all, nor did they talk to anyone who actually medically treats a high number of First Nations women and their babies.

The study is a data dredge.

That, in itself, is bad enough (and to be fair, in the paper, they did acknowledge some problems with this method).  There are so many other things that make it worse.

First, let's look at what "infant macrosomia"actually is.  In this study, it's simply babies greater that 4 kg in weight.

That would be 8.8 pounds.

The average birth weight for a healthy, full term, baby is between 7 and 8 pounds (males average slightly higher than females).

Low birth weight has long been the primary concern; in fact, reducing the number of low birth weight babes is often used as a measurement of a nation's success in improving the general health of the population.

Here, they're talking about babies in the 90th percentile or higher.

For the record, my own kids were born at 9 pounds and 9 pounds 6 ounces, which put them in the 95th percentile on the growth charts.  I grow big babies.  This was considered a good thing, back then.

Now, there can indeed be concerns in regards to unusually large children, when it can be shown that this size is in some way unnatural and harmful.  For example, if the mother had T2 diabetes, or some other health complication that can affect the growth and development of the fetus.  Does this study factor in any of this?

No.

They do know that diabetes rates tend to be higher among First Nations, but that's not something they looked for in this study.  They didn't look at maternal health at all.  They only looked at birth weights.

Oh, then we have another issue.

In the first article I found, I was astounded to read that they were comparing First Nations births in Quebec with births to women who's mother tongue was French.

They were comparing an ethnic group to a lingual group.

On finding the paper, however, I discovered this report was wrong.  They were actually comparing ...

... two lingual groups.

Since this was a data dredge, they defined who was First Nations and who wasn't by what was listed as the mother's primary language.  (They excluded women who spoke Inuit.)  They do go on to describe how this might not be all that accurate, since some First Nations women in Quebec might actually have French as their mother tongue, but figured that this worked out all right, because they had a much larger group of French speaking women (over 600,000) to compare against the First Nations language speaking group (over 5,000).

So it wasn't an ethnic group being compared to a lingual group.  It was two lingual groups being compared to each other, and one of those groups is about 120x larger than the other, which made it okay.

At this point, the study is already useless.  We have one group defined as First Nations, but we don't know if they actually are First Nations; we only know the mothers spoke a First Nations language.  Even if the mothers were actually First Nations, mothers are only half of the genetic equation; fathers aren't even mentioned.  Then we have another group defined by French being their primary language.  Again, we don't know the actual ethnicity of these mothers.  We don't know anything about maternal health.  We don't know anything about where they live, and if that might be affecting them (though the paper does speculate that First Nations women isolated enough to not have French as their mother tongue might also be isolated enough to not have access to medical care - more on that later).  Not only is it a data dredge, but the definitions of who fits into the parameters is essentially arbitrary, and there are so many factors not taken into account, the whole thing is essentially rendered meaningless. 

Ah, but that didn't stop our intrepid researchers!

On to the next point!

Let's look at the parameters they use to determine that bigger babies among "First Nations" women are faring worse.  It's pretty straightforward.  They look at the death rates, with postneonatal deaths defined as between the ages of 28 days and 1 year.  They determined that "macrosomia" babies have postneonatal death rates 8.3 times higher in the "First Nations" group than in the "French" speaking group.

So now we're looking, not at health, but at mortality.  We don't actually know anything about the health of any of these babies, just that X number of them died.  We don't even know how or why they died, other than SIDS, which has its own category.  For all we know, some of these babies died in accidents, not because of ill health.  Also, since we're only looking at mortality rates, we don't know if the bigger babies that didn't die are at all unhealthier than their smaller counterparts.

They did try to factor in the remoteness problem, acknowledging that lack of timely medical care could lead to increased deaths.  How did they do this?

They defined a remote area by looking at census data for how many people living in the postal code areas listed for the mothers commuted to urban centres for work.

*facepalm*

As the paper is concluded, the authors talk about such things as how we know about the higher rates of diabetes among First Nations, so improving treatment for mothers might help reduce incidence of macrosomian among First Nations babies - except that they didn't actually have any data about diabetes or glucose tolerance.

Then they say more research in needed.  Yeah... and more research grants, I'm sure!

When I first started reading about this, and how it was talking about there was an increased trend towards macrosomia (what a strange word - it sounds like a sleep disorder) among First Nations babies in Quebec, I assumed it was actually recent.  It turns out that all these numbers they looked at are for births from 1991 to 2001.  They explain why in the paper.  The babies in this data dredge that didn't die in their first year would be turning anywhere from 10 to 20 years old this year.  It would be safe to assume at least some of these children have died in the last one to two decades.  There's nothing to tell us these bigger babies that didn't die in their first year have any more or less health problems than average sized or smaller babies, just as we know nothing about their actual health in their first year; all we know is that they didn't die.

Oh, and the actual number of larger babies dieing before their 1st birthday in the space of ten years?

Nine out of 1,424.


In the same time period, for postneonatal babies in the 10-90th percentile, 19 out of 3, 542 died.


Adjusting to per/1000, that's a change from 5.4 deaths of average sized babies (though 10-90% is a rather huge range to compare against) to 6.3 for larger babies.


The difference between postneonatal mortality rates in the 10-90th percentile vs the greater than 90th percentile works out to be an increase of .9/1000 births over the space of ten years.


One would suspect that there's far more than infant size that's causing the disparity in death rates between the First Nations language group and the French language group. 


Based on all this, the authors state "Parents and caregivers should be vigilant about the substantially elevated risk of postneonatal death among macrosomic infants of First Nations women."  However, their data also shows that being bigger might actually have protective qualities for perinatal babies.  So... up to the first 28 days, being bigger is good, but after that 28 day mark, their size is suddenly a risk factor?

There is so much wrong with this "study," it was hard to actually read it.  The entire foundation it's based on is meaningless.  They could just as easily be telling us that speaking a First Nations language leads to higher infant mortality rates.  This, however, seems to be what passes for "science" these days.