Highly publicised but tragically flawed

Sep 11, 2015 by

BY ANDREW MULLINS –

Research in a leading journal claims that there has been a consensus on same-sex parenting for more than 20 years. Writing in this month’s issue of Social Science Research, jimi adams, of the University of Colorado Denver, and Ryan Light, of the University of Oregon, claim that there is “overwhelming evidence that scientists agree that there is not a negative impact to children of same-sex couples”. Furthermore they have detected what they say demonstrates an emerging consensus in the 90s and an  “overwhelming consensus” by 2000.

This has been widely reported as the “most comprehensive review” of studies on same-sex parenting. As the ThinkProgress site headlined it, “New Study Shuts Down Common Claim Against Same-Sex Parenting”.

The authors conclude that “The scientific community examining outcomes for children of same-sex parents has achieved consensus, and the consensus is that children of same-sex parents do not experience comparative disadvantages on important outcomes from children in other parental configurations.”

But how did they “prove” that this consensus exists?

adams and Light utilised a 2010 methodology developed by Uri Shwed and Peter Bearman to evaluate consensus on issues such as on solar radiation and cancer, carcinogenicity and smoking, and autism and vaccines. All these involve medical studies with hard data. But for social science data, this approach is deeply flawed.

Unlike what constitutes a parenting disadvantage, there is no squabble over definitions of cancer; cancer does or does not exist. To apply this instrument to soft research in the social sciences, and to reach a definitive conclusion, must be either a spectacular achievement, statistically over-ambitious, or a sleight of hand.

Such consensus-identifying methodology should not have been used in this context. Shwed and Bearman proposed this statistical method for analysis of temporal patterns in scientific citation networks. Simply speaking, the more a paper is cited by other scientists, the more scientific consensus is evident.

adams and Light seem to believe that this methodology is valid in their own field of interest, yet this is unwarranted. For example, they themselves, in their own paper, cite numerous times the 2012 papers of Mark Regnerus and of Loren Marks, strident critics of same sex parenting. Citation certainly does not necessarily equal consensus.

A method designed for consensus in “hard sciences”

In a 2012 paper, Shwed and Bearman allude to the fact that their work is designed for analysis of citations in the strictly scientific realm.  Bearman is a social scientist but his work is in the area of clinical trials. A 2014 statistical reference text insists that the work of Shwed and Bearman is only applicable to science.

When its very statistical basis was criticised by Bruggeman et al in 2012 because it failed to distinguish between positive and negative citations, Shwed and Bearman responded by saying that their method was only applicable in scientific contexts not in popular controversies, further suggesting that the paper of Adams and Light is a substantial misapplication of the approach.

The methodological choices made by Adams and Light also seem puzzling. In the absence of “definitive consensus statements” as required by Shwed and Bearman, Adams and Light opt for subjective choices that are therefore problematic “ … as a means to interpret the content of consensus on outcomes for children of same sex parents.”  This is completely unacceptable. They explain that because there is an absence of definitive consensus statements in this field (such as the Surgeon General’s finding on the carcinogenicity of smoking that Shwed and Bearman cite) they rely on subjective identification of “landmark moments indirectly connected to … consensus”: a statement from the American Bar Association, which obviously has no expertise and no new data to offer in making any statement about same sex parenting.

A flawed understanding of meta-analysis

Furthermore, as a meta-analysis, the underlying tenets of this study are questionable. Meta-analysis design is a notoriously difficult. As an expert in the field, Mark Russo, has written: “If it is well conducted, the strength of a meta-analysis lies in its ability to combine the results from various small studies that may have been underpowered to detect a statistically significant difference.” He insists that “the study design of a meta-analysis determines the validity of its results.” Yet the goal of Adams and Light is to demonstrate a lack of difference, thus stretching meta-analysis beyond its competence. That, on a given meta-analysis, a significant effect is not evident cannot be taken as proof that that effect does not exist: it is far easier to point to something that exists than to show it does not exist.

What one social science researcher regards as significant, another dismisses; that a meta-analysis is seeking to distil information from a wide range of approaches leads to further challenges. The uproar over the work of Mark Regnerus, of University of Texas, demonstrates this difficulty in establishing agreed or coherent parameters for investigation in a social science meta study.

Regnerus screened down from 15,000 young adults 18-39 to identify 163 who said their mothers had had a same sex relationship at some stage of their childhood, and 73 who said this of their fathers. In addition to the comparative family instability of homosexual households he demonstrated that children raised in a homosexual household were more likely to identify as homosexual and have had a same-sex relationship. His strident critics Cheng and Powell (2015) agreed substantially with these findings but dismissed them. As one anti-Regnerus commentator put it: these outcomes should “not considered to be objectively worse”. Hence Regnerus can be shown to have found no negative outcomes for children of same sex households.

Because data is so open to manipulation in a meta-analysis, authors need to be meticulously objective, at arm’s length from findings. According to Reuters, Light says, “I hope we’ll see acceptance of gay marriage of the courts and by the public at large.” I am not suggesting a conscious bias, but there is abundant evidence now that implicit bias can skew thinking and the question must arise as to whether such an implicit bias may have contributed to the assumptions underpinning the Adams and Light paper.

Over-selling “science”

It seems puzzling, either an oversight or deliberate, that the word “scientific” appears over 50 times in the Adams and Light paper. Regnerus, in comparison, despite a heavy statistical presence in his paper, does not once use the word. In recent literature supporting same sex parenting however the word is in abundant evidence. For example, Perrin et al (2013) in the Journal of Gay Lesbian Mental Health subtitled their article “A scientific evaluation of the no-differences hypothesis”. In one press article, Light is quoted: “We hope that these tools help judges and lawyers weigh the state of agreement on scientific issues before the courts.”

This is a paper out to prove a point.

Let us hope that such a flawed paper did not influence the US Supreme Court in its late June ruling. The facts are that the majority decision did link same sex marriage and parenting. Justice Anthony Kennedy wrote, “Without the recognition, stability and predictability that marriage offers, children suffer the stigma of knowing their families are somehow lesser. … The marriage laws at issue thus harm and humiliate the children of same sex couples.”

Was Justice Kennedy impressed by the “overwhelming consensus” that existed in the literature about the outcomes for same sex parenting? And even if he was, as Shwed and Bearman themselves point out “consensus, of course, has nothing to do with ‘the truth’”.

Dr Andy Mullins, author of Parenting for Character and an occasional contributor to Mercatornet, is past headmaster of two Sydney schools, Redfield and Wollemi Colleges. In his doctoral thesis he investigated the neural substrates of virtue. He is currently working in Melbourne. He holds an adjunct professorial position at University of Notre Dame Australia. 

Source: Highly publicised but tragically flawed

Print Friendly, PDF & Email

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.