UNDERSTANDING THE FLAWS
While it can be intimidating to look at pages of scientific jargon, with a little bit of information, it’s not impossible to learn to quickly spot fairly glaring issues. Often the same problems are repeated across the studies — one of the most obvious issues is that we don’t have long-term, quality studies on the specific population that mirrors our current group of young people seeking these interventions — teens with longstanding and/or multiple mental health diagnoses, neurocognitive challenges like autism or ADHD, a history of being bullied or abused, a history of trauma, discomfort with sexuality, etc. Conclusions drawn for one distinct group cannot necessarily be applied to a completely different group. How outcomes are measured also matters. For instance, reviewing pharmacy records to track whether study participants were prescribed medications is a more accurate measure of data than asking patients to recall this information.
Aside from flaws in design, other problems find their way into studies. These problems can result in incorrect conclusions, and studies being less meaningful than perceived.
NOT MAKING THE GRADE
While a number of studies purporting to show that outcomes of patients on “gender-affirming care” interventions have been conducted, every systematic evidence review finds the data to be unreliable and therefore inadequate, given the tremendous risk of serious harm. Evaluated using the GRADE system, which measures how well you can estimate the outcomes from giving a treatment, the studies are all rated "low" or "very low quality." These are not simply opinions of their worth, but devastatingly poor evaluations in terms of whether the studies can reliably predict patient outcomes.
Low Quality: Confidence in the effect estimate is limited — the true effect may be substantially different from the estimate of the effect
Very Low Quality: Confidence in the effect estimate is low — the true effect is likely to be substantially different from the estimate of effect
In other words, using the study to estimate the likely effects from the intervention might not be reliable. Or is probably not reliable. For more information about the GRADE system of assessing study conclusions, see GRADE guidelines: 3 Rating the quality of evidence.
THE DEVIL IN THE DETAILS
Below is a list of common flaws in either how a study was designed or how conclusions were drawn. Becoming familiar with these flaws will help you understand the limitations of what we actually know about the safety and efficacy of the “gender-affirming care” model (See these flaws in action.):
Given the high stakes for those considering embarking on the “gender-affirming” medical pathway, reliance on shaky, biased data is a dangerous bet that no one should accept.