Lancet Researchers Unresponsive...

While I hesitate to read anything into it, I have yet to recieve a response to my request of the Lancet researchers that they provide me with the underlying statistical paper that backs the one recently published in Lancet. In order to conduct any form of replication, at minimum I'll have to see the models they estimated. But no word yet. Will update if anything changes.

Lancet Study Bombshell

Steven Moore explodes the Lancet Study in the WSJ. Not only are their cluster points small (leading to the vulnerability I mentioned earlier regarding the impact of outlier clusters), but apparently the authors collected no demographic data on their respondents. This is unbelievable. I have to say, it never even occurred to me that they wouldn't have collected demographic data on their respondents. It's like questioning whether a mathematictian remembered to carry the one in calculating an addition problem. It is so basic and fundamental that you simply presume it is there. If someone is going to conduct a study based on a survey, they have to collect demographic data on their respondents. They have to. This isn't debatable. It isn't some methodological controversy among survey researchers. Every survey researcher collects demographic data on their sample. Besides the fact it is often a subject of interest, it is absolutely necessary in order to externally validate a sample. If your sample doesn't reflect population statitiscs (say, it over-samples women for some reason) then you correct and report accordingly. If we ask the Lancet authors, "So, what level of confidence do you have that your sample reflects the true population"...an honest answer would have to be, "we have absolutely no idea." I can't understate how huge this is. There is no way to externally validate their sample without demographic data. For all we know, their respondents may all be 20 year old Sunni males....or 40 year old Shiite females...and they can't tell us otherwise because they didn't collect the data! This is amazing. Either these guys don't know what they're doing...or they are being duplicitous (i.e. their sample isn't representative and the demographics would prove it). I would normally hesitate to introduce the possibility of dishonesty (most often errors are the consequence of shoddy methods or lazy applications)...but this 'oversight' shines like the sun.