Let’s cut to the chase, shall we? This thread results from a discussion/debate you had with Robbie Canuck in the Doping in Other Sports thread. This is what Robbie said there:
You clearly do not understand the difference between a study that uses random assignment and a study that does not. Goldman's study was not a properly constituted design with random assignment, and therefore fatally flawed. Furthermore it did not control for answers from athletes versus non athletes.
You are missing the point that when the study is carried out properly it had diametrically opposite results to Goldwin's. I am lost as to the fact of why you do not seem to grasp this fundamental distinction.
The sports blog of Dr. Roger Pielke Jr. explains it further,
"A new study in the British Journal of Sports Medicine by James Connor, Jules Woolf and Jason Mazanov has tried to replicate Goldman's findings with a rigorous survey of elite North American track and field athletes. They were motivated to do the study because, "there has been little in the way of replication of the Goldman dilemma since 1995."
Titled "Would they dope? Revisiting the Goldman dilemma" the study finds results at odds with that of the Goldman dilemma:
Only 2 out of 212 samples (119 men, 93 women, mean age 20.89) reported that they would take the Faustian bargain offered by the original Goldman dilemma. However, if there were no consequences to the (illegal) drug use, then 25/212 indicated that they would take the substance (no death condition). Legality also changes the acceptance rate to 13/212 even with death as a consequence. Regression modelling showed that no other variable was significant (gender, competitive level, type of sport) and there was no statistical difference between the interview and online collection method.
Goldman’s results do not match our sample. A subset of athletes is willing to dope and another subset is willing to sacrifice their life to achieve success, although to a much lesser degree than that observed by Goldman."
Here is what Connor et al.
actually said about the Goldman study:
The first weakness of Goldman’s work is that no comparable measure of acceptance exists among the general population. That is, there are no data to suggest whether the athletes are responding in the same manner or differently to members of the general population.
I’m sorry, but why is that a weakness? Why do athletes have to be compared to the general population? Goldman was interested in how athletes, who might actually be in a position to act out their beliefs on this issue, would respond. There is no need to query non-athletes, and it’s not clear how their answers would be relevant to those of athletes.
The second weakness in Goldman’s work is found in the wording of the questions. The question presented the outcome (Olympic gold) followed by the consequence (death). As Connor and Mazanov4 assert, ‘Goldman dilemma responses may represent a positive response bias as a function of wording, necessitating replication using the counterbalanced presentation’ (p. 872). The current project tested whether the counterbalanced presentation identified effects in terms of substance legality (legal vs illegal), mortality (death vs non-death) and order of presentation effects (outcome vs consequences).
Again, why? Why does the question have to be worded in a different manner? The point of the Goldman study was to show that many athletes answered a certain question about entertaining the Faustian bargain. Maybe if the question were worded differently, the % would change, but so what? In the absence of being able to show that a particular wording of the question is better correlated with what the athletes would actually do—which neither Goldman nor Connor studied, because they couldn’t—what difference does it make? The salient point of Goldman’s study, which seems to be lost on Connor et al., is to provide a glimpse into the minds of athletes, not determine exactly how many would carry out some act. That so many would answer positively to this question, however it was worded, is remarkable.
The third potential weakness in Goldman’s work is the use of the question method.1 Goldman’s initial study saw athletes verbally answer the question while in attendance at events or training. However, there is little discussion from Goldman on the method of recruitment or how representative his samples were, and to our knowledge the studies have not been peer- reviewed. The biases associated with this method are well known in terms of interviewer effects (eg, confirmation bias), respondent effects (eg, faking good or bad) and setting effects (eg, other people may over- hear responses).
Sure there were biases possible. But in the first place, the biases were likely to decrease not increase the number of positive respondents. If you really thought other people would overhear your response, would you be more likely to say you would dope? Seriously?
And secondly, Connor et al. don’t seem to understand that their own method also introduces a bias:
A research associate solicited athletes after an event (by chance) near the track and invited them to participate in exchange for a sealed bottle of Gatorade. A walled area was set up. Participants were informed of the ethics of the experiment and asked for consent, then randomly assigned into one of two conditions, both in private screened off areas. In the first condition, participants were asked survey questions by a trained inter- viewer, replicating the procedure used by Goldman. The second condition was an anonymous online survey (utilising Fluidsurveys) in a secluded area with a laptop. The questions were identical across administration conditions. The instructions were scripted.
What this method does is explore the minds of athletes in a relatively quiet, reflective environment. But is this the kind of environment in which they make the decision to dope, or is it in the heat of the moment, the locker room environment the track, the road, when you are actually competing? How do they know that answers in this environment—which is much more like a classroom, where athletes might be much more likely to say what they feel is the right thing to say (doesn’t merely raising the issue of ethics bias the result?), than what they are actually going to do in the heat of competition—are more representative of what they might actually do?
The authors found that there was no difference using various wordings of the question, or whether they used a live interviewer or the online method. They seem to accept the validity of Goldman’s findings, in general terms—it would be hard to argue that if the real % of positive answers was a few % then, that Goldman had by chance selected a group of athletes with such a vastly higher proportion. They argue that the much higher % was a reflection of different times, different attitudes towards doping:
Put simply, the sensational reporting of the 1982 to 1995 responses to the Goldman dilemma is no longer relevant to the contemporary debate around the role of drugs in sport.
I don’t buy this conclusion as having much weight. It might be the case, and it might not be. I think the authors are really kidding themselves if they think most serious dopers would answer this question positively, particularly in the environment that the experiments set up. Just ask yourself this: If LA or any number of other dopers of recent years had participated in this study, what do you think their answers would have been?