The Froome Files, test data only thread

[Alex Simmons/RST]

Re:

Burnley (like Andrew Coggan) is on the board of the JAP, the journal that published Coyle's article in 2005.

Putting aside whether or not being on the board matters in context of each published study' content, neither were on the board at the time.

 

[thehog]

Small community, I guess. Within banking and consulting there' is "external auditing", by which a non-banking entity will review your work and practices. After the Enron & subsequent banking fallout a significant degree of standards/regulation was applied.

Peer review by name states that a entity within the same industry as your own will review your work. That itself raises potential issues with independence and very reason why external review, i.e. like the Deloitte's review of Sky/British Cycling needs to be conducted.

Interesting discussion mind you; I'd be happy to be shown that academia has similar standards as the corporate world in terms of regulation.

 

[Alex Simmons/RST]

Cheats exist in all walks of life, and the process of conducting science can always be improved, but this attacking of science as if it's to blame for cheating cyclists is a bit of a stretch. Science is a very robust process (e.g. results need to be independently replicated for a body of evidence to be built) nevertheless there are things that can be done to help improve the level of robustness overall, including improvements in the standards employed by publishers of science. One of the biggest problems that is clearly misunderstood is just how hard performing good science is.

I think these items by Steven Novella sum up the issue quite well:
http://theness.com/neurologicablog/index.php/is-science-broken/
http://theness.com/neurologicablog/index.php/faking-peer-review/

 

[thehog]

Re:

Cheats exist in all walks of life, and the process of conducting science can always be improved, but this attacking of science as if it's to blame for cheating cyclists is a bit of a stretch. Science is a very robust process (e.g. results need to be independently replicated for a body of evidence to be built) nevertheless there are things that can be done to help improve the level of robustness overall, including improvements in the standards employed by publishers of science. One of the biggest problems that is clearly misunderstood is just how hard performing good science is.

I think these items by Steven Novella sum up the issue quite well:
http://theness.com/neurologicablog/index.php/is-science-broken/
http://theness.com/neurologicablog/index.php/faking-peer-review/

Low and behold, the second peer review from the Radcliffe study, Holden McRae from Peppidine, studied at the University of Cape Town, also Univeristy of Texas which was Coyle's institution along with Coggan.

Indeed it is a small world.

What are the chances? One study, which references Coyle with two peer reviews both intersecting with the Univeristy of Cape Town.

Holden S. MacRae
Professor of Sports Medicine
Coordinator of Sports Medicine
Division: Natural Science Division
Ph.D., Physiology, University of Cape Town, South Africa, 1991
M.Ed., Exercise Physiology, University of Texas at Austin, 1984
B.S., Sports Science, University of Cape Town, South Africa, 1981
B.A., English and Physical Education, University of Stellenbosch, South Africa, 1979

https://seaver.pepperdine.edu/academics/faculty/?faculty=holden_macrae

 

[Alex Simmons/RST]

Re: Re:

What are the chances?

That researchers study and work at universities with departments that specialise in specific fields? That would depend on the number of institutions that specialise in given fields of speciality I'd have thought.

It's like asking what chance is there that two pro cyclists might have ridden on the same team or with the same manager at some time in their career (and not necessarily at the same time).

People work at places of employment that exist in their field of employment. That should be self evident and nothing more.

 

[thehog]

Re: Re:

What are the chances?

That researchers study and work at universities with departments that specialise in specific fields? That would depend on the number of institutions that specialise in given fields of speciality I'd have thought.

It's like asking what chance is there that two pro cyclists might have ridden on the same team or with the same manager at some time in their career (and not necessarily at the same time).

People work at places of employment that exist in their field of employment. That should be self evident and nothing more.

Yes, I get that, the lines will transect at some point. That's why you make "declarations" in your paper, which is not seen here.

That being the case, the club is small, it suggests peer review really isn't as independent as its made out to be.

How is the peer selected? By one's self or independently assigned?

 

[acoggan]

I don't understand this proposition that acedemia is beyond corruption or "influence". Humans by their very nature are corruptible. Whether in banking, acedemia or otherwise. It happens.

Do you understand that throwing around vague comments and discussions about the nature of corruption in the context of this thread, without any specific evidence of such in this case is illogical, distasteful and really kind of dumb? No one argues that humans are corruptible. Entirely meaningless to bring it up. Is anyone saying academia is "beyond corruption or influence"? If they did I missed it.

Swart, by his own public statements seems biased. I see no evidence that this bias was bought or paid for, much less any evidence of a particular person or group doing it. My guess would be that his bias is nationalistic in nature but I really have no idea.

You're dancing around, throwing out words like "corruption" without making any specific accusations, but smearing people by using that kind of terminology. Not sure what you get out of this game, but it's the lowest kind of argumentation. Weak sauce.

I think you missed the part of the discussion where acoggan implied that Sky couldnt exert any influence on the University of Cape Town.
Which is where the Hog's comments about corruption came into play.
After all, acoggan's implication would be true only in some parallel universe where universities and scientists are incorruptable.

Apparently you were asleep when I emphasized how piss-poor all of pro cycling is with respect to what it would cost to influence an entire academic institution.

 

[acoggan]

Re: Re:

What are the chances?

That researchers study and work at universities with departments that specialise in specific fields? That would depend on the number of institutions that specialise in given fields of speciality I'd have thought.

It's like asking what chance is there that two pro cyclists might have ridden on the same team or with the same manager at some time in their career (and not necessarily at the same time).

People work at places of employment that exist in their field of employment. That should be self evident and nothing more.

Yes, I get that, the lines will transect at some point. That's why you make "declarations" in your paper, which is not seen here.

That being the case, the club is small, it suggests peer review really isn't as independent as its made out to be.

How is the peer selected? By one's self or independently assigned?

Are you really that clueless as to how academic peer review actually works?

 

[Alex Simmons/RST]

Re: Re:

How is the peer selected? By one's self or independently assigned?

You mean you are criticising a process that you clearly don't have much understanding of?

 

[Merckx index]

Re: Re:

How is the peer selected? By one's self or independently assigned?

This is actually a good question. In several journals I'm familiar with, authors are allowed to suggest several reviewers. In fact, AFAIK, this is the policy of most journals, at least in the general fields I've worked in. The candidate reviewer has to be very familiar with the area of study, of course, but he or she may be very sympathetic to the author's work, e.g., they may share a pet theory.

This is mainly because--this goes back to what Alex said about the small world--science has become so highly specialized that there are frequently very few scientists considered capable of adequately critiquing someone else's work. One of the cardinal rules of review is anonymity--the reviewer's identity is not supposed to be known. But in fact, we frequently knew the identity of one or more reviewers, because there were relatively few researchers it could be, and we were familiar enough with all of them that we could tell from what they said and how they said it who it must be.

 

[thehog]

Re: Re:

How is the peer selected? By one's self or independently assigned?

This is actually a good question. In several journals I'm familiar with, authors are allowed to suggest several reviewers. The candidate reviewer has to be very familiar with the area of study, of course, but he or she may be very sympathetic to the author's work, e.g., they may share a pet theory.

Thanks, I thought it was a good question, clearly the defensive mechanisms kicked in on some. Don't think it would be too hard to provide a clear response to a open question... thanks for your response, I understand it to be the same. It may vary country to country. Nevertheless the author does have the ability to make the selection or encouraged selection.

Granted, it being a small world, the author will become reviewer and vice versa over time. There doesn't appear to be external review of the process which leave it open to subtle forms of abuse.

 

[Alex Simmons/RST]

Re: Re:

How is the peer selected? By one's self or independently assigned?

This is actually a good question. In several journals I'm familiar with, authors are allowed to suggest several reviewers. The candidate reviewer has to be very familiar with the area of study, of course, but he or she may be very sympathetic to the author's work, e.g., they may share a pet theory.

Thanks, I thought it was a good question, clearly the defensive mechanisms kicked in on some. Don't think it would be too hard to provide a clear response to a open question... thanks for your response, I understand it to be the same. It may vary country to country. Nevertheless the author does have the ability to make the selection or encouraged selection.

Granted, it being a small world, the author will become reviewer and vice versa over time. There doesn't appear to be external review of the process which leave it open to subtle forms of abuse.

It's also a question that could be readily answered with a little of your own research before assuming it's the root of all evil. There are a number of peer review processes, not all are blinded, many are single blinded, some doubled blinded. Choice of reviewers and review process and how they are chosen varies. It's evolved over several hundred years.

Peer review is not perfect, and it's probably not the ultimate solution for quality control, but it's the best solution we have. There are some things that could be done to improve the overall QC process but this does not invalidate science. Especially when one is simply looking to generate a conspiracy theory where none exists.

 

[thehog]

Thanks. In more simple terms; it's an industry which regulates itself. A concept last seen pre-Lehmans 2008. Nevertheless, that's a larger question, better for an episode of Geoffrey Robertson's hypothetical.

 

[Alex Simmons/RST]

Thanks. In more simple terms; it's an industry which regulates itself. A concept last seen pre-Lehmans 2008. Nevertheless, that's a larger question, better for an episode of Geoffrey Robertson's hypothetical.

Science is not an industry. It's a process whereby the validation and ongoing questioning of prior knowledge and understanding is under constant review and challenge. Peer review is but one part of the validation and QC in science. Another is the validation of results by replication of experiment by others, which is why experimental details and results are published in the scientific domain so such validation can be undertaken. That's how a body of evidence is built.

Definition of science:

the intellectual and practical activity encompassing the systematic study of the structure and behaviour of the physical and natural world through observation and experiment

 

[sniper]

Thanks. In more simple terms; it's an industry which regulates itself.

a bit of a lax peerreviewing process would explain the extraordinary confidence both Burnley and Swart seemed to have in getting those Froome 2007 data published.

This was Swart on twitter:
"we've been asked by the scientists who collected that data whether we can publish it jointly."

To which Burnley added:
"there we have it - scientists who collected the data confident enough in it to publish"

Burnley all over the place here. It's an inexplicable comment from an independent science pov.

 

[sniper]

Apparently you were asleep when I emphasized how piss-poor all of pro cycling is with respect to what it would cost to influence an entire academic institution.

i don't think anybody (except you) is talking about 'an entire academic institution'. You're making unwarranted, strawman-ish, leaps.
The question is, could Sky exert some sort of influence on the UCT.
You seemed to discard all possibility of any sort of influence from Sky, which is naive at best.
Nobody's saying it's happening. The question was: could it be possible?
The answer is clearly: yes, even if only on a microlevel.

We've seen several examples of scientists whose independent thought process seemed to be influenced by (members of) the cycling community.
Olaf Schumacher acquitting Hayles, that Dutch scientist acquitting Dekker in 2000, Vrijman Lance. Coyle Lance. etc. It's a long list.
Add guys like Bermon, Saugy, Swart etc.
Many of these guys (e.g. Swart, Bermon) have trifold occupations:
- managing athletes
- antidoping
- sports science

Correct me if wrong, but you seem to be rather insensitive to the fact that there is an inherent conflict of interest there (arguably even multiple). A CoI is not evidence of wrongdoing, we all know that. But it means a degree of skepticism is warranted.


Ow, and obviously, influence can range from microlevel influence (asking people for favors, subtly influencing the reviewing process, etc.) to more blatant forms of influence (bribery, see e.g. Vrijman).
But regardless, the bottom line is: skepticism should be default once you observe a CoI.

 

[sniper]

wow. there are oceans between these responses on the one hand

acoggan
Are you really that clueless as to how academic peer review actually works?

Alex Simmons
You mean you are criticising a process that you clearly don't have much understanding of?

and this response on the other.

Merckx Index
This is actually a good question.

reminds me of the vast difference between Burnley & Swart on the one hand, and Ross Tucker on the other, in terms of how they react to inquiries on twitter.

 

[sniper]

Re: Re:

How is the peer selected? By one's self or independently assigned?

This is actually a good question. In several journals I'm familiar with, authors are allowed to suggest several reviewers. In fact, AFAIK, this is the policy of most journals, at least in the general fields I've worked in. The candidate reviewer has to be very familiar with the area of study, of course, but he or she may be very sympathetic to the author's work, e.g., they may share a pet theory.

from my (limited) experience, some journals will explicitly ask the author to propose reviewers, as well as to advise against certain reviewers who the author thinks might be biased. In some cases, you'll be asked to explain why you advise in favor or against certain reviewers.
In some cases, there is no such possibility (i.e. the author cannot advise anything)
Importantly, in ALL cases (at least in my field), the editor has the final say.
The editor(s) can even decide to send your article to one of the reviewers you advised against.

Bottom line, in my field, if you'd suspect a rigged (or too lax) reviewing process, you'd probably first be looking for links between the author and the journal editor(s).

This is mainly because--this goes back to what Alex said about the small world--science has become so highly specialized that there are frequently very few scientists considered capable of adequately critiquing someone else's work. One of the cardinal rules of review is anonymity--the reviewer's identity is not supposed to be known. But in fact, we frequently knew the identity of one or more reviewers, because there were relatively few researchers it could be, and we were familiar enough with all of them that we could tell from what they said and how they said it who it must be.

and of course reviewers can waive their anonymity. This often happens for 'strategic' purposes. I mean: If you write a favorable (or very elaborate/helpful) review for someone, you could be temped to waive your anonymity, so that the author you reviewed will know your name and review you favorably in return next time he gets your paper. It's human nature, and it's one reason why (most seem to agree) peerreviewing is far from flawless.
(edit: from my experience, if the reviewer wants to waive anonymity, the editor can still block that and say "hold on, you have to stay anonymous". In exceptional cases, for different reasons, the editor may ask the reviewer to waive his anonymity. Here, of course, the reviewer has the option to say "thanks but no thanks")

 

[gillan1969]

Thanks. In more simple terms; it's an industry which regulates itself. A concept last seen pre-Lehmans 2008. Nevertheless, that's a larger question, better for an episode of Geoffrey Robertson's hypothetical.

Science is not an industry. It's a process whereby the validation and ongoing questioning of prior knowledge and understanding is under constant review and challenge. Peer review is but one part of the validation and QC in science. Another is the validation of results by replication of experiment by others, which is why experimental details and results are published in the scientific domain so such validation can be undertaken. That's how a body of evidence is built.

Definition of science:

the intellectual and practical activity encompassing the systematic study of the structure and behaviour of the physical and natural world through observation and experiment

indeed...what we have here is not science...its a bloke who did some tests...once...where the subject chose the timing and preperation....no conclusions can be reached other than that which we had already observed i.e. he is a rider physiologically capable of being a GT winner

not sure why swart is even being described as a scientists in relation to this piece of work...standard methodology on standard kit...which even that he seems to have mucked up

well...plus the fax obviously...needs a longitudinal basis for his conclusions

 

[Dear Wiggo]

Thanks. In more simple terms; it's an industry which regulates itself. A concept last seen pre-Lehmans 2008. Nevertheless, that's a larger question, better for an episode of Geoffrey Robertson's hypothetical.

Science is not an industry.

heheheheeh.

It's a bit early for April Fool's.

 

[thehog]

Re:

wow. there are oceans between these responses on the one hand

acoggan
Are you really that clueless as to how academic peer review actually works?

Alex Simmons
You mean you are criticising a process that you clearly don't have much understanding of?

and this response on the other.

Merckx Index
This is actually a good question.

reminds me of the vast difference between Burnley & Swart on the one hand, and Ross Tucker on the other, in terms of how they react to inquiries on twitter.

Yes, the defensive aspect was noted. On Christmas day no less. Inquisitive minds like to ask questions and learn more, that should be encouraged, the dismissive nature with some responses gives reason to ask more questions.

If the Radcliffe study shows one thing, it's a small and cozy club of authors and for peer review. Reading through the responses, it doesn't strike me that peer review is a rigid or robust process, i.e. the position they take is more one to check for any obvious errors and lack of citation rather than validating the data; is it true or not? Are there too many assumptions being made? for example. I would also question if 'blind review' is even possible given what is seen the Radcliffe paper.

Back to the 2007 Froome data, how will they validate it's data, given the peer can't see or perform the tests themselves, do the see the raw data, what's the validation process?

Based on all of that, how one gets to; "He just lost fat" is beyond me.

Throughly interesting discussion mind you...

 

[sniper]

Thanks. In more simple terms; it's an industry which regulates itself. A concept last seen pre-Lehmans 2008. Nevertheless, that's a larger question, better for an episode of Geoffrey Robertson's hypothetical.

Science is not an industry.

heheheheeh.

It's a bit early for April Fool's.

it's not the first time Alex and/or acoggan seem to conflate what science is (or is not) supposed to be with what it really is.

 

[ScienceIsCool]

Re:

wow. there are oceans between these responses on the one hand

acoggan
Are you really that clueless as to how academic peer review actually works?

Alex Simmons
You mean you are criticising a process that you clearly don't have much understanding of?

and this response on the other.

Merckx Index
This is actually a good question.

reminds me of the vast difference between Burnley & Swart on the one hand, and Ross Tucker on the other, in terms of how they react to inquiries on twitter.

You guys are really hung up on peer review.It works really well and the nice thing is that it isn't even a critical part of the process. Let me explain: I'm betting Hog buy everyone a round of drinks if he found an obvious flaw or an outright lie in one of Coggan's papers. Now imagine a couple thousand scientists around the world who feel the same way. It doesn't matter if peer review catches it. You publish some BS and your reputation is gone. Done. Over. Never get it back.

Who here would believe anything Coyle published about the efficiency of a pro cyclist? Wouldn't matter if it appeared in Nature or the National Enquirer. Likewise, I hear Pons and Fleischman took up gardening to fill their time...

John Swanson

 

[thehog]

Thanks. In more simple terms; it's an industry which regulates itself. A concept last seen pre-Lehmans 2008. Nevertheless, that's a larger question, better for an episode of Geoffrey Robertson's hypothetical.

Science is not an industry.

heheheheeh.

It's a bit early for April Fool's.

it's not the first time Alex and/or acoggan fail to distinguish between what science is supposed to be and what it really is.

The irony is not lost; is sports science a science? Is selling coaching by scientists an industry in itself?

Like you state, when is a scientist being a scientist and not a coach selling products and services? Where is the line and separation of duties?

 

[sniper]

Re: Re:

[quote="ScienceIsCool

You publish some BS and your reputation is gone. Done. Over. Never get it back.

John Swanson

[/quote]

i think you're wrong on this. It maybe holds for some researchers (e.g. Coyle), but certainly not for the majority of researchers.

in many disciplines, and for many researchers, truth is that what they write is only ever going to be read by maybe one or two persons max (if they're lucky). So really no need to worry about any damage to ones reputation, in many cases.
For many (if not most) researchers, the main goal is to have a publication list as long as possible in order to apply for funds and jobs, etc. It's the publish or perish principle, you'll be acquainted with that. And it's one of the main driving forces behind scientific fraud.