Explore Ethics in Science: Feynman's Cargo Cult & Popperian Falsification

[Andre]

Been pondering a bit about ethics in science. Not sure where to post about it. It doesn’t seem to qualify for philosophy nor is it social science. It’s just how science should be conducted. But if it has to be moved elsewhere, sure go ahead.

Basically we have of course the (Popperian type) sciencific method about falsification and the ethical basic requirement to be strictly objective, unbiased. For this the cargo cult lecture of Richard Feynman should be mandatory reading and rereading every year in all schools. Two quotes:

.. The first principle is that you must not fool yourself--and you are the easiest person to fool. So you have to be very careful about that. After you've not fooled yourself, it's easy not to fool other scientists. You just have to be honest in a conventional way after
that...

.. One example of the principle is this: If you've made up your mind to test a theory, or you want to explain some idea, you should always decide to publish it whichever way it comes out. If we only publish results of a certain kind, we can make the argument look
good. We must publish both kinds of results

Now Popper and Feynman's cargo cult should lead to this types of studies.

Abstract:

In Myself et al (2013), we propose H and formulated a prediction P. Testing the prediction, using open methods X, Y, Z with empirical data, we find NOT P consistently. Therefore we have to conclude that H is falsified and should be rejected.

Hey, anybody ever seen a study like that? I think I once did, but only once and I read hundreds. Would you believe to see real science in action here? I sure would. Obviously the authors decided that their ego's (and fooling them) were less important than the progress of science.

It must be even harder to publish something like this:

Abstract:
Our-Enemy et al (2013) propose H and formulated a prediction P. Obviously H is contradicting Myself et al (2013), therefore it is essential to test which of the two is right. Using open methods X, Y, Z with empirical data, we find P consistently. We conclude that H may be correct, which would refute Myself et al (2013)

If you happened to find that, are you going to publish it? If you did, you sure would be very high in my personal hall of fame of real scientific scientists.

More later.

 

[DragonPetter]

Of course I would publish results that refute my hypothesis. I thought this was how all scientists behave.

I can only think of corporations and governments that would go against this mentality.

Personally, I have a hypothesis that I'm in the process of developing an experiment for, and I would love for it to work out. If it doesn't, I hope to make the results available to others anyway so they don't go down the same dead end road that I could be going down or if they can see that I did something wrong in my experiment.

EDIT: This reminds me of the faster than light neutrino experiment. It took a lot of courage for them to publish their results, and even though they were wrong, scientists and engineers learned a lot from their pitfall.

 

[DragonPetter]

That Feynman speech is great, thanks for sharing it.

After reading the part of the psychology student wanting to use results from a previous experiment for her own and of the NAL scientist who wanted to use results from another experiment for his own, it reminded me of the current situation that CERN is trying to get statistical relevance for its results with the Higgs boson, where it wants to combine statistical results from ALICE and ATLAS to get a higher statistical certainty. I know those people are really smart, so they are probably doing this in a valid way, but when I first heard they could do this, I was a little skeptical.

 

[Andre]

Thanks for your thoughts DragonPetter. I can imagine for somebody for being a truthful honest physician that's a nightmare to have to conclude that neutrinos appear to defeat c. I can imagine a cry for help then, "What did we do wrong?". Perfect, Absolutely nothing wrong with that. A good example of how things should be.

Counter examples are for instance the Gothenborg geomagnetic flip, which suggested that there was a brief geomagnetic reversal 10,000 years ago. This was a real sensational discovery some decades ago, published by a renowned highly visible scientist. Obviously, such a discovery needed confirmation/duplication, which proved to be impossible. I happen to have had some discussions with the key player here. He discovered why the Gotenborg flip was "fatally flawed". However, a refuting article had never been published (avoiding to hurt ego's?). The Gothenburg flip had to just dimisnish gradually from the scientific arena.

Another example could be the 14C spike at the beginning of the Younger Dryas (Hughen et al 2000), of which I reported here, disappeared below noise levels after a thorough clean up with a new carbon dating calibration scale (INTCAL09), but can anybody find an article pointing this out? However the radiocarbon spike lingers on in discussions.

Anyway, how about the ethics and scientific method when reading about an article like this?

Abstract

Recent measured data do not concur with the generally accepted beautiful theory BT. We analyse the data and find that the observations may be obscured by fudge factors. Comparing the observations with the results of Models using suppositions X, Y and Z, we conclude that the data are likely being modified/contaminated. If we compensate for X, Y and Z the data would have been different and consistent with BT. So, rest assured, no ugly fact slaying the beautiful theory BT.

(with excuses to Thomas Huxley.)

Would you believe that? And what if the beautiful theory was your favorite? And, what if you were sceptical all the time, about the beautiful theory?

 

[Andre]

..
Anyway, how about the ethics and scientific method when reading about an article like this?

Abstract

Recent measured data do not concur with the generally accepted beautiful theory BT. We analyse the data and find that the observations may be obscured by fudge factors. Comparing the observations with the results of Models using suppositions X, Y and Z, we conclude that the data are likely being modified/contaminated. If we compensate for X, Y and Z the data would have been different and consistent with BT. So, rest assured, no ugly fact slaying the beautiful theory BT.

(with excuses to Thomas Huxley.)

Would you believe that? And what if the beautiful theory was your favorite? And, what if you were sceptical all the time, about the beautiful theory?

sceptic or protagonist, we cannot deny that such an article is not following the scientific method. It does not attempt to refute BT, rather, it attemps to salvage it, as appears to be in danger. Situations like these have been described by Thomas Kuhn, from http://www.des.emory.edu/mfp/Kuhn.html

Chap VIII C: In responding to these crises, scientists generally do not renounce the paradigm that has led them into crisis...

3. They devise numerous articulations and ad hoc modifications of their theory in order to eliminate any apparent conflict...

There you are. Of course one cannot conclude that such a symptom of crisis response automatically implies that BT is false. It can still be correct but the article is not nearly as convincing as the refutation of myself et al (2013) or the support for Our-enemy et al (2013). It could just as well be false. One may wonder about the difference in Feynmanian ethics of 'Myself et al' and the BT defenders.

 

[Vanadium 50]

Andre. it sounds like you have a real axe to grind here.

This happens so often in science that it doesn't even generate any attention - you can find exceptions because they are exceptions and thus generate attention: a man bites dog story.

Personally, I am in the midst of doing exactly this. A year ago, I wrote a paper with a theorist (I'm an experimenter) pointing out a class of models that were not receiving consideration, and proposed a method to test for them. I'm now in the midst of writing an experimental paper saying we've done the test, and here's what came out. The fact that it came up negative is no big deal. It is what it is.

 

[lisab]

I realize it's a generalization, but it's been my experience is this type of thing is more common as you move away from the core sciences (i.e., towards social science).

I remember taking a class from a history professor who was clearly in *love* with her hypotheses. The whole purpose of that particular class was to allow her to preach to us that there is no friction between science and religion, and never has been. It was clear to me (and probably everyone with a physical science background) that she didn't have that Feynman-esque objective approach to her research. It was hard to sit through!

 

[Andre]

Thanks ⁵⁰V and I think that Lisa has a point. Math and physics are likely a lot less vulnerable for ethical issues, than for instance Earth sciences, medical and social sciences.

In Earth sciences for instance we find several of this type:

Abstract

We investigate P and find T. This is contrary to current understandings. Models imply that such an anomaly may be explained by X, Y, Z, although variations may be larger than previously thought.

http://geology.geoscienceworld.org/content/30/5/427.abstract

So is it okay to explain things away? -go with the mainstream flow- or is this an attempt to make "current understandings" unfalsifiable, and hence withdraw it from the realm of real science?

 

[Andre]

Obviously the former example is striking due the use of "anomalous", but to see if it's the man that bites the dog, other examples bubble up with searches like: Pleistocene contradict. So how extensive is this contradiction and how is it handled?

Take for instance http://geology.geoscienceworld.org/content/30/4/379.abstract.

Abstract

Examination of published data reveals that a marine bed in Beijing can be dated as 80 ka or younger on the basis of abundant nannofossils. This age is ∼30 times younger than that published previously on the basis of magnetostratigraphic and biostratigraphic interpretations. The abundant nannofossils and foraminifers suggest that Beijing was inundated by the sea within the past 80 k.y. The very recent nature of this marine transgression has profound societal and geological implications and thus calls for new studies and thorough evaluation of all relevant data sets.

That seems to be an honest ethical study finding problems for 'current understandings' and not hiding behind explaining models, but what now?

 

[Andre]

Science magazin has http://www.sciencemag.org/content/336/6078/159 this week. It advocates total transparancy for computer source code. I wonder why would that be necesary?

...The publication and open exchange of knowledge and material form the backbone of scientific progress and reproducibility and are obligatory for publicly funded research. Despite increasing reliance on computing in every domain of scientific endeavor, the computer source code critical to understanding and evaluating computer programs is commonly withheld, effectively rendering these programs “black boxes” in the research work flow. Exempting from basic publication and disclosure standards such a ubiquitous category of research tool carries substantial negative consequences. Eliminating this disparity will require concerted policy action by funding agencies and journal publishers, as well as changes in the way research institutions receiving public funds manage their intellectual property...

 

[Andre]

Working on a general scientific ethic code and I'm looking at the Hippocratic Oath

..I will respect the hard-won scientific gains of those physicians in whose steps I walk, and gladly share such knowledge as is mine with those who are to follow...

No no no, that's quite unuseable, it should read...

I will NOT respect the hard-won scientific gains of those [strike]physicians[/strike] scientists in whose steps I walk, instead I will challenge them, and gladly share my doubts about such knowledge as is mine with those who are to follow.

 

[Ryan_m_b]

Science magazin has http://www.sciencemag.org/content/336/6078/159 this week. It advocates total transparancy for computer source code. I wonder why would that be necesary?

I believe it's because any other equipment used in scientific research usually is (and should be) utterly open for scrutiny. If specific consumables are used they should be listed in the materials and methods of a paper, from that readers should be able to look up the product on the company catalogue (digital or hard copy) and find out all the specifications, characteristics, components etc. This allows other researchers to figure out what exactly is going on, trivial example but it's important to reveal what kind of plates are used to grow cells in biological research in case there's a suspicion that the plate material chemistry, mechanics and topology are affecting the experiment.

With increasing use of software it's the same, the way that the software works should be open to scrutiny in case the data is flawed in some way. Obviously software companies are far more averse to this than other companies because if even if I were to tell you everything about a well plate you still probably won't go to the bother of making your own but a source code can easily be used as is.

IMO a way round this could be for journals to agree that they won't accept papers that list a commercial software package in their materials and methods without listing a unique serial number that should be registered to their institutions. I admit this isn't a great idea.

Working on a general scientific ethic code and I'm looking at the Hippocratic Oath

..I will respect the hard-won scientific gains of those physicians in whose steps I walk, and gladly share such knowledge as is mine with those who are to follow...

No no no, that's quite unuseable, it should read...

I will NOT respect the hard-won scientific gains of those [strike]physicians[/strike] scientists in whose steps I walk, instead I will challenge them, and gladly share my doubts about such knowledge as is mine with those who are to follow.

True but the difference is that the hypocratic oath is for people who are going to be practising methods and using tools already developed by the scientific method.

 

[dabble]

Ethics in science is the same as ethics in the rest of the world. Either you have ethics, or you don't. I have enjoyed reading your thread, thanks for sharing it with us.

 

[Andre]

... the difference is that the hypocratic oath is for people who are going to be practising methods and using tools already developed by the scientific method.

True but even then, if Marshal and Warden hadn't been skeptical about stomach ulcers being caused by stress factors, a lot of people would not have been happy today. Note especially..

...who with tenacity and a prepared mind challenged prevailing dogmas.

Ethics in science is the same as ethics in the rest of the world. Either you have ethics, or you don't. I have enjoyed reading your thread, thanks for sharing it with us.

You're welcome. But I guess 'having ethics' is not that simple. Is it ethical to administer a overdose morphine to a terminal patient, perishing in pain, or is it a crime (against the hippocratic oath)? Is it ethical to withhold serious doubt about the correctness of the science behind some so much desired ethical correct policy, or is it unethical to do so?

Maybe a lot of people are more ethical than assumed. Only, it's other ethics, although maybe ethics should be objective regardless if it is liked or disliked, unlike morality in this definition:

“What is morality in any given time or place? It is what the majority then and there happen to like and immorality is what they dislike.”

Alfred North Whitehead

 

[Hobin]

...I thought this would be an appropriate comic. Or, alternatively:

 

[Ryan_m_b]

True but even then, if Marshal and Warden hadn't been skeptical about stomach ulcers being caused by stress factors, a lot of people would not have been happy today. Note especially..

I would argue that this is very different. They weren't acting in their capacity as doctors when the tested their hypothesis they were acting as scientists. The difference is that it would have been irresponsible for them to test their hypothesis by not giving patients approved treatments and instead giving them what they thought would work.

For example: Alice is a doctor. She thinks that current treatment X doesn't work because the suspected etiology of disease X is wrong. She continues to treat her patients with treatment X whilst engaging in medical trials to discover the true etiology, she completes this and publishes her work showing that treatment Y would be better. She and other doctors then switch to Y.

Bob is a doctor. He thinks that current treatment Z doesn't work because the suspected etiology od disease Z is wrong. He stops giving his patients treatment Z and instead gives them treatments he thinks may work. Consequently some patients die needlessly.

Do you see the difference? There is a fine but extant line between treating patients and developing treatments for patients.

 

[Andre]

... Consequently some patients die needlessly.

I don't think that the appeal to sceptism implies malpractice, taking life threatening risks. Instead it should be a high ethical duty of any phycisian to monitor clinical pictures and medical treatments and investigate and report anything he considers irregular.

 

[Andre]

This week a column in Nature that covers the scientific ethical issues in medical research. So I guess that's interesting enough to resurrect this old thread.

Beware the creeping cracks of bias

Agenda problems:

The problem, after all, was not with science, but with the poison of the profit motive...

...The belief is that progress in science means the continual production of positive findings. All involved benefit from positive results, and from the appearance of progress. Scientists are rewarded both intellectually and professionally, science administrators are empowered and the public desire for a better world is answered. The lack of incentives to report negative results, replicate experiments or recognize inconsistencies, ambiguities and uncertainties is widely appreciated — but the necessary cultural change is incredibly difficult to achieve...

Maybe it's a bigger problem than it seems:

...It would therefore be naive to believe that systematic error is a problem for biomedicine alone. It is likely to be prevalent in any field that seeks to predict the behaviour of complex systems — economics, ecology, environmental science, epidemiology and so on...

 

[OmCheeto]

Consequently some patients die needlessly.

I don't think that the appeal to sceptism implies malpractice, taking life threatening risks. Instead it should be a high ethical duty of any phycisian to monitor clinical pictures and medical treatments and investigate and report anything he considers irregular.

This reminds me a bit of my thread in medical science. The "ethical" portion of the title, which was never delved into, involved the needless death of a patient, and how that death prompted the patient in the thread to perform what some may consider an unethical act.

I can only conclude that ethics, in all its forms, is kind of funny.

 

[Andre]

Thanks for your view Om, No doubt that saving a patient is the most ethical thing to do for a doctor.

The question here is more like: We find result R, this can be caused either by A or by B. Our boss wants it to be A, we have grants to prove A, Mankind and Earth are going to be saved if it is A, so it must be A. I'm going to be rich and famous if it is A. Don't ever mention B, that's going to be very bad.

That's the gist of the OP, quoting the cargo cult lecture of Richard Feynman.

.. The first principle is that you must not fool yourself--and you are the easiest person to fool. So you have to be very careful about that. After you've not fooled yourself, it's easy not to fool other scientists. You just have to be honest in a conventional way after
that...

.. One example of the principle is this: If you've made up your mind to test a theory, or you want to explain some idea, you should always decide to publish it whichever way it comes out. If we only publish results of a certain kind, we can make the argument look
good. We must publish both kinds of results.