Questioning The Conclusions Of Science

[naomi24]

This question arises from the discussion in R&S on the dubious practice of Water Divining. Sometimes the conclusions of science result not from positive evidence that the subject is invalid, but from absence of evidence. Whilst I know the scientifically minded will say ‘until evidence is forthcoming, I won’t consider the possibility’, but the question is do those who accept the conclusions of science ever waver and consider the possibility that evidence could exist that science has missed – or has overlooked – or is currently technologically incapable of recognising or testing?

[naomi24]

Jom, see Larry's post at 10:41 Wed 24th Jul 2013.

[vetuste_ennemi]

I want to know why Larry didn't find the water main.

[naomi24]

Pass.

[Matheous-2]

Naomi24. May I thank you for one of the best and most enjoyably entertaining posts I've seen in AB for years! It really made my Saturday night / Sunday morning!!
I would have loved to throw in my twopence worth, but I'm afraid I don't have the stalwart intellect or scientific background you guys apparently have!
Just for the record Naomi- I think you won!

[vetuste_ennemi]

Ah, we're not all spoilsports!

[jim360]

Naomi didn't win -- but then no-one else did either.

[jim360]

Should just add, while I'm thinking about it. Naomi asked, and I'll agree, that we move on from this subject. So unless someone else wants to resurrect this thread, I'll keep my word. But still wanted to thanks Naomi for a debate that has helped me, I think, to sort out my views on this and how to express them. Hopefully, if this comes up in the future, I'll be able to put it to the test. But despite the disagreement, despite the fact that we still do not agree, I think it's been a very useful debate.

[Hypognosis]

@naomi
//Sometimes the conclusions of science result not from positive evidence that the subject is invalid, but from absence of evidence.//

Looking at your question from another angle today, so I'm curious as to whether you're referring to pronouncements, or generalizations, made by scientists when interviewed by journalists or whether you've seen a specific research paper that draws its conclusion based on a lack of results - the failure of the experimental procedure?

My understanding is that most experiments are constructed around a "null hypothesis", which is worded to be the opposite of what the researcher is attempting to prove and the observed results are supposed to succeed in disproving the null hypothesis at the nn% condfidence level.

Do you consider that methodology the same as "positive evidence that the subject is invalid"?

[naomi24]

Matheous-2, thank you. I’m very pleased you enjoyed it – and you should have thrown in your twopence worth if you wanted to. I, for one, would have welcomed it. Next time perhaps?

Jim , you’re welcome. It’s good to throw ideas around. ;o)

V-E. ha ha! I know. ;o)

[naomi24]

Hypognosis, Pronouncements or generalisations? A bit of both I think.

//Do you consider that methodology the same as "positive evidence that the subject is invalid"?//

Not at all, but if, unbeknown to science, the methodology is flawed in some way, positive evidence that the subject is invalid cannot be truly ascertained – it simply appears to have been ascertained.

[Hypognosis]

I trust we're not talking at crossed purposes here? I was talking within the narrow frame of reference of one experiment, not the wider frame of science in general and it's treatment of certain subjects, which is what I [i]think[/i] you're concentrating on.

For example, the null hypothesis is that a chemical is ineffectual at treating a medical condition. The medical trial shows that 10 out of 100 patients who received it (and not the placebo) had a positive outcome.

So, the medicine failed 90 times out of 100. Is that absence of evidence that the medicine is effective?

[naomi24]

Hypognosis, I don't think we're talking at cross purposes.

//So, the medicine failed 90 times out of 100. Is that absence of evidence that the medicine is effective? //

No, it's evidence that it's effective in some instances.

[jim360]

Doesn't that assessment change, rather, if in the same survey 12 out of another 100 people got better when having placebo?

[naomi24]

Jim, that would depend upon whether you are testing the effectiveness of a drug, or the psychological effect of taking a placebo – two different tests.

[jim360]

Oh, OK -- well, not really, but never mind. That's not really the right way to interpret such data. If people get better after taking a drug, but more people get better having had nothing or placebo, then you cannot argue that the drug is at all effective -- given effectively no intervention, some people got better anyway. You seem to have overlooked the effect of regression to the mean.

Please, read Ben Goldacre's "Bad Science" when you have the chance.

[Hypognosis]

//No, it's evidence that it's effective in some instances. //

Quite so. I guess I should use similar wording in reference to dowsing, rather than just pooh-pooh it, out of hand.

Anyway, the point I was pressing is that scientific statements are at the 95% confidence level or, if you prefer, there is a 5% risk of throwing away a perfectly valid line of research.

If the medical condition was invariably fatal then changing 100% mortality to 90% mortality makes the medicine worthwhile. If it was a treatment for a minor ailment, you'd not waste any more research funds on it.

I know you're keen to learn how dowsing works but who do you imagine would stump up the research funding for it?

The water companies are already making practical day-to-day use of dowsers. Evidently they accept that it works and lack further curiosity. The gubmint is broke, so they won't fund it. Who does that leave?

[naomi24]

Jim, regardless of what Ben Goldacre says, the test is to ascertain whether or not a drug is effective. If you introduce placebos into the equation, and a percentage of people get better, you wouldn’t know whether the drug is effective or not. Your test would be invalid.

[naomi24]

Hypognosis, I don't imagine anyone would stump up the research funding for it.

[jim360]

Then all medical science is invalid, by that standard, Naomi.

In order to ascertain if a drug is effective, you have to compare it to something else. Placebos are known to be "effective", to some extent. If you invent a new drug, then the measure of effective is either:

1. Better than placebo (which is effectively equivalent to better than nothing), or;
2. Better than currently available drugs.

Very often, pharmaceutical companies test against placebo, which is an easier test to pass. But you still have to test it against placebo, because that is the baseline.

If you wanted to test a medicine against nothing, that would be unethical anyway. You have to give the patients that you aren't testing the drug on something, after all.

[woofgang]

jim, is there a difference though between a test where the volunteers are told that they might be given a drug and they might be being given an inert substance that won't help them, and a test where they are told that might be being given an existing drug that works to an extent, or a new drug that might work better than the existing one.