How Do You Spot When An Opinion Is Not Based On Good Science?

[vinika]

How do you spot when an opinion is not based on good science?

[Clanad]

Although obliquely touched on by jtp and especially Khandro within this thread, it deserves emphasis to define "Science".

Science, at it's root is the structured collection of data. That's not to say that task is easy or simple in most cases. But the collection of data is then open to interpretation by others… and this is where the sparks will fly… even among the most dedicated peer.

This in no way degrades or argues against "Science", but the understanding of methodology is an important component.

[jomifl]

Clanad, science is not about the collection of data, data is only collected to prove or disprove a hypothesis. Some areas of science are data intensive , others not so. The essence of science is ideas and creative thinking.

[Clanad]

jomfil… do you not agree that data comes in differing modes? All science is data driven one way or another… In fact the entire Scientific Method stands or falls on data.

One of the best essays on the Scientific Method I've read explains it this way:

"...There is no such thing as proof in science. It doesn't matter how many experiments agree with your hypothesis, or how much data you have. All concepts in science are fundamentally tentative. What does change as we accumulate evidence is that our level of confidence in our ideas increases. As more and more evidence accumulates which supports an idea, and none appears that significantly contradicts it, we become very confident in that idea. This is the situation with things like Newton's Laws, Einsteins Constant and the theories of atomic structure. We assume that they are at least very close to The Truth. But we never, ever decide that we know for sure that they are the truth…"

I'd ask… politely, how is the essence of science only ideas and creative thinking? Ideas are so much wind without the sails of data to provide motion and creative ideas are only worth the data that support or destroy them...

[jim360]

Data is certainly important, and a theory will live or die on the data that confirms or refutes its predictions. I think the paragraph you quote is probably one of those points that is "right but misleading". There is of course uncertainty in Science and no theory is ever perfectly correct -- on the other hand, the probability that certain theories are wrong (at least within their region of validity) essentially approaches zero with the passage of time and the gathering of more supporting data, and after a while the only thing left is for a better theory to come along that extends the old one rather than replaces it.

This is essentially the same picture as in the paragraph cited, but with just a bit more optimism. Nothing is known in Science for certain -- but, after a while, the difference stops mattering that much.

[Peter Pedant]

You check the data and see if the conclusions follow.

SOme advertise the fact they arent based on science but on something else (revealed truth) and an example of this is creationism - I dont think the biblical basis of creation is based on 'science' even in the minds of those who support it.

Unfortunately this leaves a large area where the person has set out to deceive. Such as the prof of geology at chandigarh who planted fossils half way up a local mountain so that it would support his own theories

Or Piltdown - see wiki on this -

The research supervisor I was a tech for - repeated every experiment that underpinned his research interest and found about 50% couldnt be repeated. He said that was about par for the course.

[jomifl]

Clanad you are putting the cart before the horse again. I could go outside my front door and spend a day gathering data but unless I had decided what data to gather, how to gather it and where from it would be useless. A data set has to have some common factor such as all being from one species or one production run etc. Randomly measured objects would give very little useful information, other than it was possible to measure a random collection of objects.

[jomifl]

Clanad, science is supported by data not driven by it. You stated that there are various ways of interpreting data but good science decides what data is needed to prove or disprove a hypothesis which pre-exists the gathering of the data.

[Clanad]

So, jomfil, we're only arguing about semantics[i here, no? I never alluded to [i]when] data was collected… only that any idea lives or dies via the data (and it's interpretation).

Seems you've gone from "...science is not about the collection of data…" to "...Some areas of science are data intensive , others not so…" to"...Randomly measured objects would give very little useful information, other than it was possible to measure a random collection of objects…" (which I never said to begin with). Finally, arriving at "…science is supported by data not driven by it…", which was my point to begin with.

One of the sure methods to bumfuzzle a science projects is to follow your directive that "...good science decides what data is needed to prove or disprove a hypothesis which pre-exists the gathering of the data…" Data mining to arrive at a desired outcome isn't good science, no?

One source (Live Science) says it better than I could… "As sufficient data and evidence are gathered to support a hypothesis, it becomes a working hypothesis, which is a milestone on the way to becoming a theory."

[jomifl]

Clanad, it is not a semantic point but a point of logic which I was hoping you would be able to understand. At no point did I suggest data mining, what I suggested was deciding what data is needed to make a decision one way or another and then looking for it. If you don't understand science please don't pretent that you do.

[Clanad]

Ah… there's the jomfil I've come to expect… At any rate, have a good day!

[jomifl]

any time clanad :o)

[Hypognosis]

@vinika

Step 1)
Determine whether the opinion is being parroted from somewhere else (eg they're just passing on info from some book/webpage they've just read) or whether it's somthing they have come up with themself.
Step 2)
Ask for directions to the source material, such you can read it yourself.
Step 3)
Analyse the data
Step 4)
Formulate your own hypothesis
Step 5)
If -your- hypothesis is better at explaining how -their- data came about and also makes more accurate predictions of the future results of differently designed experiments...
Step 6)
You go on to obtain funding, conduct these additional experiments and demonstrate your hypothesis to be the best fit for -all- applicable data produced to date, then your hypothesis stands a chance of becoming the 'new paradigm'.

In short, a researcher with a radical new theory up their sleeve not only has to prove it correct by producing their own data set, they are also tasked with overturning all previous research in their field.

The Helicobacter pylorii story (stomach ulcer causing bacteria) was a case of one researcher versus hundreds (past and present) but that was probably the exception to prove the rule!

In the meantime, this may help explain why science, seen through the distorting lens of the media, appears to be adversarial, dishing out contrary advice, every third Thursday.

[Hypognosis]

Incidentally, Ben Goldacre's latest campaign is to try and force disclosure of medical research which was known to be 'work in progress' and, with startling regularity, ends up, mysteriously, not being published...

With regard to the impression of a funding gravy train or 'fads' determining who gets funding. Consider the alternative: an egalitarian system where everyone gets an equal allocation of funds. I will exaggerate for effect but suppose someone was to submit an experimental design for "research into the effect of furniture arrangement/alignment on patient recovery from pulmonary illness", then they'd be entitled to their share of funds and I dare say an increasingly long queue (of increasigly flaky projects) would be forming behind them too.

Government funding for research is a bone of contention. The public moan if the Daily Flail happens across expensive research into proving the blindingly obvious, as happens every so often. As if the overall budget wasn't feeble enough, public opinion about what is 'valid research' and what is not, places quite strict boundaries on what areas science will explore within the forseeable future.

For everything else, there is no escaping the fact that vested interests underpin every research project. Some multinational companies plough up to 20% of their turnover back into R&D to produce new products or improve existing ones. In the UK, it is reckoned that 6-8% on R&D is typical.

Don't draw too much from that. It may be that this is all they can afford, because they're not on the same scale as the global players so it's not a systemic failing - the attitude to importance of R&D - which I'm criticising. To be honest, it is something I read and am merely repeating but have forgotten where I found it.

[jim360]

I might be mistaken but I think jomifl and Clanad are trying to make the same point but are misunderstanding how the other guy is doing it. I mean the thing is that firstly the results of the data gathered shouldn't be deliberately twisted to give the desired result.* The second thing though is that data can#t be gathered unless you have some idea of what you are looking for, so the methods of conducting the experiment and analysing the data will reflect that. If, say, there is a prediction that speed due to gravity goes as time squared, you'd measure speed against time and then test your data against the prediction v/t^2 = constant. Now of course a fairly conducted experiment will not make the data fit this prediction, but will note if it does or does not fit it it and by how much, and (in the case of something as tested as this one) try to explain in terms of random and systematic error why it didn't (if it didn't).

So the key process to data gathering would be that the way in which you look for data is guided by theory/ hypothesis, but the data once gathered and properly analysed is the thing that should be taken to be "true", and the theory should be adjusted to fit the data.

Theory motivates the search, data guides the theory, and the feedback loop continues.

[Khandro]

jim; Off this topic, but will you help me with some maths please? - it's from another thread, and the question is analogous;
World population is 7 million. Say everyone likes ice lollies, which come in just two colours; red and yellow, but only 2% like the red ones and of those only 0.1 % hold them in their left hand. Given the same %age of left-handidedness in the general population. How many of the 98% of yellow lolly-eaters hold them in their left hands please?

[Hypognosis]

@atalanta

interesting point you made, which I've raised in a separate thread: -

http://www.theanswerbank.co.uk/Society-and-Culture/Question1313992.html

[woofgang]

just to point out that an opinion doesn't have to be based on good science to be correct, ie to accord with the facts as we know them.

[Hypognosis]

@khandro

World population has passed 7000 million (7 billion). The b and m keys are quite close together, so hard to tell if that was a typo or an arbitrary choice of population size.

The answer is, in part, embedded in the question.

0.1% of the set {yellow lolly eaters} will hold it in their left hand.

Alternatively: -
0.1% x 98% of the set {whole population} will hold it in their left hand
= 0.098%

0.1% x 2% of the set {whole population} will hold their RED lolly in their left hand
= 0.002%

Which add up to 0.1% left handed in the set {whole population}

[Khandro]

Hypo; Thank you, and yes I meant 7 billion.
As you see maths is not my strong point, so what you say is; 0.1% of 98% of 7 billion is the number of LH yellow-lolly eaters ?
And for a lolly of your choice the answer is ........... ?

[jim360]

0.1% = "one in a thousand" -- so in a population of 7 billion then 7 million would be left-handed. Then 98% of 7 million = 7 million - 2% of 7 million, which is 7 million minus 2*1% of 7 million, which is 7 million minus 2*70,000 = 6.86 million.

I think 0.1% is a gross underestimate of the proportion of Left-handed people, though.