Quizzes & Puzzles2 mins ago
How Do You Spot When An Opinion Is Not Based On Good Science?
How do you spot when an opinion is not based on good science?
Answers
Best Answer
No best answer has yet been selected by vinika. Once a best answer has been selected, it will be shown here.
For more on marking an answer as the "Best Answer", please visit our FAQ.
Usually it's based on anecdotal experience - in other words someone's own experience.
Sometimes it's from selectivity bias - people ignore instances that weigh against the opinion and highlight those that support it
Sometimes it's based on very small data sets
or occasionally it's people playing fast and loose with statistics
..And I say that as a man with more than the average number of arms
Sometimes it's from selectivity bias - people ignore instances that weigh against the opinion and highlight those that support it
Sometimes it's based on very small data sets
or occasionally it's people playing fast and loose with statistics
..And I say that as a man with more than the average number of arms
It's one of the toughest skills -- generally speaking, opinions that are not based on good science are:
-- unfalsifiable: the person will not take any arguments on board, or misinterpret or ignore contradictory evidence;
-- unjustified: really, Science involves the attempt to explain observations while also predicting outcomes of further observations, so if there is no testable prediction that doesn't make the opinion worth much, and there ought to be some theoretical basis for the opinion too;
-- opinionated: personal experiences can be perfectly correct, but are not Science by their very nature of being personal. Ideally a Scientific observation should have little to no dependence on who made that observation. If you saw something, and described how, then I should be able to make that same observation to within experimental errors by repeating the method.
There is some debate over this, but I think that testability, falsifiability and predictive power are three of the major criteria that distinguish Science from non-Science.
-- unfalsifiable: the person will not take any arguments on board, or misinterpret or ignore contradictory evidence;
-- unjustified: really, Science involves the attempt to explain observations while also predicting outcomes of further observations, so if there is no testable prediction that doesn't make the opinion worth much, and there ought to be some theoretical basis for the opinion too;
-- opinionated: personal experiences can be perfectly correct, but are not Science by their very nature of being personal. Ideally a Scientific observation should have little to no dependence on who made that observation. If you saw something, and described how, then I should be able to make that same observation to within experimental errors by repeating the method.
There is some debate over this, but I think that testability, falsifiability and predictive power are three of the major criteria that distinguish Science from non-Science.
Depends on what you mean by String Theory. As a fundamental theory of nature it's not really Scientific by my criteria, no. As a mathematical method for solving problems in real physics, then yes it certainly is Science. People (myself included) have sometimes made the mistake of regarding String Theory as either a fundamental theory of Nature, or nothing at all. It has turned out in recent years that it is a very powerful halfway-house, a heuristic method for solving difficult problems that are very much testable.
It's hard to provide examples without going into details, but loosely speaking the statement can be that String Theory can be used in the same way as Archmiedes used something called a "mechanical method" to obtain results which he could then prove in a more rigorous way. So String Theory becomes a mathematical tool -- and there's nothing wrong with mathematical tools, really.
It's hard to provide examples without going into details, but loosely speaking the statement can be that String Theory can be used in the same way as Archmiedes used something called a "mechanical method" to obtain results which he could then prove in a more rigorous way. So String Theory becomes a mathematical tool -- and there's nothing wrong with mathematical tools, really.
vinika....what a very, very good question and unfortunately my answer will not be well received but only goes to show my lack of insight into scientific research.
My comments are basically focused on medical research.
I have indeed read the contributions by Jim and Jake the Peg but i must confess that i have not understood one word, but that is due to my lack of knowledge rather than the clarity of their posts.
As medical students, we were first introduced to medical statistics in 1953 when a publication called "Statistics for Medical Students" was suggested by our Professor of Medicine and Dean of the medical College.
We read it and i can honestly say that 99% of us did not understand one word of the gobbledegook and gave it up as a bad job.......luckily medical statistics didn't come up in our final examinations.
However, it looked good, it fooled us and we accepted medical statistics as a part of research without questioning........and so statistics developed as a speciality mainly, perhaps solely, by academics.
Doll and Hill produced their paper in the 50's (i think) into smoking and Lung cancer and that guaranteed statistics a clear run.
However.....in the last decade or so questions have been asked of statistics, has the input been accurate, have the results been "manipulated." We have see a prof of Medicine at a prestigious University bravely announcing that the "5 pieces of fruit " for a healthy cardiovascular system, was just "plucked out of the air."
Positions in the NHS have been made based more on publications than of experience and interview........and some of the publications submitted have been .....rubbish....just pure RUBBISH and we are seeing more and more of these papers presented for publication.
However....established units, well regarded personnel, provide the best opinions that we have based on the science that we have........"robust" is the term that clinches the research.
My comments are basically focused on medical research.
I have indeed read the contributions by Jim and Jake the Peg but i must confess that i have not understood one word, but that is due to my lack of knowledge rather than the clarity of their posts.
As medical students, we were first introduced to medical statistics in 1953 when a publication called "Statistics for Medical Students" was suggested by our Professor of Medicine and Dean of the medical College.
We read it and i can honestly say that 99% of us did not understand one word of the gobbledegook and gave it up as a bad job.......luckily medical statistics didn't come up in our final examinations.
However, it looked good, it fooled us and we accepted medical statistics as a part of research without questioning........and so statistics developed as a speciality mainly, perhaps solely, by academics.
Doll and Hill produced their paper in the 50's (i think) into smoking and Lung cancer and that guaranteed statistics a clear run.
However.....in the last decade or so questions have been asked of statistics, has the input been accurate, have the results been "manipulated." We have see a prof of Medicine at a prestigious University bravely announcing that the "5 pieces of fruit " for a healthy cardiovascular system, was just "plucked out of the air."
Positions in the NHS have been made based more on publications than of experience and interview........and some of the publications submitted have been .....rubbish....just pure RUBBISH and we are seeing more and more of these papers presented for publication.
However....established units, well regarded personnel, provide the best opinions that we have based on the science that we have........"robust" is the term that clinches the research.
Thanks Sqad -- it's hard to make some of my post clear because I don't do much String Theory work myself so don't understand it perfectly, but I suppose the main point is that it is a mathematical tool more than a scientific theory.
The problem with Statistics is a good one to raise. There are I think two separate problems, actually -- the first thing is that some statistics are just utter rubbish, or have been deliberately manipulated or massaged or otherwise made to give a particular impression when it was known to be a false one. Some people have cynical reasons for lying. It's most obvious in the medical industry, where naturally a drug that looks good will make you more money than one that does not.
There is a second problem, and that is that frankly most people just don't get it. This is not to insult anyone's intelligence, but a full statistical analysis is often very difficult to follow, and uses complicated methods that, while well-understood by those who use them, are almost impossible to follow for everyone else. How then is the layman to evaluate these sorts of tests? They just can't, really, and almost inevitably I suppose a certain amount of "faith" enters -- you trust that the people who perform these tests aren't lying through their teeth and relying on you not to bother looking over the work thoroughly, and usually this is the case.
The issue in the above is not that "most people are thick", by the way -- it's simply a true statement that most people haven't devoted a large amount of time to learning the mathematical techniques necessary to understand how a particular result was achieved. And usually it's hard to explain how it was, too. In one example I saw yesterday it was shown in a very sketchy way how it was possible to extract six different values from the same data, when my own relatively basic knowledge of statistical methods would lead me to expect that it's usually one measurement at a time. How this is achieved I don't entirely follow, but it is at any rate a complicated mathematical procedure. Most people just don't have the time to learn this even if they wanted to.
The end result is that most of the Scientific work going on at the moment is possible to explain to the layman, but not necessarily easy to justify. And as long as that is the case, then it's very hard to distinguish the "real Science", where the statistical tests have been correctly applied to the correct data with nothing embarrassing hidden away deliberately, to the pseudoscience and to the people who are lying to you behind the veil of "we did this properly". And while that remains the case, lots of potentially dangerous myths will hang around. Those people who say that sometimes scientists lie or are dishonest are perfectly correct, of course. But it can then be used to justify other, even more dubious claims. In the world of Physics this isn't all that dangerous -- people will just believe something that's wrong -- but in the world of medicine where lives and livelihoods are at stake, we can see all sorts of horrific consequences. The MMR scandal in this country, the anti-vaccine movement in general, rebellions against Polio in Afghanistan and African countries as some sort of Western conspiracy, the flagrant misuse of statistics in cases of Sudden Infant Death Syndrome in the 1990s that led to innocent mothers being convicted of murder... and so on and so forth.
The answer to the original question might seem a little defeatist, and maybe even a little pompous. But I think that the best way to be able to tell good Science from bad is to have a decent amount of scientific training. Equally, doctors are usually better placed to know good medicine from bad. This doesn't always hold, of course. But usually Scientists know how to do Science better than non-scientists do.
The problem with Statistics is a good one to raise. There are I think two separate problems, actually -- the first thing is that some statistics are just utter rubbish, or have been deliberately manipulated or massaged or otherwise made to give a particular impression when it was known to be a false one. Some people have cynical reasons for lying. It's most obvious in the medical industry, where naturally a drug that looks good will make you more money than one that does not.
There is a second problem, and that is that frankly most people just don't get it. This is not to insult anyone's intelligence, but a full statistical analysis is often very difficult to follow, and uses complicated methods that, while well-understood by those who use them, are almost impossible to follow for everyone else. How then is the layman to evaluate these sorts of tests? They just can't, really, and almost inevitably I suppose a certain amount of "faith" enters -- you trust that the people who perform these tests aren't lying through their teeth and relying on you not to bother looking over the work thoroughly, and usually this is the case.
The issue in the above is not that "most people are thick", by the way -- it's simply a true statement that most people haven't devoted a large amount of time to learning the mathematical techniques necessary to understand how a particular result was achieved. And usually it's hard to explain how it was, too. In one example I saw yesterday it was shown in a very sketchy way how it was possible to extract six different values from the same data, when my own relatively basic knowledge of statistical methods would lead me to expect that it's usually one measurement at a time. How this is achieved I don't entirely follow, but it is at any rate a complicated mathematical procedure. Most people just don't have the time to learn this even if they wanted to.
The end result is that most of the Scientific work going on at the moment is possible to explain to the layman, but not necessarily easy to justify. And as long as that is the case, then it's very hard to distinguish the "real Science", where the statistical tests have been correctly applied to the correct data with nothing embarrassing hidden away deliberately, to the pseudoscience and to the people who are lying to you behind the veil of "we did this properly". And while that remains the case, lots of potentially dangerous myths will hang around. Those people who say that sometimes scientists lie or are dishonest are perfectly correct, of course. But it can then be used to justify other, even more dubious claims. In the world of Physics this isn't all that dangerous -- people will just believe something that's wrong -- but in the world of medicine where lives and livelihoods are at stake, we can see all sorts of horrific consequences. The MMR scandal in this country, the anti-vaccine movement in general, rebellions against Polio in Afghanistan and African countries as some sort of Western conspiracy, the flagrant misuse of statistics in cases of Sudden Infant Death Syndrome in the 1990s that led to innocent mothers being convicted of murder... and so on and so forth.
The answer to the original question might seem a little defeatist, and maybe even a little pompous. But I think that the best way to be able to tell good Science from bad is to have a decent amount of scientific training. Equally, doctors are usually better placed to know good medicine from bad. This doesn't always hold, of course. But usually Scientists know how to do Science better than non-scientists do.
There are 'absolutely necessary truths'; those truths that are true no matter what. Examples might include: that yellow is a colour, that 1+1=2, nothing is a pyramid and also sphere, any heap of sand contains at least two grains of sand etc. Upholding those truths is not an opinion, it is ontologically factual.
An opinion is a belief based on grounds short of proof, a view based on probability.
"Good science" is well chosen epithet, it may be good, but it may also not be true. Scientific theories are continually being modified. Scientists propose theories , make hypotheses based on them, and then try to prove that specific hypothesis true or false through either experiment or careful observation. If the experiment or observation matches the prediction of the hypothesis, the scientist has gained support for the hypothesis (and therefore the underlying theory), but has not proven it. It's always possible that there's another explanation for the result.
An opinion is a belief based on grounds short of proof, a view based on probability.
"Good science" is well chosen epithet, it may be good, but it may also not be true. Scientific theories are continually being modified. Scientists propose theories , make hypotheses based on them, and then try to prove that specific hypothesis true or false through either experiment or careful observation. If the experiment or observation matches the prediction of the hypothesis, the scientist has gained support for the hypothesis (and therefore the underlying theory), but has not proven it. It's always possible that there's another explanation for the result.
Further to Khandro's comments which are essentially correct: the more evidence in support of a theory, the lower the probability that that theory is incorrect. Indeed, most current theories of Science are best-described as "correct, so far as they go" -- in other words, there are some known and understood limits at which the theory stops being valid, or experimental/ theoretical constraints that limit the level to which the theory is known.
Thus, for example, Newton's Theory of Gravity has been replaced by Einstein's Theory of General Relativity, but this replacement is only really needed (a) when gravity is particularly strong (so, near to Black holes for example), and (b) when objects are moving very fast, near to the speed of light. thus slow-moving objects far enough away from gravitational sources are described very well by Newton's theory, and thus at those limits the theory is to all intents and purposes correct (even though it's also strictly not the full theory).
The other final points, of course, are that just as "good science" can turn out to be wrong, so can opinions be based on bad science and turn out to be right (though not very often) -- and indeed there are some things that can be known without Science at all. The point worth stressing, though, is that "bad science" is right far less often than "good science" is.
Thus, for example, Newton's Theory of Gravity has been replaced by Einstein's Theory of General Relativity, but this replacement is only really needed (a) when gravity is particularly strong (so, near to Black holes for example), and (b) when objects are moving very fast, near to the speed of light. thus slow-moving objects far enough away from gravitational sources are described very well by Newton's theory, and thus at those limits the theory is to all intents and purposes correct (even though it's also strictly not the full theory).
The other final points, of course, are that just as "good science" can turn out to be wrong, so can opinions be based on bad science and turn out to be right (though not very often) -- and indeed there are some things that can be known without Science at all. The point worth stressing, though, is that "bad science" is right far less often than "good science" is.
jomifl, in his simple exclamation above accounts for nearly all "bad science". It is the need to uphold/verify a predefined hypothesis that fills the world with acres of bullsh1te. Sadly scientists are often involved because they need funding. Therefore we tend to get science "fashion" in the sense that scientist feel duty bound to give "correct" answers to get funding.
I think there's a difference between saying that some areas are researched more than others because they are "hot topics" and thus get funding, and that the results of the research reflect that desire for money. In the first place, experimental research is still valid research independent of the outcome. Either finding what you are looking for, or not, is a result that is equally worth the trouble so long as the experiment was conducted properly. It may be that you aren't surprised by the outcome. In theoretical research, meanwhile, scientists are still constrained by what is mathematically allowed, and will be soon caught out if they deliberately fudged any of their work.
That said, the issue of how to get funding is in some ways a real one -- but I don't think that it is as bad as you make out. After all, I (and most of the rest of my Theory colleagues) basically got funding without explaining what I was going to be doing! Not much possibility of bias there, though it might crop up later. Again, though, it's possibly somewhat dependent on the subject, and certainly some ideas attract less funding and attention than they properly deserve (and vice versa) depending on how fashions proceed. In the long term, these things tend to even out, though.
That said, the issue of how to get funding is in some ways a real one -- but I don't think that it is as bad as you make out. After all, I (and most of the rest of my Theory colleagues) basically got funding without explaining what I was going to be doing! Not much possibility of bias there, though it might crop up later. Again, though, it's possibly somewhat dependent on the subject, and certainly some ideas attract less funding and attention than they properly deserve (and vice versa) depending on how fashions proceed. In the long term, these things tend to even out, though.
Do all you can to check the credentials of the person making the claim. Has s/he earned a science degree ? Does s/he hold a recognised post teaching or researching the subject ? Is that person's work quoted or published in reputable science journals ? If that person has written a book, what do the reviewers say about it ? Was it published by a well-known publisher, or did the person making the claim publish it privately at his/her own expense ? Is the person a member of an academic body like the Royal Society of Chemistry ?
Related Questions
Sorry, we can't find any related questions. Try using the search bar at the top of the page to search for some keywords, or choose a topic and submit your own question.