Another polling question that misleads is the one about whether the country is heading in the right direction or not.
Aside from the questionable practice of at least one pollster whose question refers to 'on the right track' the main fault with the question is the lack of a follow up to clarify what the 'right' direction would be.
So you get a result which says that, for example x% say the country under this government is heading in the wrong direction, but this answer does not differentiate between those wanting a right wing direction and those wanting a more left wing direction.
Most (of the breathless) poll reporting shows a very high level of statistical ignorance. This is notable in the way single numbers are given and the margin of error only added (generally inaccurately) as an after thought.
Good poll reporting would show results in bands. For example if a party's poll number is 45% and the margin of error is + or - 3% then the result should be shown as being 95% likely to fall within the band 42% to 48%.
Then there is the excitement over single poll results, when it is the trend that is what really matters.
Here is a bit more on margin of error that Roy Morgan includes at the end if their poll reports. The table hasn't formatted well, but note that the magnitude of the margin of error is different for different polling numbers - a point skimmed over by other pollsters when reporting the poll results.
Margin of Error
The margin of error to be allowed for in any estimate depends mainly on the number of interviews on which it is based. Margin of error gives indications of the likely range within which estimates would be 95% likely to fall, expressed as the number of percentage points above or below the actual estimate. Allowance for design effects (such as stratification and weighting) should be made as appropriate.
Another polling question that misleads is the one about whether the country is heading in the right direction or not.
Aside from the questionable practice of at least one pollster whose question refers to 'on the right track' the main fault with the question is the lack of a follow up to clarify what the 'right' direction would be.
So you get a result which says that, for example x% say the country under this government is heading in the wrong direction, but this answer does not differentiate between those wanting a right wing direction and those wanting a more left wing direction.
Most (of the breathless) poll reporting shows a very high level of statistical ignorance. This is notable in the way single numbers are given and the margin of error only added (generally inaccurately) as an after thought.
Good poll reporting would show results in bands. For example if a party's poll number is 45% and the margin of error is + or - 3% then the result should be shown as being 95% likely to fall within the band 42% to 48%.
Then there is the excitement over single poll results, when it is the trend that is what really matters.
Here is a bit more on margin of error that Roy Morgan includes at the end if their poll reports. The table hasn't formatted well, but note that the magnitude of the margin of error is different for different polling numbers - a point skimmed over by other pollsters when reporting the poll results.
Margin of Error
The margin of error to be allowed for in any estimate depends mainly on the number of interviews on which it is based. Margin of error gives indications of the likely range within which estimates would be 95% likely to fall, expressed as the number of percentage points above or below the actual estimate. Allowance for design effects (such as stratification and weighting) should be made as appropriate.
Sample Size Percentage Estimate
40% – 60% 25% or 75% 10% or 90% 5% or 95%
1,000 ±3.0 ±2.7 ±1.9 ±1.3
5,000 ±1.4 ±1.2 ±0.8 ±0.6
7,500 ±1.1 ±1.0 ±0.7 ±0.5
10,000 ±1.0 ±0.9 ±0.6 ±0.4
20,000 ±0.7 ±0.6 ±0.4 ±0.3
50,000 ±0.4 ±0.4 ±0.3 ±0.2