Methodology


The 2019 Lowy Institute Poll reports the results of a national survey of 2130 Australian adults between 12 and 25 March 2019. The survey was conducted by the Social Research Centre (SRC), using the Life in Australia™ panel – currently the only probability-based online panel in Australia. Members of the panel were randomly recruited via their landline or mobile telephone (rather than being self-selected volunteers) and agreed to provide their contact details to take part in surveys on a regular basis. SRC uses a mixed-mode approach for the panel, including online surveys (89% of respondents) and computer-assisted telephone interviewing (11% of respondents) to provide coverage of the offline population (households without internet access). The order of questions in the questionnaire was different from the order presented in this report.

On a simple random sample of 2130 responses, the margin of error is 2.1%. Where a complex sample is used, the ‘design effect’ measures the additional variance in comparison with a simple random sample. The design effect for this survey is estimated at 2.40.

For the 2019 Lowy Institute Poll survey, a completion rate of 79.1% was achieved. Taking into account the recruitment rate to the panel and attrition from the panel, the cumulative response is 8.9%, which compares favourably with international probability-based panels. Unlike other commercial online panels in Australia, the probability basis of the Life in Australia sampling method means results are generalisable to the national population and sampling errors and confidence intervals can be calculated.

The use of the Life in Australia panel for the 2019 Lowy Institute Poll represents the final stage in a three-year transition in the methodology for Lowy Institute polling, which until 2017 was conducted solely by telephone. From 2005 to 2011, the Poll was conducted by landline only. From 2012 to 2017, it was conducted using both landline and mobile numbers. In 2017, SRC administered four key questions from the Poll to an online sample of 2513 respondents in parallel with the telephone survey of 1200 respondents, which was reported in the 2017 Lowy Institute Poll. This parallel survey provided valuable comparison information between the two methodologies. In 2018, the Lowy Institute Poll was a combination of a telephone-only sample of 600 respondents and an online sample of 600 respondents drawn from the SRC Life in Australia panel.

In order to ensure comparability of the 2018 Poll with the telephone responses of our 2005–17 Polls, the response sets were weighted and blended using the following approach: first, weights for the telephone respondents (50% of the sample) were calculated accounting for the dual chances of being contacted by landline or mobile, and reflecting key population characteristics. Then, for each online respondent, the most similar telephone respondent was found using a range of survey variables, and that person’s telephone weighting was used as the ‘base weight’ for the next step. Finally, the telephone and online responses were then combined into a single data set, and the results then weighted to reflect the demographic profile of the Australian population aged 18 years and over based on Australian Bureau of Statistics population data.

The transition to a predominantly online survey panel for the 2019 Lowy Institute Poll mirrors shifts in survey methods by highly respected polling organisations internationally. The Pew Research Center has recently moved the majority of its US polling online, primarily through its American Trends Panel, another probability-based online panel. The Chicago Council on Global Affairs, which has been conducting nationwide surveys on foreign policy since 1974, moved to online polling in 2004. The decision to move to an online survey methodology was made because of the declining productivity of telephone surveys. The number of fixed-line telephones is steadily decreasing, and more of the population has moved to mobile-only households which facilitates call-screening. A decline in contact rates and, to a lesser extent, cooperation rates, has led to reduced response rates for landline and mobile phones.

Some questions in the 2019 Lowy Institute Poll are new and are not affected by the possibility for ‘mode’ differences (where respondents may answer the same question differently in an in-person telephone interview compared with an online survey which removes the human element and provides more time for consideration of responses). However, the majority of questions in the 2019 survey are ‘tracking’ questions that have been asked in previous Lowy Institute Polls, allowing us to compare public opinion on a single issue over time. Footnotes and dotted lines have been used in the charts in the Poll to represent this change of mode, which can elicit slightly different responses.

Data quality checks for the online portion of the sample included checks for ‘speeding’ (completing the survey rapidly), ‘straight-lining’ (providing the same answer to every question of a bank of items with the same response options), and the number of non-substantive responses given (don’t know or refused). There was no single determining factor in deciding whether to exclude a case: factors were considered together and included the degree of speeding and the difficulty of straight-lining a particular bank of items.