City Builders Association Limited · Rōpū Hanganga Tāone
Advocacy document · October 2026

Fit-for-Purpose Evidence

Standards Auckland Council must apply to all data used in analysis

The Audit and Risk Committee is asked to make officers apply consistent fit-for-purpose assessments to all quantitative evidence cited in reports: sampling method and limitations recorded, non-representative surveys treated as the views of those who responded, and the standards the Council already applies to its own research extended to external material. Sent 16 October 2026.

A comprehensive advocacy document — prepared by Mark Pervan, Founder, City Builders Association

Prepared by
Mark Pervan, Founder, City Builders Association Limited
Date
October 2026
Sent to
Mike Giddey, Governance Advisor to the Audit and Risk Committee — 16 October 2026
Purpose
Ask the Audit and Risk Committee to recommend that officers apply consistent fit-for-purpose assessments to all quantitative evidence cited in reports.
Legal base
Local Government Act 2002 ss 76–79 and 82; Auckland Council Significance and Engagement Policy; Stats NZ quality dimensions; Research Association of New Zealand Code of Practice.

This is the full document as prepared and sent on 16 October 2026, with the original section numbering, including the advocacy article at section 6. It is described as sent only with its date and recipient, and as lodged only once a reference is published; no reference has been issued yet. Listed in the projects register.

Executive Summary

Auckland Council currently lacks enforceable standards governing the quality of external data — particularly surveys submitted by advocacy groups — that is used in analysis and decision-making. This creates a material risk to the integrity of advice papers and the reasonableness of decisions under the Local Government Act 2002.

Convenience samples (such as surveys taken at a bike rally), self-selected online polls, and poorly designed questions (such as “Do you think safety is important?”) routinely enter summary reports and can influence options analysis. These are not representative of Aucklanders in general and should not be treated as such.

This document sets out the legal basis for applying standards, the minimum criteria that should be required (drawing on Stats NZ quality dimensions, the Research Association of New Zealand Code of Practice, and the Council's own commissioned research standards), and practical changes needed. It includes the full advocacy article and the proposal submitted to the Audit and Risk Committee.

1. Problem statement

There are no binding formal standards that require Auckland Council to reject, discount, or critically weight poor-quality external data before using it in analysis or decision-making. Officers review and summarise all feedback. Decision-makers must consider the views and preferences of people likely to be affected (LGA 2002, ss 76–78 and 82). There is no parallel requirement to assess whether the evidence is fit for purpose.

Examples of the problem:

  • A survey conducted at a bike rally measures only the views of people who chose to attend a cycling event. It says nothing reliable about drivers, older residents, or non-cyclists.
  • A question such as “Do you think safety is important?” is almost content-free. Nearly everyone answers yes. It has no discriminatory power and is classic leading or vague design.
  • Advocacy groups rarely disclose sampling method, response rate, or limitations. Claims of “Aucklanders support…” are made without methodology.

3. Existing standards the Council already applies

Auckland Council applies higher standards to its own research. The Quality of Life Survey (commissioned from Ipsos and similar firms) includes technical reports covering sampling design, quotas, weighting to Census demographics, response rates, and data cleaning. The People's Panel is an opt-in panel; some results are weighted and reports note limitations.

Internal policy advice standards (from the 2013 Quality Policy Advice initiative, aligned with the national Policy Quality Framework) expect analysis to be evidence-based, objective, and clear about limitations.

3.1 Stats NZ quality dimensions

Relevance, Accuracy, Consistency, Timeliness, Accessibility, and Interpretability. These form the basis for assessing whether data supports informed decision-making.

3.2 Research Association of New Zealand (RANZ) Code of Practice

Requires research to be legal, honest, truthful, and objective, and carried out in accordance with appropriate scientific principles. Reporting must distinguish findings from interpretation.

5. What should change

  • Require officers to record, in summary reports and advice papers, the sampling method and key limitations of any external survey cited as evidence of wider public views.
  • Treat non-representative or poorly designed surveys as expressions of the views of the people who responded, not as estimates of Auckland-wide opinion, unless the methodology supports that inference.
  • Align the treatment of external quantitative evidence with the standards already applied to the Council's own commissioned surveys and with Stats NZ quality dimensions.
  • Make methodology disclosure a practical expectation for any advocacy group that wishes its survey results to be given weight beyond the views of its own supporters.

6. Full advocacy article

Title: Fit-for-Purpose Evidence: The Standards Auckland Council Must Apply to All Data Used in Analysis

Auckland Council must apply clear, enforceable standards to all data used in analysis and decision-making — including material submitted by advocacy groups.

Currently it does not. Officers are required to review and summarise all feedback received, and decision-makers must consider the views and preferences of people likely to be affected (Local Government Act 2002, ss 76–78 and 82). There is no parallel requirement to assess whether the evidence offered is fit for purpose. The result is that convenience samples, self-selected online polls, and questions that almost everyone answers the same way routinely enter summary reports and can influence options analysis and recommendations. That is a failure of process, not an inevitable feature of local government.

6.1 The minimum standards that should apply

Any quantitative claim about public opinion, preference, or support that is presented as evidence for a decision should meet basic quality criteria drawn from Stats NZ's quality dimensions (relevance, accuracy, consistency, interpretability), the Research Association of New Zealand Code of Practice, and the standards Auckland Council already applies to its own commissioned research.

6.2 Sampling and representativeness

A survey of people who attended a bike rally, signed an online petition, or completed a form promoted only through advocacy networks is a convenience sample. It measures the views of people who chose to participate. It does not measure the views of Aucklanders in general, drivers, older residents, people who do not cycle, or any defined population unless the sampling frame and response rate are disclosed and the results are weighted or caveated accordingly. Council's own surveys routinely report sampling method, achieved sample, response rate, and weighting. External material that claims “Aucklanders support…” without equivalent disclosure should be described in reports as non-representative and given limited weight.

6.3 Question design

Questions must be capable of distinguishing between different views. “Do you think safety is important?” fails this test. Almost every respondent answers yes. The question is vague, leading by implication, and produces data with no analytical value. Professional standards require questions that are specific, balanced, and free of leading language or social-desirability pressure.

6.4 Disclosure of methodology

Every survey cited in analysis or advice papers should state: target population, sampling method, sample size, response rate (where known), fieldwork dates, weighting applied, and any known biases or limitations. Failure to provide this information is itself a quality limitation.

6.5 Fit-for-purpose assessment in advice

Policy and project advice papers should explicitly assess the strength of evidence, including external surveys. A non-representative survey should not be presented as equivalent to a probability sample or to the Council's own weighted research.

6.7 What should change

  • Require officers to record the sampling method and key limitations of any external survey cited as evidence of wider public views.
  • Treat non-representative or poorly designed surveys as expressions of the views of the people who responded, not as estimates of Auckland-wide opinion.
  • Align the treatment of external quantitative evidence with the standards already applied to the Council's own commissioned surveys.
  • Make methodology disclosure a practical expectation for advocacy groups seeking weight for their results.

7. Proposal submitted to the Audit and Risk Committee

On 16 October 2026 a proposal was emailed to Mike Giddey, Governance Advisor to the Audit and Risk Committee. The Audit and Risk Committee advises the Governing Body on governance, risk management, and internal control and is the appropriate body for this matter.

The email requested that the committee consider recommending that officers apply consistent fit-for-purpose assessments to all quantitative evidence cited in reports, aligned with the standards already used for the Council's own commissioned research and with Stats NZ quality dimensions. The full article was included.

8. Conclusion

Advocacy groups are free to survey their members or rally attendees and to present those results. They are not free to have those results treated as equivalent to representative research without scrutiny. Auckland Council already knows how to design and report surveys to a professional standard. Applying that standard to the evidence it relies upon is a straightforward improvement in the quality of analysis and a necessary safeguard for the integrity of decision-making under the Local Government Act 2002.

The City Builders Association recommends that the Audit and Risk Committee and the Governing Body adopt the standards set out in this document.

Frequently asked questions

What standard is being asked for?

Four criteria for any quantitative claim about public opinion, preference or support presented as evidence for a decision: a sample that can carry the inference drawn from it, questions capable of distinguishing between different views, disclosure of methodology (target population, sampling method, sample size, response rate, fieldwork dates, weighting and known limitations), and an explicit statement of the strength of the evidence in the advice paper.

Why is a survey taken at a rally a problem?

It is a convenience sample: it measures the views of the people who chose to participate, not Aucklanders in general. It says nothing reliable about drivers, older residents or non-cyclists. Without the sampling frame and response rate disclosed, and without weighting or caveats, the result should be described in reports as non-representative and given limited weight.

What legal basis is relied on?

Sections 76–78 and 82 of the Local Government Act 2002 require reasonably practicable options to be identified, their advantages and disadvantages assessed, and the views and preferences of people likely to be affected considered. Information that is systematically biased or incapable of supporting the inference drawn from it is not relevant for the purpose it is offered for, and weight placed on it without caveats risks a decision that is unreasonable.

What has been sent, and to whom?

The full document was emailed on 16 October 2026 to the Governance Advisor to the Audit and Risk Committee, requesting that the committee consider recommending that officers apply consistent fit-for-purpose assessments to all quantitative evidence cited in reports. The Audit and Risk Committee advises the Governing Body on governance, risk management and internal control. No lodgement reference has been issued yet.

Does this apply to the Association's own surveys?

Yes. The document states that advocacy groups are free to survey their members or rally attendees and to present those results; they are not free to have them treated as equivalent to representative research without scrutiny. Methodology disclosure is framed as a practical expectation for any group that wants its results given weight beyond its own supporters.

Part of the Fit-for-Purpose Evidence Standards project. Spot an error? Email info@citybuildersassociation.org How do corrections work?.

Published by City Builders Association Limited (NZBN 9429053649225, company number 9426905). Director and author: Mark Pervan. Checked 23 October 2026.

Membership funds the research and does not buy editorial control or preferential treatment. A conclusion is marked lodged only when the page shows the date, the recipient, and the reference.

MethodAbout