Guide to the Census of Population, 2016
Chapter 4 – Content determination
How are the census questions determined?
Determining the content of a census is an ongoing process involving user consultations, content testing and content approval. Content refers primarily to the content of the census short- and long-form questionnaires (also referred to as the 2A and 2A-L Forms).
Before each census, Statistics Canada undertakes a user consultation and testing program to determine the census questions, taking emerging social and economic issues into account.
While consultation is an ongoing process and data users are encouraged to communicate their data needs at any time, Statistics Canada holds a formal public consultation aimed at gaining feedback from census data users, on how they use the data and to identify data gaps. Results from this consultation feed into the content testing process, followed by the formulation of recommendations on final questionnaire content and the subsequent approval process.
For the 2016 Census of Population, Statistics Canada conducted comprehensive consultations with federal departments, provincial/territorial and municipal government organizations, Aboriginal organizations, organizations representing official language minority communities, organizations representing or providing services to Canadians with disabilities, immigrant communities, academia, non-profit organizations and the general public. These data users and interested parties across Canada were asked for their requirements for census information. The content was then tested qualitatively and quantitatively.
Final recommendations regarding content were based on the census' content determination framework which assesses users' information needs (e.g., strength of user need, size of population of interest, suitability of alternative sources); respondent burden and privacy concerns; and Statistics Canada considerations such as operational factors and costs.
The final questions were presented to and approved by Cabinet, and were published in the Canada Gazette.
Below is a description of how content consultation, content testing and content approval were done for the 2016 Census.
Step 1 – Content consultation
An ongoing consultation process with census data users allows Statistics Canada to identify if the content is relevant, how the data are used, and the importance of the census to the Canadian population. A formal content consultation process is typically set in place four years before a census year. During that time, Statistics Canada invites data users, stakeholders and the public to provide their feedback on what information is used, for what purpose, and what, if any, data gaps there may be that could be explored for inclusion in the next cycle of the census.
For the 2016 Census, the 2016 Census Strategy Project included consultations with senior representatives from various federal, provincial, territorial and municipal governments in the summer and fall of 2011, in order to develop a framework for determining census content. Furthermore, the Census Marketing Team (Census Operations Division) conducted the formal 2016 Census Program Content Consultation from September to November 2012: a two-tier consultation process allowed for feedback from all census data users.
The following two-tier approach was used:
- Tier 1: A general call, broadly promoted, sought feedback from data users using a 'Discussion Points' questionnaire which asked users to rank the importance of census topics, the uses of essential topics, input on suggested modifications and questions about the level of geography used.
- Tier 2: Selected data users were asked to report on their priority census data uses and requirements. This included more than 300 organizations (senior representatives of federal and provincial governments, municipalities, community groups, academia, business and special interest groups).
Further details can be found in the Census Program Content Consultation Report, Census year 2016, Catalogue no. 92-137-X, release date: September 10, 2014.
Step 2 – Content testing
Each cycle of the census involves extensive testing. Once feedback is obtained from data users during consultation, decisions are made by subject-matter experts about content that could be tested for the upcoming census.
The first part of this process involves qualitative testing. For the 2016 process, qualitative testing of content was conducted with the help of the Statistics Canada Questionnaire Design Resource Centre (QDRC) in three testing periods: June 2013, September 2013 and January 2014, which included focus groups to test the collection of the Social Insurance Number (SIN). Statistics Canada also conducted a focus group test within the deaf community to gain feedback as to whether the language questions on the census were well understood by this particular population. The deaf community focus group was held in the fall of 2014. The QDRC also conducted a test on mobile devices in February 2016. The purpose of this test was to gain feedback on how participants responded to the short- and long-form census questionnaires using their own mobile devices.
After qualitative testing is complete, the second part of the testing process involves quantitative testing. For the 2016 Census, the quantitative content test was held in May 2014.
May 2014 Content Test
In May and June 2014, a content test was conducted. This test was designed to meet two main objectives: to assess the impact of any proposed content changes and to measure the impact of adding a Social Insurance Number question. The Content Test required the development of different test and control forms, in English and in French, distributed among 11 analysis panels (independent samples).
Various modifications were tested, including changes to the length of the questionnaire, the wording of some questions, as well as examples and reference text to aid in responding to the questionnaire.
In total, 55,000 dwellings divided into 11 analysis panels were contacted during the Content Test. Both paper and online forms were tested, and questionnaires were available in both official languages. The mode of collection for this test was respondent self-enumeration followed by enumerator-assisted non-response follow-up for some panels.
Results from the Content Test shaped the final short-form and long-form questions that were presented to the Government of Canada for approval.
Details about this test can be found in the report 2016 Census Program Content Test: Design and Results, Catalogue no. 92-140-X2016001, release date: April 1, 2016.
Changes to the 2016 Census questionnaire
The most significant change to the 2016 Census was the decision by the government to reinstate the mandatory long-form census questionnaire. In 2016, a sample of 25% of Canadian households received a long-form questionnaire, which also includes the short-form questions. The other households received a short-form questionnaire. The decision to re-instate the mandatory long form did not change the content of the long-form questionnaire.
The main content changes introduced for the 2016 Census of Population are the following:
On the short form and long form
- The questions on sex, date of birth, age and marital status were changed to an interrogative format.
- Statistics Canada informed respondents that their income information would be retrieved from personal income tax and benefits files, replacing income-related questions asked on the long form in previous censuses. Income data were obtained for all respondents and will be disseminated with the short- and long-form census variables.
On the long form
- The question on religion was not included in the 2016 Census since it is only asked every 10 years. The question will be considered for inclusion in the 2021 Census.
- The questions on activities of daily living were redesigned into one question, with sub-items providing more detail about the type of difficulties a person may have doing certain daily activities.
- The online long form had more detailed flows in Step C to help respondents determine who should be included on the census form. This took advantage of functionalities available to online questionnaires.
- Questions from the short form and long form were integrated into one form for 2016. The long-form paper questionnaire contained a transition message (introducing the long-form questions) located between the last question of the short form (Question 10) and the first question of the long form (Question 11). Question 10 and the transition text were eliminated from the online form to improve the flow. However, since the decision to reinstate the mandatory long form occurred in November 2015, it was too late to modify and re-print the paper long forms.
Step 3 – Content approval
Once Statistics Canada has determined the proposed census content, the content is sent to Cabinet for approval and an Order-in-Council (OIC) is signed by the Governor General. Once this approval has been obtained, there is a final approval process by Statistics Canada with an order signed by the Chief Statistician.
For the 2016 Census, the OIC was signed by the Governor General on January 29, 2016, and the final approved 2016 Census content and the OIC were published in Part 1 of the Canada Gazette on February 6, 2016.
The internal prescription order was signed by the Chief Statistician on February 25, 2016. All 2016 Census questionnaires are available on the Statistics Canada website.
- Date modified: