Any sampling of opinions or attitudes is subject to a margin of error, which represents the difference between a sample of a given population and the total population. The margin of error is a statistic expressing the amount of random sampling error in a survey's results and differs by sample size. For example, let's say a survey has a MoE of ±2.33%. Any percentage found in this survey has a cushion of 2.33 percent above and below that might be due to statistical or sampling error. This means that if 19% of respondents in this example survey said they did not vote in the 2024 November general election, the percentage has a margin of error of ±2.33%. In other words, the actual percentage of non-voting Oregonians falls somewhere between 16.66%-21.33%. 

OVBC always ensures that our surveys are representative of the state demographics recorded by the US Census for adults 18+. After we are done fielding a study, the sample demographics rarely exactly reflect state demographics. Surveys generally receive more responses from older respondents, white respondents, women, and those with a more formal education. To ensure that we get an accurate depiction of ALL Oregonians, OVBC weights its survey once it is out of the field to ensure that the sample closely reflects the demographics (and therefore the comprehensive views) of the entire state.  

When we mention in a memo that there is a “significant” difference between certain groups, we mean that it is “statistically significant." This means the probability of an actual difference occurring by random chance is 5% or less.

Weighting is a statistical technique that adjusts survey responses to accurately represent the target population. This technique ensures that categories of people who are often underrepresented in survey data have their voices fully represented.  In OVBC’s case, we adjust survey results to the U.S. Census Data for Oregon to make sure that our data represents the target population of ALL Oregonian adults 18+.

No survey perfectly mirrors a state of more than four million people. Just by chance, we might end up with slightly too many college graduates, too few young adults, or an imbalance across regions of the state. Rake weighting corrects for these mismatches by giving each respondent a weight: people from groups that are underrepresented in our sample get a boost, and people from overrepresented groups get a little deflated since their voices are so common. Once these weights are applied, the survey's totals more closely resemble Oregon's population as a whole. Our weights are based on U.S. Census data for Oregon released in 2023, and they align our samples with the state's population on six characteristics: gender, age, region of the state, race and ethnicity, political party registration, and education.

~~~

The following excerpt from Pew Research Center describes the raked weighting system that OVBC employs on all of its public opinion surveys: 

“For public opinion surveys, the most prevalent method for weighting is iterative proportional fitting, more commonly referred to as raking. With raking, a researcher chooses a set of variables where the population distribution is known, and the procedure iteratively adjusts the weight for each case until the sample distribution aligns with the population for those variables. For example, a researcher might specify that the sample should be 48% male and 52% female, and 40% with a high school education or less, 31% who have completed some college, and 29% college graduates. The process will adjust the weights so that the gender ratio for the weighted survey sample matches the desired population distribution. Next, the weights are adjusted so that the education groups are in the correct proportion. If the adjustment for education pushes the sex distribution out of alignment, then the weights are adjusted again so that men and women are represented in the desired proportion. The process is repeated until the weighted distribution of all the variables matches their specified targets. 

Raking is popular because it is relatively simple to implement, and it only requires knowing the marginal proportions for each variable used in weighting. That is, it is possible to weight sex, age, education, race and geographic region separately without having to first know the population proportion for every combination of characteristics (e.g., the share that are male, 18- to 34-year-old, white college graduates living in the Midwest). Raking is the standard weighting method used by Pew Research Center and many other public pollsters.” 

OVBC employs raked weighting on all of its public opinion surveys to adjust survey responses to represent the target population more accurately. There are other weighting methods (propensity, matching, etc.) that researchers can utilize to weight their data and reduce bias, so why does OVBC use raking?

• Bias Reduction: Raking ensures survey results more closely align with the actual public composition without increasing error margins. Raking adjusts sample margins on key demographics (e.g., age, gender, race/ethnicity, education, political party, region) to reduce bias and reflect the known population structure. OVBC often uses our own and third-party opt-in panels to conduct research. The raked weighting methodology helps correct non-representativeness in opt-in samples. 
• Practical and Robust: Compared to matching or propensity weighting, raking is easier to implement and still reduces bias considerably when adjustment variables are well chosen. In a study by Pew Research, it performed similarly when compared to other methods, particularly when only demographic variables were used.

• Flexible: OVBC sometimes adds variables beyond core demographics. For example, when conducting our annual Portland: Your City, Your Choice survey, we swap our typical statewide regional demographic weight with one that ensures respondents come from each of Portland's four districts. This improves alignment for regionally sensitive questions. Raking can easily integrate these additional margins into the calibration process.
• Field-tested Approach: Raked weighting is consistently used across the public opinion sphere; both Pew’s ATP and NPORS methodology for national surveys include raking in their methodologies.

Sources:

• Why and how we're weighting surveys for past presidential vote | Pew Research
• For weighting online opt-in samples, what matters most? | Pew Research

OVBC weights our statewide surveys based on U.S. Census data for Oregon released in 2021. Our weighted demographics are:

• Gender (Men, Women, Non-binary or other gender identity)
• Age (18-29, 30-44, 45-54, 55-64, 65-74, 75+ years)
• Area of State (Tri-County, Willamette Valley, Rest of State)
• Race/Ethnicity (White, BIPOC)
• Education (High school graduate or less, Some college, 4-year degree or more)
• Party* (Democrat, Republican, All other)

*Updated political party statistics are taken from the Oregon Secretary of State webpage.

No. Our data consists of 100% real people; we do not incorporate any synthetic AI-generated responses.

At OVBC protecting your data is our highest priority. For people to trust our work, we must:  

• Respect your privacy, safety, and personal identifiable information. 
• Clarify the risks and benefits of participation. 
• Ensure data is accessible to all and that we stay accountable to the public. 

We treat your information with the same care we would want for ourselves, but only you can decide what is safe for you. While we do everything possible to keep your data safe, no system is 100% perfect. To help you make a more informed decision, we have developed the following document listing our data protection practices: OVBC Data Protection & Best Practices