Valid research starts by finding the right people to participate. We call this screening. Screening allows you to require participants to answer one or more questions that you specify to any level of granularity you like.
Our panel is big, consisting of millions of people in over 150 countries. It's so big and comprehensive, we have no problem letting you require up to 10 screening questions that participants must answer correctly to participate in your study.
In practice most studies ask just one to three questions because most "targeting" can be boiled down to only a few questions. For example, if you're a credit card company, you want to ensure that people have a credit card.
Writing a good screening question consists of a question and a list of answer choices. The question can either be a single or multi-select (multiple choice) style. And like all of our survey-style questions, you can choose from a list of pre-made questions or create your question from scratch.
Screening questions are created when you create your study. A screening question differs from a regular study question in one important respect. Each screening question item has a Screen-Out checkbox. Select this box if you want to screen out (not include) who selects this item.
Writing a good screening question
A good screening question usually consists of a list of choices which don't reveal the intent of the question. So, for example, consider the difference between a yes/no question like "Do you have a credit card?" and a question that asks "What financial products you currently have" which lists five to seven options, one of which is a credit card.
Structuring your questions this way can also help when presenting your results, because you've just minimized a natural reservation people will have.
What if I don't want to screen?
You don't have to screen, and some projects do not require it since the market for the product is pretty much anyone. If you have a product like this, you'll be happy to know these studies are the least expensive and fastest to run.
When you don't ask a screening question in your study, we still ask three basic demographic questions about gender, age, and technology use, so you'll always have some information about who each participant is.
The impact of screening on cost
As you probably can guess, the more screening questions you ask, the harder it is to find participants that match a given profile. The way we manage this is by increasing the incentive (the amount of money we pay the participant). The increased incentive improves the likelihood that people will try to qualify.
The simple formula is this: for every screening question you ask, we add an additional $5 onto the per-response charge. This allows us to pay the participant more and generally ensures your study will be a success.
The impact of screening on study completion time
Because we increase the participant incentive the more questions you ask, the impact of screening on study completion time is often negligible. Of course, it all depends on what you're screening for, and the number of participants, but generally most of our studies complete in a day or two, with longer times for larger samples.
Getting an even mix
When you're doing an experience2 test to compare one or more sites or prototypes head-to-head, you'll want to consider getting what we call an "even mix" of participants on each site or prototype. Setting aside gender fluidity for the sake of an example, getting an even mix on gender will give you a roughly 50% mix of men and women on each site or prototype in your study. For other standard questions such as income or age, the mix you will get reflects the distribution of these categories in the general population. So you'll get fewer high-income people for example.
Screening and other demographic questions
Sometimes you want to collect information about who people are (their demographics), without screening. If this is the case, we generally recommend including the question in the body of the study itself (in the Study portion of our study creation process, rather than in the Screening section). That said, there are two exceptions to this.
- When you would like to get an even mix of participants on each site or prototype in an experience2 study
- When you want an even mix of participants in a basic usability test
Getting an even mix of people means we will screen the same blend of each demographic category into the test. When you're getting an even mix of people, screening questions can contain a combination of screen outs, non-screen outs, or no screen outs at all.
Why all this focus on "mixes"? Methodological sticklers will require it if you want to compare two groups, as you usually do with an experience2 test. It lets you say things like "we showed the same group of people version A and the same group of people version B, and people were 20% more likely to buy on version B than on version A."
To screen or not to screen
A good test is this: if you want the question to affect the allocation of participants within the test, make it a screening question. If it doesn't matter to the allocation, and you just want to know something about a participant, put it in the body of the study.