Which type of survey question requires coding to help discover themes from responses?

1. Respondents don’t always like them much (it takes work on their part), often resulting in brief, unsatisfactory answers.
2. They lengthen the time to complete a survey which may lower your response rate, and
3. If there are many open-ended answers, it is tedious to read through all of the verbatim comments to identify any insights

Problems One and Two can be overcome by good question wording and survey formatting.  One solution to Problem Three is to code the responses into categories to aid in the analysis. However, coding those responses is one of the most tedious and frustrating tasks marketing researchers face.  This leads us to the topic of today’s blog.

Coding categorizes open-ended responses into groups that can then be used in analysis. The coding process is open to the judgment and interpretation of the coder, so it is something that must be done diligently and with a standard process. While there is software that can be used to help you code open-ended responses, there is no substitute for human intelligence and judgement to make sure the codes are appropriate.

Here’s a process to help you code open-ended responses:

1. What kind of question are you dealing with? Are the responses truly from an open-ended question or are they responses to an “other, please specify” question? In the latter case, you are looking for answers that could be coded back into existing categories as well as additional categories that should be added. If they are true open-ended questions with no existing responses, you’re starting from scratch. In either case, the process is generally the same.
2. Next, reading through the responses will help you get a feel for potential response categories. Depending on how many respondents you have, how many answered the question, and how long the verbatim responses are, you may choose to read only a sample of the responses. Try to read at least 10%. If you are only reading a sample, select randomly through the data set so that you don’t get a bias with early responses vs. later responses.
3. Identify potential response categories. At this point, you may have “too many” categories that you will need to combine or narrow down later.
4. Go back to the open-ended responses and try coding them. Once you have them all coded, try to combine similar responses. Try not to have more than 10 categories, with no individual category receiving less than 5% of responses.
5. Cover all responses. Even if you have an “other” category, every response must go somewhere.
6. Create codes that anyone reading the report will understand. One or two-word code descriptions really don’t provide any value or additional insight to research.
7. If a comment includes multiple topics, code it into multiple categories. For example, “Orange juice is delicious and nutritious” would be coded into both the “Delicious” and the “Nutritious” categories.
8. Sometimes separating the codes into positive, neutral and negative groups can add valuable information in analysis.
9. Repeat as necessary.

Open-ended question responses can be revealing and you may want to consider preparing a verbatim reports (simply a word document with all of the verbatim responses laid out by question.) Additionally, pulling out illustrative verbatim responses in the report can highlight key findings and bring your research report to life. Click here on how to write a research report that gets read!