Yesterday I spoke rather spontaneously during the final session of the 2010 Silicon Valley P-Camp - the third annual Silicon Valley ’un-conference’ focused entirely on Product Management topics/issues.

Spontaneous talks are not uncommon at an un-conference…the whole concept is that anyone can submit a topic for consideration and attendees vote on what they are most interested in. I came to the conference with a topic I thought of that morning and 20 handouts that I printed before I ran out the door (I thought that bringing any more than that would be a bit presumptuous!) and lo and behold my topic was chosen by popular vote to be one of the 12-15 afternoon sessions!

When I say ’spontaneous’, I really mean that…this is the first time I have EVER facilitated a 45 minute session with absolutely NO agenda, just a broad idea of the topic: “Choosing the Right Methodology for Gathering Customer Input”.

We ended up with 30 people all crammed in a small meeting area to explore this topic, specifically focused on using customer input for product development/marketing/launch purposes.  The session was very interactive and one of the things we discussed before getting into the meat of the topic was why product managers so seldom actually integrate any customer research into their work to launch great products.

I came with my own theories, but we had an active dialogue and below is a list of some of the reasons that came up…I’ll be blogging about some of these in the future, but wanted to provide a summary list here:

We talked about each of these obstacles, and debunked most of them, but it was important to ‘go there’ before talking about how to choose the right tools.  According to The 280 Group’s 2009 Product Management survey, the top 3 sources of influence on product management decisions are:

Why do sales and service representatives feel compelled to tell customers how to answer a survey? Does the company want to know what the customers really think, or is the company trying to build positive publicity by claiming superior ratings?

The answer to the second question exposes the company’s culture and customer experience management motives — whether they are striving to be customer centric (eager to know and act on what customers really think), or happy to be self centric (eager for positive publicity). Maybe the motive behind the satisfaction survey depends on the sponsoring organization; perhaps a Marketing-sponsored satisfaction survey will naturally lean toward PR objectives, while a Quality-sponsored satisfaction survey will naturally lean toward continual improvement. Regardless of the sponsor, here’s why it’s best to pursue a customer centric survey strategy:

1) Investment: Surveys are an investment of customer time and of company funds, manpower and time – aren’t there more straightforward (honest) and cost-effective ways to build positive publicity? From a statistical view, manipulated surveys are worthless. Even the positive publicity is not sustainable, if it is inaccurate. Telling customers how to respond to a survey makes the survey results invalid, and the whole effort a waste of everyone’s time and money.

2) Customer Management: Many companies are concerned with respondent fatigue issues, so it’s essential to design surveys wisely and use results wisely. And since customer expectations can rise after they participate in a survey, it’s wise to have a well-established process in place to act promptly and systemically on survey results.

3) Growth: Marketing is overlooking lucrative opportunities to heighten their value within the organization if they do not view their role as a voice of the customer conduit into all functional areas across the company.

The answer to the first question reveals weaknesses in the company’s performance management strategy — either imbalanced scorecards or poor training of employees. Customer experience management scorecards should balance lagging indicators and leading indicators, with greater weight placed on the latter. Leading indicators are metrics that are actionable at the manager and worker levels, with a strong (predictive) tie to the customer survey ratings, and which can be measured before customers experience their effects. Survey results are lagging indicators because they reflect what customers have already experienced. If sales and service employees know their performance is being measured primarily by leading indicators, and secondarily by lagging indicators, their compulsion to tell customers how to rate them will be lessened. With the proper setup of customer satisfaction incentive pay, employees should be trained to respect customer’s pure assessments of the business and its related services, and to welcome constructive customer feedback.

The practice of coaching customers on satisfaction surveys should end! Customers feel insulted to be told what to say, especially in our Web 2.0 world, where customers are now accustomed to thinking independently and voicing their true opinions. To end this manipulation, go to the root cause of it: the company’s motives and/or the employees’ bonus calculation. Customers will reward you well for doing the right thing the right way.

Contact Lynn to find out how to customize these tips to your situation.

Lynn Hunsaker, founder of ClearAction, specializes in customer-centric culture-building, customer data ROI, and cross-organizational engagement to deliver brand promises. Lynn has rich hybrid skills and executive experience in marketing & market research, process improvement & quality, organizational development & change management, and leadership & teaching. She managed customer experience programs for more than half of her career, and her executive roles include Head of Corporate Quality, Director of Marketing & Business Development, Manager of Voice of the Customer, Manager of Customer Services, Manager of Strategic Information, and Market Research Manager. She has worked with Accountants Inc, Adecco, Anritsu, Applied Materials, Cadence, Deltek, FormFactor, Hospira, Merck, MicroWarehouse, Sonoco, SunPower, and many other companies.

In preparing for a session of the Power of Market Research class that I teach at UCSC Extension Silicon Valley,  I was brushing up my ‘spiel’ about how to determine sample size for research studies (surveys, focus groups, etc.)…and realize that this is a mysterious concept that could use some plain talk.

Sample size is a very important element of any primary research.  Very rarely can you get input from everyone within your target group, so you need to figure out a way to ensure that you have a ‘sample’ of the group that is big enough and representative enough of the entire group to provide ‘reliable’ results—in other words, how many people do you need in order to predict how the entire group you are researching thinks/feels?

When determining sample size, you want to think about the reliability of your consolidated results (the input of all respondents/participants) as well as the reliability of the data you receive for any groups/segments you are analyzing.  For example, if you are going to segment your results by household income or annual revenue, you need to make sure that you get enough respondents in each category of income or revenue to provide reliable results.  This will mean that you need many more respondents than required to provide reliable results for the consolidated group as a whole.

Click here for a tool to help determine the sample size you need to get reliable results, based on the size of your target group and the confidence interval you are aiming for.  Tip: I suggest using a 95% +/- 10% confidence level as a good goal for how reliable you want your data to be.  Using that approach, for any target group consisting of 2000 or more people, you would need about 90 respondents.

Hopefully this info will help you increase the validity of your data and, more importantly, your own confidence in the data you gather.