A Bi-Monthly Newsletter
Volume 6, Issue 2, March 2003
STC WVC Home>Newsletter Table of Contents>Surveying End-Users for Usability
 

Surveying End-Users for Usability

I had suspected for at least two years that there were problems with my company’s manufacturing process and inspection process documents. Because of my workload, I had been unable to do any usability research.

I wanted to research several usability issues and other documentation problems. These issues resulted from adopting translated Legacy documents from our Japanese parent company, which was necessary to earn ISO 9001 certification, and a lack of technical writing staff. The problems I suspected were as follows:

1. The existing documentation was written for the wrong audience.
2. The documentation was too general.
3. Work instructions for many products consisted of a number of small, cross-referenced documents instead of a large product manual.
4. Some needed documents had not been written.
5. Some documents had not been updated to reflect processes that had changed.
6. Informal conversations with the manufacturing, warehouse, and quality staff that use the product documents indicated that our suspicions were correct. We wanted to obtain more formal proof. We decided to survey these document users so we wrote two surveys.

Surveying Direct Users in the Field

One of the surveys was intended for the general operator, warehouseperson, and inspector. It was a check-the-box, comment-on-your-answers-if-you-wish survey. We asked what document types the operators used and their reasons for not using the documents if they didn’t.

We asked whether they wanted product manuals or the existing small product documents, whether they wanted text only documents or text adequately supported by pictures, or just flow charts and posters. We also asked whether they preferred printed documents or whether they would use electronic documentation if the documents became available online.

Surveying Team Leads

The other survey was intended for the leads in the production, warehouse, and inspection areas. This survey was in question and answer format. We asked similar questions to those that we asked on the check-box survey and also asked several other questions. These questions asked them what the most important information was that they didn’t have, what other information they needed but didn’t have, and what other information they would like to have but didn’t have.

We organized three forums at which we planned to offer them coffee, juice, and cookies for their time and their answers. Since our plant operates around the clock, we decided to offer three forums at different time slots to catch the greatest number of our target audience. We held one forum on a Wednesday at mid-morning, one on a Friday morning just after midnight, and the other the following Monday. We passed out surveys to other shifts and departments that were unable to attend the forums because of their production schedules.

We also posted notices of the forums throughout the plant several days before the forums were scheduled to occur. Despite this, we discovered that it was necessary to visit the manufacturing areas and the warehouse in person to remind them of the forums. As a result, the forums were well attended. Additionally, many of the people unable to go to the forums returned surveys.

In all, we received approximately 90 short-form surveys. Of these, about 50 were from the forums and about 40 from the general distribution. We also received 12 long-form surveys.

The survey results converted our suspicions into facts. Especially in the long-form surveys, the document users agreed that they preferred comprehensive product manuals with step-by-step processes liberally supported by pictures. They preferred a great deal of detail suited to manufacturing operators rather than the relatively general documents that would suit engineers and highly trained operators.

The users also confirmed our suspicions about why they didn’t use the documents: Half of the users cited outdated information and nearly half cited unavailable (unwritten) documents as reasons that they did not use the documents. About one-third of the users cited wrong information and about one-third cited too many cross-references as reasons they did not use the documents.

When we report the results of the survey to our management, our recommendations will mirror the user preferences for well-illustrated comprehensive product manuals. We will also acknowledge the reasons that the users do not use the documents. Before recommending that the company hire more technical writers than the two we now have, we must obtain the support of the manufacturing supervisors and their leads and explain how hiring more technical writers will help the company make more money.

Russell Pike is a senior technical writer for JAE Oregon, which manufactures electrical components in Tualatin and in Tijuana, BC, Mexico. Russell can be reached at rcpike0401@aol.com.