ACRL College & Research Libraries News

College & Research Libraries News

RESEARCH FORUM: Conducting a mail survey: Some of the things you probably didn’t learn in any research methods course

By Marilyn Wurzburger

Head of Special Collections Arizona State University

If you need input from your colleagues to “fill in the blanks” for a research project, the obvious way to collect such information is a SURVEY! Innocent-sounding words. What precisely do you want to know? How do you identify which colleagues you want to reach? How many responses do you need for a “fair sampling”? What percentage response can you expect? Easy questions? Well…

I dived into my first survey with my mind on the anticipated data and with happy thoughts of some of my colleagues who might respond. Nevertheless I did come back to the above questions. Here are some of the approaches I took (or perhaps should have taken).

Designing the survey questionnaire

First, determine exactly what information you want to obtain and formulate questions that attempt to elicit valid answers. (Those of us who had courses in educational methods or actual teaching experience, will have already learned some of the pitfalls here.)

In finalizing your questionnaire, try to group your questions in logical order. Use major categories; group like questions and try to direct the respondent’s thinking toward a logical analysis of the problem (s) you’re researching so that the survey direction will be somewhat apparent. Have at least one colleague who knows something about your subject try to answer your survey questions; then ask someone outside the field who has good writing/editorial skills to critique your phraseology, sentence structure, grammar. These assistants may help identify your use of inappropriate jargon that might not be universally understood.

You may still find that terms that you believe have the same meaning for everyone working in the field, surprisingly (or maybe not so) do not. Try to avoid these ambiguities, but remember there will always be a “semanticist” who is more anxious to point out a misused or ill-placed comma or word than answer the questions. These replies are fun to read, and will alert you to areas in which interpretation might be needed when analyzing the results.

As questions are set up in final format, make sure that there is ample room for answers. Nothing is more frustrating than trying to fit a descriptive phrase into space that allows only one word. Consider eliminating as many variables as possible. Use a list or multiple-choice form for the most obvious options; respondents merely check the answer that best applies. Then, only the unusual or exceptional cases need be described in phrase or sentence. This organization gives some assurance that the replies are based on a similar frame of reference. It also helps greatly in correlating answers for the compilation of data—a formidable task in any case.

Selecting recipients of the survey

Above all, obtaining a list of probable respondents—those who share your concerns or interest in your project—is the first requisite of any survey. It takes time to answer a survey; hence you must ask yourself why anyone would spend that time for you. If your survey will provide data that can be useful to those whom you contact (and you promise to share the results with them) they may be very happy to comply with your request. If you are trying to further the quest of “pure scholarship,” you again will probably receive a degree of cooperation, but bear in mind that an esoteric subject such as comparing bibliographic points of rare books in our collections may strike those not interested in such matters as being a little like discussing the number of angels that can stand on a pin!

In my case, information on library security (particularly in special collections) was the goal of my research. The information I needed could best be provided by those people who are charged with the operation and overseeing of special collections in academic libraries. Although I could identify these people by studying the basic reference sources such as Guides to Special Collections, where department heads are listed, or by writing people identified in the literature, writers of articles, etc., I chose to use a listing of attendees of our specialized Rare Books and Manuscripts Section (RBMS) Preconferences during the past three years. I hoped that since I had met most of these people at the preconferences there might be a name recognition factor and thus a higher response to my plea for help. To those names I added others—personal friends and colleagues holding similar positions—who would be appropriate respondents.

The question of what constitutes a “fair sampling” is a knotty problem and has no pat answer. If your survey were to cover only special collections of rare books on dentistry, your reference base would be small. After all, how many special collections of dentistry exist in libraries in the U.S. or even around the world? If you located 25, it would be a relatively easy task to survey all 25. However, if you were asking a question that could be answered by every public library in the United States and abroad, you would obviously have to do a statistical sampling and use complex mathematical formulae to ensure that your sampling techniques would, indeed, yield significant results. In such a case the sample number would be much larger than 25. If your funds and time are limited, try to narrow the questions or problems to those that can be directed to a specific identifiable audience. If your audience is too broad, the responses may not reveal a trend or meaningful pattern.

“Packet” design

Before mailing time comes around, be sure to take into consideration the weight of the material that you’re sending out. Printing your survey on both sides of the paper reduces a six-page survey to three pages; adding a cover letter plus a return envelope puts you into a borderline category of the maximum that can be sent for one first-class stamp. You’ll want to avoid a packet that takes double postage for both outgoing letter and/or reply (unless you want to divert at least two–weeks’ grocery money to pay the postage tab). Furthermore, if the survey and accompanying materials are too “weighty” (and I mean this more in physical than a philosophical sense) you’ll frighten off even the most dedicated respondent. Bigger is not necessarily better in this case, so strive to be concise and to the point when making the final selection. Enclosing a stamped self-addressed envelope should help to ensure a better response.

Cover letter

Once your survey is developed, write your cover letter. These letters may end up going to people you’ve never met so you will have to introduce yourself and your subject of research. Tell the recipient why you want the information and what you plan to do with it. Be honest from the beginning. Tell them how long (approximately) it takes to complete the survey.

Technicalities

You may wonder whether to address questionnaires to a person’s home or work address. For some, things are less hectic at home and more time can be devoted to thoughtful consideration of your document; for others a query received at home might not receive consideration at all. From informal discussions with colleagues I’ve learned that the majority would prefer receiving work-related questionnaires at work, but many stated they would give equal or possibly more attention to a document received at home. Much will depend on how effective you are in gaining the attention of the recipient. If you’ve hit on a timely topic, you stand to garner a reasonably good response, but if the topic looks like a rehash of well-worn themes, don’t expect much.

A reasonable supposition might be that if nothing has appeared in library literature during the last five years covering the topic from the angle you’re probing, you may have a timely topic and get good responses.

The logistics of sending a survey involve more details than you might expect at first glance. You’ll probably want address labels carrying your name and address for return envelopes as well as for the envelopes being mailed out. You can prepare these efficiently by using a memory-typewriter. Set up a master to print a vertical column on a page of blank labels (i.e., type in the name and address to be reproduced as many times as there are labels in a column). The typical label sheet (8x11) consists of three columns of 11 labels each. Program your memory typewriter to produce one column; then repeat that program two more times to fill the 3 blank columns. You can then produce a “paste-up” that can be photocopied on label stock as many times as you wish. You will need two of these labels for every survey being sent out. Addressing the envelopes to your colleagues poses more of a problem.

Unless you plan to do a lot of follow-up, you’ll need those addresses only once—hence, as I see it, it’s as much work to set up a label program in a computer to run them off, as it is to type directly on label stock (or even hand-write, if your penmanship is reasonably readable).

With a promise of a fine dinner (or something equally appealing) try to enlist a friend to assist you in getting your survey ready for posting. The survey has to be collated and stapled, the cover letter added, the label and stamp attached to the return envelope, and the whole thing folded and inserted into the main envelope that’s been previously addressed, stamped and labeled with your return address. One hundred or more surveys will take two people at least six hours (each, not total) to get such a packet ready for mailing. The final step—sealing the master envelope—takes additional time.

Approximate costs

Photocopying costs for a six-page survey plus a cover letter for 125 packets will run approximately $25. Envelopes, when purchased on sale can run as little as one cent each. At the current rate of 22 cents per letter, first-class postage, for 100 letters and return envelopes, will cost $44. Miscellaneous costs of labels, staples, paper, new ribbon for your typewriter (or printer if you’re composing on computer or word processor) have not been figured in. But for a rough estimate, it will cost in the neighborhood of $100 to mail a survey to 100 people (or $1 per person surveyed in this size sample). If you promised to send the survey results to respondents, be sure to count additional money needed for postage, envelopes and photocopies. Allow an estimated 30-40 cents for each return letter, depending upon the number of pages in your reply.

If you’re applying for funding, such an amount will cover the bulk of your expenses for a no frills mailing. (If you add in the cost of “dinner at the Ritz” for your friend as payment for help rendered, you may need to allow for a slightly higher budget.) Once you’ve mailed your survey you’ll be beset by doubts, and think of several modifications you wish you’d made to the original document. Just forget it!

Timing

The time of year when you send out your survey may affect the response you’ll receive. If the survey arrives during the middle of Christmas holidays or summer vacation it might be buried—other demands will be taking higher priority. Moreover, your return date should not overlap into these periods. Consider the school term when contacting academic librarians—neither the first nor last weeks of the term are propitious times. If you have your survey ready at a time that coincides with a meeting or conference that will attract people you’re trying to reach, you could plan to distribute your survey to attendees of the conference, saving postage costs. However, you do risk having the surveys lost or misplaced when they are shuffled into a briefcase, along with conference handouts, possibly not to surface until some time after the meeting. Also, this method may not give you a controlled sample.

Wrapping it up

When your surveys begin to come back to you I can guarantee you’ll be excited! It’s a professional thrill to realize that your colleagues and friends have taken their valuable time to assist you in your research project, and you’ll feel a special bond with them.

Start tabulation right away—it’s much easier to do a few replies at a time, as they arrive, than wait to do them all at once. If you decide to analyze all replies together, after the majority have been received, you may wish to again enlist the help of a friend. Tabulating is a time-consuming process. Depending upon the nature and depth of your questionnaire, you should allow from 15 to 30 minutes to record the data from each one.

Percent of response

If you’re fortunate, I’ve been told, you will receive a response from about 30 % of those you contacted. If you’ve targeted your audience carefully, you may receive a greater return. In my own case I mailed 121 questionnaires on January 19 and had received 57 responses by April 2, or 47 %; 60 replies by May 1, or 50%.

Added benefits

You’ll probably receive responses that will make your day—colleagues who realize they have some of the same concerns you do. The next time you meet any of these people at a library conference, you’ll greet each other a bit more warmly, sharing a bond of having helped each other try to solve problems of mutual concern.

While it may take more hours than you anticipate to prepare and/or collate the results, a survey can open new doors for you. If you are able to use the information gleaned as a basis for an article, be sure to give credit where credit is due. If another colleague or institution has been particularly helpful and would appreciate recognition, give it!

You may even be asked to share the results of your research with other colleagues at local, regional or national conferences. So, dear first–time- or tenth–time-surveyer, be prepared to add a “new dimension” to your life. A survey is well worth the effort!