Unit 6

Connaway & Powell Chapter 5

The Questionnaire

Planning pg. 145-146

• define problem
• consider previous research
• hypothesize a solution
• identify information needed to test hypothesis
• identify potential respondents
• select best technique for collecting data

Advantages

• respondents are encouraged to be frank
• helps eliminate interviewer bias
• elimination of variation on questioning
• especially mail questionnaires allow for respondents to do them in their own time
• can be made so data is easy to collect and analyze
• large amount of info in a short time
• relatively cheap

Disadvantages

• elimination of personal contact
• issues with ambiguous questions
• responses mostly from those that are highly opinionated about the subject
• harder for uneducated participants
• resistance to mail
• non-response is high
• electronic only will not get to those who have limited access to technology

Types of questions pg. 148-149

• factual
• opinion and attitude
• information
• self-perception
• actual past or present behavior
• protective questions

never ask more questions than are necessary

Wildemuth Chapter 18-28

Transaction Logs
Used for user-system interaction behaviors & system-mediated interpersonal interactions (pg. 166)

server-side studies more prevalent

Advantages

·                     record of events as they actually happened

·                     data not dependent on human memory

·                     draws on a large volume of data

·                     can be used for both quantitative and qualitative

·                     used in both experimental and field study

client side logs much more resource intensive

Limitations

·                     context in which the event takes place is not logged

·                     technical limitations

·                     server-side hard to determine individual users

·                     cumbersome to process such large amounts of data

·                     ethical concerns

Process of Transaction Log Analysis pg. 169

1.            identification of appropriate data elements and sources

2.            data collection

3.            data cleansing and preparation

1.            User and session identification

2.            classifying and coding information behaviors

3.            classifying and coding pages and their characteristics

Think-aloud protocols

cognitive process through verbal walk reports of thoughts during experimentation (pg. 178)

Advantages

·                     easy to collect

·                     makes possible investigation of reactions, feelings, and problems

·                     able to see subject work through the process rather than hearing about it later

Disadvantages

·                     concerns about validity

·                     task performance may be slowed

·                     protocol may influence the thought process related to the task

·                     conflicts between theory and practice - no standard

Tips for using think-aloud protocol

·                     prepare warm up tasks

·                     minimize interaction between experimenters and subjects

Variations on the Method

·                     concurrent vs retrospective - think aloud while performing or after task

·                     individual vs collaborative - collaborative may be more natural to human behavior

·                     alternatives to face-to-face

·                     protocols used in combination with other methods

Direct Observation

What people do

focused on behaviors

·                     obtain permission to do the observation

·                     determine your sample/when to observe

·                     which people will be observed, how much detail, how each unit of behavior is defined

·                     observation schedule

Issues in Direct Observation

·                     behaviors not steady, may be hard to catch

·                     context may be important but hard to observe

·                     people don't like being watched

·                     other ethical issues

Participant Observation

from page 199

·                     the researcher is a participant in the setting

·                     participation leads to a better understanding of the people and processes involved

·                     better understanding can lead to better theories

Researcher takes on two roles

Consider how the two roles will be balanced

·                     passive, limited, participant-as-observer

·                     complete or active participation

Pitfalls

from page 202-203

·                     effects of the observer on the setting

·                     possibility that members of the culture will lead researcher to incorrect conclusions

·                     barriers that the characteristics of the setting and method can place in the way of collecting data

Research Diaries

capture life as it is lived

distinct in that they require repeated self-reporting over time, can vary widely

solicited vs unsolicited

·                     unstructured

·                     semi-structured

·                     structured

·                     interval-contingent

·                     signal-contingent

·                     event-contingent

Strengths

·                     ability to research things that might not otherwise be accessible

·                     information is captured at or close to time of occurrence

·                     can be unobtrusive

·                     participants are familiar with the idea

Weaknesses

·                     familiarity with format may lead to preconceived notions and information limitation

·                     diarist may not focus on events that benefit research

·                     significant burden

·                     many recording errors can occur

Design options

·                     establish lines of communication

·                     select willing participants

·                     collect portions of the diary at intervals

considerations

·                     amount of structure

·                     trigger for diary entry

·                     length of time the diary is kept

·                     length of time between recording

·                     technology used for the diary

Unstructured Interviews

no predefined theory, hypothesis, or questions

intention is to expose unintended themes or help get better understanding

disadvantages

·                     time consuming

·                     figuring out the level of control

·                     analyzing the data

Semi-structured Interviews

·                     predetermined questions

·                     question order can be changed

·                     questions can be reworded or not used

interview guide more flexible than questionnaire

developing the interview guide

·                     outline major topics

·                     list questions for each topic that need to be asked

o                  essential questions

o                  extra questions

o                  throw-away questions

o                  probing questions

·                     avoid these

o                  affectively worded questions

o                  questions that cover multiple topics in one question

o                  complex questions

·                     pretest the guide

specific techniques

time-line interviewing

critical incident teqchnique

interviews should be recorded

Focus Groups

a group assembled by researchers to discuss the topic of the research from the perspective of personal experience

strengths

·                     conversation give fuller perspective on a topic than interviews alone can give

·                     mimics natural setting in which people form opinions

·                     efficient

Issues to consider in conducting focus group research

1.            getting focus for the focus group

2.            selecting a moderator

3.            identifying and recruiting participants

4.            conducting the group sessions

5.            analyzing the results

can conduct focus groups online

Survey Research
Used to investigate many different topics such as

·                     program quality

·                     worker satisfaction

·                     information behavior

1 question / 1 response for each variable

Designing the survey

pg. 257 set of items, statements or questions, used to generate a response to each stated item.

brief and simple to complete

consult other studies before designing

pg. 257 19 principles

·                     ask only what can be answered and what is necessary for the objective

·                     ask in complete sentences

·                     use neutral language

·                     give consideration to specificity

·                     avoid double barreled questions

·                     voluntary participation means respondents might not, and should not have to, answer each question

2 types of questions

·                     open ended

·                     closed ended

Organization

suggested 3 parts

·                     introduction - build rapport, easy to answer

·                     substantive questions - essential questions

·                     classification questions - gather basic information

consider physical appearance - progress bar for online surveys

Testing the survey pg. 259

·                     pretesting - review of survey by experts or members of target audience

·                     administration of survey to a sample from the target audience

Administering the survey

Many ways to administer

·                     mail survey

·                     e-mail and ask respond by e-mail

·                     e-mail and request they fill out surveys on a website

·                     arrange online chat

·                     post survey on website

·                     phone people, ask them to reply by phone

·                     auto-phone people and ask them to respond by interactive phone application

Methods to increase response rate

·                     multiple, personalized contacting about the survey

·                     working with specific groups to help with the survey

·                     offering an incentive

Measuring Cognitive and Affective Variables

construct- something put together by scientists to help describe data and attitudes

operationalization - indicators and level of construct defined

develop an inventory

·                     use an existing inventory

o                  faster

o                  more efficient

o                  allows you to see how inventory was used before, strengths and weaknesses

o                  how realiable

o                  validity

reliability - how consistant it measures the construct

Developing new measures

Done if you can not find an appropriate variable to use in your study

examples of constructs already defined for LIS

·                     satisfaction with information systems and services

·                     attitudes towards information systems and services

defining construct

1.            completely satisfied

2.            mostly satisfied

3.            neither satisfied or not

4.            somewhat unsatisfied

5.            completely unsatisfied

Likert Scale

statement then respondent agrees or disagrees with statement

response style or response mode - bias based on how people want to answer

to counteract sometimes reverse the numbers in the ratings

Semantic Differential scale

pairs of adjectives across from each other on a continuum

Other approaches

can you complete this task? "yes" or "no"

Developing a measurement instrument pg. 281

1.            define the construct

2.            develop the items

3.            create an inventory from the set of items

4.            pilot test the inventory