Unit 5

Connaway and Powell Chapter 3

Basic vs applied research
Basic -

• theoretical 
• focusing on testing hypothesis and developing new general knowledge

Applied -

• provides information that is immediatly useable
•  for specific problems, may not be applicable beyond the original scope of the study
•  can lead to revision of theories

Both add to the body of knowledge

Action research
Direct application to the workplace of the researcher wherese applied may be applicable to the profession as a whole.
pg. 72

• practical
• orderly
• flexible
• adaptive
• weak in internal and external validity

Basic steps of applied research (pg. 72)

• defining the problem or setting the goal
• review the literature
• formulate testable hypothesis
• arrange the research setting
• establishing measurement techniques and evaluation criterea
• analyzing the data and evaluating the results

Evidence based research

Make decisions based on current data

Evaluative research

Not about discovering new knowledge, looks at the application of knowledge.

• Summative
• effects of program
• quantatative
• determine if program will continue
• Formative
• done while program is running
• qualitative
• what needs to be changed in the program right now

Specific types of evaluative research

• standards
• cost analysis
• performance measurement
• stats
• questionnaires
• interviews
• observations
• unobtrusive questions
• diaries
• more

Specific research methods

• Survey - used most in descriptive studies
• Experimental - manipulation of one variable to see how another variable reacts.  For causal relationships
• Historical 
• Operations - application of scientific method to management.  Resource allocation, inventory, competitive strategy
• Modeling - determine performance of a real system by observing performance of the model
• Systems analysis - similar to operations but focuses on the entire system
• Case Study - organizational structure and functions or performance.  Multiple data collection techniques used.  Well suited to collective descriptive data.
• Delphi study - sequential questionnaires that get more refined as they go.  Policy level decision making.
• Content analysis - determine the content of a book, film, etc.
• Bibliometrics - specifically for library and information science.  Being used to study Internet searches
• Task-based research - research about a specific task.  Used in learning/teaching especially
• Comparative librarianship - comparing specifics about different libraries that have different environments.
• Technology-based research methods - protocol analysis, forms of transaction monitoring, 

Ethics of research

General guidelines 

• Institutional review boards
• balancing costs vs potential benefits for participants

Guidelines for LIS professionals

• Keep user anonymity
• Publication issues
• credit
• plagiarism
• falsification and fabrication of data

Ethics for researching on the internet

• public vs private information
• informed consent
• existing guidelines are limited

Scientific and research misconduct 

• legal definitions of misconduct
• institution guidelines

Connaway and Powell Chapter 4

Survey Research

Generalize from smaller group to larger group

• sample- smaller group
• population - larger group

Data in a survey can be particularly susceptible to bias introduced during research.

Differences between survey research and other research pg. 108

• contemporary data
• no manipulation of variables
• good for large number of cases
• studying personal factors
• exploratory analysis of relationships

Types of surveys

When choosing type must keep in mind question, sources of information, nature of the data and more to determine correct type of survey.

Exploratory survey

• often qualitative
• clarify concepts
• establish priorities for future research
• types pg. 108-109
• literature surveys
• experience surveys
• analysis of "insight-stimulating examples"
• can only suggest insight
• considered a first step

Analytical and Descriptive surveys

• analytical survey
• quantitative data
• need statistical analysis to be understood

Other types of surveys

• cross-sectional
• trend study
• cohort study
• panel study
• approximation of longitudinal study
• parallel samples study
• contextual
• sociometric
• critical incident

Basic Purpose of Descriptive surveys

usually strong at testing relationships between variables than exploratory research

Basic Steps of Survey Research

• formulating an objective
• selecting data collection techniques
• where safeguards against bias are put in place
• selecting the sample
• consider statistical and practical differences within sample and population
• collecting the data
• data should be "cleaned" (checked over) as soon as possible when its done being collected
• analyzing and interpreting the results
• plan for this early on in the survey process
• survey research designs
• descriptive is the most straight forward
• analytical is more sophisticated and may require more complex design
• static group design
• panel design

The Cost of Survey Research

Tends to be inexpensive but there is a list of ways to make it even less expensive included on page 115.

Sampling

Basic terms (pg. 116)

• Population
• Population stratum
• Element
• Census
• Sample
• Case
• Sampling frame

Types of sampling

• non-probability
• accidental sample
• quota sample
• snowball sample
• purposive sample
• self-selective sample
• incomplete sample
• probability 
• simple random sample
• systematic sample
• stratified random sample
• cluster sample

Wildemuth Chapters 7-15

Chapter 7: Case Study

Research study focused on a single or few specific cases

11 key characteristics (pg. 51-52)

1. phenomenon is studied in a natural setting
2. data are collected by multiple means
3. only one or a few entities are examined
4. complexity of the unit is studied intensively
5. more suitable for exploration, classification, and development of hypothesis
6. no experimental controls or manipulation are involved
7. investigator may not specify the set of independent and dependent variables
8. the results depend highly on the skills of the investigator
9. as the investigator develops new hypothesis, the site selection and data collection means can change
10. useful for why and how questions
11. focus is on contemporary events

Can be used as a pilot study for data collection methods or to develop familiarity

May be used to follow up an exploratory study

weakness - too specific and findings may lack generalized application

Designing a Case Study

• define research question
• perform literature review
• identify unit of analysis
• select case or cases for study
• plan data collection procedures

Identifying Unit of Analysis

The major entity you will be studying

most studies focus on individuals but may do groups, or organizations, projects, or events.

Selecting a case:

• theoretical sampling - trying to replicate a theory
• statistical sampling - selecting representatives of a population of interest

multiple-case study also called comparative case study - combining of two single case studies

• literal replication - trying to find similar cases expecting similar outcomes
• theoretical replication - cases are different, trying to expand scope of the study

Collecting data:

• analysis of existing docs
• interviews
• direct observation - most frequently used
• participant observation
• examination of physical artifacts
• questionnaires

triangulation - combining of data from different data collection methods

4 types of triangulation

1. data triangulation
2. investigator triangulation
3. methodological triangulation
4. theory triangulation

Strengths and Weaknesses of Case Studies

Biggest weakness - lack of generalizability of findings

Greatest strength - particularization and really understanding your specific subject and its environment

Can be used to test theories

4 criteria for determining quality of case study

1. Resonance criteria
2. Rhetorical criteria
3. Empowerment criteria
4. Applicability critera

Chapter 8: Naturalistic Research

Approximate natural, uncontrived studies (pg. 62)

Challenge of validity

There are no controls with this type of research, so there may be validity issues

Doing Naturalistic Research

Doing direct observation in the field, watching people at a library and seeing how they use the library on their own.

Can help find patterns and to develop theories

Degree of Naturalism

Can be varying degrees of "natural".  Observing may not be as natural as actually partaking in the activity to see how the participants really feel or what they go through.  IE asking a reference question at the library, not just observing.

Type of Insight

Gain insight into people's naturally occurring behaviors in a natural stetting  (pg. 65)

Specific Techniques

• Field observation - only observing not participating
• continuous monitoring
• sampling
• Ethnography - anthropology and sociology.  in-depth study of a culture
• Contextual Inquiry - learn quickly over a few days by job shadowing the participant
• Cognitive Work Analysis - multiefaceted
• 7 steps dimensions used by Fidel (pg. 67)
• environment
• work domain
• organization
• task in work domain terms
• task in decision-making terms
• that task in terms of strategies that can be used
• the actors' resources and values
• Quasi-experiments - breaking people into different groups that you can't control. For example, observing people who have taken a class, and those who haven't.

Chapter 9: Longitudinal Studies

Long study, done over time.  Studying a process over time.

Advantages

• observe changes over time
• can examine duration of a phenomenon

Data Collection

Can collect almost any kind of data.  Main rule is that data collection must remain consistent over time.

Challenges

• challenges with the sample
• hard to measure the same variables over long periods of time
• outside events affect the study

Chapter 10: Delphi Studies

Forecasting future events based on expert opinion.  Named after the Oracle of Delphi from Greek Mythology.

Refining input from experts to understand something in the present or predict something in the future.

Characteristics of the Delphi Method (pg. 84-85)

• considered more efficient and accurate than some other models
• allows more people to be in the group, and controls information to help the group stay focused
• allows for a diverse population with polarizing views to contribute without fear of argument or reaction
• strong personalities can not dominate the study
• group pressure to conform is avoided
• body language and other non-verbal ques do not affect the participants
• time and cost are reduced
• several rounds of data are collected
• participants are asked to provide justification if their answers fall out of the range of the group consensus.

Conducting a Delphi Study

1. Pick your sample group of experts
2. First round, open-ended questionnaire
3. Calculate responses from first questionnaire, present second questionnaire with questions based on findings
4. Calculate responses and formulate a third questionnaire, ask those that choose answers different than the norm to explain their answers.
5. Finalize findings

Criticism of the Delphi Method

• Lack of statistical tests
• No description of participants
• How experts are selected
• Explanation is needed only when answers are outside the norm.  More explanation could be included to get a better understanding of the responses.
• Different levels of anonymity could be used for the study

Things to Avoid When Conducting a Delphi Study (pg. 87-88)

• The researcher forcing his or her opinion on the group with the wording of the questions
• Inadequately summarizing and presenting group responses
• Manipulating consensuses by ignoring arguments rather than letting them happen
• Not properly compensating participants for the amount of time they will be investing
• Cultural misunderstandings 

Chapter 11: Quasi-experimental Studies

• Used in natural settings
• Some control of experimental conditions can be had, but not total control
• Done in real-world settings instead of a laboratory
• More confidence that findings may be attainable in the real world
• Applied research rather than theoretical

Specific Designs pg. 94

3 Types

• time series design
• nonequivalent control group design
• counterbalanced design

Time Series Design

Tests for changes over time due to treatment.

Nonequivalent Control Group Design

Two similar but different groups are studied.  One who has treatment and the other who does not.

Counterbalanced Design

Multiple groups and multiple treatments are tested.

Risks to Design and Interpretation

This experiment is to demonstrate how the treatment affects the subject.  To make sure this is true, researchers must exclude other variables that might affect the results.

Risks affect both internal and external validity.

• Internal - did the treatment affect the dependent variable
• External - Can the findings from a specific group be generalized?

Selection Bias - groups are different from each other in a systematic way.  Mostly caused by the inability to randomly assign subjects.

Morality Effects - participants drop out and the study isn't even 

History Effects - if someone interacts with the control group in a way that wasn't anticipated between the pretest and the post-test.  To lessen this, reduce time between pre and post tests.

Testing Effects - Different testing effects can affect results.  Time between tests, similar test items....

Chapter 12: Experimental Studies

Some variables are manipulated and then their effects upon other variables is observed (pg. 105)

Characteristics 

• control
• randomization

Three Experimental Designs

1. Pretest-posttest Control Group Design - groups assigned randomly.  Pretest is given, intervention introduced and posttests are given.
2. Posttest-only Control Group Design - 
1. No pretest.  
2. Can't track changes in individuals
3. No history effects
4. pretest can interfere with the intervention
3. Factorial Design pg. 107 - simultaneously investigates all of the effects from the independent variables on the dependent variables.

Randomization must be done systematically

Validity of Your Experiment

See Internal and External validity above

Threats to Internal Validity

• loss of participants
• cross-contamination of groups

Threats to External Validity

• pretesting can change the factors of the experiment
• attributes of population are not well represented

Lab vs the Field
The lab helps with internal validity
The field helps with external validity

Within- versus Between-subjects Designs
Are groups independent or overlapping

Ethics
The specific environmental setting may be hard for participants to deal with
The intervention may be beneficial and therefore giving it to only one group and not the other may be unethical.

Sampling for Extensive Studies

Representative samples are a must for a successful study.

Probability Sampling
Sampling frame - list of all eligible elements in the population (pg. 117)
Two characteristics

1. "Every element of the population of interest has a known non-zero probability of being selected in the sample" (pg. 117)
2. Elements are selected randomly

• Carefully define your population
• If desired population is not available define target population and actual population being used
• Specify unit of analysis
• Construct sampling frame - a list of elements in the population
• Select specific elements from your sampling frame

Simple Random Sampling

• Single sampling frame
• All elements in frame have the same probability of being chosen
• Number elements
• Use random number generator to create a random list
• Match the numbers on the list to the elements they represent

Systematic Sampling

Identify the first element to be included.  Then use random number n and included every nth element after the first element, then the next, and so on until you get a population size you need.

This is done when the sampling frame is too large for simple random sampling.

Stratified Sampling

• Population divided into strata
• Random elements are selected from each strata
• Will need to define your strata
• May need a smaller sample size
• Decide on sample size for each strata

Cluster Sampling

• Clusters of elements not individuals
• Multistage cluster sampling is possible
• minimize cost

Non-probability Sampling

When random sampling is not possible

Quota Sampling

• Which characteristics of the population are of interest
• Set up quota for each characteristic
• Recruit until you've met your quota
• No random selection of elements from a sampling frame

Purposive Sampling

• People chosen from population of interest based on their characteristics
• potential for bias

Snowball Sampling

• For  sample members that are hard to identify or when the topic is sensitive
• Identify one or a few members
• Then ask those members to identify other members for participation
• Non-representative sample

Convenience Sampling

Recruit people because they are available

Sample Size

Use as small of a sample as possible to keep your study efficient and cost effective.

Balance goals for accuracy against cost

Wider variance requires larger sample

When testing hypotheses (pg. 122):

• effect size you want to be able to detect
• the smaller effect you want to detect, the larger sample you will need
• Type I and Type II errors

Usability Testing

• Special case
• one argues small number is all that is needed
• other argues that a large number is needed to test usability

Effects of Non-response on a Sample

• Try to improve response rate
• Non-response throws off sample
• Use appropriate data collection procedures to improve response rate
• Lower the burden of response
• Offer incentives
• Can compare respondents with non-respondents
• Non-responders more likely to have responded like late responders than early responders.

Key concepts Unit 3

Connaway and Powell Chapter 2

Deductive method - reasons towards observations.
Inductive method - reasons from observations.

Criteria  for research (pg. 23)

• Universality
• Replication
• Control
• Measurement

Steps in research

Identification of the problem

Before you research you need to identify and clearly understand your problem,  If the problem is not understood and narrow enough, researching will be unable to answer it correctly.

Reading research on a subject can often help identify further research to be done.

Characteristics of a problem (pgs. 43-44)

• represent conceptual thinking, insight, and inquiry
• variables should represent a meaningful relationship - causality
• researcher's interest
• represent an area of research that is relatively new
• will research contribute to knowledge in the field - how important is it?
• is it manageable?  Can you actually research it given your constraints?

Statement of the problem

Write the problem down in clear, concise, accurate, terms.

Identify subproblems (pgs. 45-46)

Can be a way to deal with the bigger problem in smaller pieces.  When added together should be summation of the whole of the problem, not go beyond the problem.  If they go beyond the problem that's an indication that the problem is not narrow enough.

Most research problems can be broken down into 2 - 6 subproblems.

psedu-subproblems - problems more about the procedure than about the problem.

2 step process

• break main problem down into components
• identify key words

Metatheory

The philosophy behind research theory

Classification of data

• can help point out gaps
• can help identify relationships

Theory  - interrelation of a set of variables based on rules of logic, or a set or propositions that explain why events occur the way that they do.

theory can:

•  help organization
• explain relationships
• act as a guide

theory does

•  summarize knowledge
• provide explanations
• predict occurrences

a good theory should:

• provide means for its own testing
• should be well grounded and be able to explain what is being studied
• should be complete but stated as simply as possible

Hypothesis

An expectation that will be tested by researching the problem.

Types of hypothesis (pg. 51):

• Working or research
• Final
• Particular
• Causal
• Alternative
• Null
• Inductive
• Deductive
• Non-directional
• Directional
• Multivariate
• Bivariate
• Univariate

Developing a hypothesis - usually from theory and knowledge of subject

Concepts - used to define parts of the research.  Avoid circular definitions which define terms by using those same terms.

Wildemuth Chapter 2

Two ways research questions originate:

• your own experience or observation of the subject
• logical gaps that exist in the field

Research question (pg. 12)

"A statement of what you wish to know about some unsatisfactory situation."

Once question has been stated consider your experience with the subject and the perspective of any stakeholders involved.

Start reading about the subject to determine if your question has been answered, and gather more information.  Do searches including checking citations.

4 matters to consider:

• Is there uncertainty about the answer?
• Is the question important?
• Will study be useful no matter if hypothesis is found true or disproved?
• How feasible is it to answer  your question?

Stating your research question:

• Write it down
• Be explicit
• Should be a single sentence
• Should be able to define all concepts and components

Study feasibility

• Do you have access to those people you need as participants?
• What equipment and material resources do you need?
• Political support needed to conduct study

Wildemuth Chapter 3

Evidence-based information practice:

• Decisions based on evidence available
• Constantly seek ways to improve
• Most focus on using information from already completed studies

Librarians should look to collaborate with nearby universities in order to do more research. (pg. 22)

Formulating a LIS practice-based question

• focus on core activties, not just new technologies
• management, education, and reference services

Carrying out the study (pg. 23)

Practice-based research is usually done in a place the researcher is involved.  So the researcher will have to keep his or her objectivity.  Plan for any problems that may occur.

Applying the results

Results can be applied two ways:

• modify or improve what you already do
• make recommendations that can influence practitioners

It is suggested to always publish findings so they can be of use to others.

Wildemuth Chapter 5

Defining hypothesis (pg. 33) - A conjectural statement about the relationship between two (or more) variables.  Capable of verification.

Sources of hypothesis:

• Direct experience with the phenomenon
• Evidence provided by prior studies

Testing the hypothesis:

• define all parts of the hypothesis; make sure there are no vague words
• try to state hypothesis as a null hypothesis

Williamson Chapter 3

The beginning stages of research

pg. 51

qualitative research phrases a problem in a way to indicate the type of understanding that will be achieved by the research rather than within the context of a relationship between two variables.

Literature Review

In quantitative research the theoretical framework for a study should come from literature reviews (pg. 61)

The literature review helps the researcher understand the background and context of his or her research subject.
It can help determine which is the best choice of research method to pursue for a specific topic.

Do not limit your search to only your specific discipline.  Information about a topic could be found in many different areas.

Steps for writing a literature review: (pg. 63)

1. Categorize into subject/topic
2. Begin with an introduction
3. End intro with content overview
4. Organize body under headings which relate to your research.
5. Critically anylize literature under three headings
1. conceptual
2. anecdotal
3. research
6. Write a conclusion
7. End with the research question you will work on
8. Check that you have written a critical and evaluative review.

Key Concepts Unit 1

Wildemuth Chapter 1

     INTRODUCTION

• Research to improve the professional practice is called evidence based practice - a take on evidence based medicine.

      CHAPTER 1 (pg. 5-23)

• Research can be defined many ways but is usually considered an investigation & inquiry process.
• Not as linear as many would like to think it is.
• Basic vs. applied - fundamental and theoretical vs. solving specific problems.
• Research elements (pg 10):
• questions or objectives - hypothesis
• literature search and review
• research design
• analysis of findings
• interpretation and reporting
• Why research?
• assist others in understanding workplace issues
• add to knowledge of the field
• to maintain services that are of need and non-stagnant
• for the accountability
• to maintain and improve professional status
• to provide a body of research and findings for other professionals

Powell, Baker, Mika 

      

      Reasons for research by information professionals: 

• improve problem solving and decision making
• to make librarians active and critical consumers of research
• to better equip librarians with the ability to help others with their research

      Reasons LIS professionals are not more involved in research: (pg.50-51)

• There is poor communication between active researchers and practitioners; 
• Practitioners feel overwhelmed by the professional literature; 
• The great amount of research jargon discourages practitioners from reading the research literature; 
• Practitioners have all that they can handle keeping up with emerging information technology and service responsibilities;
•  R.R. Powell et al. / Library & Information Science Research 24 (2002) 49–72 5. There is inadequate education in research methods; 
•  There is the perception that research is lacking in practical applications or mission orientation; 
•  There is a lack of funding; 
• There is a natural resistance to change or new research ideas.

Williamson Chapter 1

Traditionally research has been focused on quantitative data.

Uses of research pg. 7

• problem solving
• development
• evaluation and improvement
• provide information before offering a new service or system
• theory building

Basic research 

• focused on theory building and developing of hypothesis
• not interested in how the information is applied

Applied research

• problem solving
• focused on application of the research
• used in information environments and businesses

Elements of a research project pg. 10

• research questions and objectives, hypothesis when applicable
• literature review and/or theoretical frame work
• research design
• description of the sample
• methods and techniques of collecting data
• analysis and findings
• interpretation, application, and conclusion

Connaway and Powell Chapter 1

Library research tends to be applied research

Shortcomings of library research pg. 5

• research done has not been shared widely
• librarians have not kept up with research results
• profession is content with non-research reports
• audience for research journals is limited
• dissertations are not connected to work done before or work that will come after, they are too isolated
• impacted has been hampered by too few copies of articles to share

The future of library research

the best way to solve the current issue is with education - make sure to put focus on research during the LIS education process

Unit 2

Williamson, Burstein, and McKemmish

Deductive reasoning - moves from general principals to particular instances.
Inductive reasoning - begins with particular principals and moves to general statements.

Positivism Central Themes

• natural sciences and social sciences should be studied the same.
• all scientific knowledge is based on experience.

The world is a collection of observable events and facts.  It is a linear process, and most types of research done in this style are either experiments or rigorous surveys.  Uses hypothesis and testing to determine generalizations. A hypothesis is put forth and either proven or disproved. 

Interpretivism 

Naturalistic inquiry - field work in natural setting.

• They believe the social world is different than the natural world.  
• They make sense of their world on their own terms and believe in multiple realities which are individually constructed.  
• Constructivism, critical theory, and phenomenology (the study of experiences of phenomena) are some major paradigms.  
• Focused on the beliefs and feelings of those involved and how they interpret their reality.  They take on a less linear approach and do not use hypothesis.

Using Qualatative and Quantative research for the MLIS profession

User surveys (qualitative)

• Who is using the library and why?
• Who isn't using the library and why not?

Performance measures (quantatative) - what has been accomplished by specific programs.

Archive management (both)  - evolution of the system, how to preserve records digitialy

Information systems (both) - their roles and uses in society

Introduction to Imperial Research

Based on observations.
To avoid misleading results focus on: why, whom, how, and when.

• why - establishes need for study
• whom - entire population or just a sample
• how - objective tests, interviews, direct observation, or new measure

experimental research - giving treatments and seeing if they make observable changes in behavior.

• experimental group
• control group

true experiment - when the subjects are divided into two groups by random.

non-experimental studies - observe participants to see how they exist naturally.

one popular type is survey or poll

causal-comparative study - non-experimental but trying to identify possible causes of a current situation by looking at the past of those in the experiment.  Can have downfalls especially if participants are not chosen carefully, but is sometimes the only ethical or practical option.

other types of non-experimental studies:

• census
• case study
• longitudinal research

quantitative research - researchers gather data that is easy to quantify and use for statistical analysis.

qualitative research - researchers gather data that is analyzed through informed judgement and major and minor themes are identified.

variables - a trait with two or more categories.  All variables have mutually exclusive categories (if you belong to one you can not belong to another).

• independent 
• dependent

All experiments have at least one independent and one dependent variable.

research hypothesis - prediction of outcome

• directional
• non-directional

Quantitative vs Qualitative

outcomes

• quantitative - presented as quantities and/or numbers
• qualitative - trends or themes, not based on statistics

samples

• quantitative - larger, random
• qualitative - smaller, purposeful

when to use each

• some questions will lend themselves more to one or the other
• when little is known about the topic quantitative is usually preferred.
• when subjects belong to a closed off or little known about culture qualitative is best.
• participants not available for extensive interactions quantitative is preferred.
• time and funds limited, quantitative
• when hard numbers are required quantitative is preferred.