An index of utility can be distinguished from an index of reliability and an index of validity in that an index of utility can tell us something about:
how consistently a test measures what it measures in a particular context
whether a test measures what it purports to measure in a particular context
the practical value of the information derived from what a test measures
none of these
Test validity:
has nothing whatever to do with test utility
sets a ceiling on test reliability
sets a ceiling on test utility
none are correct
It is probably the case that a self-report test of integrity:
is a valid test
is a reliable test
is not a psychometrically sound test
has little utility
A utility analysis is conducted using:
expectancy tables
Taylor-Russell tables
Naylor-Shine tables
all are correct
The specific objective of a utility analysis will dictate what sort of information will be required, as well as the specific:
methods to be used
expectancy tables to be used
Naylor-Shine tables to be used
Rise-and-Shine tables to be used
In the now classic utility analysis for the telephone company conducted in the 1980s, Cascio and Ramos concluded that:
the use of a particular approach to assessment in selecting managers could save the company millions of dollars
the use of a new approach to assessment in selecting managers was of little utility and would cost millions of dollars
charging for 411 calls to “Information” could result in millions of dollars in new revenue for the company
customers would make more calls if they could understand their phone bills
The statistical tool that is ideally suited for making selection decisions within the framework of a compensatory model is:
multiple regression
the Brogden-Cronbach-Gleser formula
expectancy data
utility analysis
The Angoff method of setting cutting scores relies heavily on ________.
data and empirical findings
the judgment of experts
the scholarly research literature
both data and emprical findings and the scholarly research literature
A problem with using the known group method of setting cut score is that:
there is no consistent method of obtaining contrasting groups
strong deterrents to test user acceptance of the data are in place
no standards are in place for choosing contrasting groups
test users must be personally familiar with each member in the known group
Which is NOT an economic cost typically factored in a test utility analysis?
legal costs of doing business
fees paid by test-takers for testing services
computerized test processing services
supplies of blank test protocols
In the context of utility analysis, which is NOT a direct economic benefit to a company?
increase in worker morale
less product being trashed as waste
increase in the quality of workers’ performance
both an increase in worker morale and an increase in the quality of workers' performance
The term "bookmark method" refers to an IRT-based method:
of marking items with regard to difficulty in a book of items
of setting cut scores based on expert judgment
with possible drawbacks such as floor or ceiling effects
all are correct
When the selection ratio goes down:
top-down selection policy can become discriminatory
hiring becomes less selective
competition for the position is likely to increase
both a and b
The more complex the job:
the more people differ on how well they do it
the less people differ on how well they do it
the less selective the hiring process
the more the need for item-mapping procedures
Using a cut score of 50 on a predictor test a researcher finds a base rate of 1.00. This means that when a cut score of 50 is used:
50% of applicants will perform successfully on a criterion measure
100% of applicants will perform successfully on a criterion measure
100% of applicants will fail on a criterion measure
50% of applicants will fail on a criterion measure
Validity is to ________ as utility is to ________.
consistency; accuracy
usefulness; consistency
usefulness; accuracy
accuracy; usefulness
Which of the following is NOT an assumption of utility analysis?
the value of people and their performance can be estimated
psychological tests are always preferred over other means of assessment
the performance of people in organizations can affect organizational viability
large amounts of information can be integrated to make good decisions
An educational psychologist conducts a utility analysis of a teaching program used to improve the handwriting of very young children. The measure of utility in this analysis will most likely be:
increase in performance level
decrease in costs
reduction in accidents
increase in revenue
The term used to describe the proportion of people in a population who possess a given characteristic is:
success rate
base rate
selection ratio
sensitivity
The "miss rate" is equivalent to:
the number of incorrect classifications/the selection ratio
the base rate/the selection ratio
the success rate/base rate of successful performance
the number of correct classifications/total number of classifications
An empirical standard used to divide a group of data into two or more distinct categories describes a:
cut score
predictive yield
norm-referenced test
hit rate
Which of the following statements is FALSE regarding cut scores?
The setting of cut scores impact utility though the accuracy of decisions made based on test scores
In multiple hurdle selection, there will be a cut score for each predictor used in the selection process
An instructor who assigns an “A” to the top 10% of students in a class is using a fixed cut score
Absolute cut scores are preferred when applicants must demonstrate a minimum level of competence
Research cited in Chapter 7 suggested that when drivers chat on cell phones while driving:
they are usually apt to engage in text-messaging as well
they drive just as well as when they are not on a cell phone
how they drive depends on their ability to multitask
they are less likely to look in their rear view mirror
Costs associated with testing include all of the following EXCEPT:
administrator time
use of test facility
return on investment
test administrator’s time
If criterion-related validity is .40 and base rate is .60, the false positive rate is likely to be highest if the selection ratio is: