Chapter 7: Utility

An index of utility can be distinguished from an index of reliability and an index of validity in that an index of utility can tell us something about:
1. how consistently a test measures what it measures in a particular context
2. whether a test measures what it purports to measure in a particular context
3. the practical value of the information derived from what a test measures
4. none of these
Test validity:
1. has nothing whatever to do with test utility
2. sets a ceiling on test reliability
3. sets a ceiling on test utility
4. none are correct
It is probably the case that a self-report test of integrity:
1. is a valid test
2. is a reliable test
3. is not a psychometrically sound test
4. has little utility
A utility analysis is conducted using:
1. expectancy tables
2. Taylor-Russell tables
3. Naylor-Shine tables
4. all are correct
The specific objective of a utility analysis will dictate what sort of information will be required, as well as the specific:
1. methods to be used
2. expectancy tables to be used
3. Naylor-Shine tables to be used
4. Rise-and-Shine tables to be used
In the now classic utility analysis for the telephone company conducted in the 1980s, Cascio and Ramos concluded that:
1. the use of a particular approach to assessment in selecting managers could save the company millions of dollars
2. the use of a new approach to assessment in selecting managers was of little utility and would cost millions of dollars
3. charging for 411 calls to “Information” could result in millions of dollars in new revenue for the company
4. customers would make more calls if they could understand their phone bills
The statistical tool that is ideally suited for making selection decisions within the framework of a compensatory model is:
1. multiple regression
2. the Brogden-Cronbach-Gleser formula
3. expectancy data
4. utility analysis
The Angoff method of setting cutting scores relies heavily on ________.
1. data and empirical findings
2. the judgment of experts
3. the scholarly research literature
4. both data and emprical findings and the scholarly research literature
A problem with using the known group method of setting cut score is that:
1. there is no consistent method of obtaining contrasting groups
2. strong deterrents to test user acceptance of the data are in place
3. no standards are in place for choosing contrasting groups
4. test users must be personally familiar with each member in the known group
Which is NOT an economic cost typically factored in a test utility analysis?
1. legal costs of doing business
2. fees paid by test-takers for testing services
3. computerized test processing services
4. supplies of blank test protocols
In the context of utility analysis, which is NOT a direct economic benefit to a company?
1. increase in worker morale
2. less product being trashed as waste
3. increase in the quality of workers’ performance
4. both an increase in worker morale and an increase in the quality of workers' performance
The term "bookmark method" refers to an IRT-based method:
1. of marking items with regard to difficulty in a book of items
2. of setting cut scores based on expert judgment
3. with possible drawbacks such as floor or ceiling effects
4. all are correct
When the selection ratio goes down:
1. top-down selection policy can become discriminatory
2. hiring becomes less selective
3. competition for the position is likely to increase
4. both a and b
The more complex the job:
1. the more people differ on how well they do it
2. the less people differ on how well they do it
3. the less selective the hiring process
4. the more the need for item-mapping procedures
Using a cut score of 50 on a predictor test a researcher finds a base rate of 1.00. This means that when a cut score of 50 is used:
1. 50% of applicants will perform successfully on a criterion measure
2. 100% of applicants will perform successfully on a criterion measure
3. 100% of applicants will fail on a criterion measure
4. 50% of applicants will fail on a criterion measure
Validity is to ________ as utility is to ________.
1. consistency; accuracy
2. usefulness; consistency
3. usefulness; accuracy
4. accuracy; usefulness
Which of the following is NOT an assumption of utility analysis?
1. the value of people and their performance can be estimated
2. psychological tests are always preferred over other means of assessment
3. the performance of people in organizations can affect organizational viability
4. large amounts of information can be integrated to make good decisions
An educational psychologist conducts a utility analysis of a teaching program used to improve the handwriting of very young children. The measure of utility in this analysis will most likely be:
1. increase in performance level
2. decrease in costs
3. reduction in accidents
4. increase in revenue
The term used to describe the proportion of people in a population who possess a given characteristic is:
1. success rate
2. base rate
3. selection ratio
4. sensitivity
The "miss rate" is equivalent to:
1. the number of incorrect classifications/the selection ratio
2. the base rate/the selection ratio
3. the success rate/base rate of successful performance
4. the number of correct classifications/total number of classifications
An empirical standard used to divide a group of data into two or more distinct categories describes a:
1. cut score
2. predictive yield
3. norm-referenced test
4. hit rate
Which of the following statements is FALSE regarding cut scores?
1. The setting of cut scores impact utility though the accuracy of decisions made based on test scores
2. In multiple hurdle selection, there will be a cut score for each predictor used in the selection process
3. An instructor who assigns an “A” to the top 10% of students in a class is using a fixed cut score
4. Absolute cut scores are preferred when applicants must demonstrate a minimum level of competence
Research cited in Chapter 7 suggested that when drivers chat on cell phones while driving:
1. they are usually apt to engage in text-messaging as well
2. they drive just as well as when they are not on a cell phone
3. how they drive depends on their ability to multitask
4. they are less likely to look in their rear view mirror
Costs associated with testing include all of the following EXCEPT:
1. administrator time
2. use of test facility
3. return on investment
4. test administrator’s time
If criterion-related validity is .40 and base rate is .60, the false positive rate is likely to be highest if the selection ratio is:
1. .10
2. .50
3. .95
4. impossible to answer based on the data provided

Chapter 7: Utility

Choose the correct answer for each question.