Computerized Testing for Pre-Employment

Dispatch/Call-Taker Selection

During the 2000 conference of the National Emergency Number Association (NENA) in Salt Lake City, James E. Kuthy of Biddle Consulting Group (then Biddle & Associates, Inc.) presented a seminar that described their new CritiCall computerized testing and evaluation of public safety dispatcher candidates. The company already markets such products in the law enforcement and firefighting fields, and expanded it to include public safety communications.

Kuthy listed the advantages of computerized testing in general:

• administration procedures are the same for all applicants (in theory)
• need for test administrators may be reduced
• results can be immediately available
• test may be administered to all-in applicants
• test can be modified or changed based on needs

He also listed the disadvantages of computerized testing in general:

• one computer needed for each test-taker
• may not be appropriate for all areas of the job
• typically costs more to develop
• may require special skills of administrator
• may require special equipment to administer and/or score

He said there are four types of computer-related tests:

• computer-based testing (CBT) - test given on a computer that was not specifically designed for a computer; essentially, a computerized written test
  • advantages: quicker to score, standardizes the test environment, allows for administration options [may set time to respond to each question, whether to allow answers to be changed] has similar validity to written tests, makes your agency look "modern"
  • disadvantages: requires a computer to administer,
• computer-adaptive testing (CAT) - a computerized test that tailors itself to the ability of the test-taker
  • not likely to be used for dispatch testing due to the technical requirements for creating this type of test
  • discovers the level where a person begins to encounter personally challenging questions
  • uses fewer test items per test by adapting the questions based upon the applicant's performance on previous questions
• work sample testing using a computer
  • advantages: immediate scoring and feedback; acceptance by applicant as "job-related"; accurate scoring; individual administration; measures computer-related skills; may reduce answering errors; test might be changed as job changes; some types of work sample tests may be adapted to be used for different positions
  • disadvantages: requires one computer per applicant; may be more expensive to administer; adding new types of measures or changing test content can sometimes be costly; computers are sometimes subject to technical difficulties
• testing computer skills without a computer - typing test; paper and pencil; cassette tapes,; video-based tests,; simulators and training devices; oral interviews; employment application/work history; reference checks; other tests (personality, etc.)

Kuthy explained that a legally defensible dispatcher test development requires more than just dispatcher experience and good intentions. Computer and multi-media testing must be held to the same requirements as other tests:

• Federal Uniform Guidelines for Employee Selection Procedure
• Civil Rights Act of 1964 and 1991
• Americans With Disabilities Act
• Age Discrimination in Employment Act
• Executive Orders for federal contractors
• State and local laws

He then suggested some questions one should ask about computerized testing:

• Did I obtain the published validity report before using the test?
• Is the test appropriate for applicants with no dispatcher experience?
• Is the test difficult to administer?
• Does the test measure more knowledge, skills and abilities or more aspects of the applicants than my current testing method?
• Will the test be accepted by the applicant as being job related?
• Will I need additional specialized equipment to administer the test?
• Does the vendor offer free or nominal charge support, upgrades and fixes after the sale?
• Can the vendor provide expert testimony to help defend the test in court, if required?
• Can the vendor supply legally defensible, specific test validation for your agency?
• What is the vendor's reputation and track record?

He then stressed several issues under the headline "Important!!"

• Demand to see the validation report
• Have it reviewed by an expert
• Try the test before you buy it
• Make sure the test addresses the federal and state laws, and professional guidelines
• Use a test evaluation checklist

And, the United State Equal Employment Opportunity Commission advises that it's the responsibility of the testing agency, not the test developer, to determine that the validity evidence is adequate to meet the federal Uniform Guidelines on Employee Selection Procedure.

[back to Job page]