New Haven Independent

In a unanimous 4-0 vote Friday, literally shouted over Fire Commissioner Kimber’s booming cries of “injustice,” the Civil Service Commission approved a ranked list of 195 eligible candidates for entry-level positions on the city fire force.

After a presentation by the consultant who designed the entry-level test, commissioners concluded that objections — while deeply felt — were at heart due to a lack of communication, not due to a faulty test.

“I haven’t heard one scintilla of information that suggests that anything wasn’t fair” or valid about the test, concluded James Segaloff, chair of the commission, after hearing from consultants who flew into town Friday to clear up the whirlwind of doubts and concerns that have been stirred up over the city’s new fire recruitment process.

As a result of a court decision prompted by charges of discrimination, the city is using a whole new process to fill the 26 vacancies on the fire force: It must hire from a ranked list this year, instead of picking from a list of candidates who passed a pass/fail test. (Click here for background on the testing process.)

Chris W. Hornick (pictured), president of CWH Management Systems, told the room in his 28 years of conducting testing for public safety jobs, “we’ve never, ever been challenged in court.” He wheeled in his travel suitcase into City Hall Friday to defend the New Haven testing process in front of the Civil Service Commission, plus a few fire commissioners, firefighters and union reps.

The data he showed addressed a slew of doubts that have popped up over the testing process, which included a written, an oral, and then an agility test.

Was the process doctored to achieve a specific racial outcome?

“We didn’t try to doctor this process so that we could get more blacks in the system,” said Hornick. The makeup of the initial pool of 1,096 applicants who took the written test almost identically matched the makeup of the final ranked list, he said.

In the first pool, 69 percent of applicants were white, 17 percent were black, 12 percent were Hispanic and 2 percent were of Asian or of mixed race. The top 32 candidates on the ranked list mirrored that makeup: 70 percent white, 23 percent black, 7 percent Hispanic. Makeup for the top 53 candidates showed very similar results.

Was the oral test consistent?

Yes, he said — all 770 applicants who took it answered the same seven questions, graded on a well-defined rubric. Graders’ scores showed high correlation with each other, and the scorers testified they didn’t know the applicants sitting before them. The only discrepancy was that, due to a time crunch and an unexpectedly large applicant pool — testing nearly 800 people for an oral exam is “cumbersome,” to say the least — some panels had two people, while most had three. Hornick said analysis shows the number of panelists had no impact on candidates’ scores.

He tackled the question of greatest controversy: Why was the oral weighted more than the written? News that the final ranked score would be based solely on the oral test came as a big shock, to fire commissioners and candidates who had not been informed of how the scores would work.

Hornick said ranking the score on solely the oral is the practice CWH recommends to all its clients “because it’s the best way to be as inclusive as possible, and make sure everyone had a fair shot at the job.” The oral test allows the city to judge candidates on interpersonal and emotional skills in a way the written test can’t, argued Hornick.

The fire union and the city disagree on how important an oral exam should be — click here for a previous story detailing their positions.

As to the expectation that the written test would be counted in the final ranked score, Hornick admitted “a lack of communication occurred.” “Those who have been involved in the process [including Hornick] take full responsibility,” he said.

Hornick’s take is, the lack of information got magnified into a larger sense of skepticism and anger over the process. “Some people have felt disenfranchised or not involved in the decisions,” Hornick said, referring to the fire commissioners and fire union, both of whom are stakeholders, but don’t have power over how the ranked list is created. “Not understanding how this was gonna work creates doubt over the whole process.”

Kimber (pictured), a fire commissioner, proved that point in a booming oration before the board.

“It was never told to us that the oral exam would be the deciding factor in the entry-level test,” Kimber said. “This is why the city is in such an uproar — because there was a communication gap.” He urged the board to delay voting on the list until all complaints were aired, and until he and others could have input through a public hearing.

Egan got up and made the same request, citing doubts that “call the whole process into question.”

In a brusque back and forth, Segaloff pushed him to “give us one fundamental example you have that is a problem with this exam.” Egan mentioned the inconsistency in the number of panelists, inherent “subjectivity” of oral exams, and the unfairness to candidates going into a test without knowing how it would be scored.

“When you start the race, you want to know where the finish is,” Egan charged.

“I don’t mean to be obnoxious, but so what?” replied Segaloff (pictured), who stressed he was just trying to see if there were indeed problems with the test’s validity. “I just don’t follow where that has anything to do with the results of the process… you think that was an injustice?”

“Absolutely,” said Egan.

Segaloff disagreed. “It’s bad,” he agreed, to sit for an oral exam without knowing what the stakes are in terms of scoring. But “the fact that you don’t know how it would be weighted shouldn’t impact the results.”

He and other commissioners concluded none of Egan’s or Kimber’s complaints pointed to flaws in the exam — just flaws in the administration’s communication. “Listening to the presentation, it sounds like things were on the up and up to me,” opined commissioner Frank LaDore before the unanimous vote.

Segaloff proceeded to call for a vote, literally yelling over Kimber’s booming voice as he made a last-minute attempt to stop them. “I understand I’m out of line,” bellowed Kimber. “I think what this board is doing is a grave injustice!”

As debate continued in the hallway after the vote, Chief Administrative Officer Rob Smuts (pictured at the top of the story at right) said he’d learned “you run into problems if you don’t include people from the beginning.” He wasn’t the CAO back when the decision was made on Feb. 16 on how to weight the two tests, but it’s a lesson he’ll keep in mind for the future. “You learn lessons — this is a good test, even if the communication will be improved.”