By Chris Kieffer/NEMS Daily Journal
After Mississippi’s students take their state tests this year, officials will run them through a special software that notices test irregularities.
The Caveon program flags tests with a strange number of erasures changing wrong answers to right ones. It also looks for patterns in which several students have a large number of the same answers.
The program is part of the state’s line of defense against cheating on state tests. High school students took their tests last week, and third- to eighth-graders will do so this week.
Previously, the state has used the software only on the examinations taken by high school students, who must pass their four state tests in order to graduate. However, the issue of cheating on state tests has become more prominent in the wake of a recent investigation by the Atlanta Journal Constitution newspaper revealed widespread cheating on Georgia state tests.
For the first time, Mississippi will use it for all of its state tests.
“While we have concerns about some of these type behaviors, we really believe it is in very rare instances,” said James Mason, director of student assessment with the Mississippi Department of Education.
“The reason we are doing this is not because we have this perception that there is wholesale cheating going on across the state. We think it is an important step so that the community has confidence in the results we are reporting. We want people to have confidence and believe in the integrity of our testing system. We owe that to the public.”
When the software was used for high school tests, it would look for patterns that indicated students had cheated off each other. If so, their scores were invalidated. If an investigation proved an individual student was unknowingly cheated upon, his or her scores would still be allowed.
With elementary tests, there is not a strong incentive for students to cheat off each other. The scores are more meaningful for teachers, schools or districts, who are evaluated largely upon how well students perform on the tests. So the software will look for examples where those groups have altered tests or coached students on answers.
The program analyzes probabilities, Mason said, looking at patterns that are statistically unlikely. His department likes to say individuals have a one in a million chance of being hit by lightning and a one in 30 million chance of a DNA false positive. Their chance of being falsely flagged on tests is one in a quintillion, or one in a billion billions, he said.
The software also notes unusual student gains, he said.
“We will be looking at whether we see unusual patterns among groups of students at a class level, school level and district level,” he said. “We haven’t set any hard thresholds of actions we will take at different levels, but we will see what data reveals this year.”
For high school students, the flag rate has decreased, Mason said, going from more than 5 percent of tests in the spring of 2006 to around 1 percent in the most recent data, from the spring of 2010.
In addition, the state also mandates an elaborate process to ensure the security of tests when they are being handled and when they are being taken.
“Security is a huge word when it comes to testing,” said Debbie Jones, testing coordinator for the Lee County School District. “ … It is a process you have to go through because you are accountable for these tests.”
Once districts receive the test booklets, two officials must do a complete inventory of all of the material they’ve received, even checking individual serial numbers and bar codes on each booklet. School counselors must follow the same process when they receive the tests, which are then locked in a place where teachers don’t have access.
On test day, teachers must sign out the tests they receive. Those tests are matched to specific students by serial number. Teachers must also submit a seating chart used for the test.
During the test, there are three groups with specific roles.
Each classroom has a test administrator – a certified teacher – and a proctor, an uninterested party whose job is to monitor the room.
“The proctor is a neutral outsider who observes students and test administrators to ensure nothing happens to violate test security,” said Tupelo Schools testing coordinator Lea Johnson.
A hall monitor escorts students to the bathroom and also checks classrooms to ensure nothing questionable is occurring.
After the tests are completed, a complete inventory must again be done at both the school and district level before they are sent away to be scored.
“I would say we have the same kind of chain of custody procedures you would see on the law enforcement side with evidence,” Mason said.
District testing coordinators are required to attend training in Jackson about the procedures twice each year. They must then lead a training for school personnel, who lead their own training on campus.
Also, each district and each school is required to establish its own test security plan each year. Those plans must be approved by the school board.
“I don’t want us to miss anything,” Johnson said. “Not only does my job ride on this, but so does the reputation of the Tupelo Public School District, and that is critical.”