As delays in getting scores and data from TNReady for the 16-17 schoolyear continue, the state’s testing vendor, Minnesota-based Questar, met with TEA officials to provide an update on the status, the challenges the company had in its first year, and what they are working on going forward.
Test scores go through a three step process before being used in the state accountability system. Raw scores are generated and then analyzed by the department to establish scale scores. Scale scores are used by the state board to determine cut scores to set various levels of proficiency. These cut scores are supposed to be voted on by the board by the end of summer.
Once cut scores are adopted, the TNCompass should begin to populate TVAAS data for educators for the 2016-17 school year. Last year this did not happen until late fall.
Such a delay in data creates problems for teachers applying for tenure, administrators setting evaluation schedules, performance pay distribution, hiring decisions for non-tenured teachers, and issues such as promotions and transfers.
“So much has been built into test-score data, and when they are delayed or fail it causes great disruption. I think it is time to reevaluate not only the whole testing schedule, but how and why we use test data in all aspects of Tennessee education,” said TEA President Barbara Gray.
Questar officials outlined many of the issues the company faced while implementing the test.
Having enough questions was the first hurdle. Tennessee is supposed to be working to develop its own testing content. Since the decision to switch to TNReady in 2014-15 to follow state standards, the department should have been engaged in developing questions using Tennessee teachers. It is one reason why the state will be able to publish a majority of test questions in any given year under the TEA-backed testing transparency law.
Questar held question-generating sessions with Tennessee teachers last fall, though turnout was spotty for some grades and subjects and, because of time constraints, were only held in Nashville. Questions developed last fall were included in the test, but not counted in student scores because questions must go through an evaluation process after their first use.
Since TNReady failed last year, the state owned few normed questions and had to rely on copyrighted material purchased last year. These questions cannot be published, and the department has indicated it will seek relief from publishing questions under the transparency law.
“Questar did outline their strategy on engaging more Tennessee teachers to create content for our tests, and outlined their process to ensure the questions we generate are fair and measure what we are supposed to measure,” said Gray. “It is disappointing the department didn’t reach out to us to help spread the information about this opportunity for teachers to be question writers and build a better assessment. We will certainly be asking when and where these opportunities are and letting teachers across the state know.”
Another delay was due to the test being on paper, causing logistical problems for schools and the company alike. Questar stands ready to move to an online assessment, which will help alleviate timeline issues largely caused last year by printing and scanning difficulties. Next year online is mandatory for high school students, and will remain optional for grades 3-8. Unlike the previous testing vendor, there were no problems for the school systems that opted to take assessments online, and Questar officials told TEA that it is ready to handle increases in online usage and will have staffacross the state ready to troubleshoot.
In its conversation with TEA, Questar committed to having productive conversations with educators about how best to ensure the test measures what it is supposed to and serves to improve instruction. As Tennessee teachers generate a valid question bank, and questions are being published annually so that parents and teachers can evaluate the assessment system itself, it is possible that confidence will improve for TNReady.
But after years of problems, including possible delays yet again this year, it is a huge task.
“The problem is high-stakes decisions being made with tests,” said Gray. “After years of problems, now is the time to rethink what we are doing.”