Maintaining Reliable Assessors

Quality and accuracy are not taken lightly in our department. It is in our name after all! Our assessors go through rigorous and continuous training and reliability checks to limit rater drift, eliminate bias, and ensure a high-quality assessment each and every time. Our assessors are trained in a wide variety of assessments so we are able to utilize the tool or scale that best fits a project’s need. For the Environment Rating Scales, our department receives training directly from the Inter-Rater Reliability Team. To learn more about the Inter-Rater Reliability Team, click the button below to visit their webpage. 

ERS Reliability

After an assessor completes the initial trainings on the content of the scales, they then go out with a state anchor for three consecutive observations to obtain reliability. Once reliable, assessors complete reliability checks every 10th observation. Reliability checks consists of assessors observing in the same classroom at the same time and scoring independently. After scoring, scores are reviewed and discussed. Assessors must show their scores align within one point of the state anchor’s score. The reliability process is modeled after how ERSI conducts reliabilities for state anchors nationally. 

CLASS® Reliability

Each assessor is first certified by Teachstone® through their required training. Assessors are then required to pair up to observe and code classrooms every 10th observation; this is called “Double Coding.” Similar to reliability, assessors code independently and discuss scores and the scores must be within one for each dimension. Additionally, Clayton goes through calibrations that are hosted by Teachstone® to measure coding accuracy. To learn more about CLASS® and Teachstone®, visit Teachstone’s® website by clicking the button below. 

Assessors may not observe independently if they do not maintain reliability or accurate double codes. Throughout the year, our state anchors and project managers provide continuous training to ensure consistency and continuous professional development on the assessments we utilize. These rigorous reliability, double coding, and training processes are unique to Clayton Early Learning, as our organization values having the highest quality, most qualified staff across our organization.

connect