SchoolDigger Ratings of Mississippi Schools: A Brief Critique

Many of us may have seen the ratings website SchoolDigger with its ranking of schools in Mississippi based upon their Mississippi Academic Assessment Program (MAAP) scores.  You might also notice that these ratings based upon the same assessment give different rankings from the ones I may share on this blog.  Well, truthfully, no one, with the exception of myself, may have paid much attention to the difference.  However, I wanted to mention what it is SchoolDigger is showing and why, in my opinion, its methodology is not the best way to look at assessment results for our Mississippi schools or districts.

SchoolDigger is a great website in many ways.  It gives information on free/reduced lunch rates and student to teacher ratios, which are both valuable to know for our schools.  But, we need to understand what they are showing and using to determine their school rankings and ratings.  SchoolDigger primarily uses the “average standard score” to rank schools.  Basically, the “standard score” they are referring to is the MAAP assessment scores converted to a normal percent-style grade that most of us are familiar with from our days in the classroom.  This conversion of the score (scale score) to a percent (standard score) is not really the problem.  The problem comes into play when SchoolDigger then takes all of those scores for every student in the school taking that particular assessment that year and averages them together to give their “average standard score.”  The site then uses this “average standard score” to decide which school has performed more or less successfully.  However, there are two major issues with using this means to determine the success of any Mississippi school or school district on any particular MAAP test that make it a poor means of measurement.

First, the big problem is this is an “average.”  Averages are great for some things, but not for others.  Using the SchoolDigger methodology, you could have a hypothetical school with 100 students taking the test.  We will refer to this example test as “Test X.”  Of those 100 student who took “Test X” that year, 50 students made a “standard score” of 100.  The remaining 50 students all made a standard score of 60 (what we would typically consider a failing grade).  Now in this example, the “average standard score” for the school would be an 80, which sounds pretty good.  So in this hypothetical, half of the kids in the class passed and half of the kids in the class failed, which most of us would agree would not be the making of a good, much less a great, result for the school.  But, the “average scale score” result shows an 80, which would be a “B” score in traditional terms.  You can see the issue here in that average scores do not really show how successful the school is in educating or attempting to educate all of their students.

Now keep in mind in the above hypothetical, the schools “average scale score” used for ranking is an 80.  Then, let’s go on to imagine that we are comparing them to another school which also tested 100 students on the same “Test X” during the same year.  At the second school, of the 100 students taking the test, 99 of the students made an 80 and one single student made a 79.  From a common sense standpoint, the second school was far more successful than the first with virtually every student having a “B” level of performance on the assessment!  However, the second school’s “average scale score” is only a 79.99, which puts them lower in the ranking than the first example school (where half of the class failed) with its 80 “average scale score.”  I think you can see from this example and many others that could occur, “average” scores are a very poor way to attempt to rank a school, if the goal is to educate every child.

Secondly, there is nothing remotely referencing “average scale score” or anything “average” in relation to student scores in the Mississippi Accountability Model.  The Mississippi model uses a variety of ways to get “points” for the school which are used to determine the school’s rating (A – F ratings).  Taking away graduation rate and a few other factors which have nothing to do with the MAAP or other standardized tests created by the state, there are only two things the state is concerned with:  growth and percent scoring in the top two levels on the test.  The top two levels of scores that a student could possibly make (Level 4 and Level 5) are basically considered as being “proficient or above” scores on the test.  These are the targets the state wants every student to reach.  Every goal that our schools have in regards to our testing results are geared toward getting all students to at least a Level 4.  On every single test our state administers for accountability, they only want to know did the student grow from their level the previous year and/or did they reach the target of Level 4 or 5.  “Average” scores have nothing to do with this and cannot be used to tell you what percent of students were able to reach the minimum of the Level 4, which is the target given by the state.  This without question makes “average scale score” or any type of “average” score of zero value in relation to our Mississippi Accountability Model on which all schools are judged.

Thus, the means which SchoolDigger is utilizing to rank our schools are in my opinion of little practical value, since they do not tell us the percentage of the class or group who actually are meeting the target level of performance.  The SchoolDigger “average scale score” is also of no value in relation to our Mississippi system of determining school accountability and grade levels.  While growth of the same student from one year to the next, would be the ideal means to make any sort of judgement as far as how much a student actually learned in a year and teaching effectiveness, this information is not available to the public in the released Mississippi test scores.  Without such growth data, the only accurate and relevant way to rank our assessment results for schools and districts is using the percent of students who were able to score a Level 4 or Level 5 on each of the assessments given in our schools by the state.  This is why you will never see an accountability report by the Mississippi Department of Education with any mention of “average” scores being used to demonstrate proof of success on our MAAP assessment scores by our schools or school districts.  The SchoolDigger data using “average scale score” is interesting to look over, but is of no real value for ranking success or failure of our Mississippi schools.

-Clint Stroupe