EDITORIAL: Bad Test Scores, Part One

Many of our Daily Post readers will have received their 2023 ballots by now, and may have noticed a couple of ballot measures related to school funding, and some candidates running for the School Board.

One of the measures, Ballot Issue 5A, will extend a ‘Mill Levy Override” to provide $1.7 million annually in additional property tax revenue to the Archuleta School District, in perpetuity.

Two of the School Board races are contested. In District 2, challenger Christa Laos is running against incumbent Bob Lynch. In District 3, challenger Jessica Trujillo is running against incumbent Amanda Schick. I reached out to Ms. Laos, Mr. Lynch and Ms. Schick to comment on the topics I will be discussing in this editorial series, and got comments from Mr. Lynch and Ms. Laos. I have not been able to reach Ms. Trujillo, for lack of contact information.

I also reached out to Superintendent Rick Holt, and I’m hoping he might be able to weigh in, with his own comments.

Disclosure: I currently serve as a volunteer on the Pagosa Peak Open School board of directors.  This editorial reflects only my own personal opinions, and not necessarily those of the PPOS board as a whole.

Public schools are going through a period of heightened controversy here in America, if you can believe the mainstream media, and here in Pagosa, an organized group protested certain School District policies during the COVID crisis. I’ve not heard complaints, locally, around participation on sports teams by transgender individuals, nor have I heard that parents are trying to get certain books banned from school libraries. Those controversies may be alive and well in other communities, but perhaps not here?

Nor have I heard much discussion, publicly, around bad test scores.

I suppose there are at least three ways that state-required standardized test scores can be considered “bad”.

1. The students didn’t learn their lessons, and failed the test, or the schools didn’t do a good job of preparing the students for the tests.

2. The scores were ‘bad’ — that is, relatively meaningless — because the tests didn’t truly measure what they professed to measure.

3. The scores were ‘bad’ — that is, harmful — because they damaged the future prospects and expectations of students and parents.

Earlier this month, the Pagosa Springs SUN newspaper shared a chart of recent ‘CMAS’ test scores — the ‘Colorado Measures of Academic Success’ scores generated by the Colorado Department of Education (CDE)… the folks in Denver who test our local Pagosa kids, and public school students all across Colorado in grades 3 through 8, on their knowledge of mathematics and the English language.

The CMAS tests are administered annually in the spring, but results are not provided until September.

The chart shared by the SUN also included PSAT and SAT test score averages.  Those tests are given to grades 9, 10, and 11.

Here’s an excerpt from that chart.  The percentage of Archuleta School District (ASD) students who ‘met or exceeded’ were below the state average.  In some cases, significantly below.

CMAS results: Percent of students who met or exceeded expectations

The Colorado Department of Education (CDE) likes to make things simple to understand, in certain instances.  For example, as shown on the chart above, less than one student out of ten (8.3%) in Grade 5 in Pagosa, met or exceeded CDE ‘expectations’ in mathematics.  Statewide, on the same test, 36.5% of students met or exceeded expectations.  The average scores among ASD fifth graders, was exceeded by the students in Durango, Bayfield and Alamosa.  (We note these are ‘averages’, and do not reflect the performance of any individual student.)

We could suggest an explanation for the ASD results, if we were so inclined.

Choose one:

1. We have raised a crop of fifth graders, locally, who have a poor aptitude for learning mathematics.

2. We have hired fifth grade math teachers who have a poor aptitude for teaching mathematics.

3. Most of the fifth graders with a decent aptitude for mathematics were absent on the day of the test.

4. The test itself was faulty, and didn’t measure what it claimed to measure.

5. All of the above.

Meanwhile, the average math results for ASD students in fourth grade (29.5% ‘met or exceeded expectations’) were the best in the region, and similar to the statewide averages.

Students in the fourth grade mainly attend Pagosa Springs Elementary School.  Students in the fifth grade mainly attend Pagosa Springs Middle School.  (Some students attend Pagosa Peak Open School, the District-authorized charter school.)

We could suggest an explanation.

1. We have raised a crop of fourth graders, locally, who have a good aptitude for learning mathematics.

2. We have hired fourth grade math teachers who have a good aptitude for teaching mathematics.

3. Most of the fourth graders with an poor aptitude for mathematics were absent on the day of the test.

4. The test itself was faulty, and didn’t measure what it claimed to measure.

5. All of the above.

An explanation that appears feasible in both instances — fourth grade, and fifth grade — is that the CMAS test itself was faulty.

The tests given to Colorado students in grades 9 and 10 are the national PSAT tests developed and administered by the College Board.  The SAT test, given in grade 11, also comes from the College Board.  From their website:

The SAT is a valuable part of the college admissions process because it’s a strong predictor of college success. We maintain its strong predictive validity in three ways:

1. Basing test design on a solid foundation of recent research 
2. Continually testing item types, test questions, and test forms 
3. Regularly reviewing student performance metrics 

This is a significant statement, in my book.  The SAT is designed to predict “college success.”

But only about 60% of Pagosa Springs High School graduates end up applying for college, and only about 30% actually end up with a four-year degree.

Once they have a degree, about 40% of the graduates end up working in a job unrelated to their college degree.

In other words, Pagosa Springs High School is required to administer SAT tests that are relatively meaningless for the vast majority of students. Which suggests that, ultimately, the test scores are relatively meaningless, in most cases.  Yet our high schools are ranked, based on those test scores.

Here is a sample math question, from a website that tutors students to succeed on the SAT test administered to our 11th graders.

Hypersensitivity reactions are classified by type, which are characterized by immune system response. The four most common hypersensitivity reaction types are type I, type II, type III and type IV, which occur in both males (M) and females (F).  A study was conducted in which each participant experienced a single reaction.  The table above shows the distribution of hypersensitivity reaction type and gender for the participants.  If one participation who is female is chosen at random, the probability that the participant experienced a type IV hypersensitivity reaction is 1/3.

What is the value of ‘y’?

I’m curious how many of our Daily Post readers — in particular, perhaps, those with college degrees — could confidently answer this question?

But even more importantly, how many of our Daily Post readers — in particular, those with college degrees — have ever been faced with a situation like this, that required solving for this type of ‘y’ variable in a hypersensitivity reaction calculation?

One person out of a 100?  One person out of 1,000?

Something that neither the CMAS nor the SAT test measured: the overall ability of any particular student to contribute to their community in a meaningful way.

Read Part Two…