Most likely, you notice the line and how it peaks and then drops quite a lot for a while at the end.
Now, take a look at this graph. What do you notice?
Now consider this: both graphs were created using the exact same data.
How then, can they be different? Which one is right?
Breaking the Rules
Let's take a closer look at the first graph and we'll see a few things that are against the "rules" of making graphs. What rules, you ask? Well, the same rules that are applied to the ACT, SAT, GRE, and Missouri state tests, including high school level EOC (end of course exams.) And the same rules for graphing that Rockwood starts teaching students in elementary school.)
The intervals on the left (y-axis) are irregular. [93.2 to 90.8 (difference of 2.4); 90.8 to 85.2 (difference of 5.6), 60.2 to 23.8 (difference of 36.4)]
The intervals on the bottom (x-axis) are also irregular [there is no space for 2020]
Actually, that's really all there was to it - but it’s enough. This graph made a very common error, which was using the data points as the interval spacing.
We still have two points that don’t fall within the range of the others (for 2021 and 2022) which prompts the question: what happened there?
Context Matters
To answer the above question, we consider the context of what the data represents:
These graphs show data for Rockwood student performance on the Algebra 2 EOC (end of course exams). Test results are grouped into categories representing proficiency (advanced, proficient, basic, and below basic).
To put this in context, 2021 EOC tests were administered during the COVID pandemic, when students would have received foundational knowledge through asynchronous remote learning for portions of Algebra 1 and Geometry. (Algebra 2 is taken after Algebra 1, and typically after Geometry.) Unsurprising that there would be a drop for 2021 proficiency scores.
But what about 2022? Even in the properly formatted graph, that seems significantly different compared to the others.
Let’s add some more context. Typically, around 600 students take the Algebra 2 EOC exam. However, in 2022, only 42 students took the exam.3 Forty-two.
Why does this matter?
If you have ever shopped for something online, you have probably checked the reviews from other purchasers. Which scenario would you think is more accurate: a two-star rating with 1000 reviews, or a 4-star rating with 10 reviews? What about a two-star rating with 10 reviews vs a 4-star rating with 1000 reviews?
Generally speaking, more reviews means higher reliability in the ratings. Likewise, the opposite also holds: fewer reviews, or fewer than usual scores, has lower reliability.
Or, if you want a Science Fair example - if you are measuring the growth of 50 plants under different watering schedules, you don’t suddenly measure only 10 halfway through the experiment.
Back to the graphs though.
Why did so many fewer students take the Algebra 2 EOC in 2022? Is this unusual or is it going to be true in 2023 and the future as well?
Again, context matters. The students who normally would have taken that exam in 2022 had not been able to take the Algebra 1 EOC back in 2020, as no EOCs were conducted that Spring due to the effect of COVID on schools and student learning. However, DESE requires students who wish to apply for the A+ program (which provides free college tuition) to take the Algebra 1 EOC. Therefore, even though students were enrolled in Algebra 2 (or Honors Algebra 2) for the 2022 testing cycle, they took the Algebra 1 exam instead so as to be eligible for A+.2
We do still have a point that appears way lower than the others. But now we also have indicators that it may be a singular point, and not a part of a trend. One data point is just that, a single data point. It has the possibility of being a big change in the trend, or it could be an outlier due to extraordinary pandemic-related circumstances. To be definitive, we will need to wait for more data, and see where future points fall. For that, we'll need patience.
What is the purpose for this discussion?
Let’s look at those two graphs again side-by-side:
They paint two different pictures of Algebra 2 EOC test scores, with the incorrectly-formatted one giving the appearance of a more drastic and longer-term decline, whereas the properly-formatted one shows data points of concern that have answers in the context of the recent COVID pandemic. These two different pictures lead to two rather different conclusions. This is just one example for one exam. Check out down below for some other corrected graph comparisons.
Visual aids are frequently used to communicate information. In the present digital-age, articles or social media posts that have graphics included are known to be attention-grabbers so the audience will slow down and read the message rather than scrolling or flipping past. Graphs can be a simple way to show trends in data and steep changes in a line are going to be more attention grabbing than a flat one. However, since graphics are no longer subjected to professional vetting and editing prior to being published, it is up to the viewer to use critical thinking and evaluate the sources of information. Consider the source. Cross check information from other sources. Question content intended to trigger emotional reaction.
As for the graphs presented here, and those being spread across the region as examples of falling test scores; the bottom line is: test scores matter, but properly interpreting and understanding them matters even more.
-------------------
Here we see more examples of data presentation that gives the impression of declining test scores, however when properly formatted, that decline is either less extreme, or even non-existent. Gaps in the data occur when an exam was not given (either due to COVID, or during “field testing” when DESE was implementing changes to the standards and/or tests themselves.) All data in below graphs is unverified. The purpose is for comparison only of the data when graphed correctly.
Mathematics:
---------------------
Citations:
St. Louis County Family Association. (n.d.) Rockwood - Academic Performance. https://www.stl-family.org/rockwood.
Missouri Department of Elementary & Secondary Education. (n.d.) Missouri Comprehensive Data System: District/Charter Report Card. https://apps.dese.mo.gov/MCDS/Visualizations.aspx?id=29
Rockwood School District. (2023, February 9). Rockwood School District - Board of Education meeting Feb. 9, 2023 [video]. Youtube. https://www.youtube.com/live/C6D5BwGYgns.
