The Grading Podcast

138 – Too many A’s

===

Sharona: So I wanna give kudos to you about this one. ’cause I found the article and I just, the article I have horrible feelings about. And we can talk about, I think it’s an absolute garbage article, but you went and found the research behind the article. So I have a little bit of a mea culpa and shame to say we teach a class where you’re supposed to take a news report and then you’re supposed to take the original research and compare them. And I had not yet gotten to the research. So the article is horrible, but the research is actually fascinating.

Boz: Welcome to the grading podcast, where we’ll take a critical lens to the methods of assessing students’, learning from traditional grading to alternative methods of grading. We’ll look at how grades impact our classrooms and our student success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach and instructional designer. Whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week, you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Boz: Hello and welcome to the Grading podcast. I’m Robert Bosley, one of your two co-hosts, and with me as always, Sharona Krinsky. How you doing today, Sharona?

Sharona: Well, I’m finally dry. The rains have stopped in LA and I got to go out last night with a good friend for a really good meal, and we went to the LA Phil to see Dudamel conducting Beethoven. It’s just really nice to have some time to, to hang out with friends, so I’m very grateful this morning for good friendships. How about you?

Boz: So how was, how was the show?

Sharona: The show was beautiful. It was the LA Philharmonic with a couple of choirs from Spain. One of my challenges, and luckily this friend tolerates it, is classical music does tend to put me to sleep. So I stayed awake for most of it, but I did get a couple of naps in the middle. Sorry, LA Phil, but, hey, I’m paying for my ticket and I’m there. But yeah.

Boz: Well, if it, because I know the friend you went with, if it makes her feel any better. My mom puts on this podcast when she can’t sleep, so.

Sharona: I’m gonna bring that up to her and say, you know, I fall asleep to classical music and Bosley’s mom falls asleep to the sound of our voice. Hopefully we’re not putting other people to sleep with this podcast. How are you doing?

Boz: I am actually really excited about this episode. I kind of wanna jump into this but before I do, I am going to do a disclaimer ’cause we’ve had people reach out to us before when I get fired up, I can sound like I’m accusing or blaming people, so I apologize if I come off that way. But this is likely an episode to get me a little bit riled up. What we’re gonna be talking about. So a couple weeks ago on episode 135 we did an episode on alternative grading and trauma enforced pedagogy, or the pedagogy of kindness.

Sharona: Can I clarify? Yeah. Trauma informed pedagogy not trauma enforced pedagogy.

Boz: Did I say trauma enforced?

Sharona: You did. I think you’re already on a tear, but I just had to clarify. Trauma informed pedagogy.

Boz: Oh, it’s gonna be one of those days.

Sharona: I think you should leave that in though.

Boz: But, and that was a really fun episode to do, but we started off in a very different place. Like when we started doing research for that week, we actually started with this Harvard article about giving too many As and I said on episode 135, this is an article that I really wanted to come back to. And we are, let’s.

Sharona: Exactly. No, I wanna say, we’re gonna put a gift link to this article. It’s a New York Times article. It is behind a paywall. So we will put a gift link in the show notes if you wanna read this article.

Boz: All right. So let’s jump right into this, shall we?

Sharona: Let’s.

Boz: So first just, what was the name of this article?

Sharona: So the article is called One Solution for Too Many, A’s Harvard considers Giving A plus grades and the subheading, which I think is very important also, Harvard University has been trying to cut back how many a grades professors give. Now 53% of grades are A’s down from 60%.

Boz: Yeah. And this is also coming from and is also referred to or referenced to, in another article, Harvard’s College grading system is failing report of grade inflation says. So there’s a lot about this grade inflation. So, and we’ve done an episode kind of on grade inflation before. I think you said it was episode 80 something.

Sharona: Yeah, so episode 88 is called Unearned Grades, remaking the conversation about grade inflation. And in that episode we actually talked about the fact that we feel that traditional grading has both issues of grade inflation and grade deflation from a mathematical standpoint. So we wanted to change the language to call it unearned grades, and that’s where that episode went. But one of the things we kind of didn’t talk a lot about is in traditional grading, if we sort of start from that perspective, is grade inflation really real and is it a bad thing? Right? We didn’t really address that. We more address the math problems.

Boz: Yeah.

Sharona: So what do you think? Is it real and is it bad?

Boz: All right, so I’m gonna look at this argument from two different standpoints. I’m gonna start with just looking at the assumption that too many A’s means a bad thing. Like is it bad? Is it wrong? Is there something harmful with having a lot of A’s, or a lot of passing? And I’m always intrigued by this kind of argument and, I’ve even heard both high school and college professors, I don’t think I’ve ever heard elementary and middle school educators saying this, but bragging about their fail rates as a sign of high rigor in their class. So I wanna kind of address that first.

Sharona: Okay. So.

Boz: Is necessarily on its surface, having too many A’s in a class, a bad thing.

Sharona: So I’m gonna give the answer that everybody hates, which is, it depends.

Boz: Depends on what.

Sharona: What is the point of an A? What does an A mean? So, we’ve talked a lot on this podcast about the two uses of A’s. Or, well, of grades.

Boz: Of grades, not just A’s but of grade.

Sharona: Not just A’s grades. And those two uses are ranking and sorting and communicating, learning. Those are the two possible things that grades are in theory supposed to do.

Boz: All right, so let’s look at this argument from both briefly at least, or broadly from both perspectives. ‘Cause you know, how I feel about what the purpose of grades should be and the ranking and sorting argument. And, and, you know, I, I do not think grades should be used to rank and sort students at all. I don’t think there’s any place in our grading for that. You have a little bit of a different stance.

Sharona: Well, it’s, yeah so my stance comes from, first of all, can I just say the fact that we’re talking about Harvard in the absolutely outsized impact that Harvard has had historically on both the grading conversation and education in this country overall is kind of stunning. But grades were designed by Yale and Harvard to rank and sort historically.

Boz: Yeah. Well, so for any of our listeners that have never been to our mathematics is the misuse of grading PDs or lectures.

Sharona: Exactly.

Boz: We, we go into a good chunk of our like hour and a half to two hour presentation is on the history of grading.

Sharona: Right.

Boz: And you’re, you’re right. There’s. Harvard and Yale. And it’s not just grading, I mean the, the whole reason why traditional high school classes and math go algebra, geometry, algebra two goes back to them.

Boz: Yeah.

Sharona: So just the idea that it’s so outsized from so much of a small population is just stunning. But going back to it. It was designed to rank in sorts and it was designed to do that in part, to communicate to the families of the students that were being sent there, how they were doing relative to their peers. ‘Cause this was all very elite. And Dave Clark has written about this as well, that today is there a need to rank and sort because are there points within either our educational system or the places where grades get used, that there’s some sort of true scarcity that we need to distinguish between people based on grades? That’s the argument that is made, whether it’s graduate school admissions or scholarships or other kinds of uses.

Boz: And, and I will concede the fact that yes, there are some real scarcities within our system. I just don’t think it’s, the grades is where the ranking in sorting should be, but.

Sharona: And I don’t, I don’t disagree with that statement. But we also don’t have a replacement right now. So there’s a practicality to them, to me that I say that there might be a judicious use from specific courses or specific moments within a educational continuum. But the fact that we do it everywhere all the time is horrific to me.

Boz: Okay. So I will concede the point that maybe grades have a time, place, and location where ranking and sorting is legitimate. It is in very unique places. It is not in elementary schools. It is not in middle school. It is not in high schools. It is not in most college classes.

Sharona: Right.

Boz: Would we I would agree, agree on that. Okay. But just to complete the argument, if ranking and sorting is the main point of grading is too many A’s a problem. Yes.

Sharona: Absolutely.

Boz: Okay there. I’ve said it. We’ve conceded that point. If you think the main purpose for grading is ranking and sorting students against each other, not against the learning they have in that class, so I could be a complete dunce but if I’m less of a dense than everyone else in my class, I deserve an A. Or I could have, and I’ve actually had a class that I had a 92% in the class and I barely passed ’cause everyone else had 93, 94’s. Both of those don’t show anything about my actual learning. It’s about how I ranked towards everyone else.

Okay, now let’s say it’s not. Now, let’s say ranking and sorting is not the main purpose of grading. The main purpose of grading is to communicate some level of proficiency or understanding of the materials essential to that class. So i.e. student learning. In that situation is having too many A’s an issue?

Sharona: So I’m gonna say in general, it’s not an issue. You want everybody ideally to be able to achieve full subject mastery, right? That would be the amazing thing.

Boz: Not only is it not a problem, it should be our goal. That should, yes. As an educator, if my grades are about student learning, my goal should be everyone has an A. ‘Cause that means I have taught every single student everything I needed to teach them in that class. That it’s not only is it not a problem, it should be the freaking goal.

Sharona: Yes, I would agree. It should be the goal. Well, it should be, the goal really should be, I’ve set a bar and everyone reaches the bar. The only reason I’m hedging a little bit is I don’t want to fall into the trap that the A is the goal. Because then we can do all kinds of weird things with our standards. If our goal is to put an A on a piece of paper, we can define what they have to learn at lower levels. And I’m not wanting to advocate for that. So I’m just being a little hedgie. I’m sorry, I have to be because I think a big problem with this grading conversation is we have too many places where we say the goal is the A, so I wanna flip the conversation. The goal is subject mastery at a sufficiently high bar, and my bar is pretty gush darn high, and I wanna get everybody over the bar.

Boz: Okay. You’re,

Sharona: I’m sorry, I just have to say it because

Boz: No, I, I can, I, I, I understand what you’re saying. I also think the lawyer part of you is coming out.

Sharona: No, it’s the mathematician more than the lawyer.

Boz: No, that’s the lawyer argument. That’s.

Sharona: But I would agree that that full mastery of my content or, not perfection, but if every single one of my students can get through all of my content at a level that I consider proficient and therefore they get an A, that is absolutely goal goal for me.

Boz: Okay. So going back to this Harvard issue then that too many gray, too many A’s are being given out. And they did do some research and they, and I mean there is some math behind this claim that more people at Harvard are getting A’s now than they did, you know, a couple decades ago.

Sharona: Well, and I think the problem is they actually have a definition problem in this article. It quotes by Harvard’s standards, A grade of A is supposed to be reserved for work of quote, extraordinary distinction unquote, that shows quote, full mastery of the subject unquote said the October report. So their very definition mixes, ranking and sorting. ’cause extraordinary distinction is only judged against other people with full mastery of the subject.

Boz: So, in my scenario that I kind of played out earlier . The dunce, but less of a dunce than everyone else wouldn’t get it. And someone that has full master or full proficiency in the class. But so does everyone else means I also don’t deserve it. So it is, it is doing, saying that that’s what an A is supposed to mean. It’s supposed to mean not only do you have learned what you were supposed to have learned, but you’ve learned it better or somehow more exceptionally, however that’s defined than the rest of the class. Is that my interpretation? Is that correct? Is that what you’re also reading?

Sharona: I think that’s what Harvard is saying, their A is supposed to mean.

Boz: Okay, here’s my biggest problem with that First, you’re right. It’s, it’s mixing both purposes of grading, the communication of learning and ranking and sorting. But I don’t know. I’ve been an educator for 20 something years, and one of the biggest things that we have been taught and that I would been preached to me and, and really hammered, is that we’re supposed to be teaching these things called the 21st century skills.

One of those is communication, and another one is collaboration. So if I ams only supposed to be able to get an A, if not only can I show that I’ve, I’ve gotten proficiency in all of my content, but I have to show that I’m better than you. How is that encouraging communication? How is that encouraging collaboration?

How is that encouraging those 21st century skills? Yeah.

Sharona: It’s not, but it’s more complicated than even that. Right? So we have I agree with you doing this extraordinary distinction thing and, and having students rank and sort is going to work in direct contradiction. But we have other issues here too, because at a place like Harvard, the people coming in are used to being at the top of that ranking and sorting in every institutional every educational institution that they’ve been in.

Because that’s how you get into Harvard.

Boz: Yeah.

Sharona: You get into Harvard by being the top of the top, and now you’re gonna further distinguish. So, if I am an employer, am I gonna take a straight A student from a really good public institution like a UCLA or a UC Berkeley, or am I gonna take a C student from Harvard?

Boz: Yeah.

Sharona: You know exactly it. It’s so, so, so now you’ve taken a rank and sort system and now you’re re-ranking and sorting the absolute top of that rank and sort against each other, which is just sounds toxic as hell to me.

Boz: And let’s go back to what the point you were bringing up earlier. This started two centuries ago at Harvard when there were six faculty member and hundreds of students.

We’re still using that system now, where we’re dealing with millions of students across the country.

Sharona: Yeah.

Boz: Yeah. I think there’s an issue here.

Sharona: And then it gets worse though.

Boz: Okay.

Sharona: Because there are so many pressures coming in. On these grades other than ranking and sorting and measuring learning.

Boz: Okay. So that, that kind of pressure is what I wanted, wanted to go in our next place.

So looking at just the base argument of is a having too many A’s a bad thing, let’s now go to the other side of this, which is the grade inflation.

Sharona: Mm-hmm.

Boz: And if grades are going up, so we’re having grade inflation, is it a bad thing? And there are some studies that are showing that these higher grades aren’t necessarily showing higher learning.

So where is this pressure coming from? Like, why do we see this grade inflation and some of the pressures you were just talking about is part of it. So, but, but I wanna

Sharona: clarify one thing though. When the words grade inflation are said. They don’t just mean higher grades,

Boz: they mean unearned.

Sharona: They mean higher unearned grades because grade inflation, just more of something is not necessarily bad the way we’ve talked about, but there’s an underlying implicit assumption that these grades are not earned.

Okay. It’s too easy to get them.

Boz: Okay, good point. You’re right. You’re right.

Sharona: So, so I wanna clarify that because that’s where these pressures come in.

Boz: Yep.

Sharona: So the first pressure that I wanna talk about is actually on the negative side. It’s the fail. The fail pressure, right? So the pressure for me to not fail students out of their university level math course, there’s a ton of research out there that’s showing that that is one of the biggest.

Reasons, supposedly reasons. I say that in quotes because I don’t agree that it’s causal. But when students fail their first math class or their math class in college, they leave college. And that’s a huge impact on their lifetime learnings. Okay. Earnings, not learnings. Learnings too, but earnings. So we have a lot of pressure as faculty to reduce our fail rates.

Boz: And, and that pressure, that specific pressure that you’re talking about there, where is that pressure coming from?

Sharona: From the administration of the university.

Boz: Okay. I actually thought you were gonna take it somewhere else.

Sharona: Okay.

Boz: I thought you were gonna say that’s an internal pressure. ’cause we as educators know that we know that, you know, at least, ’cause you and I both teach at freshman level math classes.

We are both well aware of these facts. We’ve seen the research, we know the numbers. So there is an internal pressure that I don’t wanna fail my students ’cause I don’t want to be the reason that they drop out.

Sharona: So I would agree that some faculty have that pressure. I have that pressure to some degree, but I also, if a student doesn’t attend, if they don’t come, like I, I, at some point there’s a, a certain piece of it on the student.

But as a course coordinator, I definitely got the pressure from my administration and, and not my departmental administration, but my dean’s office and my university level administration.

Boz: Yes. So, and, and I think this pressure is multifaceted because you, you saw it as a coordinator evaluating your, your whole course, this whole course design.

I think we see it on individual educators. In fact, it even talked about that in the Harvard in, in the, that Harvard story that if you were the flip that if you were known as being too hard of a grader, you would get the negative student evals and students wouldn’t enroll in your course.

Sharona: And that’s really super serious because if students don’t enroll in your course and you’re someone like a contingent faculty member, you don’t get courses, which means you don’t earn your salary, you don’t have a job.

Boz: Yep. And that kind of level of pressure of looking at fail rates, looking at DFW rates doesn’t start and stop in college. Like that pressure goes all the way down. In fact, almost every state that I know of, there might be a few out there, but most states that I’m aware of have some sort of school ranking system that is publicized, whether it’s on yearly reports, whether it’s on, like here in California we have a Cal, what’s called the California Educational Dashboard, where it is constantly up and showing, and some of the things that it uses to rank those schools are grades.

It is the GPAs. So yes, teachers have pressure. In fact, as much as I hate to say this, I helped put those pressures on when I was still at Sant as a intervention support coordinator. One of my roles was to do a fail a grade analysis report to my admin, which the report itself. Was it putting on pressure, but what people did with that, what some administrators did with that, definitely did.

But yeah, we see that kind of pressure. So as long as whether it’s individual teachers, departments, schools, or entire systems that are going to be, be judged by GPAs or by grades, there’s going to be pressure on educators to do that grade inflation.

Sharona: Absolutely. So we have the fail rates, we have enrollment, we have course evaluations.

But one of the things that’s interesting to me is that all of these pressures and all of these things, when you suddenly say you wanna do alternative grading, they get turned around and you get pointed at and being told, oh, these things you wanna do, they’re going to feed into this problem. They’re going to cause grades to be inflated.

They’re gonna let people pass who don’t deserve to, you know, you just want better course evaluations ’cause you’re nicer as a teacher.

Boz: Okay. So I’m thrilled that you brought that up, but I wanna go back to the Harvard one real quick.

Sharona: Okay.

Boz: How does that article suggest fixing some of this grade inflation into easy A’s in Harvard?

Sharona: Well, but that’s why I wanted to, before I said that, that’s why I wanted to bring up the other thing, because the first thing that most people accuse individually, alternative graders, if you bring this to someone who’s skeptical is they say one of these things, right? But then Harvard’s undergraduate Office of Undergraduate Education in talking about fixing their grade inflation problem.

Said some of the potential steps raised in their report, including giving more weight to students subject mastery rather than their effort and including the median grade for each course on students’ transcripts for context.

Boz: Oh. So they’re saying, let’s make the grades actually measure students’ learning.

Hmm. Interesting. And they’re saying, let’s use a mathematical measurement or a statistical measurement that more correctly reports and measures, centers of skewed data sets.

Sharona: Right. ‘

Boz: cause that one low grade and an average is going to drag that average down no matter what. It’s an average is a skew sensitive measure of center where a median is a skew resistant measure of center.

So let’s use that at instead. Wow. These are all alternative grading principles.

Sharona: Now, I think though, one thing, and I don’t know that people object to measuring the mastery, the the issue I think comes in with some of our other practices because one of the things that’s gonna get exposed if you measure mastery is many of your students maybe don’t actually have it.

Even the students that were getting A’s or B’s. Because of the weirdness with how traditional grading works. So this need to allow for a retake, this need to learn from your mistakes that we consider as part of some of our best practices. I can kind of see why people for whom they’ve been relying on the obscuring factors of traditional grading might be a little hesitant.

’cause if you actually had to measure your student’s mastery, they might not do as well at the very beginning as you think they do.

Boz: Yeah.

Sharona: And then that’s your fault as the teacher, right? I don’t think so.

Boz: But yeah, this article, like I said, it, it brings up kind of the complexity of this. I mean, you and I are having this debate on like four different sides now, and I do wanna make it clear.

I do think that grade inflation can be a bad thing. Teachers that, educators that just simply give a grade that is unearned. Which is why, you know, we keep saying yes, our grades have gone up, but some of our bars like what it takes to get an A in my class is actually more rigorous and harder to do than what it was 10 years ago.

But as we were looking at this and as we were looking at the inflation, you found another really interesting article.

Sharona: So I wanna give kudos to you about this one. ’cause I found the article and I just, the article, I have horrible feelings about it. And we can talk about, I think it’s an absolute garbage article, but you went and found the research behind the article.

So I have a little bit of a mea culpa and shame to say we teach a class where you’re supposed to take a news report and then you’re supposed to take the original research and compare them. And I had not yet gotten to the research. So the article is horrible, but the research is actually fascinating and not horrible.

Boz: So let’s talk about the article first. It’s easy A’s lower pay, grade inflation’s hidden damage, an analysis on more than a million high school students. This article came out not that long ago. When did this come out?

Sharona: Couple weeks ago.

Boz: February 9th.

Sharona: The article came out a couple weeks ago.

Boz: Yeah, February 9th. So by the time this episode comes out about a month ago.

Sharona: Yeah. And, and this article, and if it says what the research, what it says, the research says, this is a little scary because it says. That lenient grading or grade inflation, which I don’t agree those are the same thing, but they say lenient grading or grade inflation is actually harming students leading not only to worse academic outcomes, but also reducing their employment prospects and future earnings.

That is quite the statement.

Boz: Yeah. Which, okay, if that’s true, that that’s a big thing we wanna be concerned with.

Sharona: Now I do wanna point out in the article, they do say it has not yet been published in a peer reviewed journal and may still be revised. So I do wanna take the research with a grain of salt.

Looks really good, but it has not gone through a peer review process.

Boz: Oh. And in fact the article that when you shared it and when it was released, they’ve already come out with the correction.

Sharona: Oh, I didn’t notice that. So what the article did or the research did?

Boz: No, the article.

Sharona: Oh, okay.

Boz: So they published this like we said, February 9th. I am looking at it. We are recording this on February 21st. They have already had to come out with a correction to something they said.

Sharona: And the correction’s pretty hilarious.

Boz: The correction is hilarious ’cause it’s a big deal. So this is the correction. There is a sentence in the report that says that the lifetime loss per student is $160,000. That that’s how much loss income is caused by this grade inflation or grade leniency.

Sharona: Yeah.

Boz: I had to correct that because that’s not per student, that’s per teacher, which means the per student was between 42 and $132 per year.

Sharona: Per year

Boz: from 160,000 to 42 to $133.

Sharona: Well, and then it, and it’s weird because they went also from lifetime to per year in that correction.

Yeah. Original sentence was $160,000 in lifetime earnings. And then the correction says it’s not per student, but for all the students who were taught by that teacher that year per student, the income loss ranges between $42 and $133 per year. So why didn’t they go to Lifetime on that one? Oh, because you’re talking a thousand dollars to maybe $5,000.

For lifetime, it’s a lot less impressive. So by switching units it obscures it even more, which is super fun.

Boz: Yeah. And it wasn’t just per student taught in that class. It was per student taught by that teacher per year.

Sharona: Yeah.

Boz: So it wasn’t just the class.

Sharona: So as I started reading this, I had all kinds of issues with this article, some of which you told me are not actually a problem.

So this is what happens when you have a very statistically driven piece of research read by someone who’s not a statistician. And even though I teach intro stats, you are much more of a statistician than I am. So this research was looking at Los Angeles students and students from Maryland over a 10 year-ish period in each nine year in each group.

Boz: Which is true. Well, if you are making a comparison of the two groups, it’s bad. There’s a lot of possibly confounding or lurking variables.

But they weren’t doing that here. They weren’t comparing the two groups. They were saying they saw similar similar things and similar issues with this lenient grading in both groups. So they’re taking two very different groups and applying the same independent variable and seeing similar results.

So that’s actually not a bad thing. That’s a, a way to strengthen a statistical argument. ’cause you’re seeing it in very different groups.

Sharona: Yeah. So I’d like to read a paragraph from the newspaper article or the newspaper, the, the media article and get your reaction to this statement.

Boz: Okay.

Sharona: So it says, students taught by lenient graders defined as teachers who gave higher grades than expected based on standardized test scores and prior student performance did worse.

Later in high school in Maryland where there was data through college and into the workplace, these students were also less likely to attend college or be employed and earned less. So what’s your reaction to that whole statement? What do you wanna unpack from that? Because there seems like there’s a lot in there.

Boz: Well, my, my first, and this is kind of what we teach our students in our class. My, my first reaction to that is. I want to see how those conclusions were actually made and are those conclusions actually supported by the data? Because you can make some, and we see it all the time in, in media. I mean, we, we do a whole unit on bad graphs.

We see bad conclusions made from good data. So yeah, that’s my first reaction is, okay, how are we measuring this? And is that claim actually backed up by the, by the research?

Sharona: So what did you find?

Boz: So I went and found the research.

Sharona: Right. But when you went and looked at the research, what’d you find was going on?

Boz: So,

Sharona: I, is this an accurate statement or,

And it very clearly says that they are not trying to say this is causation, that simply there needs to be more research on this.

Sharona: And one thing I wanna point out is that unlike sort of this vague definition of grade inflation, they actually had a very specific way of coming up with the list of teachers that they felt, had this issue, this definition of lenient graders, it’s not a vague thing at all. I don’t have the math to double check them, but they actually do it mathematically, right?

Boz: First they actually define this two different ways. They define the grade leniency in two different ways, which I do think is important ’cause we’ve kind of been discussing grade inflation as this one whole thing where this really did bring it up into two distinctively different styles of grade inflation.

And I think it’s important. So the first one, which they call pass inflation, this is where grades in a class are easier to get C’s compared to you know, this indicator, this index that they used. And I’ll explain that a little bit more later. But the other grades, especially the A’s are not. So these are the students that are the teachers that will pass anyone.

But getting an A is still hard to do. So that’s what they refer to as pass inflation. Grade inflation is when all of the grades are easier to get or have a higher index score that again, I’ll explain in a second. So those are two different things, and we were kind of talking about that earlier when we were looking at the Harvard’s, you know, too many as versus just the pass rates that you, you were seeing so much pressure on as a coordinator.

So they did on both of these they calculated these two different indexes differently using a value added type of statistical formula. It does look. Mathematically, they were at least attempting to do some pretty sound math. I did not have the time ’cause I would probably take me a few days to break out some of my old stats books and go in and to really look at it.

But I recognize some of this. I, I recognize the fact that this is a value added type of index formula. So yeah, it looks like the math is, is legit. They’re taking in several different indicators past student performance and other classes, past performances on standardized tests, current subject grades and those those different tests.

Sharona: Mm-hmm.

Boz: And a few other things. So yeah, the mathematically they. It’s got some, it’s got some heath to it. It’s got some, some legitimacy to it.

Sharona: And what I appreciate about this is one of the biggest problems we find when we start to talk to people about stuff is people throw away these term, throw around these terms, like grade inflation, but nobody ever defines it other than, oh, there’s just more as than I feel there should be.

Boz: Yeah.

Sharona: And in this case, they actually tried to put some math to it. Now I have some issues with the math because it’s based on the mathematics of traditional grading, which I spend a lot of time trashing.

Boz: Okay. So before we get to that, ’cause I, that’s a really big point. I, I, I want to dive in and really spend some time on that.

But before we do that, I just talked about the legitimacy of some of the math that was involved here. The math might be legitimate, but the data that was put into this. I have some issues with, I have some questions about, and any good statistician knows, you’ve heard this term.

Sharona: Garbage in, garbage out.

Boz: Shit in shit out.

Sharona: Yeah. Garbage in, garbage out.

One of the first things that I have issues with is it uses the math and English CAHSEE scores as part of their test assessments to te to look at the current. Performance.

Sharona: So can you please explain what a CAHSEE score is? What does CAHSEE stand for?

Boz: I apologize.

Sharona: I was just waiting for you to do that.

Boz: I know.

Sharona: I even know it.

They started to take it typically their sophomore year. Some schools did it even before that, but this was supposed to be a minimi, a minimal proficiency or minimal skills test for graduation. Here is my issue with, with using that we no longer use the Casey. We haven’t used the Casey for quite some time.

And the reason that we don’t is because it had a lot of issues. Some of those issues was that it was found to measure irrelevant skills. It was measuring things that in the modern classroom where we are looking at, you know, 21st century skills, when we’re looking at common core skills, it was not measuring that.

So it wasn’t, so the fact that students might have been doing worse on it than projected, if it’s not measuring what it’s supposed to be measuring, who cares?

Sharona: Well, and so to clarify, for our non mathematicians out there, what this research did is its definition of lenient grading. Is that there was this mismatch between the grades in the classroom and the scores on these tests that these so-called lenient graders had higher scores than other teachers relative to this test.

Well, if you have some teachers who are more adopting these modern skills and it’s got more of a, you know, trauma-informed pedagogy classroom, or a more active learning classroom or whatever, and this test is not measuring those things, that is one big reason that this definition, this very definition of leniency might be a problem.

Boz: Oh, and it gets worse.

Sharona: Okay,

Boz: so

Sharona: please do share

Boz: those. The, the math and English CAHSEE were two of the tests. Two more of the tests were the 10th grade science CST, which stands for California Standardized Test. And the 11th grade, the 11th grade science, CST.

Sharona: Okay.

Boz: Here’s the issue with using grade level science.

Not everyone takes science in the same grade. And in fact, in in schools that have high population of EL students, they are often not taking the same sciences that a lot of, you know, English only students would take their freshman year because they’re having to take additional el LD or English classes.

So they do eventually take the science class, but whereas, you know, a, a school that or students that are English only or English fluent might take biology in ninth grade, an e student would likely not take biology until 10th grade.

Sharona: So, wait a second. So a 10th grade science test, which might include both biology and say anatomy, physiology, or chemistry, whatever a typical sophomore class would be is testing both types of students and one type of student has two years of science and one type has one year.

Boz: Oh, it’s better.

Sharona: And that’s what you’re basing it on?

Boz: No, it’s better than that. Okay. That 10th grade that, so like I was explaining, an English only student might be taking biology in the ninth grade, whereas students that that are EL students are likely taking that in the 10th grade. The 10th grade California CST test was mostly about chemistry.

Sharona: Oh. So it’s the 10th grade science that none of the EL students have had. Lovely. So on the one hand, again, the math that they did looks solid. The data that they’re basing it on, not so much.

Boz: Yep.

Sharona: Well, and then I have an issue, even though, because as we’ve said before, just because something is statistically significant doesn’t actually mean it’s really actually practically significant.

Okay.

Boz: Before you do that, okay. I’m not done.

Sharona: Oh. Oh gosh. Oh, please do continue.

Boz: One of the other indexes, or one of the other things that went into this was the L-A-U-S-D graduation rates. During that time, again, the CAHSEE was being used and was preventing large groups of people from graduating, mostly impacting EL students.

Sharona: Because we had a high school exit exam, which we no longer have.

Boz: Which we no longer have, and those graduation rates weren’t readjusted. So we got rid of the CAHSEE.

Sharona: Yeah.

Boz: And eventually students that were prevented from getting their diplomas because of kc they did eventually get them like this did happen. It does, it’s not reflected in, in those graduation rates.

Sharona: Well, and correct me if I’m wrong, but didn’t some of the standardized test scores that they were looking at here also include the ACT and SAT

Boz: and no SAT and PSAT did not include the ACT.

Sharona: Okay.

Boz: Because so few students in LAUSD actually take it.

Sharona: Right, but the SAT, you mean the test that measures family wealth more than anything else? That one,

Boz: yeah. That one,

Sharona: so, so that one, that’s the one we’re gonna use as a baseline for a district that is large, that is largely Hispanic, that is high levels of English language learners.

We’re gonna use a test that is driven by a family’s wealth as its highest correlation factor as the baseline to say whether these grades are unearned.

Boz: Yep.

Sharona: That’s just, I love data

Boz: getting lower scores than, than expected on SAT/PSAT, on the CAHSEE, on the CSTs. Those are all like negatives that are going into this formula that are saying that these high grades weren’t earned, that those are indicating those teachers were being lenient graders, whether it was lenient in the pass.

Inflation style or in the grade inflation. So it affected both of those measurements. Okay. Sorry, that was the last mathematical point I wanted to make.

Sharona: Okay. Well, my issue then though is that their definition of lenient was a 0.2 or more difference in essentially the GPA of a student versus, you know, the GPA for that class.

So the, the, the conversion of their grade level to the four point scale versus other students. And given that I talk for an hour and a half on grades as the misuse of mathematics and that I can prove that the weighting of different categories is the biggest impact on this stuff. That the fact that a teacher has a higher GPA it, it’s just how do you even compare these.

It’s, it’s talk about garbage in, garbage out.

Boz: And it, the research in the full report, not the article, but the full report, it brings that up. It recognizes that and brings that up as an issue that how everything that’s done in a classroom is filtered into a final grade can be drastically different from teacher to teacher.

So they did do try to do some normalization measurements to help with that. But they, they did, they recognized that and they said that, yes, that’s an issue.

Sharona: So here’s where things really go off the rails for me.

Boz: Okay.

Sharona: And it’s not in the research itself. We’re gonna go back to the critiquing the media thing that we tell our students to do.

There is a sentence in this article, and this is the kind of article that will get thrown at me all the time. The study cannot directly explain why higher grades lead to worse outcomes. So they’ve already misstated the research. The research does not say higher grades lead to worse outcomes. I believe it says they are correlated.

Boz: There’s some correlation.

Sharona: Some correlation,

Boz: and it needs to be explored more.

Sharona: Right. So the article says, the study cannot explain why higher grades lead to worse outcomes, which is a lie, but the mechanism is not difficult to imagine. So now we’re just gonna completely ignore data and science, and we’re going to make up a hypothetical scenario and say this is why this is the problem.

Boz: And, and this is a, a classic error in logic, error in statistics. I actually observed an English class a few days ago that this is the exact thing they were talking about, distinguishing fact from interpretations of fact.

Sharona: And now this whole thing, this whole article, this whole, everything is about grades in general.

So tell me what class they give me this scenario. And if you had to guess they’re about to say, it’s not hard to imagine. I’m going to imagine something. Which class are they gonna pick to imagine this in?

Boz: Oh, if it’s high school, it’s algebra one. Hands down.

Sharona: Yes. So in a class with a lenient grader, a savvy student may quickly realize she does not need to study hard or complete all the homework.

If she earns a B in algebra one without learning how to factor or solve quadratic equations, the knowledge gaps follow her into geometry and beyond. She may scrape by again. Over time, the deficits compound confidence erodes learning slows in college of the workplace, the consequences show up as lower skills and lower pay.

So first of all, why is it a she? Okay.

Boz: Oh, that’s a good point. I hadn’t even picked up on that.

Sharona: Yeah, I got that one. And why is it the factoring in quadratic equations. Why is it knowledge gaps following? I mean, they’re just sentence after sentence after sentence. And yet this is the kind of thing that somebody who is skeptical of alternative grading is absolutely gonna throw at us and we need to be ready to defend it.

And that’s one reason we wanted to dive into this is say, hold on. And I think that thing, it’s not hard to imagine, actually. Yes, it is. The research shows that this is very difficult to understand why these things are happening and how they’re happening.

Boz: Yeah. And, and like I said, one of the big results of this research that the article is quoting is that we need to do some more research into it.

There are some interesting things there. They did not look at differences of genders that, so they said there should be some more research to see if there’s any kind of reaction difference they need to like. Do more research just on this whole thing because they, like the research said, we showed there’s some interesting results that deserve more research.

They never tried to put a causation. They said our results show interesting results that need more research into it. But going back to looking at this article, and I know we’re running outta time and I don’t care if we go long. This article, when you were talking about people throwing things back in our face, it even does it here, it’s quotes that using equitable practices might le might be the cause of this great inflation.

Sharona: Well, does it say that or do they say that? A survey documents teachers saying this.

Boz: Okay, you’re right. It, it says surveys. Teachers indicate that equitable practices, that research that was done, that this whole thing is done on, never says anything about the grading practices. And in fact, going back to the Harvard, a article, how did they say to defend against grade inflation Using, using alternative grading practices, making the grades actually about the student learning, taking out the behaviors.

Sharona: The fact that so many misrepresentations go into this. That retakes means unlimited, that forbidding zeros means that we’re inflating grades. And you know, we’ve had all these conversations a million times. But yeah, there is such a backlash and such a push against equitable grading.

And one of the first things that people say is, where’s the research? So I do think this research is important and we’ve gotta be hypercritical about stuff like this because, like you said, the data that went into this and the stuff that they’re using to measure. Very questionable.

Boz: Yeah. Well, and And even more so the conclusions, not from the researchers, but the conclusions from the research to the article.

These claims that were not actually part of the research, but taking things from that and making their interpretations and then people thinking that that’s research. No, that’s an interpretation on research. And we could argue probably spend another hour, hour arguing about the legitimate legitimacy of some of those conclusions.

Sharona: Absolutely. So I guess for me, what this just reinforces is we still need clarity on purpose of grades. Before we can really analyze any of this stuff, we need to know what our grades are supposed to be doing for us. And then we need to know how are we gonna measure if the long-term outcomes for students are good based on the changes we’re making.

Yeah. So I have a really good feeling, but we do need more research.

Boz: It also brings up, I absolutely hate that argument about great inflation and that alternative graders cause grade inflation. And again, this article even did it, but the research, and I have not seen any research that talks about grade inflation showing any kind of connection between alternative graders and that inflation have grades been going up over the last 20 years.

That is an undeniable fact. It is in both the high school level and in the college level. That is the undeniable fact. The reasons for it, whether it’s good or bad, those are debatable, but there has been no research. And if, if I’m wrong, if, if you’re aware of some that directly ties this idea of unearned grades and higher grade inflation with alternative graders, I’d love to see it.

And I’m sure there’s, there’s definitely alternative graders out there that do it. Just like in traditional grades. There’s, there’s educators out there that will use, that can use the traditional system to inflate or deflate grades. It’s the same thing, whether it’s intentional or not with, with alternative graders.

That’s why part of our PDs, we start off by really dissecting all the issues with grading, because we’ve seen too often when you don’t do that, people recreate some of those ones that they weren’t aware of. They fix the issues that they were aware of. But there’s other ones I don’t, and sometimes those.

Those issues do lead to grade inflation. Sometimes it leads to grade deflation. It can happen. Sure. But show me the research that a large part of the national wide grade inflation is due to alternative grading practices.

Sharona: Well, I think it’s gonna be really interesting to see where Harvard takes this, going back to Harvard.

I mean, ’cause this piece of research is very interesting. But I wanna go back to Harvard because they actually have some very strong alternative graders at Harvard and they have an entire support page on their center for teaching and learning, supporting alternative grading practices. So if Harvard goes alt grading really goes, it’ll be curious to see if the narrative changes.

Boz: Yeah. ’cause I mean, they started, like you said, Harvard and Yale started this whole mess, you know, 250 years ago. But maybe Exactly. They start the, the, the big new chapter as well.

Sharona: Exactly.

Boz: All right, well, this has been a really fun conversation. Like I said, I hope I, if I got a little worked up, I didn’t offend anyone. But this has been the grading podcast with Boz and Sharona, and we’ll see you next week.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode’s page on our website, http://www.thegradingpod.com. Or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the Contact us form on our website.

The Grading podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.

Boz: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State System or by the Los Angeles Unified School District.