The Grading Podcast

133 – to display or not display

===

Sharona: At the very end, they say this. “Regarding implications for teaching, our findings indicate that consistency in grading practices may be as critical as the practices themselves. The initial display of grades appears to exert negative effects. Thus, educators who must implement grading are advised to mitigate these effects by thoroughly preparing students for the receipt of grades, such as explicitly linking grades to learning outcomes.”

Boz: Welcome to the Grading podcast, where we’ll take a critical lens to the methods of assessing students’, learning from traditional grading to alternative methods of grading. We’ll look at how grades impact our classrooms and our student success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach and instructional designer, whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week, you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Boz: Hello and welcome to the Grading Podcast. I’m Robert Bosley, one of your two co-hosts and with me as always, Sharona Krinsky. How are you doing today Sharona?.

Sharona: I’m doing okay. I mean, I am grateful to be in California where we are not currently experiencing the Sub-Zero arctic temperatures across two thirds of the United States.

Boz: Yes. Which in is included in that is my home state.

Sharona: Yeah. And I was in Green Bay, four days ago, God has it only been four days, so I just missed this arctic storm by like a hair. So I am definitely feeling grateful, even though when I look out my window today, it’s gray. It’s like June gloom in January.

Boz: Oh, really? I’m looking out right now to nothing but blue skys and sunshine.

Sharona: Oh my God. No, it is completely marine layer gray here. So yeah, so I’m doing well. How about you? How are you doing?

Boz: Not bad, although I am really curious ’cause. A) the semester started for you and we talked a little bit about the new semester, so, my week was fine. I’m really interested in hearing a little bit more about how your first couple of classes went. ‘Cause you teach three days a week, but because of the holiday you, you only had two class sessions, right?

Sharona: Right. I only had two class sessions and we talked a little more than just a little bit last week about my class.

Boz: Yeah.

Sharona: Yes, I think it is appropriate to, to talk about how it went before we get into the meat of this article. So overall, I think the classes went quite well. I did much of what I expected. So I started, so a little bit of context this course meets three times a week for 50 minutes, and then it has a back-to-back workshop three times a week, 50 minutes. So I’m supposed to teach a hundred minutes in a, or two 50 minutes in a row. But I did, of course, one of the first things I did is I asked the students, do you want me to do 50 minutes, 10 minute break, 50 minutes, or do you want me to go all the way through? And they all said, go all the way through. So I’m teaching an hour and 40 minutes. And they, if they need a break, they can have it. I did manage to get into one of the active learning classrooms on my campus, so have a fabulous setup for my room.

Boz: You got your rolling chairs and. Did you?

Sharona: I definitely have rolly chairs, but it’s even more extreme. I have these big I have these round table sets that seat six with big screen TVs at each station. So there’s like seven or eight table stations and I can control from the central module what I’m sending from the computer to every one of those stations, which is really pretty amazing. So I started, I did my usual stuff. I do my visually random grouping, so I have their names on cards. I’m constantly moving them around. The first couple weeks we started with our community norming, which I’ll talk more about in a moment. We went on from that to the asset bank activity. So they are getting a chance from day one to start to express themselves and get to know each other a little bit. And then I started right in on day one with the grading conversation. So we started talking about grading. These are also freshmen mostly, so they’re a little more hesitant to share some of their thoughts on grading.

Boz: Yeah. They’re still new to the college scene.

Sharona: Right. And as we know, building trust takes time.

Boz: Yeah.

Sharona: So they’re not gonna be super forthright right at the beginning. And then the second day we went into some more activities on not only building community, but starting to introduce elements of grading, elements of revisions, things like that. And I also did touch on some math content, which was a real eye-opener for me. But I did wanna go back to that community norming. So the very first question I ask about my, from of my students is, what kinds of behaviors do you want from the instructor? Because that’s the most important thing is I want them to feel they have the power to tell me what to do.

Boz: Yeah. And we talked about that on the last episode those series of questions that, that you were gonna ask and that being the first one.

Sharona: Yes. And I always basically get the same answers, respectful, communicative. I got two this time. I got calm back to back with energetic. We had have a little conversation whether I was gonna be calm or energetic or both. But there’s one that I always get, and it always breaks my heart, is I always get asked to be kind and I don’t, I mean, I know it’s the case. Why should my students have to ask me to be kind?

Boz: Hey, you know, I had a similar experience in one of my last evals where someone had came in and observed my room and, made the comment about how novel and nice it was to see that there was a little bit of laughter in my class, ’cause we were looking at a scatter plot and looking at kind of a. Of, I think it was height and expensive cars and making a joke about a one that was a, possibly an outliner and that comment hit. ’cause I mean, it was a nice comment. It was meant to be complimentary of my class, but yeah, the, why is that such a shocker? Why is, one of the key things that students wanna see is that you’re just kind that is really kind of depressing when you think about it.

Sharona: Yeah, so as we were coming in to do the research for this week, that’s been on my mind, the emotions that my students are bringing into the classroom, especially because this is a course that if they’re there, they probably have failed before. We know they’re coming. This specific class failed. This specific class.

Boz: Yeah.

Sharona: And they’re coming in with known mathematical deficits. So this is a very, very challenging class for the students because the math content that we start with goes all the way back to like sixth grade, maybe even earlier. And that’s why I said, when I said that, it was a little eye-opening for me on the math side. We ran into some of those deficits head on, on the second day of class, like the very first two questions that I put up and that we were working on as teams, just like face first into those deficits. And so the emotions are running really high. So when we were preparing for this episode, it was like, what am I gonna talk about? And we came across this piece of research that I’m excited that I want us to talk about today.

Boz: Okay, so I wanna give a warning to all of our listeners. We might get nerdy on this one.

Sharona: Yes, I was gonna say, we’re gonna nerd out.

Boz: This is definitely a, this is not an article about research, this is the actual research. So, yeah we might get a little bit nerdy on this one. I wanna apologize in advance if this we get too far down into the statistical or experimental design weeds of this. But these are things that you and I, absolutely love to talk about and to look at and really getting into the details of actual educational research. So you’ve been warned. I

Sharona: have to say

Boz: you’ve been warned,

Sharona: warned, but I do have to say, I think one of the reasons we’re so excited to talk about is this is a extremely well designed study in, in my opinion, that’s one of the things that we nerd out about is we’re looking at all the details of the research design, the methodology. Dang. They did a good job.

Boz: Yeah, because we, and a big part of the reason we’re even having this discussion is because of, how complex and how subjective grading can be. So it’s really hard to set up an experiments to be a legitimate measure of some of these things. ’cause there’s so many, lurking variables or, things that can affect the results that it’s really hard to do this kind of research and have validated results and man did this group that was doing this go every aspect they can to try to, show that these groups were statistically comparable in pretty much every facet to show that these results are as valid as possible in this kind of research.

Boz: Yes. So I do want to, talk about what that means, displaying grades versus non-playing. Because when you first mentioned this article to me, I was like, displaying grades. Isn’t that a FERPA violation? But we’re not talking about like displaying grades, a. On a board or for everyone to see what, what exactly were they talking about with displaying grades?

Sharona: So they were talking about on specific individual assessments or assignments in a class, whether or not to put a grade on a scale of zero to 10 points on either the paper. Or on the individual electronic assignment for the student. So it was displaying an individual assignment grade to the student who submitted the assignment.

Boz: Yeah. So putting that total or end grade on the paper or however it was, shown to the students or not.

Sharona: And this goes back to the study by Butler that we talk about all the time about feedback grades, both, none. It’s the same kind of thing. In this case though, the addition, because the Butler study was primarily about performance and motivation. This one now adds that emotional outcomes piece to it. Yeah, and that was why I was so drawn to this after my experiences this week.

Boz: Alright, so with the danger of going too far down into the weeds, I do think we need to talk a little bit about how this research or this pseudo experiment was set up. Again, let’s try not to get too far into the weeds, even though you and I geeked out on it. But just in general, what was the setup of this?

Sharona: So let’s start with who the students were. This were ninth grade and 10th grade students. In a set of schools in Spain, and there were two ninth grade classes, two 10th grade classes total of 99 students.

Boz: The different classes had similar makeup between genders. I do believe it was a little bit heavy on the female side, but it was that way in both classes, so right.

Sharona: And what they decided to do is take these four classes and break them up into two experimental conditions. So what they were gonna do to all the classes is they gave them a baseline assessment and where they gave them grades and feedback to everybody. And then one group was gonna get two more assessments where they got both grades and feedback, followed by two assessments where they only got feedback. And the other group was gonna reverse that and get two assessments with only feedback and then two assessments with both grades and feedback.

Boz: Yep. So that’s the general design of it.

Sharona: And. They did all four of the classes, like both ninth grade classes, I believe were taught by the same teacher. And both 10th grade classes were taught by a different teacher, but the same, so they controlled for that as well. And there was one difference between the ninth grade and 10th grade in that the ninth graders submitted their work electronically. The 10th graders submitted on paper.

Boz: And this was done not by the experimental design. This was done because that’s what those particular teachers normally did anyways, so that was an element that was left like that for the convenience of the teachers. So that was not one of the experimental differences.

Sharona: And what they did is they took one of the ninth grade classes and one of the 10th grade classes and put them in a group. And then took the other ninth grade class, the other 10th grade class and put them in the second group so that the experimental conditions that we talked about, the two groups as whole groups were very similar.

Boz: Yes. In fact, they.

Sharona: So they controlled for all those changes.

Boz: Yeah. In fact, they did several different statistical analysis like a mixed ANOVA and to show that any of the factorial differences between the groups were to be expected. There was nothing that was standing out as a difference between any of the four groups.

Sharona: So that’s their groups. So they’re gonna do these two groups with these two con conditions. So what they did is they came up with two research questions. And the first research question had two possible hypotheses that they were testing, and the second one had four. Do we wanna share all of this, or is this too going into the weeds too much?

Boz: No, I think we should at least share the, what were the actual research questions. So what were they looking at?

Sharona: So they basically were looking at one research question related to academic performance and one relate related to emotional outcomes. So their first research question was, how does displaying versus non-playing grades affect students’ performance while keeping feedback comments?

Boz: Okay, so that was one actually looking at per academic performance and then the one that kind of drew your attention and really brought you into this.

Sharona: Research question two was, how does displaying versus non-playing grades impact students’ emotions while keeping feedback comments? And I think one of the reasons that I’m drawn to this, in addition to what I said about my students talking about kindness, is this entire study is acknowledging the complexity of grading as it relates to the student experience. That it’s not just about performance, but that students’ emotions are a huge component driver of their overall academic success. Not just in a given class, but like at a university or in a high school. And so given this complexity of grading, bringing emotion into it I think is hugely important.

Boz: Yes I agree and we talk a lot about student motivation and that’s actually one of the big, kind of pushbacks against some of grading reform and, taking some of the non-academic factors out of grading is that, oh, this is how you motivate students to do it. Looking at the emotional aspect of this I think, has got some really interesting results. And in fact, when they looked at the emotional, they broke it up into two different groups of emotions, positive emotions and negative emotions. So for example some of the negative emotions could be, feelings of anxiety or nervousness about grades or not having control of grades. Positive emotions the feeling that they were being successful or motivated feelings of accomplishment. So we’re looking at both these positive emotions and also the negative emotions.

Sharona: And part of the reason they’re doing all of this is that there’s the direct performance on the assessment. That’s that academic performance. Piece that they’re testing, but those emotional responses contribute to overall academic performance throughout the course. So it ties back, both of these things actually do tie back to student performance. So otherwise, if we didn’t know that, it might be like, so students are negatively emotionally impacted. So what, that could be one of the things people say. It’s no, those negative emotions and those positive emotions turn back around into performance, even if not immediately on that particular topic or that particular learning outcome, whatever. It’s exactly.

Now I want us to share one more thing before we start to talk about any of the results of what they happened. But in addition to checking that the two groups of students were very similar because of the design of combining one ninth grade and a 10th grade and one group, et cetera, they actually went further to make sure that the groups didn’t vary on some things that had nothing to do with being in ninth or 10th grade or submitting electronically or not. So they looked at students’ goal orientation. They looked at a bunch of personality traits and they looked at students’ receptivity to feedback, and they checked those at individual students levels, but then compiled those into the groups. So made sure that the condition one group and the condition two group were not different on any of these dimensions.

Boz: Yeah and you talking about looking at all these different dimensions, they really did. So they’ve used a lot of validated and vetted tools that was actually really interesting. And some of these, I think when we were reading this we’re like, Ooh, I wanna learn more about that now, but.

Sharona: Yeah, that’s actually been the biggest downside of this podcast is we keep finding entire new areas of research that I’m like, I don’t need to be an expert, but I wanna know just enough. That happened with the teacher assessment identity. I wanna learn more about goal motivation theory and its distinctions. And now I’m like, oh, I gotta look at personality traits. And now there’s a whole new one that we’re about to talk about called Control Value Theory. I’m like, Ugh, I’m gonna spend the rest of my life just reading research papers with psychological theories or something. It’s not good.

Now they did one last thing, though, to confirm the validity, which is they gave a baseline task to both groups and gave both grades and feedback to both groups in that baseline. So regardless of if they were gonna get grades and feedback, and then no grades and feedback or vice versa. They gave everyone a baseline one that made sure that their academic performance between the different groups was comparable and that they got the same condition going in.

Boz: Yeah,

Sharona: that they’d seen grades and feedback.

Boz: And the reason they used. The displaying grades and feedback as the baseline is because that is what both of these teachers involved were typically doing anyways, so they tried to use as much of what was normal for the teacher, which I also really appreciate as an educator that would love to be and has done, participated in some of this research. If you’re gonna ask a teacher to do that the more you can allow their normal routine to be part of the design, it’s really nice. Yeah, exactly. They looked at the academics, so they’ve done the academic to show that those are similar. They’ve done, emotional scales using something called the Perkrun achievement emotional questionnaire. They’ve looked at, like you said the achievement goals. So the amount of work, like I said, that they put in to show these groups were statistically similar to each other was really impressive. Like they really went out of their way to try to take out any of those, confounding or lurking variables out of their results.

Sharona: Yes. So now let’s describe what happened in terms of I wanna share what the student experience was as they went through the process, and then from there we’ll get to results. The students come up to one of these tasks that’s going to be included in the study. They did a few things. The researchers instructed the teachers to clearly indicate to their students before they took the task. Whether or not grades were gonna be displayed or not. So students knew going into the particular assessment whether or not what they got back out was gonna include the grades or not.

Boz: Yep.

Sharona: And teachers were instructed that when hand they hand out the corrected work, they needed to emphasize the written comments represented the most valuable feedback in working towards improvement, regardless of whether they were getting the displayed grades or not displayed.

Boz: Yep.

Sharona: And then finally, they were also told to communicate to students that were in the non display phase that they should focus on leveraging this feedback. So this all happened before they took assessment and when they were hand it back.

Boz: And one other thing with the groups, when the grades weren’t displayed, the teachers still explained and emphasized that the assignment itself or assessment was in fact graded. It just wasn’t being displayed. And all of that was made very clear to the students involved. So there was no surprises. There was no, blinding of the subjects where I knew or I didn’t know if. I was gonna get something returned to me with or without a grade. So all of that was very clear and laid out to the students.

Sharona: And then once they received that information, they took a survey. And the survey was based on prins control value theory which is the theoretical background of this particular study. And emotions, like you said, are categorized based on whether they’re positive or negative. And in typical CVT also they did activation level activating versus deactivating. They excluded deactivating versus deactivating from this study to simplify and streamline things. So they were only looking at positive or negative emotions, and they had a set of emotions that they were checking for. So their emotions were the list of emotions. Every single one was ranked by the students on a scale of one to five, where one was, I experienced it very slightly or not at all, and five was I experienced this emotion strongly or very much, very extremely, I guess it was. And they also specifically used adjectives instead of noun. So I’m proud versus pride. So pride would be the noun of the adjective. Feeling proud would be what they were actually checking it for. So students were asked how, like things like studying makes me feel irritated or I’m ashamed, or some of those things, those were the actual sentences that students had to respond to.

Boz: Okay. Are we, do we wanna start? ’cause there’s a lot to it. Do we wanna start looking at some of the results?

Sharona: Yes.

Boz: Okay. So I think.

Sharona: Results are the thing now.

Boz: So I want to, I think we should talk about some of the like baseline results before we get into some of their interpretation of the results. And I wanna start with that research question two. I wanna start with the emotions. ’cause that was the part that, really brought you into this. And I find this very interesting. One of their measurements of specifically positive emotions, and if you actually look this up this research up, this is their figure two in their actual research. But it has some interesting results. So for the group that started off with just feedback? No grades displayed, just the feedback. They come in after the baseline with, pretty positive emotions. There’s a little bit of decrease in, in the positive. Going from, one to the second assessment, but as soon as you add the grades back. So this, I’m talking about the group, the first two assessments, no grades, just feedback. The third one, they have grades introduced. There is a huge decrease in positive emotions with that group, like a dramatic decrease. With the group that started off with getting grades go, going to their second one. There is also a diminish of positive results or positive emotions. It is not as dramatic, but there is still a significant drop. So adding in, showing the grades and the feedback is lowering students’ positives emotions with the group that then has the grades taken away. So they got the grades and the feedback the first two times, and then don’t get it the second two times. They have a steady and fairly substantial increase in positive emotions.

Sharona: Yeah, so both conditions, introducing grades and taking them away had the expected impact. So introducing grades tanked their positive emotions, taking away the grades increased the positive emotions. So both conditions showed what we would expect if grades were bad for positive emotions,

Boz: and we see the same, basically the same thing with the negative. We see this, decrease of negative emotions with the groups that did not have the grades displayed at first, and then a huge increase in negative emotions. On the third assessment where they are, the grades are reintroduced.

Sharona: And in addition, one thing you didn’t mention is the positive emotions. Kind of the, at the very first assessment, they were basically the same, the level.

Boz: Yeah.

Sharona: But in the negative emotions, they were not. So the display condition had significantly more negative emotions From the beginning.

Boz: From the beginning, yep.

Sharona: As opposed to the non display group.

Boz: Yeah. And both of them in the negatives did dip, going from one assessment to the second with the group that had the grades displayed. It is a very, in fact it’s not statistically significant. If you look at the tables, it is very minor where you have this drastic drop with the non displayed grades, and as soon as the, that group one. When they go to their third assessment and all of a sudden grades are now taken away, taken back away, and it’s only feedback. That a huge drop in the negative emotions.

Sharona: Yeah. So independently, that’s one of the things I had to learn like about the growth mindset scale. You can have fluctuations on your positive emotions and fluctuations on your negative emotions. And they’re not in contrast with each other, right? You can have an increase in positive and an increase in negative. You could have an increase in positive and a decrease in negative. Like they’re not on a scale from positive to negative. These are independent emotions. I think that’s important to know.

Boz: But what I found really interesting in this. In both the positive and negative emotions, the group that started off with the grades not being displayed and then having it reintroduced, have the more dramatic changes, like in the more dramatic loss of positive emotions and the more dramatic in crease in negative emotions. So that group going from what they traditionally would’ve seen to all of a sudden not having grades displayed and having all these positive effects because of it, but then getting reintroduced to having the grades, their changes were so much more dramatic than the other group. I found that really interesting.

Sharona: And it relates to this theoretical construct, so it’s not just. That the emotions are there, but the control value theory, which is used heavily in educational settings, specifically talks about students’ perceptions of control and their perceptions of the value of what they’re doing. That’s why it’s control value. And so it says that by this, that this research, they have been able to directly link. The emotional responses to the academic performance, and they’re able to see that this perception of a loss of control or a restoration of control and value, that these things can fluctuate, right? So a student’s perceptions of their control and their value over their educational learning can be altered. In response to changes in the learning environment. So we, this is where grades may actually have the biggest impact is students’ perceptions of control and value.

Boz: Yeah. But I found that really interesting that, ’cause I’ve never thought of it this way, that displaying the grades actually was one of the things that took away the student’s feeling of control of, their academic journey. I never looked at that and would’ve thought, by putting this letter grade or this percentage that I was affecting students feelings of control. And this is, again, going to that CVT or a control value theory that they were using. So that was a result that I had never thought of, even though I’ve been pushing for this for so long, I’d never actually thought of that aspect of it.

Sharona: And it leads me to a question that I’m grappling with for myself is, I use emojis as marks and my emojis are pretty much a check for you’ve gotten to where I need you to go. Then I have other marks for, you’re not there yet. Here’s what I want you to do next. So like I have a mark for revising their existing work and I have a mark for you have to retake something. I wonder if those marks and their affiliation to the fact that it’s more a communication of what you have to do next. Either you’re done or you have to do something. I wonder if that actually has the same impact on the perception of control. Does it help students feel like they have control or does it take it away or neither?

Boz: See, and I would, ’cause we talked a little bit about this before we hit record, I would argue that what you are talking about with the marks that you use. And what was done in this research was a, which was the typical like they used the typical Spanish version, not maybe not necessarily typical American, but they used a zero to 10. They used a number which we know as soon as you see a number, people are gonna try to do math on. With yours and with the emojis and, not even using terms like proficient and near proficient and like even taking that language out. I don’t think you can make a, one-to-one equivalence of this. I don’t think what you do would have necessarily anywhere near the same results of these marks that they’re talking about because they are so dramatically different.

Sharona: I agree with you. I’m just hoping that at least at a minimum, I’m mitigating the negative impacts though.

Boz: Yeah.

Sharona: Of what we see in this research. Because if not if putting those marks is taking control away, then that’s something that I would have to consider about whether or not to continue doing that.

Boz: Yeah,

Sharona: so I think it, it’s just an interesting place because that’s one of the things, one of the criticisms we get sometimes is, okay, you’re not using numbers, you’re using mastery or proficiency or meets expectations. Who are you to set the expectations? Who are you to decide that it’s good enough? And it’s that’s our job.

Boz: Yeah,

Sharona: But it’s also maybe not, so it just was, it’s just something that as I’m reading through this, again, this was interesting ’cause it’s not directly designed to be about alternative grading, although some very interesting results coming at the end that end up tying it to it. But it’s just, it’s causing me to continue to question my practices. Yeah. Because just because I think these practices are better doesn’t mean that. I’m not realizing that there’s something that’s still not going great. Yeah.

Boz: There was one other interesting thing I saw, especially in these emotion results and they talk about ’em in like their discussion of results, which was. At the last one there was whether it was, the positive emotions going back up or the negative emotions going back down a little bit or even back up in one case. But there was this kind of stabilization where. A lot of the results became more similar to where the baseline was, and they, in their discussion, they call it the bounce back effect, which I interpret that as students are gonna adjust and evolve to whatever environment they’re in. Our students are extremely resilient. And I think some of this shows that, and that they talk about this in some of their discussions, but which also brings this, back to, I was having discussions in some of my coaching this last week that, students are gonna rise up to your expectations. Like they’re going to adjust to the environment that they’re in, and they, we see that here too, all right. So we’ve been talking about really some of their emotional findings, but what about the academic findings? What happened in those two groups there?

Sharona: So the academic performance, which again, if you’re looking at the article is in figure one, the difference at the third assessment, which is where the conditions shift, right? So for both groups, the first two assessments are in either display or non display. And then in the third one, they shift. The difference is very significant in both cases. Both in the case when they went from grades to no grades, and when they went from no grades to grades and. It did what we expect if, if you’re people like us who believes that displaying grades is possibly bad, but I was shocked at the degree to which it did it.

Boz: Yeah. And again, one of the big pushbacks that we get all the time for a lot of the grading reform is the motivation. Students aren’t gonna try if they don’t have all this stuff. These results, yeah. The, when the students had that change and the change went from not seeing, not having the grades displayed to displayed versus displayed to not displayed. That is the biggest difference between the academic performance with the ones going from displayed to Non-displayed having the highest, like the highest results of any of them, and the one that was now getting the grades, having the lowest. So not only was it not what we, you know what. Some people would argue, but it was quite the opposite. It had a huge negative impact.

Sharona: But then something a little strange happened at the fourth assessment. And that’s what you were talking about on the emotional side.

Boz: Yeah. We see that same bounce back effect to where they’re almost identical on the fourth one.

Sharona: But what’s interesting to me is in the case of the non display so the ones who went from displaying the grades to non-playing the grades, both of the assessments when they weren’t getting the grades were very high. And they were at the end of the time. So you can also argue, okay, students are getting better over the time, things like that. But in the, going from non display to display case where that third one just tanked. They really rebounded, like they rebounded higher even than the other condition. So that was just a really interesting when they bounced hard. Like they, they hit that display really badly performance wise, but then they kinda adapted and adjusted and took it way back up,

Boz: which does bring an interesting question. And a question that actually worries me. ’cause what I’m seeing is that there is benefit to not displaying the grades and just doing feedback, but there’s a lot of harm once students start to get used to that and then go back to getting it now that harm is short-lived because of this bounce back effect. I’m wondering if there’s something similar, if we are taking the students that are, been doing traditional grading and we get ’em for a whole semester and we’re doing alternative gradings and we’re having these kind of changes, what happens when they go back to a traditional class?

Sharona: So I think one mitigating factor in that is that we’re only one of four classes in a semester. So even though we might be doing this entirely alternative grading, I think at the same time they’re still experiencing two or three other classes where they’re getting the traditional display. So it’s not like they go six months without getting grades displayed. They’re only not getting ’em in our class.

Boz: Yeah.

Sharona: So I’m thinking that factor mitigates

Boz: And if anyone hears that, that’s not an argument not to do traditional grading. That is an argument to get non-traditional grading in more places, more often, faster.

Sharona: And in fact they say this at one point in the paper, they say their findings indicate that consistency in grading practices may be as critical as the practices themselves.

Boz: Absolutely.

Sharona: So I think we would be in more trouble if we do more of these shifts in a semester than ’cause students are adjusted to the fact that different instructors grade differently. They expect that. So that’s almost a consistency is every semester at the beginning I have to adjust and they know that and so hopefully that also is, but within a single instructor, be consistent.

Boz: Yeah.

Sharona: That’s one of the things that we take away from this.

Boz: Alright, so what were some of the other discussions and findings from this?

Sharona: So one of the ones that I think is a little bit of a smaller finding, but I did think was important is that there are some conventional dichotomies, right? High stakes versus low stakes that this is a common one. And oh my God, it comes up all the time in grading.

Boz: Yeah,

Sharona: Homework is low stakes because they can do it more times than whatever, and then when you look at the amount of time a student has to spend on a set of homework to try to get it up to a sufficient. Score so that it doesn’t hurt them in the overall grade is ridiculous, and that’s time that they’re spending on homework that is not necessarily directly related to what they still need to learn. So high versus low stakes is not as obvious as you might think based on grading practices and even having grades or not having grades does not distinguish whether it’s high stakes or low stakes.

Boz: Yeah. Although I wanted to add to a little bit to that because when it’s talking about high stakes and low stakes, that was in the view of the student. It is not necessarily, as an educator, when I hear high stakes versus low stakes, I have ideas of, what that means. But this is talking about the viewpoint of the students of this assessment being a higher, low stakes and the reason that’s important is because we see with the high stakes, a lot of the negative emotions are increased, the anxiety, the nervousness, the like dread. So, it’s not. The it, we’re not saying that, putting grades on or doesn’t, not putting grades on affects the assessment if it’s a higher low stakes, but it was the student’s view.

Sharona: And part of that is actually the impact of the feedback. So even if you don’t have grades, there is emotional and motivational impact of feedback and. It’s extremely complex.

Boz: Yeah. And that was actually what are their findings, right? That

Sharona: So it’s the content and the delivery of the feedback may be as or more important than whether grades are displayed or not. So they say that this complexity underscores the need for educators to carefully consider how feedback is framed and delivered. Recognizing that the stakes of educational assessments extend beyond the simple allocation of grades. So what you say, the sort of judgment that is embedded in your feedback can be extremely impactful on students.

Boz: Oh yeah. And I mean we’ve done whole episodes on how to give feedback and we’ve had lots of other episodes that it comes up, but there is an art form to giving feedback and i’ve seen it with, with my own daughters especially my oldest that just graduated last year from high school. Some of the feedback that was just, red X’s or this is wrong, or how negatively impactful that can be. And we see it when we’ve calibrated our grading and our statistics class at Cal State LA that the feedback, knowing how to give effective and efficient feedback, but also showing and teaching our students how to use our feedback is incredibly important. And yeah, this research backs that up that, even with the absence of grades, especially harsh feedback or ineffective feedback can still have that same negative impact on students’ emotions and future performance.

Sharona: And I would actually argue. The same is true in reverse if you don’t give feedback and the only thing a student is getting is a number, a score on something.

Boz: It’s horrible.

Sharona: But we saw something recently on Slack. Did you wanna share about that conversation? That we saw one of our community participants had with their daughter because the daughter had gotten some zeroes.

Boz: Yeah, I actually, I’d almost like to pull it up and read it, but yeah, we got this really nice post on Slack. We don’t have the permission to say their name, so we’re not going to, but if you’re listening. That post made both of our days. That was such a nice posting. Did, but yeah, it was basically someone talking to, having a conversation with I believe it was their daughter, correct?

Sharona: Yep.

Boz: And, the daughter was really upset because their math grade was low, or at least not as high as the child was hoping for. This writer looked at it. Her assessments were great, like every one of them were stellar. She was just missing a couple. And because of that, it had this, huge negative mathematical impact on her overall grade. And the student, the daughter thought she was bad at math. And the writer, the parent was like. You’re not bad at math, you just got a couple of missing assignments. I’m not worried about this at all. And the relief that it gave their child, it was just, it was really interesting. It was really nice post. We should reach out to see if we can get their permission to mention more of it, but thank you for posting that.

Sharona: And I do wanna read one particular quote from the parent.

Boz: Okay.

Sharona: It says, do you see why I’m not worried about your math grade? Those zeros didn’t give us any information about what, they were just missing pieces of information. ‘Cause the student was getting perfectly adequate scores, but had these missing things and the kids said, so it’s really okay. And the mom’s yeah. And so that’s the thing is ’cause we see a lot of arguments about these zeros. Those zeros are feedback. Those are feedback to students, and if you don’t clarify what those zeros mean, they interpret it all wrong. So feedback. Super important. One last thing that I wanted to mention, because this came at the very end of the article. We’re getting very close to time. But this just again, this whole article, no mention of alternative grading anywhere in the whole thing.

Boz: I wouldn’t necessarily say that they don’t mention alternative grading, but they don’t explicit

Sharona: say it.

Boz: A lot of the research questions and a lot of the abstract, a lot of the is based on work from people like Guskey.

Sharona: Yes. But they don’t specifically mention alternative grading. And at the very end they say this. Regarding implications for teaching, our findings indicate that consistency in grading practices may be as critical as the practices themselves. The initial display of grades appears to exert negative effects, thus. Educators who must implement grading are advised to mitigate these effects by thoroughly preparing students for the receipt of grades, such as explicitly linking grades to learning outcomes. And for me, that kind of came outta nowhere in the article.

Boz: Yeah, because that’s,

Sharona: and I’m like,

Boz: that’s what are the pillars of a alternative grading and how it’s different is our grading is linked to that, not on percentages, points and averages.

Sharona: So making those grades super explicit about what they measure can help mitigate that display. I found this whole article fascinating. There’s so much more. And like you said they link Guskey, they link Brookhart, they link Butler and Neeson. There’s, yes. The, a lot of this was designed, but I wanna just shout out these authors for even doing this study.

Boz: Yeah.

Sharona: Because we need this kind of high quality research. To be done of this incredibly difficult topic.

Boz: And to go on that, I wanna read one of one last sentence from the research. This was close to the end of the, or just before their conclusions part, our design, echoing the insights of Brookhart and Guskey, reveal the complexity surrounding the effects of grading. Suggesting our research methodologies should reflect this complexity, but just it going back and pointing out that. Grades and grading really is a complex, nuanced and multi-level conversation, which is why grading reform is so difficult. It’s why I am still blown away every time we have conversations with any kind of like pre-service educational program. How little our time is in actually researching and studying and being taught the art of grading, like why in every K 12 and quite frankly, higher ed program as well. Why there isn’t a, art of grading 101, 201 and 301. I don’t understand.

Sharona: I don’t understand it either. But I am grateful that people are starting to do it, and I’m super grateful for the community of people who attend the conference, listen to this podcast, engage with us because we’re all on this journey together. To learn how to do this thing called grading better.

Boz: All right, and with that, I think we’re gonna end up putting a pin in this one. So I want thank everyone. Hope we haven’t bored you too much by geeking out on the statistics and the experimental design, but you have been listening to the Grading Pod with Boz and Sharona, and we’ll see you next week.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode’s page on our website. http://www.thegradingpod.com, or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the Contact Us form on our website. The Grading podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.

Boz: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State System or by the Los Angeles Unified School District.