The Grading Podcast

82 – Second Chance Quizzes – Bri and Rob research

===

Sharona: So this is a student named Adriana and she said, "honestly, even when I got the questions wrong on the quiz, instead of being all stressed out that I had done terrible on it, it was more like a minute of clarity. I looked at it as being helpful to get questions wrong because now I could figure out why rather than focusing on, Oh my gosh, I’m not going to do well in this class." So that, "Oh, I can figure it out." That’s the key. That’s competency. I’m capable of figuring it out.

Boz: Welcome to the grading podcast, where we’ll take a critical lens to the methods of assessing students learning. From traditional grading to alternative methods of grading. We’ll look at how grades impact our classrooms and our students success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist, and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach, and instructional designer. Whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week, you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Boz: Hello and welcome to the grading podcast. I’m Robert Bosley one of your two co-host and with me as always Sharona Krinsky. How are you doing today Sharona?

Sharona: I’m doing ok I’m very happy that I’ve sort of cleared the decks on my massive list of to dos this week. I’ve been really inspired by Robert Talbert’s intentional academia stuff with like getting things done and every once in a while I fall into that process and then I fall out. So this was a week of actually following the David Allen process of getting things done. So that makes me happy and less anxious. How about you?

Boz: I’m doing all right. It’s getting into the grind of the semester. So, but yeah, Everything’s going all right.

Sharona: And we have some exciting announcements for this week.

, Yay! And remind everybody, Sharona, when is this conference?

Boz: And the registration is not the only thing that is still open. What else is still open right now that people can sign up for?

Sharona: So we are now in our abstract submission phase. So if you would like to present something this year at the conference you can do so. We’re looking for all kinds of different presentations and talks. We have some new formats this year. We accept all kinds of different presentations, work in progress, classroom practices, research, faculty development, panel, question and answer type things. So go to the website, check on the abstract submission page, and you’re going to see all kinds of information about the types of ways that you can participate in the conference this year.

Boz: And we also offer different timeframes, right? Like 20 minute up to an hour. So if you don’t think that you’ve got something that you can spend a whole hour talking about, there are smaller ones that, I think the smallest timeframe is 20 minutes, is that right? Yes. Yeah. So we’ve got a variety there. And we’re looking forward, like we said last year, we were completely blown away with the amount of abstracts that we got. It was humbling. It was extremely gratifying just to see how this little conference has blown up in just a few years.

Sharona: Exactly.

Boz: And speaking of that, we’re also asking for volunteers because this has blown up so much. So, if someone wants to volunteer, what can they volunteer for and where do they do that?

Boz: Yeah, we were blown away by how many people wanted to participate last year and unfortunately had to limit that just because of the constraints that only a handful of us doing this can do. So we look forward to seeing you at the conference virtually. We hope to get as many if not more abstract submissions this year and hoping to kind of increase the internal part of this with with getting some more people with hands on to help run this thing.

Sharona: Exactly and speaking of being blown away, we were blown away by something here on the podcast.

Sharona: Absolutely. It was just, I’m just stunned. And I know that it’s not that we’re so great. It’s that people are craving results. People want to know, does this stuff work? Does the stuff not work? Do you have any evidence? So having that evidence was pretty critical.

Sharona: Yeah, if any of my administrators are listening to the pod, fair warning. I’m going there.

Boz: But you did make some pretty big changes. So I wanted to take this note from Dr. David Clark and of course we’ll link this blog, which, Hey, shout out to, Dr. Robert Talbert and David Clark. If you guys are not reading that blog, grading for growth, yet, you really should. Like they’ve been doing this a lot longer than we’ve been podcasting. They’ve got a ton of great articles there. you and I have guest written for them. They have a recent one with Robert and Dr. Susan Blum, who was one of our keynotes last year.

Sharona: Yeah. It’s very inspiring. Also full disclosure, we often take inspiration from their blog posts for some of the ideas of our podcast episodes. And I hope vice versa as well. So.

Boz: Oh yeah, no, no, we’ve done a couple of episodes where that they were straight out like blog reviews from some of their posts.

Sharona: But yeah, so to go back to what you were saying though, I do want to reflect on the rest of my semester. So if you’ve been listening to the pod for a while, you know, I took on this expanded course coordination job that included four different flavors of pre calculus and also calculus one. So I now coordinate a total of seven courses, a couple of which they overlap with each other because it’s a supported and a non supported version, but still there’s enough differences between them, that we measure a lot of things.

Boz: So, so back up, cause I know what you mean when you say different flavors, but can you kind of explain that for people listening just so they do get an idea of how different, even though they have the same title of pre calculus, how different these courses really are?

And so following some guidance from the Dana center, most of the Cal States, including Cal State LA, decided on a STEM calculus pathway and then a either non STEM or or non calculus pathway, which was usually statistics, and then possibly a third pathway that was for really sort of very, very general mathematical topics. It could be things from finance to voting theory to basic probability to geometry. There’s a variety of topics.

Boz: So kind of your math for a liberal arts type of pathway.

Sharona: Exactly. And at the time, there was money from the chancellor’s office. So all of these different courses had course coordinators, at least at Cal State LA. And for all of the very first semester of each of these three pathways, we had choices. So you could stretch the class, you could add a co requisite support, or you could do both, were sort of the three ways that we address the fact that we were going to be putting students from all different preparation backgrounds and all different skills directly into the same course.

Boz: It does qualify as the gen ed. So students might be taking that course that aren’t STEM majors, but.

Sharona: Technically it has an associated lab. Technically it’s a five unit class with a one unit co requisite. Okay. but it’s really blurred together the way it is done structurally.

Boz: Okay. So that’s why, like I said, I wanted to kind of take some time because you and I know what we’re talking about and anyone that’s in the Cal State LA system for sure knows what we meant when we said different flavors But I wanted to make sure our listeners did just because those are pretty different courses I mean.

Sharona: Exactly. That is the goal that a student who successfully makes it through whatever sequence they choose is prepared to succeed in calculus.

Boz: So kind of different starting points, different pacings, but the end, the finish line, should be about the same, and I say should be, because.

Sharona: Right. Real life butts in, but that is the goal. That is the goal. I took all these extra pre calcs and calculus over six days before the start of the semester last semester.

Boz: Oh, wow. So you had plenty of time to redesign the entire sequence, right?

Now I was doing it part time in the past and full time now, but still it was a lot. So, yeah. So. When I took this over and you were there right by my side, sometimes hitting me over the back of the head, but usually being supportive and trying to figure out what was I going to accomplish? Because even though this was handed to me so late, there were still expectations that this was going to prove to be a beneficial move on the part of the Dean’s office. So I had to have some results of some kind. I had to do something that was going to justify because this is a one year position and we’re in major budget crises. So trying to justify why to keep this person around who is not in the classroom and is doing all this work that seems very invisible. So the good news is I did a good job.

Boz: And I want to clarify something because we’ve gotten feedback on this before. When we say grade norming, we are norming the way the professors grade. We are not trying to do some normalization of the students grade. We’re not, I mean .

Sharona: Right, we’re not creating a normal curve.

Boz: Yeah. Anyone that’s been listening to us for any time at all should know that that would be so against our philosophies. But no, this process is calibrating the way the teachers grade. So if you and I are looking at the same wrong problem or wrong student work, you’re not taking off five out of 10 points and I’m taking off two.

Sharona: Exactly. And we did better than I expected in that process because I had no time to train anybody. We’re still on points. We’re still using, per problem, a 10 point grading scale and I was trying to get it within about a two to three point band, usually two point band. I was trying to get everyone within a point or two of each other for any given type of work.

Boz: Which I’ve said this before in a couple episodes when you were talking about it at the beginning of the semester I’m gonna say it again. I know just us being friends outside of work and this podcast how frustrating you were finding this last semester. That change alone, I do not think you give yourself credit enough for, even though you weren’t able to pull it off at a hundred percent with all the professors, that change alone, and the fact that you’ve got a good chunk of them is nothing short of monumental.

Sharona: Thank you.

Boz: And anyone that’s been listening to me for a while knows that I keep going back, even though grading reform really is my passion. I got started through the DuFour’s work with PLCs and working with PLCs, at the higher ed level, they’re usually called FLCs, but getting groups of educators together and having these conversations and it is so powerful. Teaching can be such a silo and such a kind of lonely place, that you really don’t understand the benefits of really working, planning, and, and going through everything that we do as a group of fellow educators, it really is powerful. If, if you’re not doing it, I really encourage you. There’s a ton of things out there on PLCs on FLCs. It really is the best way to, in my opinion, to improve overall. Instruction caliber at an institution, whether it’s a K 12 or a higher ed.

Sharona: And I want to do a shout out to one of my instructors. I don’t have permission to name her, but she did bring in a proficiency scale that helped this process because when you’re doing points based grading, there’s just too many mistakes to try to decide this is worth a point, that’s worth two points. So we went with a five level proficiency scale of: mostly correct, more correct than not, pretty even, more incorrect than not, and mostly incorrect.

So that was our five level scale that we expanded into basically two points per band. So a nine or a ten was mostly correct, a seven or an eight is more correct than not, and we we basically felt as a team that problems need to be more correct than not to be a passing problem. So that’s how we got seven eight nine ten and we just didn’t bother within the band and if everyone agreed they were in the set nine to ten or even if they were all in the eight to nine like I just didn’t bother with it, But that was sort of the theoretical structure So for anyone in stem who has these complicated problems that students could get wrong so many ways, doing that sort of mostly correct, more correct than not, like that’s a good way to get started

Boz: The the rubrics can be really powerful tools But, you either make them individualized for every assessment and can give a lot of details, which provides a lot of feedback to the students that you don’t have to do additional to, or you do really generic ones like what you were just describing and be able to use it on all of them. And then have to give a little bit more personalized feedback. So it’s got advantages and disadvantages either way. I personally like, doing what you kind of did and keeping my rubrics pretty generic, just because I have seen so many weird mistakes in my teaching life that I don’t think I can make a rubric that could incorporate.

Sharona: Well, clarify because we do jump back and forth on this. You’re talking about proficiency scales specifically for those of you that make rubrics for which are multiple scales on one assignment.

Boz: Well, yeah, I mean, I’m calling it a rubric because it’s the final thing that I used to grade. But yes, that rubric might have one proficiency scale. It might have three if I’ve got three learning targets that I’m assessing. Right. All right. So that’s the first big change.

Sharona: I also like the generic, the more generic ones when you’re working with a lot of instructors, because the more detailed, the more you might get resistance. Yeah. So when people have autonomy, I get to decide what mostly correct looks like it. It’s just a little bit easier to get buy-in.

Boz: All right, so that was one big change, right? What was the second one?

Sharona: The second one I really thought about, we have these four pillars. What can I actually do? And I felt that multiple attempts at showing evidence of learning was the lowest hanging fruit. So I did a couple different ways to try to bring that in. One is we did some exam replacement things where, depending on the instructor, the final exam could act strictly as replacement or part of it was part of the grade, but that part of it was replacement. I encouraged instructors to do what we called second chance quizzes, which is if they gave a quiz to re give it or something very similar and let students try again.

And then we added what we called retakes. If a student scored less than 70 percent on a midterm exam, one of the ones that was really in the middle of the semester. If it was way at the end, we didn’t have time. And they did a corrections and reflection assignment. They could come retake the exam and I proctored those retakes. So I’ve set myself up in a room all day on a given day, about two weeks after the initial exam, and any student who had completed their corrections and reflection and scored less than 70 percent could come in and retake it.

Boz: Yeah. Now we’ve actually talked about this idea quite a bit. I mentioned, Robert and David’s, blog earlier. They’ve written about it. I think we’ve had, what, three or four episodes that specifically talked about reassessment. I know we talked to our good friend and original organizer of the conference, Dr Kate Owens back in what, episode nine? That was wow, that was a long time ago. Most recently we did episode back on episode 70 with Becky Peppler. But then we also had one in between. Yes. So going, going back to episode 41 with Brie Tripp and Rob Furrow, we had this one and there’s a reason I’m bringing that one up in particular, but I love that one, talked about second chance grading, how this small general approach, to changing grading can have such a big impact.

Sharona: Yeah, and actually the title of that episode is second chance grading, how a small gentle approach to changing grading led to research in alt grading. That was the title of episode 41.

Boz: And why are we bringing that up right now?

Boz: Yeah, this was written by Brie and Rob and a couple others. The title of this is Undergrad STEM Students Perceptions of Grading Practices Reveal that Quiz Retakes Positively Impact Drivers of Self Determination.

Sharona: Exactly. Now, this is published in CBE, the Life Sciences Education Journal. It is a free article, so it’s not behind a paywall. And I’m really excited to dive in because they talked about their research in episode 41 and what they were trying to do, but now we have the results.

Boz: Yes.

Sharona: Where do you want to start with this one?

Boz: Well, I think I want to start with one of the interesting things, and this came up in their episode, so I don’t want to spend a ton of time on it. If you want more information about it, go back to episode 41. But one of the things that came up in there was having the framework to build this research off of. And we’ve talked about that. In fact, The Center for Grading Reform is looking at trying to branch out into another type of conference that’s purely on research, but they did this based on Self Determination Theory. So this is where they framed this research. Again, I don’t want to go into too much details, but I do think we need to talk a little bit about, Self Determination Theory.

Sharona: Yes. So in their paper, what they say is that Self Determination Theory characterizes several affective behaviors, factors, that affect the degree to which a behavior may be externally or, or autonomously motivated. So these are the, you know, when we looked at research, a lot of us looked at growth mindset. Some of us looked at performance goal research. But in this case, they were looking at self determination because one of the challenges we have with grading is it’s such an extrinsically motivating factor. And so there’s a lot of debate about grades being demotivational because of the extrinsic nature. So Self Determination Theory is looking at what things might make things more intrinsically motivated. And there’s three things, perceived competence, level of autonomy, and a sense of relatedness to others.

Boz: Yeah. So those are the three factors that subject Self Determination Theory looks at and their research Brie’s and Robert’s research. Part of it was quantified data, but part of it was qualitative based on some interviews, and they tried to break those student responses from the interviews into those three categories of Self Determination Theory.

Sharona: But before we get to those categories, they picked one specific intervention, in the grading, and that’s what they were trying to measure if it impacted these elements of self determination.

Boz: Yeah. And what was that element? Just the retest, just the opportunities to do retakes.

Sharona: Exactly. And they were the ones that originally coined the phrase second chance quizzes, at least as far as I became aware of the language. I’m sure second chances have been around a million years, but they were the ones that first did it in my hearing related to grading.

Boz: And we’re of course linked this in the show notes. if anyone is ever like just wanting to go and look at how you write a good research article and man, this was so well written and detailed out. But yeah, if you want a lot of details on some of their methodology and stuff we’ll link this. I don’t think we have time to get into a huge part of that. But basically they used two different course types of courses, statistics course and a neurobiology course. I think originally they had like what, 112 participants in the two classes for the for the quantified data.

Sharona: Quantitative.

Boz: Quantitative data. Now for the interviews course, anyone that’s ever done any kind of research knows with students, it’s harder to get bigger numbers. So their numbers were lower for being interested in that part of it, but they did get participants from both classes. So they’ve got some slightly different classes involved, and they had some really interesting results. And = I wanna talk a little bit about their results and then go back to your results.

Sharona: Exactly. So do you want to start on the quantitative side or the qualitative side of their results?

Boz: I really want to look at the qualitative. Like I said, we will link this so you can go in, anyone that’s interested can go in. There was some really interesting things from the quantitative side, which might come up when we talk about yours. But yeah, I want to talk a little bit more about the qualitative, the interviews.

Sharona: Okay. So let’s start with where they started. So of those three elements of self determination starting with competency. It says that over three quarters of the student participants in the interview, reliably made connections between alternative grading practices and perceived competence. So I picked out two quotes, but do you have one you want to talk about that one or you want me to read my quotes?

Boz: No, go ahead and read yours.

Sharona: Okay. So this is a student named Adriana. And she said, "Honestly, even when I got the questions wrong on the quiz, instead of being all stressed out that I had done terrible on it, it was more like a minute of clarity. I looked at it as being helpful to get questions wrong because now I could figure out why rather than focusing on, Oh my gosh, I’m not going to do well in this class." So that, "Oh, I can figure it out." That’s the key. That’s competency. I’m capable of figuring it out.

And then she also goes on to say, "I’m also more motivated because if I study this more in depth, not only do I understand it better for the final," I’m sorry, this is the other student, Isabella. "Not only do I understand it better for the final, but I’ll be able to do better for my second chances quiz." So without any prompting on the competency language, the students didn’t know this. This was all coded later. They both were like, Oh, I can do something about this.

Boz: And just to clarify in case anyone’s like freaking out because you’re using student names. These are pseudonym names that are in the research. These are not actually.

Sharona: I’m literally reading from the paper, so I’m not revealing anything.

Boz: Yeah, these are actual student quotes, but these are not the actual student names. These are the names that were used in the research. Exactly.

Sharona: So that’s what I saw on competency. Do you want to take the next one, autonomy?

Boz: Well, that was the one I really kind of, not surprised, but I really liked. So 72 percent described the increase in student agency with lower stake assessments are implemented into course.

Sharona: So before you go on though, I want to just point something out. Lower stakes doesn’t mean low stakes.

Boz: Exactly.

Sharona: Okay. What you do when you take, say, a midterm exam, which is a very high stakes assignment, but you give an opportunity to try it again? That action lowers the stakes, even though it’s still the same percentage of your grade. It’s still the same high stress testing environment.

Boz: So here’s one of the direct quotes from one of the students. "I was about to take the final, because then I was like, Okay, I’m going to retake these two quizzes. So I need to focus more on that material when I am studying. And that worked out well for me. So during the quarter, I didn’t really make much of an impact because I was trying to ignore it and just focus on, like, doing well in the moment. But then once I knew, okay, these are the ones I have to retake, then when I was studying for the final, I catered more towards that material." So giving the students that autonomy of going, okay, this is the direction I need to study. This is where I need to focus my time in.

Boz: And that choice is motivating to students because it’s, Okay, this is what I am choosing to do, not what I’m being forced to do. And because I’m choosing to do it, I’m going to be more motivated in doing it.

Sharona: So then the third piece that they did was relatedness. So the degree to which students feel connected to their peers and instructors. Did you have a particular quote that you liked in that one?

Boz: Oh, all of them are good, but

Sharona: Right. So the one I liked is "this class was very refreshing because the grading system made it feel like the instructor cared about the students. It’s like they just really wanted us to learn and retain the information and not so much on getting a good, that good grade." And I have found this too, that when I’m talking to students and I’m like, don’t worry about it. Let’s learn the material and retake it. The student’s like, Oh, they’re not worried that I messed up and that they’re going to permanently bake this in. They just want me to learn it and they’re going to give me another chance. So that improves that sense of relatedness to the professor.

Boz: Yeah, and another one that, another student quote that I want to read that It wasn’t maybe directly related to these three components of self determination, but I think is incredibly important. It’s one of the things that we’ve been saying on this podcast forever. "The retakes really did help and make a difference because I was able to solidify my learning towards the end, it really did give me another opportunity to dive back into the material again and relearn it the right way. I think it really does solidify your learning. To this day, I’m still explaining Pathways to people, and I remember so much from that class compared to other classes that I have taken. She was talking about learning. She wasn’t necessarily talking about her grade." That is one of the quickest things that I have found with my students that changed. Discussions change from points and how do I get more points to what do I need to learn more about. And this quote, I think is, just amazing that it’s first, again, she was talking about her learning in this. She was not talking about how, yes, this made my grade. gave me a chance to make my grade better. She was talking about how it solidified her learning.

Sharona: And it’s amazing to me because as they say in the beginning of this particular article, they did not go away from points and percentages. They just, quote unquote, did second chance quizzes.

Boz: Yeah, and in fact, again, we didn’t talk about it, you can see it, it’s linked, you can go and look, but, the grading percentages for these two courses, you know, are fairly different. I mean one’s got a project that’s 28 percent of the grade. The other one doesn’t. One actually had some grade on attendance. So yeah, they were using traditional grading in weighted categories and they’re both fairly different. So very different courses, different grading structures. And 78 percent increase in competency, 72 percent autonomy, 72 in relatedness. This is great article. I really do encourage people to go and read all the details. Cause this is very robust and very, very well done.

Sharona: Right. Now, I did want to get to the quantitative piece a little bit as well. I also wanted to point out that "just" doing the second chance quiz actually brings in two of the four pillars because it brings in retakes, right? But then it also brings in the feedback loop. Because even if you don’t give a lot of personalized feedback, there still is a feedback loop, even if students are just learning from their own mistakes. So, this one action brought in two pillars at once. But I did want to mention that the numbers of students who took advantage of these retakes, so the percentage of students, so the two courses over the course of three years is in their data set. The percentage of.

Boz: And hold on, because I misspoke earlier. It was a for for the quantitative part. It was actually 213. I think I said 113. It was 213 students involved in just the numeric part of the data collection.

Sharona: Yeah. I think the 112 was the students that were invited to participate in the interviews of which about a quarter accepted and then fewer were able to actually complete. So a couple of statistics here, the percentage of students taking at least one, retaking at least one quiz ranged from 76 percent to 95 percent of the class. So very high uptake on the opportunity to retake. The percentage of retakes who improved at least one quiz score ranged from 81% to a hundred percent.

Boz: Yeah, that’s massive numbers. So of the ones that retook at, at bare minimum, four fifths of them improved. And these were not small improvements either, were they?

Sharona: No, no. The median average percentage increase. ranged from 5 percentage points to 7. 4 percentage points.

Boz: So we’re talking a good, a good chunk of increase.

Sharona: And look, 5 percentage points, that’s possibly a grade or a grade and a half. Yeah. You know, depending on how the weighting goes.

Boz: They’ve got a lot of other quantitative results and I kind of want to look at some of those and also some of yours because although yours wasn’t technically done as research controlled as much as this one was, how many students did you have involved in the, not counting the statistics,

Sharona: right.

Boz: Because that’s a full, that’s a full alternative grading. Just the ones that you coordinated that the biggest change for the students was second chance opportunities on quizzes.

Sharona: So I had, 1, 2, 3, 4, 5, 6, I had about 1, 300 students, maybe 13- to 15-hundred students, that were in one of the classes where we allowed some form of retake reattempt, not including statistics. So this, this was a massive undertaking. I don’t, I, what I do know is that of the students who were eligible to retake, so the number of students who had scored 70 percent or below, only about a quarter in any given assessment actually took advantage of it.

Boz: So that is a drastic difference between your results and the results of the study, right?

Sharona: Now, one of the reasons I think is because we did not intentionally did not communicate this retake process until after the first exam had happened due to concerns of people not taking that exam seriously and just opting for the retake.

Boz: And let me clarify, because if I was listening right now and not knowing you and not knowing what happened, I would be yelling at you right now. That was not your decision alone. That was kind of a compromise you had to do to get this actually in the course.

Sharona: And what’s difficult about, it is, but what’s difficult about this is that I see both sides because I’ve seen both sides of student behavior. I’ve seen a lot of the, Oh, well, I’m just not going to take it seriously the first time, or I’m not going to take it at all the first time. And that’s not the point because you’ve shortcut the feedback loop, but you have to have a willingness, some level of skill and understanding, to talk to students ahead of time of the importance of that feedback loop.

Boz: Yeah, we’ve talked about this on several episodes. If we’re not explaining to students the purpose of this, and I actually I think this came up with Becky Peppler ‘s episode 70 that we talked about earlier. If you’re not talking to students and explaining the purpose of your grading policy and your retake policy. Yeah, students are going to are going to misuse it. Are going to do some silly things that that isn’t going to benefit them.

Sharona: And so because of the environment I was in, we were not doing that. We were absolutely not having these conversations with students about feedback loops and learning from mistakes, at least not in an authentic way.

Boz: Yeah. And quite honestly, there’s sorry, you’re gonna have to add more to your show notes.

There was a the last I think the most recent January 20th post on grading for blog. Not all limits are the same. Talks about

Sharona: Grading for Growth blog.

Boz: What did I say?

Sharona: Grading for blog.

Boz: Oh, I said grading for growth blog. Okay. Grading for growth blog. David Clark did another great article that’s called not all limits are the same. And it’s talking about that very thing that you were right now with, you know, especially if you’re doing unlimited reassessments, which neither you or I advocate for at all.

Sharona: I used to do that. There’s a reason I don’t.

Boz: That’s why we don’t advocate, advocate for it on this podcast because we did do it before and it.

Sharona: Yeah. It’s not sustainable and it’s not helpful to students. But yeah, so we had about a third who of those eligible take it on during the retakes. However, with the final exam also playing in, I don’t have the data to tell you how many students actually took a retake at some point. Additionally, I was encouraging my instructors to give second chance quizzes, and I think many of them did. So, I don’t have that data. We just didn’t track it. I do, however, have the data on my pass rates.

Boz: Yep.

Sharona: So again, I made two changes, some form of re takes reassessments throughout and the grade norming instructor calibration process. And I had improvements that were any, the least improvement was my supported, first semester pre calculus, because that is the students coming in with the biggest gaps in their background. It still improved about ten, about ten percent year over year. 10% of the fail rate. So if the fail rate were, say, had been 30%, it would be 27%. Okay? So the fail rate would’ve been, it, it, the fail rate itself decreased by 10%. So it would’ve gone from 30% to 27, for example. That was on the low end, on the high end the fail rate was cut by a third in,

Boz: that’s impressive.

Sharona: Yeah. In one, two, three classes. And cut by a quarter in some of the other classes.

Boz: So the one, and I find that really interesting, that the students that are taking the course that gives them the most support, theoretically, this isn’t always the case, but theoretically, these are the students that are interested in a STEM field, but for one reason or another, need a little bit more support with getting ready for calculus. That one being the lowest and I’m going to make a completely unsupported statement by data, but just based on my experience and based on some of the findings that, Brie and Robert had, or Brie and Rob have, had you, because these are likely the students that have the least amount of competency, at least internally feeling that way, if you’re looking at, Self Determination Theory. Had you explained those retakes earlier, had you explained that policy? I bet those numbers would have been better.

Sharona: I think so. And we are making one change this semester that my instructional team is not super happy with, but we’re going to allowing everyone to do retakes instead of capping it at the people who scored 70 percent because part of the challenge is they scored 70 percent or less, and the best that they could get on the retake was 70%.

Boz: Which is really interesting because that is one of the results that came out of Brie and Rob’s. One of the quantitative analysis is looking at which students retook and which students didn’t. The amount of students that scored good grades, that scored above 80%, even above 90%, that still elected to do the retakes. I mean, look at that. If you’re, I don’t know if you’re looking at that on that histogram, but those numbers were, I was surprised to see that students that were scoring, you know, between 95 and 100 were still retaking the test.

Sharona: So that one doesn’t surprise me partially because of the university context they’re in these tend to be students who are very grade driven, but I think what’s important for me actually is the percentage of students between 70 and 90. That was extremely high, like almost everyone between 70 and 90. And I think part of that is because we know just how damaging even something in the 70 percent range is, to be able to pass the class because of the way these weighted averages work. So it’s just, I think the skewing is rough.

But then if you look at the students below, say, 70%, or actually below like 85, 80%, like every student did retakes, like all of them. There’s no one who was in the no retake category. So, yeah. And also if you look at the, there’s a graph here that talks about their initial quiz average and then their average improvements. And you definitely see a lot in that say 50 to 80%, if they retook it, many of them went quite a bit higher on their improvements.

Boz: We’re talking 30 percent, like improvements of 30 percent.

Sharona: So, this stuff works and it really has the opportunity to impact those averages that we rail against so much. So I see that we’re getting close to time. I really like the one thing I want to go back to the article. Self Determination Theory was not something I was super familiar with prior to this article, and speaking to Rob and Brie. So I’m very curious to see what direction this research takes because we’ve got some people looking at motivation theory, some people looking at goal theory, some people looking at growth mindsets. And now some people looking at self determination.

Boz: Yeah. Which you said you were, I had never even heard the term before we were talking to Brie and Rob. Like I remember as we’re doing that interview, I’m looking up, I’m like, what did they call that self something? I was trying to Google it just so I knew a little bit about it while we were discussing, but yeah, that’s come up, not just in theirs, but in others, this framework of what to do the research on. I think self determination is one that a couple of people have talked about that I think is a promising and is a framework that can definitely be used. There’s a couple more out there, but that’s part of the reason why, you know, the grading center or the center for grading reform is. Looking at doing a new type of conference in the future, with how things are in education right now, it might be further in the future than we wanted, but trying to bring that research, cause that’s the other thing that they’ve talked about that others have talked about is we have all these pockets and not a lot of agreement. And not a lot of just bringing, you know, big resources together and bringing big pods of things together to really get a massive type of research going and agreements on those.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode’s page on our website, www. thegradingpod. com, or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show, or would like to be considered as a potential guest for the show, please use the contact us form on our website. The grading podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.

Boz: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State system or by the Los Angeles Unified School District.