The Grading Podcast

Sharona: Well, and I think that starts with what does it mean to get an A in your course?

Bosley: Yeah. And when you’re making these decisions, that absolutely is where it starts. What is the purpose of the course? What do you need the students to leave with? If a student is leaving with an A, how is that student similar or different than the student that’s leaving with a B?

Welcome to the Grading podcast, where we’ll take a critical lens to the methods of assessing students’, learning from traditional grading to alternative methods of grading. We’ll look at how grades impact our classrooms and our student success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles. Faculty coach and instructional designer, whether you work in higher ed or K-12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week, you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Hi everyone. Welcome back to the pod. In this episode, we’re going to be talking about the decisions three and four of the grading architecture, which are how students will show sufficient evidence to complete a learning target and how instructors can wrap up to a final grade. So, Welcome back to the Pod.

I’m Sharona, and Boz how are you doing today?

Bosley: I’m doing really good. I’m actually very excited about this episode. This is kind of a tying up a three episode series. You know, back on episode five, we really went into the overview of the grading architecture. Then our last episode, episode seven, we really took a deeper dive into those first two decisions of how are you going to collect evidence, how are you going to assess your learning outcomes, and then how are you going to grade those?

This episode kind of completes that sequence, not that this will be the only, the last time we talk about this, obviously we’re going to go into a lot more depth in later episodes, hopefully having some guests on to kind of talk about specific ways of how they do each of these different decisions.

But yeah, I’m excited about this one.

Sharona: And I think you especially like the third decision, right?

Bosley: I think the third decision is unique because it’s the one that is completely absent in traditional grading. So that third decision of how do you know when a learning target is actually at the level that you want to count it?

In traditional grading, because we don’t do, our grading system is usually based by categories, test, homework. It’s never looking at that, a single learning target so this decision is unique to alternative grading. And that can sometimes be a little bit problematic for new practitioners to kind of wrap their heads around.

Sharona: Well, and I think the other thing that these two decisions really get at the heart of, and is a little bit of a mental leap for a lot of people is do we care when, during a term, the student shows sufficient evidence of learning, to check something off is completed? I mean, both checking it off early and checking it off late.

And I think the answer is no, we shouldn’t care.

Bosley: Well, yeah. And that kind of goes, you know, directly to that fourth pillar that we talked about back in episode four, that eventual mastery matters. And we’ve talked about Ken O’Connor before and some of his things, but he has a, we do this parachute, his parachute example.

Both we talked about it on the podcast and in a lot of our trainings. But he also has another one that he likes to do in his trainings where it shows four different students, one that is starting off already with a lot of these skills and just kind of maintains that, that high level of achievement throughout the course or the semester. And then other students that start at different points, but they all end at the same point. And his meaning or his goal with that is to say, you know, the student that starts off with a lot of gaps or a lot of missing things, but by the end still reaches that same goal as the student that started off with all the material already there, all the holes already filled, if they end at the same spot, shouldn’t their grade reflect that in a traditional grading? It absolutely does not.

Sharona: Well, and as you said then, and I was thinking about that parachute packing problem, this goes back also to course context and institutional context. You know, if you actually had a parachute packing class, and at the end of the class you have two students, one of whom is consistently packing it well, and one of whom is still jumping all over the place.

Like, those are two different students, but in like a calculus class, as long as they’ve shown they can do it, I’m less concerned about the jumping all over. So I think context also matters, but the end of the term really is the most important point at which we’re looking at this, and I think we’re going to see that as we go into these decisions in a little more detail.

So decision three is figuring out when we have sufficient evidence that a student has completed the content, whatever we mean by that.

Bosley: Yeah. So since our philosophy of grading is based on how much of our learning targets that a student has gotten a level of achievement or proficiency, or mastery, whatever you want to call it, our grading system needs to be set up in that same way that we’re not judging how much homework or what a student did on assessments, but on these learning targets. So that third decision is, okay – we have all these assessments on a specific learning target. How do we say, okay, they’ve gotten proficiency, they’ve gotten mastery, they’ve gotten the level of achievement that you are expecting for your class and your situation.

And in that episode five where we did the overview of grading architecture, we talked about a couple of different ways to do that, so I kind of want to take a minute and review those three methods that we talked about, although those are definitely not the only three, and then we can kind of get into more detail of all three.

But the first method we talked about is what I like to call the Guskey method. And that is just looking at the overall patterns and breadth of all of the different assessments used for that learning target and making a judgment call based on those patterns you see, instead of some sort of mathematical algorithm.

Sharona: And something I want to add there, you know part of this alternative grading world is this thing that is called ungrading, right? And I especially like the Guskey method because who is looking at that pattern of learning? Can be the student initiating that, which would be much more of an ungraded structure, right? The student is initiating, this is where I think I should get my final grade based on all this evidence. But it can also be used by the instructor. So either one, particularly in the Guskey method, is suitable to a student driven final grade discussion or a instructor driven.

Bosley: And if you want some kind of more details or even some of the research that backs this up even though Guskey has, I don’t know, a dozen or maybe more books out now, two of them are, are really good on that. And that’s On Your Mark and Get Set go.

Sharona: Then there’s a third book by Guskey that he just revised into a third edition, which is Implementing Mastery Learning.

Bosley: Yeah, that’s actually the book I am currently reading. Joe and I both are reading that together.

Sharona: That would be a definitely a stretch goal for this podcast is to get Dr. Guskey on as a guest, so stay tuned. We’re going to try for that one.

Bosley: Yeah. We did have him on as one of our keynote speakers to the grading conference that was for the K-12 world. And anyone that’s ever met me and, and heard me talk about this knows that I just have a huge professional crush on Guskey. Not only is he an incredibly intelligent well versed person, but he is just such a nice, like down to earth guy. He, he’s great. If you ever get a chance to, to meet and talk with him, please do. He is absolutely phenomenal.

Sharona: Well, and I think that the opportunity to meet some of these people who’ve been so influential in our work and knowing that we’re putting the energy and effort into this podcast and getting to give back to them after they’ve done their life’s work is, it’s part of what makes this such a joy for me.

Bosley: Absolutely. All right, so enough beating around the bush. Let’s, let’s get back into this.

Sharona: Okay. Little sidebar. That’s okay. So our second method that we talked about was the N times, meaning that a student has to show a sufficient level of skill on a learning target. N number of times where N is one or more, definitely an integer number of times we’re not doing 1.3 times.

Bosley: Yeah. And one of the key things about that method is it should be a predetermined amount. This is one of the things that leads to being able to explain the system to a student to get buy-in, which buy-in is a very important part of this that we’ll do multiple episodes about. And those different learning targets could have like different N times. I know this is the method that you and I both use in the coordinated statistics class.

And one of the things we’ve talked about is we do have three learning targets that are assessed more often than the others and talked about wanting tomaybe have those at a higher N times. So that you can do that. I know we are kind of limited by our learning management system, our LMS.

Sharona: Yeah. I mean, we could set it more, but honestly at this point, there are more opportunities just because of the structure of those learning outcomes. It’s not like we need to see it more often. Yeah. So there’s not a justification that way. We do have a learning outcome that’s only one time, like most of our learning outcomes are two times we have one that’s one time, and that’s our habits of mind standard, because that one is just a preponderance of the evidence kind of thing.

They accumulate so much evidence and as soon as they get to that evidence level, they’re done. So it’s calculated in the grade book as a one time, but it’s more mechanics than anything else.

Bosley: All right, so that was, that was the N times method. And then the other one we talked about was the decaying average.

Now, in our second episode, we talked a lot about why averages should not be used, but a decaying average is different. A decaying average, which puts a lot more weight on the later assessments and isn’t punishing students for early mistakes. Now, we, the one big caution, we talked about it in that episode five was if that mistake, if that student stumble, is one of the later assessments using a decaying average it can be very damaging. Where say I was using the Guskey method, I could see that was just an outlier. That was a bad day. That was, you know, just something unusual happened and I can still see from the rest of the evidence, no, this student got it. He just had a bad day, or she had a bad day that day. The decaying average, if that bad day is towards the end, it is going to be very detrimental to the grade.

Sharona: Well, and I have real issues still with the decaying average that actually, even when we used to use it, we had to do a lot of careful thinking because these proficiency scale scores, they are still categorical data. I mean, technically we should not be averaging them. The only purpose of the decaying average is to allow for a level of automation in the grade book tracking.

So if you, if you do two most recent times or three most recent times, it’ll have sort of the same effect. So there is a way to use the decaying average that I think is okay, but it still has that problem of turning categorical data into quantitative data that I don’t like. And so, first of all, you should only use this if you’re going to assess… So this really bothers me that it’s still turning categorical data into some form of a quantitative data and trying to average things. The only time I would consider using this is on a single learning target, and really it’s not averaging, oh, hey, I got a four on my proficiency scale and a three on my proficiency scale, so that means that, that a student is actually halfway between the three and the four. It’s just really averaging the number of times that they got a certain proficiency scale level. So it’s not quite doing what a lot of people might think it’s doing. And that bothers me a little bit.

Bosley: But I do think even using it mathematically like you were saying, I do think there is some instances where that actually might be appropriate. One in particular is if you are doing a course that might be like a last course before some sort of qualifying test. For instance, we work with engineers that have to, you know, take this qualifying test and if I’m the last class before that, because my goal is for them to pass that engineering test, then it might make sense to really put that heavy weight on those later assessments because I need them to kind of have that pressure because of the pressure of that qualifying test.

Sharona: Right. And I can certainly see that I just, I spent too much time in the bowels of the learning management system, getting the mathematics to work correctly, and they don’t work the way you think they work with the decaying average. And so getting the automation on the gradebook set up properly is actually really difficult in the learning management system. So I just have a lot of cautions. And I think there’s other ways you could do that, where you could literally just be like, Hey, you have to get it twice, but one of those two times has to be on the final, or it has to be like, there’s other ways to do it without turning categorical data into quantitative. I just, I’m very uncomfortable with the decaying average.

Bosley: Yeah, and like I said, I agree with you on that. But I still also have seen people that want to use it and have justification for it. And you know what, if it works for you and your system and the purpose of your course, and you can justify using it, use it. I’m never going to do it again. But that doesn’t mean, and that’s again one of the real benefits of alternative grading is it does let you really customize your grading into a way that fits you as an instructor, that fits the purpose of your course, that fits the needs of your students. If you just take time to examine all those things, you can really set this grading structure up to support all of those things a lot more so than any kind of traditional grading system would.

Sharona: I think what this is speaking to is, in a way, the decaying average is more about grade tracking than sort of the holistic way we’ve been describing things, right? And so this idea that a late grade hurts you more, that I can totally get behind, it’s the mathematics of the grade tracking that I think bothers me the most, and that it kind of lends to one of the most important things that we need to think about with all of these methods is tracking this, because if the communication with the student breaks down and if the student can’t understand how the grade is being tracked and how it’s being checked off for sufficient evidence, you’re going to have some major issues.

Bosley: Yeah. And one of our biggest advices that we give in all of our trainings, you know we just did finished up that intensive 35 hours that Kate Owens was also one of our facilitators, and I can’t remember how many times she said her biggest advice is keep it simple, make it more simple, and then when you think you’ve got it simple enough, go back and simplify it again.

And this is where it can be, you can add a level of unnecessary complexity and that’s going to have effect on your student buy-in. And buy-in, because this is a new system, buy-in is a important thing that we’re going to do a whole episode. Probably more than one. Not probably, we will end up doing more than one. Just looking at what groups and how you build buy-in.

Sharona: Well, and not just buy-in, but you know, philosophically, the purpose of doing this is so that the grades actually enhance student success and support student learning. And if the student can’t understand the grading system, then it’s not going to do that.

And we talk about simplification. I’ve been spending the last five years trying to simplify our system in statistics, I feel like it’s the simplest one I can think of, and the minute I start explaining it to anyone, I realize how complex it still is. And I’m just like, okay, is there anything I can do further?

And then I keep trying to make it even simpler and it’s just, it’s hard to simplify it as much as we need to.

Bosley: So let’s kind of get into, we’ve already talked a little bit about the advantages and disadvantages of the decaying averages and some situations that you might actually consider it.

So let’s go back and let’s look at the Guskey method and the N times, and let’s talk about some situations that one might fit better than the other, or one might benefit you more than the other.

Sharona: Well, I nominate you to talk about the pros and cons of the Guskey method.

Bosley: So especially looking and comparing it to the N times. One of the things that the Guskey method will, I think, lend itself to is when you’re in a situation where the longevity of the learning and the longevity of the students hanging onto that knowledge is important. So, a while back, we had an interview with Joe Zeccola, a nationally board certified English teacher here in Los Angeles.

And he does use the Guskey method, and part of that is because he’s doing a writing course, a writing course that those skills do not end at his class. In fact, those skills really never end. I mean, at no point in time do you really ever stop being a writer. People have to write and communicate in almost any facet of life no matter what you are doing.

So that longevity, whereas compared to, say, our statistics class where we do use it N time, we’re not as concerned with, okay, six months later, can the student really perform a hypothesis test again? You know, do they have enough base knowledge that if they had to do it, can they go back and kind of reteach themselves?

Sharona: Well, and also like in my linear algebra, yeah, I only check twice. But the reality is the early skills in the course are used throughout the later part of the course. So I mean, I could check 17 times and once they get it by the second time, they’ll probably get it the next 15 times because they have to use it over and over and over again to get the later stuff.

So. I just don’t need to keep checking. Because they’re going to keep most of those skills, they just have to.

Bosley: And another kind of situation for when the Guskey might be a more appropriate method is if they’re having to utilize those same skills over and over in very different situations.

So again, going back to that writing, being able to write an effective introduction. That might look very different if you’re doing an argumentative paper compared to an analysis compared to some sort of fictional writing or all of those. It’s still, the skill is still very important in all of them, but they’re also very different.

So needing to have it not just in the analytical paper, but also in the argumentative paper might be a time where having the Guskey method and not just the N times would be a lot more beneficial. I don’t like saying the word beneficial, but a lot more appropriate fit.

Sharona: So what do you think some of the cons are of the Guskey method?

Bosley: Definitely one of the biggest cons is it is harder to explain. I think, at least in my opinion. It’s harder to explain to the students. Because, if I’m going to be assessed six times throughout the course of the semester and I’ve gotten a nearly met, a met, and another nearly met, I might not be as clear about what my grade is compared to an N times where I can definitely look at it and go, okay, I’ve got three more chances coming up, but so far, I’ve only gotten it once. I know I need it one more time. That’s one of, I think, one of the biggest disadvantages. And that’s not just for the student.

Sharona: Yeah. It was occurring to me, tracking could become a problem. And I worry about implicit bias also. You know, if we just sort of do this holistic, oh, it’s just sort of what I feel about the student at the end, which is not what we mean by the Guskey method, but it could certainly be interpreted to be that, then you’re bringing in all kinds of equity questions into play.

Bosley: And I think that is actually something that might be pointed out here. Any of those, and with alternative grading, it can be done and set up in a negative way that actually does more harm than good, or that actually increases some of those negative things that we talked about within the traditional grading, the issues with traditional grading. Like you can set this up and actually not just replicate those, those inaccuracies and those issues, but you can actually amplify some of them, right?

Sharona: And that’s why if you’re new to mastery grading or alternative grading, and you’re just jumping in with this episode, we definitely recommend you go back and look at, I believe it’s episode two, the problems with traditional grading.

Because if you don’t really know, like a lot of us got into this because we just sort of felt really icky about traditional grading. We’re like, oh, this is clearly bad, but we couldn’t articulate what was bad. When you know the details, it becomes easier to avoid the problems. And I definitely think implicit bias is one of those problems.

Bosley: Yeah. And if, if you go back to our episode with Robert Talbert, he tells the story about how he got into alternative grading, what took him away from traditional grading. And it was this, you know, very specific case with a student. Which I think a lot of us, that’s how a lot of us get here.

So we recognize maybe one of the issues with traditional grading. It doesn’t necessarily, you know, show us all of them. And like you said, if we’re not aware of a lot of these different issues, you could end up recreating them. So even those that are highly mo highly motivated and want to change a system.

If we don’t understand all of the negatives and not just the one or two that we personally saw, like what Robert was saying with his one student that just crushed him and forced him to completely reexamine grading you’re going to, you could miss something. You could end up recreating what you are trying to prevent.

So, and one of the other issues with the Guskey method, there’s more than just you the teacher and the students that might need to have those grade communications with. There’s other people, especially in the K-12 setting, you have admin, you have counselors, you have parents that also need to be able to look at and understand kind of where a student is. The Guskey method is more difficult to do that, I think.

Sharona: Well, and we’ve talked about it, it occurs to me that might be more difficult, particularly in an environment where you have interim grades, which I know you deal with. I don’t. Although some higher ed faculty do.

Bosley: Yeah. Absolute bane my existence. We will definitely have episodes about how you deal with those interim grades.

But yeah, that does make. Although I don’t think that’s just a Guskey issue. I think that’s really an issue with any of the methods.

Sharona: It’s definitely an issue with any of them, but I can at least, I mean, I can at least make an argument, say on N times, I can look at an administrator and say, well, look at the schedule of assessment.

We haven’t had any assessments yet in week five, so I’m basing my interim grade on this other thing, but you can see how this is disconnected. Whereas the Guskey method, it’s like, well this, there’s this preponderance of the evidence essentially. And I can see an admin looking at me and saying, well, what’s the preponderance of the evidence right now?

And I’m like, how do I push back on that? So I, I just, not being someone who lives in that environment, I can kind of see that. I would think the guskey method would be even harder, but it’s maybe because I don’t use it, I don’t have the familiarity with it.

Bosley: Yeah. But so those of you that are listening, especially in the K-12 world, because all of us have interim grades or those of you in the higher ed world that might be in an institution that requires those, look out for that episode. It will be coming down the pipeline later on. But we will be doing a whole episode on how to do those interim grades.

Sharona: Or if anyone’s figured out how to blow the system up and not have to do them at all. Like you were in a situation where you had to do interims and you got out from underneath the burden, that’d be great to hear about too. So, okay. So we’ve talked about pros and cons of Guskey. We’ve done pros and cons of decaying. What about N times?

Bosley: Another one of the big benefits, I think to the N times that is a direct con of the guskey method, a lot of the LMS systems now have, if they have any kind of mastery built into their program, N times is one of the ways that it can track it for you.

So the tracking of it becomes a lot easier to do. Although there is a lot of systems, a lot of tools out there, and we’ve talked about that in that episode five. There is a lot of tools that can help with either of these methods, but utilizing those LMS systems, whether it’sCanvas or Schoology or Moodle, the N Times is one of those that is usually built in.

Sharona: I also feel like the N times is the one easiest to hack in a traditional points-based grading system. Because you can literally use ones and zeros. One is you got it. Zero you didn’t. And you know most of the averages and things like you can do simple counts. Do you have a two? Because you have a one plus a one. Well you got it twice. Do you have a three? Because you have a one plus a one plus a one. You got it three times. So I do think it’s also the most hackable.

Bosley: And I think the biggest benefit though to the N times, it is probably the easiest to explain and for students to look at and know exactly where they stand.

Sharona: Yeah. And also, I mean, there’s, but there’s pros and cons for how many times. Because N can be any integer. So when I think about N times and I think about the cons, to me it’s more about the specific decision. The main con of N times is it makes it very discreet, right?

So it runs the risk of laundry listing things. Right. But I also kind of, I know a lot of faculty who have an issue with the one-time mastery because, well, two reasons. One is it can make the stakes a lot higher because you have got to get it or you don’t. And so there may be a more incentive to cheat. And the other con of the one-time is a lot of false positives.

So they managed to get it on the one problem they saw because they happened to get it right, but it was almost a fluke. So a lot of people don’t like one time. I don’t like one time for most of my stuff, although I use it in history of math. I like two times for any of my skills-based stuff and because they have to show it twice, it’s reasonable more than once. Anything more than twice gets unwieldy usually calendar wise,

Bosley: And that’s what I was going to bring up with one of my other cons is you do need to predetermine these amounts, and this is where especially a new practitioner can end up being too ambitious and really drowning themselves in either assessments or grading.

So depending on how you’re assessing. Now, we talked about in our last episode, there is a lot of ways to assess learning targets that isn’t the traditional quiz or test, and if someone is going to use a lot of different ways, then having a higher N makes sense. But if you’re using a lot of those traditional quizzes or tests, this is where you really can end up drowning yourself in grading if you’re not cautious.

Sharona: And I think it also, N times is the one that has the biggest interplay with decision four on the rollup. So I think when we get to that, we can also talk about some of the ways that the N times plays in. Because you mentioned it might be on different assessments, so it could be, well, you have to get it once on homework and once on a quiz, and once on a portfolio or something. So it might be three times, but not three of any kind. So that could, that could affect things as well.

Bosley: Well, let’s, let’s go ahead and jump right into though that decision four. So the decision four is how do you take the collection of learning targets, you know, mastered or not?

And change that in, wrap that up into a final grade. Because we don’t have the percentages, you know, we’re not defining our A as a, you know, 90 to a hundred percent or a B as 80 to an 89.5 or 89.9. Since we’re not doing that, how are we going to then define our A, B, C, and down?

Sharona: Well, and I think that starts with, what does it mean to get an A in your course?

Bosley: Yeah. And, and when you’re making these decisions, that absolutely is where it starts. What is the purpose of the course? You know, what do you need the students to leave with? If a student is leaving with an A, how is that student similar or different than the student that’s leaving with a B?

You know, are you defining that as how much knowledge they’re taking? Are you defining that on how likely they are to be successful in the next course? If you’re in a sequence of courses. So that this, those kind of decisions are what’s going to really drive not just this last decision, but really the last two decisions of the grading architecture.

Sharona: Right. And so for me, this is an area where I may not have examined it quite as much philosophically, but I basically have defined an A as you got the vast majority of the content, which is somewhere in the neighborhood of having completed 90% of what I want you to complete both in the learning targets and or the tasks that I assign.

So for linear algebra, it’s you’ve gotten 26 out of the 28 learning outcomes to a completed stage. That’s an A and then an A- is 25, I go down from there. But in history of math, it was you got nine out of the 10 learning targets, but you also completed all four projects.

Bosley: Okay. And that kind of shows two different ways of doing this. So the first one you were talking about, where you just count up how many they get, that is one of the three methods we talked about in that episode five, and that is that simple count. We, however many learning targets you have, you define your A at some break off point. Doesn’t have to be at 90% it, but whatever you think is appropriate for you as an educator, your course and your students.

You’re just counting those, it doesn’t matter which ones they are. Doesn’t matter when they got them, doesn’t matter if they got them, you know, if you’re using a a four level proficiency scale, doesn’t matter if they got them at 3s or 4s or advanced or proficient, whatever you call it. It’s just how many you get. And that’s exactly what we do in that quantitative statistics course that we both teach in.

Sharona: Right. I basically use a form of the simple count in all my courses. The only variation is in history of math, they also have to complete a number of projects in addition to a number of learning targets.

Bosley: Yeah. So that’s one method. What is one of the other methods that we talked about on, back on that episode five?

Sharona: Right, so another one we talked about is the bucket method. Now there’s, there’s different kinds of buckets. The one I’m talking about right now is the learning target buckets. So in a lot of our engineering courses, the faculty have defined a sort of fundamental bucket. And in order to pass the course, you have to get all the learning targets in that fundamental bucket. And then they might have an extension bucket. And depending on how many of those you get in the extended bucket determines the B or the A.

So they sort of started from passing the course is a C. You’re not allowed to move on if you don’t get a C and that fundamental bucket is what you have to get to get a C. And then they built from there. But there’s a different type of bucket.

Bosley: Yeah. And another version of that, and I’m not going go into huge detail because we did that on episode five, but is the content itself. Is breaking it up into different areas of your content, you know, so the example I think I, I gave in that episode five is my Algebra two class, where I had one bucket that all of my learning targets around graphing and then another bucket that was around solving equations and another one around mathematical modeling. And to get the C they had to have so many from each bucket, and then every grade was defined from that. So many out of this bucket, so many out of the modeling bucket. So that’s another way of, of breaking them up. But they’re both the bucket method.

Sharona: Right. The pro of that method, for me, is you can assure that there’s some of each of those content areas in there. So if you need to make sure they hit a little bit of everything, they can’t completely ignore one section. Where that gets, forms a bit of a trap is when they far exceed your expectations in some of the buckets, but then fall short in another bucket. So we see that particularly in that fundamental versus extended. You have to think about the situation, which you might think cannot happen, and if you think it cannot happen, it will happen. But they miss one of those fundamentals, but they get everything in the extended.

So they have everything in the whole course except one. But the one they don’t have is in that fundamental, which says that they shouldn’t pass the course.

Bosley: Yeah, and it is same thing with, you know, the content breakdown, like what I was describing. You know, you, you’ve set it up and defined it in such a way that, you know, your C is this level, you know, maybe the student needs to get two out of the five that are in the graphing. But the student has gotten every single learning target in every single bucket except for that one, and they only get one. So should they get the D? Should they actually get something else? Because they far exceeded all the other ones, but it’s just that one graphing bucket where they fell short of the C definition. So does that mean they get a D? Whereas had they gotten that one, It would’ve been an A or a B. So that is kind of the, one of the cons of the bucket method is it can add a level of complexity. And we’ve talked about on this podcast several times, the issues with complexity, with trying to keep it as simple as possible, especially as a new practitioner.

Sharona: And so I would challenge anyone doing this to really challenge your assumptions. We tend to think in these absolutes, particularly in the STEM fields, I think. But it could be in any field where it’s like, well, they can’t possibly succeed without this thing. And then someone does everything except that thing. Every single thing. Except that one thing that you thought was just impossible to do without, and what do you do? And what would you have done in a traditional grading system? Because making this… the point is to, in some ways, reduce the penalties and give a more holistic look at student learning and communicate a more holistic look at student learning.

But some of those rules can be unexpectedly brutal when someone doesn’t fit in them. And so my number one out for that is always give yourself a clause in your syllabus that says, essentially I reserve the right to increase your grade beyond the, you know, if you do these things, you’re guaranteed to get what I told you in your syllabus, but I reserve the right to give you a better grade.

You can’t reserve the right to give them a worse one. That doesn’t work.

Bosley: Yeah. Yeah. And that’s, that’s one of our other, you know, any training that we, that we do our last bits of advice and that’s always one of them, is give yourself the out and the freedom to increase a student’s grade. Because your grading architecture somehow broke down for that student.

And that student, at the end of the day in your professional opinion deserves a grade other than what, a higher grade than what you’re grading architecture states it should.

Sharona: And I don’t remember who said it exactly, but one of the people that we have learned from one of the books I think said, Do not let the algorithm overtake your professional expertise. You are the professional in the room.

Bosley: That is guskey all the way. That is one of his big things throughout most of his books is never give up your professional judgment to any kind of mathematical algorithm. That we are the trained professionals, so use it.

Sharona: Yes. Now there’s one other wrap up that we haven’t addressed yet, which is actually utilizing the proficiency scale levels in various ways. Usually this is done when you have two or more levels on your proficiency scale that are considered satisfactory, although it could be you’re also wanting everything to be progressing, I don’t know. But do you want to talk about that one?

Bosley: Yeah. And, and I don’t, this one I would kind of disagree a little bit with you on. I think to really utilize this method, you do need at least a four level proficiency scale, right? You do need two levels of success.

Sharona: Usually you need at least two levels of success. I could see a situation where you didn’t, but…

Bosley: Yeah. So this is utilizing looking at the different amounts of Satisfactory versus Exceeds, so your grade is defined not just by how many learning targets they got, but how many learning targets they got at a satisfactory level compared to how many they got at an exceeds level.

So, for example, your C might be they had to have at least eight of the ten at a 3, or at a meets level. And the B is, they had to have those same eight, but at least three of them had to be actually at the exceeds level. And the A is they actually need nine out of the ten at a 3 or 4, but at least five of those had to be at at the exceeds.

Sharona: And I’ve seen a lot of people add in something on the below and, and I think that’s, personally, I think that’s problematic. I think when a student is struggling, adding that into the final grade and preventing them from getting a higher grade based on their success because they’re so far below something. I just, to me philosophically that goes against what I believe. I really just want to be looking at evidence of success. I don’t want to downgrade someone because well, they almost got there versus they’re nowhere close. Like…

Bosley: Yeah, you get that same issue I was talking about with the bucket method. Where, let’s give the example of kind of the one I just did, where you’ve got 10 learning targets. Their A was defined as having all ten at a 3 and at least five at a 4 and nothing 2 or below. Well, what happens with that one student that has nine of them and they’re all at that exceeds level, but that one learning target, that one thing they got a two on, right? Like, does that student then end up getting bumped all the way, you know, all the way down…

Sharona: To a C? Like they, it’s not even that they just get bumped down to a B or an A minus, but they jump several grade levels because of one thing.

Bosley: So that that issue kind of comes up both in this method and potentially in the bucket method, which is again, the easy shield to that is what you said, do that clause, give yourself that out. Like Guskey says, don’t give up your professionalism, your professional opinion, to some sort of algorithm.

Sharona: Even then though, if you even include this in your grading system, it can create a tremendous amount of stress with the student. It can do a lot of damage to your relationship because they can see this too.

They can see this, and they’re freaking out that they have this one thing that for whatever reason, they just can’t get there. And their quality of work is so good that if they’d been in a traditionally graded system, they’d be at a 94, 95% because that one thing would’ve been overshadowed by everything else they did.

So it’s just, check your assumptions. Check your assumptions. Check your assumptions. Yep. Students are going to do things differently than you expect. The other thing, assumption I’ve seen a lot of times is the student is not going to be able to get these harder things until they get these easier things.

Bosley: We see that a lot, especially in math and in some of the physical sciences like Physics or, or chemistry or engineering. Yeah. We see that a lot.

Sharona: And the assumption is, the reason I’m putting these harder ones in the A or B level is because there’s no way they can get them without the one, the learning outcomes, the learning targets that are in the C level. And that’s just not the case. Because for some students it just makes more sense in a more complicated environment, or they finally learn the easier stuff when they’re doing it. So another question we get a lot in, we’ll have a whole episode on calendaring, is how do I handle the fact that some students are still assessing the early stuff and I’m moving on in the class? And my argument is, move on. Keep teaching. Because something that you say on the later stuff might be the thing that that student needs to get the earlier stuff.

Bosley: Yeah. I actually give an example of this. This came up in one of our trainings. The last time I taught algebra two, I didn’t just had one, I had multiple students that could not grasp linear functions, like they just had an incredibly difficult time doing linear functions, the simplest function there is to graph, other than maybe a constant function. Until we got to the exponentials and for some reason the seeing exponentials for the first time and really not being fearful of it, and that once they got that, that made the linear make sense to them.

Like I didn’t reteach the linear, it just, it finally clicked. And again, the human brain is really weird that way. Where it can, like you said, if you don’t think it can happen, teach enough classes, you will find somewhere where it does.

Sharona: Exactly. And I mean, one of my, one of my sons. The way his brain works when it comes to mathematics, and he’ll probably kill me when he listens to this episode. Actually I don’t think he’s listening. He knows it’s going on, but I don’t think he listens. We actually have to sometimes go into much more advanced mathematical concepts because his brain will reject the simplified version without the more complicated version. So I literally, you know, he’s in sixth grade and I’m having to prove the quadratic formula, using completing the square, because his brain will not memorize the formula without it. And then once he, he didn’t remember the proof, but once I sort of reassured his subconscious that yes, this was mathematically legitimate. Then it would allow the recording of the formula into his brain. It was, it was a real trip. I gotta say.

Bosley: Well, but it’s, it’s the same thing that, you know, I know in the math world, we like to joke about when does someone really learn calculus?

Sharona: The first time you teach it.

Bosley: Exactly.

Sharona: I think that’s true in every discipline, though. I’m not, I don’t know. English, history professors, you want to weigh in? When’s the first time you learn your content? Is it when they teach it the first time? Because, Yeah, definitely. The first time I taught calculus was when I first started to understand it. Absolutely. So we’ve done decisions three and four. These tie into pillar four of our system, right? So pillar four states, reattempts without penalty.

Bosley: Or eventual mastery matters.

Sharona: Eventual mastery matters, eventually success.

So, that’s the goal with a lot of this stuff, right, is that students learn through mistakes. We all learn through mistakes.

Bosley: Yeah. That, that’s, we try something, we, you know, we fail at it. We figure out why. I mean, this is how you learn to walk. This is how you learn to ride a bike. This is, this is how you learn almost everything that we learn.

Sharona: This is how you learn to record a podcast.

Bosley: Yes. You guys have no idea how many mistakes we’ve made.

Sharona: So many,

Bosley: and still making them.

Sharona: Still making them. So I’m excited. With this episode, we really have completed the four decisions, at least the first pass through them. I’m pretty sure we’ll revisit them time and time again. We’ve now really tied into three of the four pillars of alternative grading. So what do we have coming up next, you know?

Bosley: One of the things that go with these two decisions is this concept of reassessments. You know, are you doing reassessments? Are you doing revisions? How do those interplay with these last two decisions? And I know one of the episodes we’ve got coming up is another interview, with Kate Owens, who we mentioned earlier, and really diving into some of the different ways and different ideas about those reassessments and those revisions and how you give students multiple attempts to show their proficiency or their mastery of learning targets.

Sharona: Exactly. And then the other, there’s two more I’m excited about Boz that we have coming up. One is we’re going start diving into feedback. Because the heart of this whole thing, even though we sort of left it for last of the pillars, but the heart of this whole thing is feedback loops. And if you’re going close a loop through this process, the feedback you give is absolutely critical, and we have got to learn how to do that better, in my opinion.

Bosley: And we’ll get more into that on the episode. It’s not just learning how to do it better, it’s learning how to make the students understand the value of it and actually utilize it. I mean, I’ve been an educator in LAUSD for, this is going on my 19th year. We just started our 19th year at the time of this recording. I’ve always given feedback and one of the things, Ms. Whistler, if, God, if you’re still, if you’re listening, she was my master teacher when I did my student teaching. And, you know, she taught me a lot about feedback, but none of it translated to the students taking that feedback, all that time and energy and effort spent, really ever translated into students making a lot of success out of that feedback until I changed to this kind of alternative grading system.

So it’s not just about how we give it, it’s about how we teach our students to utilize it.

Sharona: Exactly. And that I’m excited to start to dive into that. And then the last thing that I just want to mention before we sign off for today is we have that episode with Dr. David Clark coming up. He is a co-author of Grading for Growth, but one of the things that we’re planning to talk to him about is the concept of artificial scarcity.

And I think that’s at the heart of some of the challenges with traditional grading. Some of the challenges with grading at all. One of the things that I think is a very, for me emotionally challenging and intellectually challenging piece, is really looking at the concept of artificial scarcity as it relates to grades and our grading system and the reasons that we grade students. So really great stuff coming up.

Bosley: Yeah. And if anyone that caught the episode with our interview with Robert Talbert, that name David Clark might sound familiar. That is because multiple times Robert Talbert talked about how much of a genius Dr. David Clark is. So that, that’s a fun episode.

Sharona: So we will see you around the pod, maybe? Right Boz?

Bosley: Absolutely.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode’s page on our website. http://www.thegradingpod.com, or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the Contact us form on our website.

The Grading podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.