The Grading Podcast

97 – the 3 hurdles

===

Sharona: So when do you think we would do some of this and, and how would that play into the grading?

Boz: I love this question ’cause I do think all three methods have times that one might be more appropriate than the other.

Welcome to the grading podcast, where we’ll take a critical lens to the methods of assessing students’, learning from traditional grading to alternative methods of grading. We’ll look at how grades impact our classrooms and our students’ success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach and instructional designer. Whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Boz: Hello and welcome back to the Grading podcast. I’m Robert Bosley, one of your two co-host, and with me as always, Sharona Krinsky. How are you doing today, Sharona?

Sharona: I am doing quite well. I have no idea once again what state or time zone I’m in, but ’cause I had a week full of travel. But yeah, I was in Kingston, Ontario this week at Queens University in Canada. Yeah. For the competency-based assessment symposium. And I had a wonderful time and I just have to shout out the city of Kingston, which I don’t know how old the city is, the university is 250 years old, but for a small, relative to where you and I live, city, they had just phenomenal restaurants. Oh my god, I had so much good food this week. It was amazing.

Boz: Yeah.

Sharona: How are you doing?

Boz: So how did, how did that conference go or that symposium go?

Sharona: You know, it was amazing. It was about a day and a half. We had, I think a total of about 70 participants. And what was interesting is the first half day, ’cause it starts in the afternoon, we had a hybrid session with people in the room and people on a zoom because people have to travel either drive or travel by rail. So a lot of ’em couldn’t make it that first afternoon so that it’s hybrid. And all of my hybrid teaching skills came back. So they were having some technical issues with sound. I’m like, Hey, let me throw my laptop on there and stick my AirPod in my ear because I know that that’s how I make sure I can be heard when I’m teaching hybrid wise ’cause I had a lapel mic for in the room, but they couldn’t get the second mic for the zoom to work.

So that was kind of a trip to have my hybrid teaching skills come out, but it was amazing what a group of thoughtful educators yeah, yeah. So they’re doing a ton of good work at Smith Engineering College at Queens University. Also, we had people from the University of Saskatchewan. We had people from York. We had at least one person from Montreal. So I was able to refer him to our CEGEP episode.

Boz: Oh, nice.

Sharona: So yeah, it was, it was pretty amazing. And I was talking a lot, which, you know, it was such a, no, such a trial for me, but yeah, no, it was, it was really, it was incredible. The engineering deans of the colleges that came, they’ve been they have a whole initiative at Smith called re-imagining engineering education. So they’re doing a whole top to bottom look at their program there, re-imagining a ton of things. So I had a great time.

Boz: Yeah. I, I wasn’t able to go to this one with you, but not only did I help you plan it, I was also a part of a lot of the meetings that we did ahead of time. And like you said, a really thoughtful group, a really well organized, or at least it seemed from my end on all the pre stuff, a really well organized symposium. Great group of people. Really was bummed when A, I couldn’t go and B, the original plan was I was going to help with that hybrid being a zoom proctor, kind of like what we do with our conference. And unfortunately ’cause of my work schedule that I didn’t even end up getting to do that. So I was definitely bummed. I, I’m thrilled though to hear that the symposium went so well and it sounded like the organizers and stuff, just that, and the community in general treated you really well when you were there, so that’s great.

Sharona: They really did. Lots of, lots of interesting conversations, lots of active listening. You know how, one of the things I like about. The way that I’ve been doing a lot of my talks that we’ve been doing, a lot of our talks recently is we provide a QR code and a link to the slides at the very beginning of the talk. And so people feel less need to be writing and taking notes. So they are actively listening and it’s like I see the steam and smoke coming off of people’s heads ’cause they’re so locked into thinking about how what we’re saying is relating to themselves. So I wanna just huge shout out to Alex and Brian to the organizers and Sean Maw, one of the other presenters for just an incredible collaborative couple of days and, and work process. This was just so much fun.

Boz: Yeah. So if, you’re in that area, if you work at any of those colleges and you haven’t been to this thing yet I think this was what their third, third, fourth.

Sharona: This is their third. They keep saying annual, but I’m not convinced that it’s annual, it might be every couple years.

Boz: So if you’re in that area in one of those colleges and you haven’t been to it before, definitely look, you know, keep, keep your eyes out for when it happens again. It is definitely worth the, the time and effort to be there and to be part of that symposium.

Sharona: Right. And then we will definitely provide a link to the symposium website itself so that you can kind of keep abreast of if that’s coming in the future.

Boz: I know anytime that we work with a big group that is kind of a self contained group like that, there’s usually some issues that come up. I mean we all have those, all of these issues, but there’s usually one or two that for whatever reason is really focused on there. Was there anything, or was there any kind of surprises that came up during this one?

Sharona: Yeah, there was something really interesting that I hadn’t, I thought a little bit about in the past, but it hadn’t come up too much, which is, what do you do when you have a class where the demonstration of evidence on, let’s say a learning outcome, there’s different levels based on difficulty of the problems that you’re being presented with. Okay. And I hadn’t really thought too much about difficulty itself. In the past when we’ve done difficulty, like with engineering, usually we’ve ended up making separate learning outcomes. But this was different than that.

Boz: Yeah. So, when we worked with the engineering we were looking at that difficulty going from, you know, two dimensional to three dimensional or from one, you know, straight force to angular force. Like, so when you, when you’re talking about the difficulty here, what, what that really affects is the grading architecture. How do we do that last step? How do we go from a series of learning targets that have been proficient or not proficient on by the students, and wrap that up into a final grade.

And we’ve kind of talked about two general different methods before, but I think there might be a third one that we need to kind of look at and talk about. But those first two are, and we’ve done this running a hurdles race metaphor where we talk about the higher grade, the A versus the B, the differences the student was able to jump over more hurdles. So it’s just the amount of content, which is really what you and I do in the, in our stats class.

Sharona: Mm-hmm.

Bosley: It, it’s not about, you know, how high they jumped, it’s about how many they jumped over. And then that second one that we’ve kind of talked about in previously is instead of more hurdles, they’re higher hurdles. So it being more difficult, you know, being able to do things at a higher level of proficiency.

Sharona: Well, and let’s clarify. In the past, for me, at least in my head, by more difficult, it was, it was that they did it, the student’s performance was higher than I expected. It wasn’t that they were doing it on a different problem or a different task. It’s that the quality with which they did the task I gave them their performance was at a higher level.

Boz: Yeah. And, and for in fact if we look at, like I said, ours is the more hurdles, you know, we’ve talked to our, our good friend Joe several times and his grading architecture, it looks at the good versus the great. It looks at how, not just how many, but how high it’s right.

Sharona: But I wanna specify higher being on the same assessments.

Boz: Yeah. So it, it’s the students getting three or whatever your equivalent to proficiency is versus a a four on a four point rubric or whatever that equivalent is. Which is why Joe does use a four point rubric and oftentimes you and I don’t really use a four point rubric, even though it might have four points because of the coordination element of the course, but we really use it like a three point.

Sharona: Right. And I’ll have to say, I’m looking at one of the books that I recently got about standards based grading, and this is more K 12 oriented. Most of the proficiencies I scales I see in standards-based grading books in the K 12 level. They do have the four levels, but they have like proficient and advanced or they have meeting and exceeding. Although this particular book I’m looking at doesn’t like exceeding because that turns into at the next grade level. ’cause especially in elementary school, it’s like at grade level versus beyond grade level. So again, it’s exceeding some level of proficiency, but not that the work that they’re working on is more difficult.

And that’s the difference of what I was seeing with the engineering folks at this symposium is they actually have looked at, one group has looked at their problems and said, these problems are basic recall problems. And we might have assessments specifically on basic recall, sort of that bottom of the Bloom’s taxonomy.

Yeah.

And then we have these next level that are what they would call basic multi-step problems. So it’s not straight recall. It’s got some apply or analyze to them and they’re characteristic of the field, but they’re what you would expect any engineer to be able to do. Then you go up a level from that where you’re starting to do more writing or design type problems. So there’s still multiple step problems, but, and they’re still not quite novel yet.

Boz: Yeah.

Sharona: But they are definitely harder because you’re expected to do more. You’re now creating written narratives around them, or you’re evaluating someone else’s work on this thing. And then they go all the way up to these very difficult, often multi concept problems. So might touch more than one learning outcome because they’ve got a lot of basic knowledge that you need. And they’re tough or tricky.

Boz: So would their proficiency scale scoring or their rubric that they would I as a student would get back? Would it be based on the type of questions I was able to complete correctly?

Sharona: So I don’t have that information. And what they specifically are doing with these, what I have is their description, and then it looks like what they’ve done is they’ve sort of subdivided their learning outcomes into these different types. So they might have a learning outcome that is a basic defi design definition learning outcome. And they’ll say, this is a type A outcome. And on that type a outcome, you can still get mastery, developing mastery, competence, developing competence. And mastery, developing mastery and competence are all levels of success.

Boz: Okay.

Sharona: Oh, wait.

Boz: So hold on, hold on. So they have a four point rubric. In three levels of the rubric.

Sharona: They have a six point rubric.

Boz: Oh, they have a six point. Okay. So, that makes more sense.

Sharona: This particular example I’m looking has a six level rubric. Mastery, developing mastery, competence, developing competence, not yet competent and no evidence. Those, okay, so, so five levels plus the no evidence. And what they do is they say, okay, this learning outcome, this is a type A, this learning outcome itself falls into this bucket. And depending on how you do on the proficiency scale, they use the higher hurdles also there. Our traditional definition of higher hurdles, but then they have two other learning outcomes that they’ve said, well now we want you to just demonstrate this first one, but in a design context. So the learning outcome itself is a more difficult learning outcome, but it still has multiple levels of success. And then they have another learning outcome that is utilizing an engineering design mindset that’s an even higher difficulty learning outcome, but still with multiple levels of success on a proficiency scale.

Boz: So it sounds like, again, we’ve talked about this quite a bit, but it sounds like we really might have three different ways of thinking about this now. And of course you can combine these in any way you see fit. But we’ve got maybe three general ways of wrapping up a grade.

Sharona: I would agree. So I think another way to say that is you have the quantity of learning outcomes completed.

Boz: Let’s go into some details. So when we say quantity, so we’ve got method one, the quantity, i.e., the number of hurdles in our old metaphor, right? So what do we mean by that? Like, I know we’ve talked about it some, but.

Sharona: Yeah, for me, let’s say we have 30 learning outcomes in a course. An A means sufficient level of proficiency in say, 27 or more of the 30. And then a B is 24 to 26, and a C is whatever. So, so basically, or it could be 20, whatever the break point is.

Boz: Yeah, the number is, irrelevant, right? But it’s based on how many of our learning targets the student has gotten at least proficient in.

Sharona: And that quantity one is also one of the base ways we distinguish standards-based grading from traditional grading. So one of the ways I like to say is that in the traditional grading system, students got 80% of the points, which meant that they could have gotten 80% of the way there on everything in the course. Versus in standards based grading, the way I do it, they’ve gotten all of 80% of the material. So it’s a flip flop. 80% of a hundred or a hundred percent of 80.

Boz: And, and you know, really the big difference between traditional and any kind of alternative that uses this, I can look at and say, yeah, the student hasn’t shown proficiency in A, B, C, and D. Whereas in traditional, I can look at it and go, yeah, your test scores are low, or, yeah, you need to do more homework. But can’t get to that specificity of going, yeah, you’re not able to show me that you understand what a p value is or how to use a p value and analyzing statistical significance. It’s exactly. All right. So that, that’s one. Mm-hmm. So that’s our, that’s our number of hurdles.

Mm-hmm. Okay.

Sharona: So the second one is the quality of the student work on the same assessment or way that you gathered evidence of learning.

Boz: Yeah, so it’s, you and I have done the same assessment. We’ve had the same writing prompts, the same questions, analyzing the same thing. I do it at a proficient level, and you are doing it well above what the teacher or instructor has defined as their proficient. So you are doing it at a more insightful or graceful way or a clearer way, whatever it is that’s made it go from good to great.

Sharona: And so typically something like this would require that the way we’re specifically thinking about this group of things is there’s not just one right answer that’s all that you care about from a student. Because if a student arrives at a right answer, both students who arrive at a right answer and get the credit, this has to be something where you are looking at the way in which they’re getting to the completed thing. So what’s coming to mind in math, intro to proofs course, there’s getting a correct proof and then there’s getting an elegant correct proof.

Boz: Or maybe instead of defining it on elegant, defining it on efficient or shorter. Yeah. You’ve got to the end, but you did it in a way that maybe had some novel ideas to it and it was a more efficient proof.

Sharona: Or when I’m thinking about Joe’s, with writing, you made a rhetorical argument versus you made one that was more persuasive or more a clearly written or uses more allegory or whatever the criteria might be.

Boz: Yeah, now in our metaphor, our hurdle metaphor, that is what we would traditionally say, the higher hurdles. I think that might be a little bit of a misuse of our metaphor.

Sharona: But I don’t know, there’s a whole community out there using it, so I’m not sure if we’re gonna be able to change it, but we’re gonna try.

Boz: Yeah, we try. I think that is, instead of that being the higher hurdle, that is jumping over the hurdle more gracefully.

Sharona: Is it that, or is it just clearing the hurdle at a higher height even though the hurdle stays the same?

Boz: Okay. So.

Sharona: That’s another way to think of it.

Boz: I, I, I don’t know you ever, you ever run track?

Sharona: I have not.

Boz: When you run hurdles, you don’t wanna clear it with a lot of height.

Sharona: Okay?

Boz: You’re wanting to barely clear that thing. ’cause the closer you get to that, the faster you can actually do. You go.

Sharona: But we don’t care about faster in in.

Boz: Well, no but if we’re making that,

Sharona: it’s grading, so it’s breaking down a little bit, right?

Boz: If we’re making a metaphor about hurdles, let’s at least make it, because I know some of our old track.

Sharona: So more smoothly let’s more, more, more efficiently. More gracefully. I kind of like graceful. Me do. You’re just, look, an observer watching would be, wow, that person is a more capable hurdler than another one for whatever reason. Mm-hmm. Someone who’s not experienced. So if that’s what we’re gonna do, then what does it mean to do the higher hurdles?

Boz: Well, that’s the new one that you were talking about. That’s the, we are actually doing harder things. It’s not just that you and I are doing the same task and you’ve done it, you know, more efficiently, more gracefully. You’ve done it more elegantly with the language or whatever the criteria is. It’s not i’m good, you’re great. It is. Or my work, not me. My work is good and your work is great. It’s the task we actually did different. That I’m doing it at one level and you are doing it at a a completely different level of question.

Sharona: So thinking about this, I mean, I’ve seen this done now that I’m sort of thinking back against it. I’ve seen this done by people doing the bucket method on the wrap up, where they take those more advanced problems, but they directly tie them to learning outcomes, which this group did too. They’ll write some learning outcomes that are the same basic content, but they’re asking for a higher levels like higher DOK, or a higher Bloom’s taxonomy level of activity on the part of the student, and they’ll bucket those. Then they’ll say, okay, in order to get a higher grade, you have to have more of these things in this advanced bucket. But this was done a little differently I think so, yeah.

Boz: And, and we’ve definitely seen that with the engineers we worked with at Cal State, we’ve seen it in some others where they have, whether they call it the essential and the advanced, whatever they call it. But those, those learning targets are actually separated. What it sounds like was going on here, what you were seeing that was kind of new. It’s not that the learning targets were separated, it’s the assessment type questions were separated. And if I could do it at this level versus I can do the same learning target at a different task at a higher application or modeling or a more difficult way of doing it. Mm-hmm. So you’re right, they’re both accomplishing the same thing, which is interesting ’cause these were both engineers right now that I think about it, they’re both accomplishing the same thing. They’re just doing it in slightly different ways.

Where the group we worked with at Cal State actually had different learning targets defined, and the wrap up was based on how many , of each of these two buckets they had, versus what you were seeing at the symposium where it’s not the learning targets, it’s how they were assessing the learning targets were different.

Sharona: So one of the options that we’re starting to see is there’s the different learning outcomes that you could have, but you also could give students options on the assessments itself. Like you could have a assessment that says. This assessment for this learning outcome is at this level of challenge, this level of problem, versus this like a type B they call it, versus a type C challenge. And the higher grades, you have to have more of those type Cs successful.

Boz: Okay. So that going back to our hurdle metaphor, that’s really the higher hurdle. And I love the fact that it’s the student’s choice of whether I want to jump over the two foot hurdle, the three foot hurdle, or the five foot hurdle. Like I, I, I love that. And I think we’ve talked about something similar to that before with one of our other guests about, you know, allowing students to kind of choose which question on a learning target to answer. But I don’t think we got into this level of, okay, you wanna get a three or whatever your proficient level is on your rubric, whatever rubric you’re using. Okay. You’ve gotta answer this many difficult level two questions, but if you wanna get to that next level, you gotta answer this many difficult level three questions.

Sharona: So the only thing that I’m concerned about with that, because yes, my mathematician/ systems engineer brain loves this and the student autonomy and stuff, however, I have two concerns. One is we’re already struggling to communicate the complexity of our grading systems. And so adding the cognitive requirement of a student to decide what level to attempt worries me a little bit.

Boz: Well, I mean, I get that. I completely understand and that is a valid concern, but I don’t think it’s a reason not to do it. It is just, if you’re gonna do it, you need to understand that that’s more explanation, that’s more time of making sure the students understand to get that buy-in and that trust from the students. So it might take more work especially at the front end, but I don’t think that’s a reason not to do it just understand and be prepared. That might mean you need to spend more time on explaining and getting buy-in from your students.

Sharona: So, and then I have a second concern and I have a possible solution though. My second concern with making it strictly student choice is I worry about which students will have the confidence to make the choice to go for the higher one. I think you can end up with some self-selected bias in there. Because we already know that students do this. So instead, my recommendation might be to require that they take a certain number, that they try a certain number at these different levels. Maybe they can choose which ones to try but then make it clear it, we’re encouraging them to try and to legitimately try. There’s not a penalty. You might not achieve what you’re hoping to get, but I really worry about students self opting out.

Now that may be contextual because at the place that I was at it is somewhat selective. They do have very high performing, very grade motivated students, whereas in the context that you and I teach at, we tend to see students who are more motivated to get through than to worry about what grade they’re getting. So there’s a little bit of a motivational challenge there.

Boz: Oh, yeah. And I definitely think there’s a lot of contextual issues that would go into that, that might make it a bigger or smaller issue. I, I do, and don’t get mad at me when I when I say this, but I do think it’s funny ’cause when we hadn’t talked about this issue before, so while you were saying it, I was thinking about how I, although I completely agree that that self-selection might have some equity issues, how I would deal with it. And you were like, okay, I’m gonna force people to do so many, and if they get it wrong, fine, no penalty. It’s not gonna hurt them. Which, and I was thinking about it of Okay then that’s my role as a teacher. To push those students that I noticed on my assessments, were doing really well at one level and weren’t trying to go to the next level. That becomes part of my feedback loop. That becomes part of my discussions with those students of, Hey, you’ve got this, you’re doing this, let’s see what we can do at the next level. Let’s at least attempt it. You might surprise yourself.

Sharona: Well, and then I think that’s also a context issue and a quantity of time. I mean, you see your students so much more than I do.

Boz: That’s true.

Sharona: That’s so much longer, and I think you’re better at getting to know your students faster. Your students also show up at much higher levels than mine do. I mean, we have students that we lose in week six and we never see them again. So it becomes very.

Boz: And that is definitely, definitely contextual difference between my main world, which is K 12 and your main world, which is higher ed.

Sharona: Yeah. So. One of the questions I have though is when would you wanna do this? Because I really worry, and maybe this is, I’m sure this is because the types of courses I teach, I don’t teach a senior design capstone type course. I don’t teach a course where we’re really pushing students to the limit. I teach an elective third or fourth year course, and I teach a lot of first years. So I, in my context, I’m like, I just need to set a bar and they need to get across the bar. So when do you think we would do some of this and, and how would that play into the grading?

Boz: I, I love this question ’cause I do think all three methods have times that one might be more appropriate than the other. And again, this is very much part of the different flavors of alternative grading. This is definitely going to be a decision that you should be making that fits your personality, your teaching style, your setting, all of that. But I do think there might be some settings within the class that might have advantages for one method over the other.

But I don’t think it’s necessarily based on level of the course. I think it’s more appropriate based on the amount of information that is new versus the amount of information that we are now expecting to do really high levels of. So here’s, here’s my example. Like the difference between doing an Algebra one course where we’ve got a lot of new material. If you look at the material between eighth grade and algebra one, there’s a lot of new material. Any kind of class like that, which our statistics class also falls in. I think the quantity of hurdles, the quantity of targets mastered, has the advantage. And now this is just our, our opinion. Let’s be very clear. I, I do not know of any research that’s out there. This is mine and your opinions. So take that with a grain of salt. But I would say that’s when the number of hurdles really is probably the better option.

Sharona: So if that’s the case, and I think I, I pretty much agree with you. And that does go to all three of my classes. My first year classes are in theory new material. Pre-calculus does have quite a bit of new material in the form of functions and periodic functions and exponentials and stuff. And then my linear algebra. Is also, even though they have done systems of equations before, a lot of the work we do with vectors in spanning is new material. And then my history of math class, this is the first time most of them have seen anything about the history of mathematics. So it’s also

Boz: Well, but see, I, I would disagree with you a little bit on that one.

Sharona: Okay. Now that I said that.

Boz: If we look at the way you set up your grading architecture, I definitely think that.

Sharona: Okay, that’s a good point.

Boz: Because doing, doing something for the first time doesn’t make it new material. If I’m doing something for the first time, but it’s combining three or four tasks that, or topics that I’ve done before, that’s not new. That’s a new application. That’s that more sophisticated application. So that’s not really new material just because we’re being asked to do it in a new way.

Sharona: Well, and to be clear, the reason it’s not new material. Is because I don’t require they master any specific piece of content. Like they don’t have to know how to do a specific set of historical math problems they’ve never seen before or anything like that. So you’re right, it is not new material. I still, well, I don’t know how to call what I do in history of math, it’s specifications grading, so that’s a different animal.

Boz: Okay. So let’s, we’ve talked about the numbers. Let’s look at the, the gracefulness or the way you jump over the hurdle. So the good versus greats, when might that have an advantage over the other two?

Sharona: So, in my context, specifically, the intro to proofs course. So, and why is that? Because there are multiple ways to prove things. So there is a little bit of new stuff because this is the first time they’ve seen how to prove things. How to really prove things. But there are also more elegant, shorter, more efficient ways to do it.

And that is the skill that largely we desire to encourage.

Boz: See. And I, so I think you and I definitely have a different definition of new material because I, I don’t see any advanced proof class as new material because we as a student have been justifying our thinking for many years at that time.

Sharona: Okay.

Boz: We are doing it at a more formal defined and formal and formal language. That’s not new material. Again, we’re doing it in a more advanced way.

Sharona: Okay, fair enough.

Boz: Which is exactly why I agree with you that this kind of the gracefulness of the jumping over the hurdles, the good from great has the advantage in that kind of class. And I, I think that could pretty much be said for any class that is heavy in writing, whether it is a straight up writing course or, or a literature course or something like the geometry proofs. It’s a heavy writing class.

Anytime there’s a lot of writing involved you’ve got that possibility of going, yeah, this, this work is compared to what I was expecting and compared to the level that I would say is proficient is just so much better. Whether it’s the gracefulness of the writing, whether it’s the use of metaphors, whether it’s the use of different strategies, whatever it is, it’s just at a higher level than what I’m expecting at that point from most students.

Sharona: So let me ask you this question. When we look at the mathematical practice standards or the science and engineering practices of NGSS or the anchor standards in either English or social studies, would you say that those practices, as you go further along, especially in high school, you can expect more from the students even if their disciplinary content is new? So if you’re in ninth grade versus 12th grade and you’re expected to ask questions, the questioning standard for the science and engineering practices, could you say you’ve gotta be able to do better questions?

Boz: Well, yeah, if I’m comparing the progression through K 12 or even just through high school, again, I might be asked to do it in a different way, but if I’m still being assessed on my ability to ask, to design experiments or design questions or, in math, justifying my arguments and critiquing arguments of others you know, any of those things. It’s, it’s being done at a more sophisticated level. And I do think a lot of those, especially the SMPs, the standard mathematical practices, looking at that through this lens of being able to do it good versus great is absolutely appropriate. And we’ll see that quite often. And that’s again, part of the reason that I think most writing courses could use this as well, that good versus great. The here’s a good writer versus here’s a in the next Stephen King, you know?

Sharona: Right. And I wanna clarify for people, ’cause we are using good versus great as a shortcut. I personally would never use that language with a student.

Boz: Oh God no.

Sharona: So I just wanted to clarify ’cause people are hearing us say this and at the same time, I’m out there teaching never use adjectives like good and great.

Boz: So yeah. No, no, no.

Sharona: But anyways, so I think we’ve explored a few different times you might wanna do one over the other. I think in the sciences it may make more sense for our brains to understand what a more difficult problem looks like in the humanities and social studies. I don’t know what that would look like, so yeah,

Boz: I would’ve to think about it a little bit.

Sharona: Well refer that to other people who teach at those levels. Yeah, because I honestly, I could probably handle it at a K 12, but thinking about a university like a freshman comp versus a senior writing seminar, I don’t know, I couldn’t tell you. We’ll have to go ask Emily Pitts Donahoe or Ashleigh Fox or one of those folks to say, what does this look like over there? But it’s just been interesting. So I think what we’re gonna try to do, I don’t know if we’re gonna succeed, is we’re gonna go more hurdles as quantity. Higher hurdles might be difficulty and grace or elegance or efficiency might be. The distinguishing of what we were calling good to great. But whatever that looks like, multiple levels on a proficiency scale of a single thing.

Boz: All right. But we’ve talked about when the more hurdles might be a, a more, I don’t wanna say appropriate ’cause there’s a lot of other things, the better fit, a better fit for somebody might have a better fit. We’ve talked about when the more graceful hurly, when might the higher hurdle, the actually doing more difficult questions have a advantage over some of the others.

Sharona: So I definitely think many of the STEM fields come to mind. So physics, chemistry, biology, engineering, even math. The other ones that come to mind is things like health sciences, you know, where we’re talking about being able to do anything that requires the synthesis of a lot of discreet content knowledge. I guess might be a way of saying it.

Boz: And that’s what I was looking for. Not really the the subject matters or the types of courses, but what’s in that course. And I think you just nailed it. When you are taking a lot of discreet things in, you’re synthesizing it into one application.

Boz: Well, I don’t think it’s that we’re biasing it. I think there are applications, again, looking at the content that’s being taught, looking at, is there, like I said, or like what you said, if I’m taking a lot of different discreet skills and I’m synthesizing them together into one application mm-hmm. Will, there are, are there definitely somewhere, okay, I can do it at this pretty straightforward, can I do it at this more difficult one that’s not, it’s clear, can I go all the way up to designing my own system. So I don’t think I’m biasing it for STEM. I do think we have some more concrete examples, especially in some of the sciences like engineering or I bet I could come up with a bunch for physics, but we have more examples. I don’t think we’re biasing it. Okay. It’s just.

Sharona: Well and as I’m sitting here, I am actually thinking of my real life and some of the marketing tasks that I had to do over the years. So, for example, I launched, at one point, an end of lease program for Nissan Infiniti. That was a merging of data and writing, and it was an extremely complex project, like to the point where almost no one had the skills to do that project. And I did. So even in a non STEM field, this was a marketing project and you need to have consumer behavior skills, but you also had to have some skills of understanding how data flows into a production process and so there was a lot of stuff there.

Boz: Yeah.

Sharona: So that would be a case.

Boz: Yeah. So, you know, if I was doing a senior, and I don’t know a lot about this, so if I offend anyone, I apologize, but say I’m doing some sort of senior level marketing class. And I set up these three or four different challenges and yeah, the, it’s taking and making a marketing plan, like you said, at different levels, whether it’s just this straight up thing or can I combine all these different elements? So yeah, that would be a case where I think it might be the higher, again, I don’t wanna say appropriate, but higher hurdle might have an advantage over the other two.

Sharona: Right. It’s definitely a way that I would encourage people to consider adding to their grading architecture, let me put it that way. Whereas if you’re in one of these lower level math courses, I’m like, no, please don’t do this to the students. I would encourage you to not, but that’s again my opinion. So take it for what you paid for it.

Boz: And then I, something else I, I definitely wanna make clear, we’ve talked about these three as separate things. Can you combine them? Absolutely. Can you get to the same goal doing them different ways? Like, you know, we brought up the queen engineering that, or Smiths engineering that you had the symposium with versus the Cal State that we worked with. They both got to the same goal, right? Goal. One of them did it by defining the learning targets in different ways that that separated these essential, or these foundational skills and these advanced skills. And that’s how the grade was the markup that last step was done versus the queens that did it with the type of assessments. So same goal, same exact goal, same exact destination. Two different ways to get there. Which way is the better way? Well, which way works better for you? Which way works better for your style, your, your content? There’s a lot of other decisions that go into it.

Sharona: But I’m gonna make a plug though.

Boz: Okay.

Sharona: I’m gonna say which one is simpler for you to communicate to your students? Because we keep talking about, especially in STEM, we love our complexity and it’s too hard for the students. So if you are more able, you personally are more able to explain one more simply than the other, then that’s the one you should go with.

Boz: That’s a good point. That’s a very good point. That’s part of those, like I said, that personalization, all those choices. Yeah, that should be a big consideration as well as which one can you explain better, quicker, and more efficiently to your students to get that buy-in from your students? That’s a great point.

Sharona: So. Those of you that have heard more hurdles, higher hurdles, it’s more hurdles, higher hurdles, or more efficient hurdles. We’re going for three of ’em. And more efficient is what used to be higher, but now we’re not doing that because we think higher is more difficult. I don’t know if that’s gonna fly. We’ll see. Well I think we’re gonna have to, I think we’re gonna have to persuade Drew more than anything else. Alright, so I think we’ve beaten this one to death. And again, ultimately, where do these hurdles go though? There’s one last thing I wanted to say. This is your grade wrap up. What distinguishes an A from a B from a C, or if you don’t use A, B, C, use something else.

Boz: Well, the equivalent. What’s the difference between what’s equivalent

Sharona: between what, yeah. So what distinguishes these three things? Because that is probably going to be our episode title, just so you know.

Boz: Yeah. And that is, to put it back into some of the terminology we’ve used before, that’s the fourth decision in the grading architecture. That how do you go from the collection of learning targets that students have or have not reached proficiency to, you know, that final grade, whether it’s an A, B, C, or whatever it is that you and your institute uses.

Sharona: So I have two last things I wanna touch on in this episode. One is there are some other concerns that you might wanna think about in this distinguishing. Okay. In particular the one that comes to mind for me, but I think it’s different for you. Often if I’m ever asked to write a letter of recommendation, it’s several years after I’ve had the student. So for me, depending on again, what type, of course, if you’ve are asked a lot to write a letter of recommendation, for example, for graduate school or a scholarship, and you wanna know the quality of the work, that might be a reason to have these either more efficient or more challenging hurdles available. I dunno, what do you think of that one?

Boz: Well, that’s an interesting point. I, I do think this is probably another place where we will see a real difference between K 12 and higher ed because a, I usually am not asked to write recommendations for students that I have not had recent contact with. Like most of the students that have ever asked me a, I’m getting asked usually towards the end of their junior year or their senior year ’cause it’s recommendations for college or for scholarships and b it’s is usually students that I currently had just had, or I might have had them, you know, a couple years ago, but I’ve still had contact with them. You know, it’s not like I had them their freshman year and I’ve never seen them again and they come back to me then. So I don’t need my grade to be able to tell me that I have my personal experiences, I have my, you know, interaction with the students. So I could see if I haven’t had continued relationships and continued contact with the students, why that might, you know, the point you were making might come in handy. But for a K 12, for a K 12 educator that e even the students that I’m, you know, haven’t had since their freshman year, I’m still having contact with them. I’m still, you know, seeing them around. I’m still communicating with them. I’m still having some sort of you know, relationship with them. Otherwise, they wouldn’t be coming to me to ask for that recommendation.

And the reason they’re reaching out to me is because I was one of the few that actually impacted them. So it’s not even that they don’t have good grades in other classes, but they didn’t feel like they had the personal relationship. But if it’s been two or three years and I haven’t spoken to them, even if I had a good relationship with them during the class, I still have to go back and refresh myself. So having a grade book.

Boz: Yeah. And, and again, like I said, I, I think that’s an interesting and a valid point, just a point that might be different for a K 12 educator versus a higher ed. ‘Cause like I said, I’ve, I’ve never in my 20 years experienced someone coming to me and asking for a recommendation. You know, after having several years of no contact. Right. I’ve, even, even the students that I’ve had that have, I. Ask me for a job, you know, if they can use me for a job reference or something, they might’ve graduated several years ago, I still have some contact with them. Right. Like, you know, on social media or whatever we still have. So it, it’s, that is just a difference in our two worlds.

Sharona: Yep. So totally changing track. So I think we’ve, we’ve finished with this. Am I right? We’re good with this topic. We can completely.

Boz: Yeah. I, I, I would love to, I I would love to hear if, you know, our audience has opinions on, on these, if you’ve used one of them, or especially if you’ve used multiples, ones of these and found that you like one better than the other. Love to hear from you guys. Love to hear your, your thoughts and opinions on it.

Sharona: But send us an email at info@thegradingconference.com ’cause I have not had time to try to fix the contact us form.

Boz: I was just about to ask.

Sharona: Yep. Nope, haven’t done it. Speaking of the grading conference though. That’s what I wanted to finish up this episode with. ‘Cause we are just a couple weeks away from the conference. We’re about three weeks away, give or take, and our schedule came out. Yes. And okay, one thing y’all may not know, but we’ve got a core group of organizers and we have to assign ourselves to every single like, hour of the conference. All of us have to be somewhere working some session. And so we tend to pick the sessions that we like. And I’m afraid it might come to blows this year because we have some fantastic sessions. So I thought we might go back and forth a couple times saying what favorites we have.

Boz: Well, I wanna talk about it briefly ’cause I do wanna have a whole episode where we really get deep into this. We’ll probably invite one or two of the other organizers to join us as guests like we usually do. But yeah, this schedule, because neither you nor I were on this part of the organization, so we’re seeing this fairly new as well. But , are there a couple things that you saw that you’re already like, oh yeah, I want that one.

Sharona: Okay, so I’m gonna let you go first though, because you’re always accusing me of stealing stuff. So yes, I have several, but I’m gonna let you go first. So what’s your first one?

Boz: So there is on day one, and if you guys, when you look at the schedule, you’ll see these are kind of broken up into different tracks. There’s one on our research track.

Sharona: Which is Wednesday, June 11th.

Boz: Yeah, this is wednesday, June 11th. This is the 3:30 Eastern standard time slot. So this is the second to last track in the day one. But on the research track a student perceptions and they’ve got three different, is it three or four? No, three different, presenters coming in talking about some of the research they’ve done on different aspects of the student’s side of this. So I am really interested in that one. That’s one I definitely, I’m, I’m hoping to get assigned to so I can hear it as it’s, as we’re doing it. What about you?

Sharona: So my first one is also on day one, June 11th. Mine’s at the 2:00 PM Eastern Time slot. There is a track on neurodiversity in alternative grading, and I am really excited there are two presenters there, but that’s one of the questions we get asked a lot is how does this impact NEURODIVERSE students? So I’m really excited about that one.

Boz: Yeah.

Sharona: Do you have a second one? Because I have two more.

Boz: I do have another one. Where did it go? So this one is on day two at that 2:30 slot that yours was on. One of the other common things is we we’re gonna have a lot of panel tracks and there is a panel discussion on student student views of alternative grading. So are, are you seeing a theme with me? I’m really interested in seeing what students see this as and how they’re reacting. So yeah, that research one on day one and then this student view panel on day two. I’m, again, I, I’m hoping I can be the, the facilitator or, or

Sharona: Well, did you see who the moderator of that one is? That’s Katie Mattaini. So yeah, you might have to fight for that one. So my second one is also on day two at the 3:30 time zone. And it’s in the special topics writing assessment column that, that talk, but it’s specifically the second one there, which is called the hidden curriculum of Writing Assessment Literacy, how new Instructors Learn and what they Need to Assess Writing equitably.

So you’re focused on students. I love the faculty development side, so having a talk about what you need to teach new writing instructors. ’cause we have an equivalent in mathematics where new mathematics instructors need some skills in order to be able to do some more equitable assessments. So that, that one looks fascinating to me.

Boz: Yeah.

Sharona: So I have a last one. Do you have a third one or should I just go with my last one?

Boz: Well, I’m gonna go a different direction. I’ve got one that just, I absolutely does not interest in me at all. And that’s the third day. That’s the.

Sharona: Oh fine. Screw you.

Boz: Standards based. So, yes. I’m just kidding. That’s the one that you’re running. I’m just picking, yeah.

Sharona: The one that I’m like, I didn’t even have, we hadn’t talked about it. I’m like, I know where he’s going. Yes. I’m presenting and you should not be interested in it because we already did that on the pod. So like, don’t go to that one, if you’re coming to the conference because it’s on the pod, that’s for everyone who’s not a podcast listener. Okay. So you, since you blew your third one, I’m gonna go with my legit third one just because I love him so much.

So one of our keynotes is Jeff Anderson. He is been on the pot a couple of times, but he says the title of his keynote is Learning Grades Cause Learning Cancer. What might We Do instead? So he has created a metaphor that grades are cigarettes. And I’m like, oh my goodness. So he asked, what would you say about a medical industry that requires doctors to prescribe cigarettes to patients while claiming this practice is a necessary part of healthcare? And I’m like, oh, ow. Let’s go for the jugular. So I can’t wait. I cannot wait for that one.

Boz: Like I said, I, I’ve talked about some of the keynotes before. I’m really, really looking forward to that first one, just because unlike most of the other organizers, I am not familiar, you know, as familiar with, with him and his work.

Sharona: Yeah. That would be Asao Inoue.

Boz: Yeah. But yeah, you know, first we love Jeff. We, we absolutely love Jeff. He is so much fun.

Sharona: Jeff, we love you. You’re probably listening. We love you.

Boz: Yes. That was, that was, I think is going to be a really just killer to end on.

Sharona: I wonder how painful it’s gonna be for the smokers in our midst.

Boz: Oh, shut up.

Sharona: You deserve that.

Boz: All right. Well, we will get into more details with looking at the, schedule overall, but like I said, I wanna get one or two of the other organizers on us to do that, but if you haven’t already, go check it out. The schedule is up there. Know that this is tentative there, you know, always we are three weeks out, so there could be some changes, but we’ll give you an idea of how much we’ve got available in the conference this year. We definitely have more work, you know, more sessions than we’ve had in the past. Thank you to all the people that are volunteering to help make that happen. We could not do that without you.

Sharona: And one comment to notice all those times are in Eastern time, so please be careful looking at the schedule they’re set in Eastern. Make your adjustments, you Pacific Coast folks.

Boz: All right, well, unless you’ve got anything else to end on we’re gonna wrap this one up. We wanna thank everyone for listening, and we’ll see you next time.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode’s page on our website, http://www.thegradingpod.com. Or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the Contact us form on our website. The Grading podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.

Boz: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State System or by the Los Angeles Unified School District.