The Grading Podcast

121 – Collaborative Grading in STEM

===

Sharona: The second thing is that the most mentioned attribute was that using a community generated rubric for self-evaluation was greater motivation for focusing on learning for learning’s sake.

Boz: You said the most. How much, like what percentage of students responded with that?

Sharona: 83%.

Boz: 83% of students said that this was more motivating.

Sharona: To focus on learning.

Boz: Welcome to the Grading podcast, where we’ll take a critical lens to the methods of assessing students’, learning from traditional grading to alternative methods of grading. We’ll look at how grades impact our classrooms and our students’ success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist, and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach and instructional designer. Whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you.

Each week, you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Boz: Hello and welcome back to the Grading podcast. I’m Robert Bosley, one of your two co-hosts, and with me as always, Sharona Krinsky. How are you doing today, Sharona?

Sharona: Well, I’m pleased to say that I have successfully managed a feedback loop of my own. So the day that we’re recording this, the Dodgers just won the World Series last night and listeners to this podcast may remember that last year when the Dodgers won the World Series, I got locked out of my apartment.

So this year. I had a plan. I had a go bag, and the minute that that last out hit, I was on my Marriott app booking a hotel away from downtown LA because I was not gonna be able to get back. And in fact, Dodgers fans went crazy right in my area. So I had a good night’s sleep. Not at home. Not at home. So this is what we call learning from feedback.

I made the mistake of trying to come home last year and got stuck and had to go to a hotel four blocks from my apartment at one in the morning. And then this time it was 10 30 at night, I was already in my hotel room. Yeah. So a lot better sleep in a great mood. ’cause I was not dealing with Dodgers fans, of which I, sort of, kind of am a mild one in in as much as I care at all.

Boz: Yeah.

Sharona: But yeah, so literally my friend was like, they just won. And I was like, Marriott App. Book hotel.

Boz: Yeah. No, it was funny ’cause there was overlap between the Dodgers game and the OU football game. So my wife was cracking up ’cause I’m in the living room yelling at the OU game, the end of the OU game, and my son is in his room yelling at the Dodgers game.

Sharona: How’d OU do? I’m sorry I didn’t look it up.

Boz: It was, it was actually a good win. It was a ranked opponent. Our offense didn’t look great, but our defense did and we ended up pulling out a really high quality win. So it was a good night for sports in the Bosley house.

Sharona: Nice, nice. It was a good night for my household where I did not have to sleep at home. So I am very blessed to have the privilege that I could afford to book myself a hotel room so that I didn’t have to go home and I didn’t wanna crash on anybody’s couch. I’m too old for that. Just too old for that. So Boz, what are we gonna be talking about today?

Boz: You’ve been kind of hinting at, a little bit on the podcast, but more off, about some things that you are thinking about trying to do next semester with some of your classes. And we came across this research called Fostering Student Agency and Motivation: co-creation of Rubric and Self-Evaluation in an Ungraded course.

Looking at this, I thought we might look at this and kind of go through it, your process, and some of the things that you’re taking from this as you’re preparing to go to a form of alternative grading that you really haven’t tried yet.

Sharona: Yes, and I kind of wanna talk about the background of why I am doing this, if that’s okay. Of course, then we can get into the article. So I’m back in the classroom next semester, so I’ve been out of the classroom for three semesters doing full fledged course coordination. I’m going back into the classroom next semester and one of the courses I’m teaching is the alternatively graded statistics that we’ve talked a lot about on the podcast.

So I’m not gonna do a lot of tweaks on that class, although I am gonna try a couple of things that we’ve talked about. We had a piece of research that we talked about a few weeks ago, I should have pulled up the episode number, but about students opting in to attendance being mandatory or attendance being optional and what they did with that.

So I was grappling with how am I gonna get students, especially in the spring semester. So students in the spring semester in this course have very likely already failed the course once. So I know that I am struggling with students who are already struggling. And I was thinking and talking to you about maybe trying to go a little bit more extreme.

I’ve been standards based graded for a long time. I’ll probably maintain some of that, but this idea of collaborative grading where I am co-creating the process of what the grading looks like in the course with the students was kind of interesting to me. But I’ve never done it. I’m like, Ooh, this is really scary.

Boz: And something that you kind of learned from experience when you first redesigned the stats class, we made a lot of mistakes that first design. Yes. So instead of just, okay, I’m gonna try this. We’re actually looking at some of the resources that are out there. ‘Cause there is a lot more out there now and being a little bit more informed.

Sharona: Exactly. And so what was interesting is the first thing that happened is a Grading for Growth blog came out on this topic a couple weeks ago, titled Students as Partners in Learning Assessment. What was really cool for me about this is that it was co-creating and it was specifically in a STEM class. ‘Cause one of the other struggles I have is some of the people that I have resources that I could reach out to for this are doing this in a writing class. And that was one of the things I was worried about.

Boz: You’ve brought that up and you’ve questioned several of our guests over several episodes about, some of these more, I don’t wanna say extreme forms of alternative grading, but some of the ones that have more choice to the students and how you do that in a STEM class.

Sharona: And how you do that when you still have a very structured set of materials that you’re intending to attempt to get the students to learn?

Boz: Yeah, because we talk about this quite often, especially in a lot of our early how to series, one of the big things to consider when designing the architecture of your grading scheme is where the class sits. Is it in a sequence? If it is, where in the sequence? And this very much is in a sequence. This is leading up to calculus. That may be the final course or may be one of many in a line of calc one, calc two, calc threes.

So that’s different than the stats class that you designed, that is a standalone class that doesn’t necessarily have a prereq or is a prereq for something else. So this is a different situation. And even the history of math that you’ve talked about before, again, was not one in a sequence. So this is a little different.

Sharona: Technically linear algebra is also not necessarily in a sequence, although it is intended to prepare people for more advanced linear algebra. Very only really math majors and applied math majors ever take that. So the bulk of students taking linear algebra, it was not feeding another course either.

Boz: It might not be technically, but there’s a lot of engineering courses that work from that, right?

Sharona: They use the content, but it’s not like “you must have this one skill”, or at least in a mathematician’s mind, the content is still structured in such a way that it builds on other things or or builds to other things. But there’s not one single path. Pre-calculus is literally labeled pre-calculus.

Boz: Yeah.

Sharona: Which we can talk about whether or not it should be, and that’s a whole different thing. But to continue with what I was saying is this blog post came out and you and I talked about, we’re gonna try to get the author Sharon Stranford, on the pod. However, as we were doing our research for this episode, we discovered that she had actually published a more extensive description than was in the grading for growth post. And so that’s where I think we wanna go today, is we wanna explore this published research that she gave about her experience doing this and maybe tie it into what I’m thinking I might do.

Boz: Exactly. And my first kind of question, to you, which does also relate back to this research is why are you doing this? I mean, I know you’ve talked a lot about grading being one of the biggest things in a teacher student relationship, that sets up a lot of power dynamics.

Sharona: Yes, absolutely. And we’ve talked before on the podcast, that there are inherent power dynamics. And if you look at some of the most successful teaching models out there in the general world, everybody looks up to like say the coaches in sports. Coaches have a lot of power, but they’re also quite often beloved or they’re the people that people go to, that students go to to try to get better, to try to get help. And I wanna be that person, but it’s very difficult. I have age over my students. I have the power to assign a grade. I have my personal identities and my personal positionalities versus my students. I don’t want to be the power arbiter in the classroom. I want my students to have the motivation for themselves and understand that the power to get the grade they want or the power to learn is in their hands. ’cause it’s really not in mine. I can try to control them through grading. It doesn’t work.

Boz: Yeah. We’ve talked about how grades are often used as ways of motivation, that we’ve seen research after research that says that doesn’t work, or control. So what you’re trying to do is going into something that is much more a collaboration between you and the students. Yeah. When we look at the research on critical pedagogy, which is something that is brought up in this research by Sharon Stranford, we see that that relationship doesn’t need to be one way. In fact, it works better when it’s a two-way relationship, where both the students and the instructors are bringing their specific lived life experience and both parties are gaining from the other.

Sharona: Yeah, I mean, I learn a ton from my students. I probably learn as much or more from my students when I give them this kind of freedom than they might be learning from me. And it’s a very heady experience.

Boz: Right. So this is the kind of the why you’re doing it.

Sharona: Well and and the other, yeah, the why also is I truly believe that if my students are retaking this class, they are committed to a personal goal for themselves, that requires this class. Especially, almost all of them are STEM majors or declared STEM majors, and I really wanna make an impact. One of my impacts on the world is to increase the number of people that succeed at their personal goals. So whatever they did before didn’t work for them, and I wanna give them the freedom and the support to make this work for themselves.

Boz: Yeah. And that’s kind of another thing that has been brought up in a, especially a class like a pre-calc class.

There are a ton of majors that require calculus and therefore require pre-calculus. So the goals in the direction that your students are going in a class like this can be in a lot of different directions. Even though they’re all stem, they could be going into math, pure mathematics, they could be going into engineering, they could be going into architecture.

There’s a ton of things. So bringing the students in as partners as to what kind of learning, what direction they’re wanting to go with a course like this can be very powerful. There’s a lot of research out there about student motivation and bringing students in as a partner into the learning and the setup of the class, how that can increase the motivation of a student.

Sharona: Exactly. Because they feel ownership. They feel like they’re not subject to the whims of the instructor, but they see a clear path, for themselves, to their own goals.

Boz: All right, so we’ve kind of talked about some of the why. Now let’s get into some of the how and specifically how are you looking at research from Sharon Stranford. So we’re not well, and yeah, you’re not shooting in the dark here. Right.

Sharona: And that has been one of the biggest things that I have struggled with, looking at collaborative grading. Because I’ve never done anything like it, I had this sort of big mass of, “I have no idea how you might do this”.

And so there were several things that she mentions that really gave me some confidence that I can use her materials to inspire my own. So I want to give you some of the stuff that she talked about this specific course and setting. She is doing this in a immunology course. This is actually a course that it’s not the first in a sequence, but it is in a STEM major, and she already had been doing a bunch of things. She had high structure, partial flip design, just in time teaching practices, which require regular pre-class work. I know how to do all that. Like I’m confident for myself that I can recreate those things because that’s what we do in statistics.

Boz: Okay.

Sharona: So even though this is a collaborative grading course, which she calls UN grading. It’s a high structure course. So for me that was critical. ’cause a lot of times when I talk to collaborative graders, I don’t understand their structure in their course, and that is a big barrier for me.

Boz: Yeah, that’s been one of your big mental barriers is looking at, not the structure in the class, but more of the fact that this is in a sequence and there are goals that you need to accomplish for the students to be prepared for that next course.

Sharona: And one of the challenges with pre-calculus especially is that it overlaps with a lot of high school curriculum. Some of the material in pre-calculus, the skills the student needs go all the way back to algebra one.

Boz: Absolutely.

Sharona: And one of the biggest mistakes that people make in teaching pre-calculus at the university is feeling the need to go back and reteach algebra one skills. And so I don’t wanna do that. I don’t wanna be one of those instructors, but I also can’t ignore that my students often are struggling with those skills.

So I philosophically believe the reason they’re struggling with those skills is because those skills are not in a context for them. So I wanna engage in class with a lot of rich context material. I don’t wanna teach students something that’s just a mathematical equation. I wanna talk about a drone, or I wanna talk about a satellite, or I want to talk about.

In fact, a funny statement happened last night. My son, who’s actually very good at math, was teasing me about using advanced calculus to reposition an image on a video wall. And I was going back to him. I’m like, no, that’s a simple translation of a linear equation. This is algebra one. He’s like, no, mom, it’s advanced calculus.

Now, this kid was a math major for a while, so he’s had advanced calculus. He knew he was wrong. But that’s the point is that I actually used simultaneous systems of equations to solve moving an image on a video wall. Most people wouldn’t do it that way, they just would move it around, guess and check.

Boz: Guess and check.

Sharona: But that’s something that people can figure out. So having this example of a high structured course was really important to me. So okay. Feeling a little better here. I have some idea of the different elements. She also uses active learning. She does weekly in-class discussions and problem solving sessions with peers.

I’m looking at all of this going, yes, this is my course. This is the kind of thing I wanna be doing. Still don’t quite understand how it works. I’m scrolling through this and going, okay, so what did she do? And this is where I love it. There’s a section called the Addition of structured UN grading, which I’m gonna use the term collaborative grading.

Boz: Yeah. And not just feedback, but the feedback from students that wasn’t positive.

Exactly. I’m not gonna say it was negative, but it wasn’t positive so. You started this episode out with practicing what we preach and engaging in her own feedback loop.

Sharona: Exactly. There are a couple things about this that really stood out to me. So I’m gonna talk about a couple of things.

One thing to understand is, just like Professor Stanford, I am going to be in a class that gives an end of semester grade. Absolutely.

Boz: Yeah.

Sharona: And so one student critique that she recalled is after a semester of singing the praises of no grades, she then asked students to tell me what grade they deserved and they felt it was a bait and switch.

So there still are grades. And then there are also some other things from the students. They said grading, and especially self grading, is hard and uncomfortable.

Boz: And I think we’ve actually talked a little bit about this in some of our early episodes, that there is research out there that this self grading can be very different based on a lot of demographic information. Race, gender, that there are groups that tend to rate their selves very high or other groups that rank themselves very harshly. So, yeah, it can be really uncomfortable and if you don’t understand what you’re supposed to be grading yourself on, it can be really hard.

Sharona: Well, and that’s the problem that I’ve always had, fundamentally, with un grading, is who decides how you distinguish a multi-level grade.

If you’re giving A versus B versus C versus no pass or whatever it is, who decides what those are? And if it’s the professor, you’ve already sort of destroyed your power dynamic. Right?

Boz: Exactly. Yeah.

Sharona: But on the other hand, how do you hand that over to the students and not become subject to critiques like you’re lowering the rigor or whatever else.

And at the end of the day, I still want my students to actually know the math. I want them to have learned the math and be able to apply it because I think the math is actually important to their future. And so is the critical thinking skills and the problem solving, the perseverance that they have to develop.

So this is one of the things I have always been uncomfortable with, but I think she come up with a bit of an answer here.

Boz: Yeah. But before we start talking about some of what she did what was some of the other feedback that she got from students? Because there was one that, when I read it, I could see myself doing in a math or STEM class. That one was the feedback that the lack of scores could lead some towards perfectionism, not knowing how good or when good was good enough. That kind of endless loop of getting in and not making progress. ’cause you’re trying to make such little improvements that you end up spending way too much time and effort on one assignment or one project and not progressing forward ’cause you’re chasing that. Perfect.

Sharona: And similarly that students wish they knew what characteristics to look for in assigning a “Good Grade”, which again, this goes back to if you’re going to assign multi-level grades, they have to be defined. Yeah. Somehow, somewhere, sometime they can’t just be, oh, what do you feel? Squishy? Squishy. Like, that doesn’t work for me. I’m too much of a mathematician. I need definitions here.

Boz: All right. So she’s gotten this feedback.

Sharona: There’s one other one I wanted to mention.

Boz: Okay. Sorry.

Sharona: They said one-on-one meetings with the professor were valuable learning opportunities and also intimidating. And that says to me that I need to do, whatever I’m gonna do with one-on-one meetings, they have to start early and often. So one thing I’ve heard a lot of people do for their first meeting is an ask me anything.

Boz: Yep.

Sharona: So that might be where I go with that.

Boz: But yeah, that, that’s interesting. That kind of contradiction of, Yes, these were really valuable. They were also scary.

Sharona: Yes. And part of that is the challenge for me is if I rely too much on my own experience, I am not gonna understand what a lot of my students go through. ’cause I’ve always been possessed with a lot of self-confidence. And I grew up with a mom who was a math professor.

So math professors were just fundamentally not scary to me. Just fundamentally, professors in general. I’m like, oh, they’re people who go home at night and argue with their kids. And so I never had that sort of, I’m intimidated by professors experience, but I know that a lot of people do.

Boz: Oh yeah.

Sharona: So. It just, I have to be careful. I have to be careful not to put too much of my own personal experience onto these things.

Boz: But have you never had that experience where you run into a student in the wild and they’re like amazed that you have a normal life?

Sharona: Oh, for sure. But my problem is that at 18, 19 years old, which is what I think of when I think of my students, like I expected to run into professors in the wild. Because part of the problem is all of my professors, my first set of math professors, were all colleagues of my mom. So they’d been to dinner at our house. They had like, I had a big problem calling them Dr. So-and-so, because I was like, Hey, Jackson’s coming. No, it’s Dr. Henry.

Like, so I just understand intellectually that my experience was not normal and I just have to keep remembering that. So it was more weird to go off to college and suddenly not know the professor’s first names. Yeah. That was a lot stranger. So so what I think is fascinating is she took all these things and she said, okay, what are we gonna change?

What are we gonna do differently? And she debated it with her students from that first semester. She actually closed the feedback loop while still doing the first semester of trying this. So those students, at that point, they weren’t co-creating it for themselves. They were helping her figure out what to do the next semester with the future students. Which I thought was, I mean, talk about empowering.

Boz: Oh, absolutely.

Sharona: Empowering a student to make a difference for another student? Like wow.

Boz: And just brilliant too. ’cause I mean, so years ago, decade, more than a decade ago, I did a training called Capturing Kids’ Hearts. It was a training for K 12 secondary, like middle through high schools. And one of the things that it encourages you to do, to get students to buy in and take ownership in the class is to come up with and co-create the classroom rules and expectations. Yeah. Which is great, i’m not saying anything, but I’m doing this with students that, for the most part don’t know me. Yeah, occasionally I would, when I was still teaching full-time in the classroom, I would often have some Algebra one that those students would eventually see me a couple years later in algebra two.

But for the most part, the students didn’t know me and didn’t know the class. She did this with a group of students that had already experienced the class, so they’re doing this co-creation from a much more knowledgeable and experienced viewpoint.

Sharona: So she made some changes and I think what’s really interesting, and I want your opinion on this, is the first one listed says regular individual feedback with room for growth. So I provided weekly individualized feedback for all online pre-class assignments, but no grade with the option to resubmit answers up to a level of mastery.

I kind of feel like that is essentially, now we’re doing it in checkpoints, she’s doing it pre-class, but isn’t that the whole revise sufficient? I mean, yes, there’s no grade, but it’s basically revise it until you get complete. Yeah. Is that not No, I do. Which I do. I already do that, especially in history and math. I did it.

Boz: But I, I’m curious though, ’cause she said with the option to resubmit answers up to a level of mastery. So I am curious ’cause if there’s no grades. By whose definition or if I did something that we, let’s say you and I were in this situation and you were the professor and you give me some corrective feedback, but it’s a minor detail. It’s something that could turn something that was really good into even stronger or better.

At what level is that resubmission getting to the point of diminishing returns? Like at what point is good, is good enough, and any additional time I’m spending, I would be better spent somewhere else.

Sharona: So I was thinking about this in my context ’cause I don’t understand what she’s doing, but let’s say I gave some pre-class scaffolded actual problems, right? And they did the work. They submitted it. I mean, it could be that I give them feedback and I might tell them these answers are correct, these other ones are not. You’ve got some arithmetic errors. Since it doesn’t directly count towards the grade, they could look at it and go, oh, I see my mistakes, I’m not gonna bother to resubmit it. But they might have a problem where they’re really struggling to even understand the approach.

And so they try something and I would give them some feedback and say, well, I don’t know about your approach. Have you thought of this? Or have you thought of that? They may want to submit it again to see if their new approach is any better. Right? So I think it could depend on the type of error, and none of it has to get to a certain point.

But if, for example, in these types of classes, everything I’ve heard is people are gonna put together a portfolio of work at the end that they’re gonna use to justify their final letter grade. Maybe you would do three or four or five iterations on that problem that you didn’t understand the approach, and then put them all in there if perseverance is one of the things that we’re have as a learning outcome.

I think that maybe that’s where some of that is playing in, where it’s like, if I kick it back and like, hey you missed a negative sign, again. Like it’s not worth doing. But if it’s, I don’t understand how to do this concept, I don’t understand what I’m doing wrong, then you might choose to, and I might suggest you might need to work on this particular problem some more, but these other ones you don’t need to worry about.

Boz: And that kind of goes to the second change that she was reflecting on in this part, which is the self-reflection and metacognition, which is another thing that we have talked a lot about. In fact, that is something that we incorporated into our stats class a little while back. This is something that Joe Zeccola has talked about pretty much on every episode he’s ever been on.

It’s something that we have found is important enough that we’ve actually, in the stats class, moved it up. ’cause that used to be something that we did for students that had struggled on the first few. Now we’re like, no, everyone can benefit from this. Everyone is doing this after our first checkpoint, getting ready for the second and possibly third if they need it.

So regardless of what you teach, how you grade, I personally think this is one of the most important tools that any educator should have in any class. But I’m thrilled that she was talking specifically about how she used that in this class. ’cause I do think that is again. Something that as you go to design your new pre-calc class is something that you definitely need to look into and how are you going to utilize that student self-reflection.

Sharona: I think what’s really important for me, that I’m really gonna have to wrap my head around, is I don’t think there’s a lot of great opportunity for metacognition and self-reflection on can I solve a quadratic equation? It, it, it, it’s just, you can or you can’t, you know, what are we gonna get there? But the key is those mathematical practices though, right? So I don’t know that I would do a lot of self-reflection on a single skill, but I would absolutely do it on, can I recognize the patterns? Can I recognize that there’s a quadratic there I need to solve?

Boz: Yeah. But you’re also talking about making your course a lot more of investigating the why and then the how instead of traditional math class, here’s how you do this. And then eventually we might get to, oh, here’s an example of when you might use it. But you’re talking about flipping that, looking at a scenario where you might need to use it and then using that to drive the student’s desire to learn about it. I think that is an area where you can utilize that self-reflection.

Sharona: Absolutely. So I think the key that I guess I’m saying as I’m sort of reflecting on professor Stranford’s materials, ’cause I looked at her learning outcomes. I mean, it’s immunology. There’s a ton of facts that they need in immunology, and there’s a ton of facts that they need in pre-calculus.

But if you focus your learning outcomes on the practices, on those mathematical practices, those bigger concepts, then that’s where that self-reflection comes in. So that’s probably the biggest shift that I need to think about structuring the course around is I need bigger learning outcomes than are currently in existence in the pre-calculus class, while still maintaining the same content.

Boz: The next thing she talks about is another one that we’ve kind of talked about as well. And I was mentioning earlier, in a pre-calc class you can have students whose end goal is a lot of different things. Pre-calc really is a course that a lot of different STEM majors go through and even non STEM majors have to go through.

And that is how she addressed some of the intrinsic motivation by having her students each select at least one of their own goals. Yes, it had to be related to the course material, but the students setting their own learning target. And that’s exactly what you were talking about ’cause we’ve talked about this offline, is looking and having your students go, I am in this course because I am an architecture major, so I can set one of my learning goals, one of my learning targets around, something towards that. Or if I’m an engineering major or if I’m a business major and I have to take this class ’cause I have to do business calc. So giving the students. The ability to make their own learning target, I think that is just an incredibly powerful intrinsic motivation tool to use.

Sharona: So we have all these things. There’s other techniques and structural things, and I’m like, okay, so this article, and I wanna encourage anyone who’s doing this work, please publish. Please publish. Anywhere. Because having the details of this stuff is so critical to allow other people like me to recreate and not have to make all the mistakes.

Can we move on to the actual what happened in her next class? Oh, that work for you? Yes, absolutely. Okay. So she has a section in this piece, and again, like I said, I really wanna get her onto the podcast as soon as possible to have her explain more of it from her own experience, especially ’cause all of these papers are written as of a couple of years ago, so I wanna know what she’s doing now. She gave ’em a week into the fall semester to explore the course structure, settle into a rhythm of pre-class work during class. And then she set aside time for a discussion about the self-assessment guidelines.

And this was fascinating and I totally wanna do this if I can figure out how. They began in a space that the instructor was not present in.

Boz: Yeah. Talk about authentic giving the students that opportunity to actually have these discussions without the fear of the professor there. I mean, like the student feedback that they said earlier, how important those meetings were, but how intimidating they were.

And you know what, I’ve done the same thing with adults. So one of the roles that I did as department chair at Santee for so many years was data-driven instruction. And we would have data conversations with names on the data. Like we were very open and vulnerable. We would look at each other’s data and go, oh wow, my class isn’t doing this well, and Sharona yours is, what did you do differently?

So we had these very open, vulnerable conversations. They started by me not allowing the admin in the room. And there were oftentimes where I would tell the admin to leave the department meeting so we could have these conversations openly and honestly, because we were gonna have to be vulnerable.

So doing this here with the students really allows them to have that authentic discussion and really take that fear part out. That is incredibly powerful.

Sharona: And I think, if I recall correctly, Asao Inoue, talked about some of that same stuff.

Boz: Oh yeah. And a lot of his work is referenced in throughout this. That was one of the people that she pulled off of.

Sharona: Now I’m gonna find this a little bit challenging because I won’t have a built-in set of peer mentors, but I do know that some of our engineering instructional student assistants, at least one of them, who are working in our mastery graded classes in engineering, is a former linear algebra student of mine.

So I may bring that person back and ask them to facilitate this for this fall until I have my own group of students that could do this.

Boz: Oh,, that’s a great idea.

Sharona: ‘Cause that’s the closest I can get at this point. And so she brought in these mentors, so she took her students from the first semester that she did this, asked them to mentor the other students, and they began a discussion of the self-assessment guidelines.

And then the peer mentors filtered that back to her and gave her, this is what we as a group came up with. And then they brought that into the classroom and the next classroom, they had a whole class discussion. So they were starting from something that the students had created outside of the perspective of the instructor.

And the instructor basically said, this is a great place to start. Let’s talk about it. And they made some changes and they added some things. So they had four components. They did effort, mastery, participation, and personal goals. And they have amazing things in this, like multiple, multiple bullet points across the board.

So it’s just interesting. I don’t know how I’ll do it for mine ’cause I won’t do it exactly like hers, but Interesting. And in some ways, some of these things I would actually worry about putting in. Like the students were essentially harsher in some ways on themselves as a class. So I personally would worry if students who were good at the game of grades might have dominated some of these conversations.

’cause I’m looking at the mastery component and one of the items is complete and on time individual submissions for all three summative of assessments. Well, I’d want, as the instructor, the flexibility to be like but what if a student can’t do it on time?

Boz: Well, I think that highlights, again, something we keep talking about.

When you’re setting up your grading architecture, looking really where your course lives. You’re now comparing your freshman level course to an upper division course. ’cause Right. This course, I think she said is a senior level or a higher, higher level.

Sharona: An upper division.

Boz: Yeah, upper division. Yeah.

So. Yeah, you’re gonna have some different students. In, in the course. So yeah, you might end up having different bars.

Sharona: Right. But I also, but I also worry, ’cause even in my history of math, there were people needed exceptions. And one of the things that I worry about with, and this may just be my discomfort and my power dynamic imbalance, I worry about a group of people who are not trained in equity, who are not trained in some of the things we’ve been working on, not realizing that they themselves are now being inequitable to their fellow students. And someone who needs more accommodation, for whatever reason, may not feel comfortable speaking up.

So I just am gonna have to think through some of that for myself as well. What happens if you have a majority dynamic and a minority dynamic within the classroom.

Boz: They consider, yeah.

Sharona: But she also thought of this, I wanna point out, so I wanna read you something that it says there was student consensus that on time submissions were important for effective in class group work. And that having all of the group present for most of these and other classroom meetings was crucial, but that everyone deserved at least one instance of life gets in the way. We tried to build in exceptions to each rule with a general philosophy that each student takes sufficient responsibility for their own learning and contributions to the group, such that they eventually found a way to catch up with the material.

So I think the big thing here is this particular class is actually heavily structured around group work. Mm-hmm. And that may be the difference. I am not as concerned with the group work side of things. I will be doing a lot of group work, but I don’t usually assess on group stuff.

Boz: Yeah.

Sharona: Because I don’t feel good about that. So that may be one way to, to deal with that. So I think that there’s so much more to this. We could go on and on and on, and I will be spending several weeks digging into this article and all the things, but I wanna see what happened. I wanna see that second semester, what she says about what did the students say?

Boz: Yeah, what? What did all these changes that she made, how did the version 2.0 go?

Sharona: So she asked the following question. Based on your experience this semester, what do you see as the advantages and disadvantages of un grading using the self-assessment rubric we created this year. If possible, give me some examples of each.

Do you feel like the advantages outweigh the disadvantages or not, and why? So this was. Interesting to me. Some of these results. Do you wanna share any of these ones?

Boz: Well go ahead. You said you had some interesting.

Sharona: Well, it’s not a knock it out of the park one way or the other. So 71% said the advantages outweigh the disadvantages.

Boz: Okay. So I would disagree with you already then saying, okay, it’s not a knockout of the park. If nearly three quarters of your class is going, yes, this class has more advantages than disadvantages, I’d call that a pretty solid knock out of the park.

Sharona: I just curious about some of these details.

Boz: Yeah, some of the details, but overall, especially if most of the students hadn’t experienced something like this before, this is their first experience, and 71% are saying they think it has more advantages than disadvantages. Yeah, that’s a good thing. That is a really good result.

Sharona: It is, but more importantly to me was the next part, and that’s what makes it knock it outta the park for me.

So 71 said that the advantages outweigh the disadvantages. But no one said the other way around. Everyone else was just more equivocal. They’re like, I don’t know, kind of balanced for me. No one said the disadvantages outweighed the advantages.

Boz: Exactly.

Sharona: So we have to be careful with data, statistics, you know, all those things

Boz: Yes to, because 71% said that it was it had more advantages. Does not mean the other. 29% said the opposite.

Sharona: And I think that the examples of the advantages are what are really interesting.

Boz: So, okay. So what were some of the advantages that the students pointed out? So th this is what the students found was advantageous.

Sharona: So the first two are ones that you and I would predict and know.

Boz: Absolutely.

Sharona: Reducing pressure in mastering all the material all at once. Cramming and forgetting to study doesn’t work. Like, yeah. Okay, great. Three is interesting. It’s something that I hope would happen, but that I’m not sure the students would realize, but they did. It builds sustainable study skills.

Boz: Yeah, and I think that fourth one is also interesting and one that we could have predicted or would hope to be.

Sharona: We could have predicted, but I don’t know that I would’ve predicted that the students would realize it.

Boz: I, I see. You think so? Especially the relationship with the instructor. I like the fact that, so what the students actually said is that it builds relationships with peers and instructors through multiple submissions and constant check-ins.

So that’s one of the things that we’ve talked about. That’s one of the things that comes up in here. Is the inherent competition that is often there with traditional graded classes. So I’m actually more surprised that the students pointed out that the relationship with their peers was better and more collaborative.

Seeing the difference with the teachers, I think can be a little bit more. Obvious to a student, but yeah, I love that one.

Sharona: I agree. I think going further down though, there’s a couple of things that are really knock it out of the park for me. Okay. So she did an anonymous end of semester evaluations. And the first thing they knocked it outta park is that 87.5% of students responded to anonymous end of course evaluations. Like that response rate is insane.

Boz: And we’re we’re talking about a fairly small dataset so that almost 13% is only three people. Like there are only three people that did not respond. And anyone that has tried to do post surveys of students knows that once a course is over, it is hard to get the students to respond to those kind of things most times.

Sharona: Yeah, so that was one thing that was interesting. The second thing is that the most mentioned attribute was that using a community generated rubric for self-evaluation was greater motivation for focusing on learning for learning’s sake.

Boz: You said the most. What percentage of students responded with that?

Sharona: 83%,

Boz: 83% of students said that this was more motivating.

Sharona: To focus on learning.

Boz: Yes. To focus on learning. One of our biggest things that we always hear about with educators that are pushing back on looking at some of the alternative grading practices, especially some of the things like not grading homework for com for correctness or policies on late work, it’s always about student motivation. If we don’t do this, the students won’t do it. Here’s one where all of this collaborative work and things that the students did. They’re saying this all led to more of their own motivation.

Sharona: Now this section on results goes on and on and on. There’s just so many things with a ton of detailed student comments.

I highly encourage everyone to read them, but there’s one last section of this and we are coming on time that I really wanna touch on.

Boz: Okay. Before you do, there’s a quote, a student quote here, that I have to read. Okay. It’s part of the results. So this is directly from one of the students.

One massive advantage of un grading was that the course focused on learning and understanding the material rather than the grades and memorization. Additionally, UN grading allowed me to want to learn the material for myself rather than to get it for a grade. This allowed me to retain more information about the class and its materials.

Isn’t that what we keep saying? Isn’t that what we want? Isn’t that what the student learning for their sake of learning, not for the grade. All right. I just had to point that quote out.

Sharona: Yeah, no, absolutely. And I wanted to sit there for a second because just thinking about that is that’s what we’re doing. That’s why we’re putting all this work in.

I do wanna highlight some other things because these are things that are really allowing me, like one of the things that is formulating in my mind. There’s so many things I like about what I currently do. I like the rubrics of complete revised or the check in the cloud. I feel like that’s communicating to the students something. It almost feels like what I really am gonna change is the wrap up. Like I might not change any of my proficiency scales. I might still use the cloud and the check and things like that. I was thinking that maybe I had to go away from those.

But in this section that says other structural elements that support un grading and self-evaluation is they do a lot of things, predictable weekly online learning module modules that allow them to create routines, et cetera, et cetera. I do that already. But a suggested cutoff date for when they were expected to have baseline mastery of the material from each learning module.

I can do that. Like that would be where my proficiency scales would come in is a way for them to get a quick check-in. Not that it becomes the grade, but just you got that check mark. You got these learning outcomes related to this content that are maybe more skill specific. They just don’t go into the grade wrap up the same way.

Boz: And I love the fact that it’s stressed it’s a suggested cutoff date. So again. We talked about this with Emily and Sarah a few episodes ago when we were doing our episode on Neurodivergence. We talked about deadlines are these weird things. Some people really need them. Some people it’s unmotivating or even demotivating.

So this suggested, so, okay, I haven’t gotten there yet. I’m struggling in this one area. Yeah, I’m coming up on this deadline, but I know okay, if I don’t make this deadline, it was suggested. So, yeah, I know I need to work a little more now ’cause I’m a little bit behind the schedule, but it’s not gonna end up hurting me if I’m able to catch up.

Sharona: And if they miss the deadline and I can see they missed the deadline, then I can say, Hey, let’s have a meeting. Let’s talk about where you’re at. Not because I’m gonna penalize you from missing it, but that my experience, my knowledge, this is where my coaching knowledge comes in. I know how you’re starting to get yourself in trouble. So let’s talk it through.

Let’s see how we’re gonna catch that up. Not because I wanna penalize you, but because if you don’t have the skill going in to the next thing or whatever, like, there’s gonna be issues. So I love the fact that I’m feeling a little bit more confident that I don’t have to blow everything up of what I have developed that works for myself with my grading systems.

Boz: Exactly.

Sharona: So there’s so much here. There’s so much here. Like I am really excited to try this. I have a lot of things to think about. I am worried about holding 30 minute individual meetings with 30 students three times a semester. ’cause it’s like 12 to 14 hours of meetings, which in my schedule is gonna take a couple weeks.

So a little worried about that. But

Boz: so you know the practice is a good practice. You know, the way you modify it, maybe doing. Small group ones, maybe doing it for smaller amounts of time. We’ve also talked about and talked with several people that do even more often check-ins, but they’re just quick rapid fire ones. It doesn’t have to be, it was 30 minutes in this course. Maybe because of the density of the the material. But in her description of her classes said that these were classes of 20 to 24. So they’re a little bit smaller than yours. Not hugely, but yeah, they’re a little bit smaller.

So, yeah. Make adjustments.

Sharona: So we’re pretty much out of time. The article does wrap up with suggestions for the future, I think for herself and for other people. But I just am so grateful that someone took the time to document this at such detail that I feel confident that I can learn from it and be able to adapt my own practices and get them growing and trying something new.

Boz: And I kind of wanted to conclude this by reading some of the conclusion from this. So her the first paragraph in the conclusion section of this. In conclusion, I have found this un grading course structure in working with students to design an assessment tool to be liberating and humanizing practice. The major advantage I see are similar to those given voice by the students, an emphasis on learning for the sake of understanding, lower stress, a more holistic and accurate assessment process. I absolutely love that.

Sharona: We have to read the last two sentences. I’m gonna continue. Okay, go ahead. So right from what you said, you said accurate assessment process.

It allows reflection on mistakes to lead to growth, a more collaborative environment and stronger instructor to student relationships. Importantly, it also brought me added joy in teaching.

Boz: That is a great way to wrap this up. We are gonna try to get this author on. I don’t know if she ever listens to this podcast, but if she does, you’re invited.

Expect check your email. Check your email. But until then, you have been listening to the grading podcast with Boz and Sharona, and we’ll see you next week.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode’s page on our website. http://www.thegradingpod.com, or you can share with us publicly on Twitter, Facebook, or Instagram.

If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the Contact us form on our website. The Grading podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.

Boz: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State System or by the Los Angeles Unified School District.