The Grading Podcast

122 – Sharon Stranford

===

Sharon Stranford: And that was really kind of what ultimately led me to ungrading. And then after, like I said, after the pandemic, I had already started to think about, I really wanted to try, I had talked to our Center for Teaching and Learning director and she had sent me some websites on ungrading, and I had been thinking I wanted to try something like that, but it was the pandemic that helped me to decide to jump in the deep end and go full on. Let’s just get rid of grades.

Boz: Welcome to the grading podcast, where we’ll take a critical lens to the methods of assessing students’ learning from traditional grading to alternative methods of grading. We’ll look at how grades impact our classrooms and our students’ success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach and instructional designer. Whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week, you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Boz: Hello and welcome back to the Grading podcast. I’m Robert Bosley, one of your two co-host, and with me as always, Sharona Krinsky. How are you doing today, Sharona?

Sharona: I am doing well. I had a couple of things happen this weekend that, one of which was really good for how I feel and one of which not so good. So I actually got sleep on Friday night. Those of you that have been hearing me talk for a few weeks, I’m having the worst trouble sleeping. I don’t know if it’s my age or what’s going on, but I just like, I’ll wake up at five o’clock or four 30 in the morning. Well, before my alarm I got like, I don’t know, eight hours of sleep Friday night, and I am a whole new person. So what did I do with that good feeling? I went and got boosted. I got a COVID booster and a flu shot. So this morning I woke up and I’m all, oh, I’m achy. How about you, Boz? How are you doing this weekend?

Boz: I’m doing all right. You know, I actually got a little bit under the weather and got sent home , the Friday before we recorded this, which is unusual for me. Although when I say sent home early, I was sent home from a meeting that was supposed to happen after work. So getting sent home early meant I left at 3:40 instead of at his normal time. Yeah. But no other than that, I’m doing well.

Sharona: Excellent. And I believe OU had a bye week this week, right? So there’s no real sports to talk about, is that correct?

Boz: Yes, we, yes we did.

So welcome Dr. Stranford, Sharon to the podcast.

Sharon Stranford: Thanks for having me.

Sharona: And one thing that most people don’t know about me is my name. Actually, my birth name was Sharon, so I go by Sharona now, and I did legally change it, but it’s interesting when I get to talk to another Sharon. So

Sharon Stranford: yeah,

Sharona: there we go.

Boz: And, and I think this is probably the closest interviewee we’ve ever had to where I live. ’cause, I live on like the edge of Ontario and Montclair.

Sharon Stranford: Oh gosh. you are just around the corner.

Boz: Exactly. So welcome we’re thrilled to have you on , but before we kind of dive into what we really want to talk about, something we always like to ask our guests the first time they come on is just, you know, how did you get involved in this crazy world of alternative grading?

Sharon Stranford: Hmm. Well I, like the bio says, I’ve been teaching for a long time and I myself am a first generation college student. So I’ve never been really comfortable with grading my students like I’ve, and I’ve always felt like teaching is a bit of , and learning is a bit of a mystery that I had to uncover for myself. It’s a pathway I had to navigate college and graduate school. And so I’ve always wanted to sort of reveal more of what happens under the hood. You know, like to lift the veil of like, this is what’s really going on. Be very frank with my students. And I was uncomfortable with grading even before the pandemic. But then when the pandemic hit, like I think a lot of institutions, we set aside grades in favor of talking to our students, getting to know our students more about them and their lives. And so I really never went back after that to any form of traditional grading.

Boz: You know, that’s interesting that you mentioned the pandemic. That has been kind of a catalyst for a lot of change, at least temporarily. I’m hoping some of that change is more permanent, although I don’t know about you two, but I am starting to see some of the, some of the changes that happened because of pandemic that continued post pandemic. Starting to go back to pre pandemic.

Sharon Stranford: Yeah, I am as well, the test optional like SAT optional which was in place for most institutions. Some are rolling that back now, and I’m sorry to see that.

Sharona: Well, and we, did a big rollback in our statistics course as a reaction to AI, which is that we pulled all of our what had been asynchronous quizzes which are the bulk of how students get assessed in our standards based class. We pulled them all into the classroom so that we could proctor at least do a light proctoring on those quizzes, which makes me very sad. But on the other hand, what’s weird is our pass rates of the on the quizzes are going way up.

Sharon Stranford: That’s interesting.

Sharona: As is our attendance. So it’s like, hmm, I’m uncomfortable. And yet, and yet. Interesting. So you started because of the pandemic, but what made you really go down the, or maybe you could explain a little bit in case someone didn’t listen to last week’s episode. What direction did you go with your alternative grading? How did you get from, I’m traditional to where you are today? What was that path like?

Sharon Stranford: Hmm. Well, tenure helps. So I’ve tenured in a full professor, so I felt like I had the latitude to really try something different. And I had been using just in time teaching since I started teaching. And so I was really used to students doing work before they came to class. And it was easy for me to see that they were putting a lot of effort into that and that that should count more and more towards their final grade. So in other words, like I could really tell how hard they were trying, and this is pre AI and so they’re doing a lot of answering questions online. And so, my tests got smaller and smaller like that as the percentage of the grade for the class. So they were getting more and more and more diluted. Then I started playing with rewriting exams because it felt like. I didn’t want students to walk away from an exam and feel like they just, you know, table that and don’t have to worry about it anymore.

And also in many of the courses I teach, information’s hierarchical. So if they didn’t really grasp something from the first third of the class, I need them to be able to understand it in order to do well in the next section. So it felt like I really had to give them opportunities to recover and to grow. So I started doing, you can rewrite exams. And then I started doing more and more group work and I realized that they are so good at teaching each other and in fact it helps those who are the teachers to learn as well. And so I started putting them in groups and letting them redo, rethink some of those exams together. And that was really kind of. What ultimately led me to ungrading. And then after, like I said, after the pandemic, I had already started to think about, I really wanted to try, I had talked to our Center for Teaching and Learning director and she had sent me to some websites on ungrading and I had been thinking I wanted to try something like that, but it was the pandemic that helped me to decide to jump in the deep end and go full on, let’s just get rid of grades. ‘Cause I was really, really tired of hearing, I have a lot of pre-med students hearing students’ concerns over what their grade was gonna be, what the number was gonna be at the end. So that was what led me to just jump in.

Boz: So, for our audience that are listening that haven’t heard that term before, can you explain a little bit what you mean by just in time teaching?

Sharon Stranford: Oh, sure. Yeah. Just just in time teaching asks students to think about the material right before you’re about to discuss it. And so it, it asks them to, it’s like a flipped classroom. It asks them to read or now maybe watch a video and then answer a set of questions. And the just in time is usually the responses to those are delivered to the instructor the night before or the morning of, say, a class session so that when you walk into the classroom instead of seeing empty vessels in front of you, that students are often a buzz with, well, what did you say for that one? What do you think the answer to that is? And then we discuss their responses. And the really nice thing about that is it takes a lot of the easy stuff out. So I’m not writing definitions on the board. I’m not going over simple explanations. I’m getting much more into, you all are struggling with this or most of you are struggling with, I know exactly what to target in the classroom.

And like I said, I’ve been doing it since I started teaching. ’cause I read Kim Novak’s book on just in time teaching and it’s part of my toolkit from like day one. And so now I can’t stand to walk into a classroom where they’re not already asking me questions.

Sharona: I have so many questions. So when you say that you don’t grade because a lot of times when I hear about ungrading, a although I know a lot of biologists who do it, actually, there’s an entire group of biologists doing this. But in my head, I can think of what it means in a humanities class, but I struggle in a STEM class ’cause I’m a mathematician to say, does that mean that we’re never telling them when they’ve gotten a problem right? If there’s a closed-end problem, are we not telling them that this is now Correct. So how do you handle closed-end feedback versus open-end question Feedback?

Sharon Stranford: Yeah. I struggled with this and I struggled a lot with finding models. So I’d love to be connected to more people in the sciences that are doing this. For me, it was a philosophy more than anything else really. It was a de-emphasizing scores and numbers. I read some work by Ruth Butler. Where she looked at motivating students and I think this was in middle school. What she did was she looked at they took a test and she gave them back the test with just comments, with just a number or a grade or with comments and a grade. And what she found was students couldn’t really absorb your comments with a grade. And with a grade, they often felt like either, great, I’m good. Don’t need to look at this again. Or, oh, I did horribly, can’t stand to look at that.

And so they weren’t really getting any kind of feedback from receiving that exam back. And I thought, I really don’t wanna do that anymore. And then I think I tell this story in the blog and in the paper. I shortly before jumping into un grading, I had a couple of students sitting at a table outside my office. And we were meeting to talk about the class. It was like office hours, like group office hours, and they’re unpacking backpacks and talking to each other about another class, another STEM class and asking each other, oh, hey, how’d you do on that? Did you study for that? You know, those sorts of conversations. And one of them says to the other oh, yeah, yeah, I, I studied for it and I think I did really well on it. And then that student says to the other one, but, you know, I really like this subject. At some point I’d like to learn it. And I just had this like, oh my God, what are we doing with this? Like, how, how are exams, and doing well on an exam, not at all motivating students to actually learn?

And so at that moment I thought, oh man, I really need to do something totally different. And so that was when I jumped into, I’m just gonna stop emphasizing numbers, letters, stop putting it on anything. And I had already been giving student feed students feedback. I was just gonna talk to them much more about like their mastery, their understanding, where they could supplement that understanding and give them opportunities to do that and try and de-emphasize numbers and letters.

Boz: I love that story. We actually talked about and quoted that very thing that comment like, I, I want to make like coffee mugs or t-shirts out of that. I think that sums up so many issues with, with education, just in general. I really like this subject. I I might take some time to actually learn it. I mean it, I absolutely love that. Yeah.

Sharon Stranford: Yeah.

Sharona: So one of the things I’m trying to dive into though, in all of this commentary is you do still have in your grading schema

Sharon Stranford: Yeah.

Sharona: That you’ve agreed upon with the student something called mastery.

Sharon Stranford: Yeah.

Sharona: Is there. A mark or something, like, I use a check mark in a cloud. I use emojis. So a check mark means you’re at what I would consider sufficient. I might still be giving you feedback, but it’s good enough. And a cloud is not yet. Yeah. Is that still a grade? Is that check and that cloud a grade in your world, or is it No, that’s, that’s part of the feedback. And it’s really just shorthand for, because I, I feel like students don’t know what the bar is. Yeah. We still have to tell them what is a bar of good enough, at least on a factual expectations for the course level. Now, the level might change for a freshman class than a senior class. I might have different expectations, but at some point I’m still the person with the content knowledge to tell them what’s good enough. Am I not? Yeah.

Sharon Stranford: Yeah. And they’d want that, right? They really do want that. And so, yeah, so where that shows up is they do weekly pre-class questions. And my course, so this is an immunology course. It’s more advanced course. And so it’s an elective, so nobody’s in there that doesn’t wanna be in there. I know they wanna learn the material, they’re interested. I set it up in a weekly format, so there’s kind of modules for each week, and there’s a set of material. It’s always a combination of some, maybe listening to things they really love, multimodal ways of learning. So some reading, some watching, some listening. And then they answer a set of questions. And then we spend at least one class period, like really going over the answers to those questions. And so in that question bank, I have a combination of some multiple choice, true false, short answer and essay questions. And it’s a pretty comprehensive set of questions. There’s anywhere from 10 to 20 questions each week, and even for the multiple choice and true false, I ask them to give me rationale and my system allows me to do that.

So they have to write to me about it. And so the system can grade those multiple choice through false questions for correct ’cause I can put the correct answer in there. And for the essay questions it can’t, but I go through that. I set aside time to actually go through all of their responses and read their rationale. And sometimes I take away the correct that the system gave them. And so that’s, they get a one for correct or a zero for not correct or not there yet. And then for the essay questions, I do the same. I give them a zero if they’re not there yet, I give them a one if they’re there. And I tried taking those multiple choice and true false questions away and just relying on essays. I tried taking away that one and zero because it still will then, for that quiz, it’ll give them a percentage correct. It might give them a 75% correct, or 50%. And I wanted to try and get away from those numbers and actually they pulled me back from that when I tried that. They said, I really need your words, your feedback that you write to me is so helpful. So, like I said, true to that, but my rationale is way off. Now I understand better after you wrote back to me. But don’t take away those quick multiple choice, true, false ’cause. It’s a quick and dirty way for me to know, okay, I understand this pretty well, and then I give them feedback and occasionally I will give them their one, but still give them some feedback. Like this is correct enough. But you might also consider the following things and they really appreciate that. So I was having some problems with they get 90% and they’d still wanna go back and redo their quiz ’cause I was still commenting and they wanted to respond to all of my comments. So we had this set a bar that was like when you hit 85% on one of those pre-class quizzes, stop. We can, we can keep writing back and forth or we can talk to each other, but just stop because you and I both are spending more time than it’s worth on this. Move on to other things.

Sharona: That’s fascinating because one of the things that I’m hearing as you talk through some of this stuff in this collaboration is, ’cause I thought I was like really far away from this, but my structure actually feels like it’s gonna fit in if I just make some tweaks actually to my wrap up where I give my students more, say in the wrap up process for the final grade.

So, yeah, I don’t know. This is exciting. So I have another question. So I am super, super curious about, ’cause I read in the article, you started the first semester, kind of fumbled the way through it got the peer mentors. But now today, what does the first couple weeks of your class look like as far as in detail? How do you introduce the fact that you’re gonna have this conversation with your students? Do you show them the previous classes rubric and let them, what do you do? What does that first couple weeks of getting buy-in look like?

Sharon Stranford: Yeah. I spend a lot of time in the beginning of the class really selling this process which I initially called un grading. Now I call collaborative grading. But as you know, the vocabulary is kind of a mess. So I just have to explain to them that I de-emphasize grades. I give them some like, here’s the student feedback from before. That really helps when they hear, oh, this is what students had to say about the way she’s teaching this class. The peer mentors are usually students who’ve taken the class with me before, so they’re really great with supplementing some of that. So I really try and sell them in the first class or two, but I don’t actually bring the rubric in right away. I have found that you need, you need to build a little bit of trust with the students and rapport with them before you ask them kind of vulnerable questions. Like, how much do you wanna hold yourself to attendance? And what do you feel like is the right schedule of like expectations for meeting as a group outside of class. And if I try and ask ’em those questions too soon before we’ve really developed a rapport with each other before they’ve gotten to know each other, it falls flat.

So the first time I tried it too soon, so now I wait for at least a week into the semester. I sort of introduce the idea and then I just launch into teaching the class the way I do. So in fact, I send them a set of questions before the first day of class so that on that first day of class, they’re still not empty vessels. They’ve actually told me a lot about themselves. So like I model, this is the way it’s gonna be all the time. You’re never gonna walk into class having not thought about what are we doing in class this day? You’re never gonna have to ask me that question. And so they get used to that. I start putting them in groups right away. I have them like in groups. So we have tables set up so they’re meeting and talking to each other. So there’s a lot of interaction in class. So I model like, this is what you’re gonna be doing. You’re gonna have to learn from each other, and you really talk to peers. And then after we’ve done that for a little bit and they’ve answered a set of pre-class questions online, I bring in the rubric. Now I bring in last year’s rubric. What I had to do the first time was they had to create that. So now I always start with the rubric from the year before, but I tell them this is just a template to begin with. You can modify this as needed. Actually, we tweak it less and less each year. I do this now.

Boz: I was just about to ask from year to year, does it get tweaked a lot? Like have you had a group that has made major changes to the previous year’s rubric?

Sharon Stranford: Not really. I had one year in which the students, when they met outside of class came back to me with, well, should we have a set of expectations for the instructor and the mentors? And and so we kind of added some expectations. Of like what what our role would be. But that kind of got integrated into the various levels of things. So for example, they could come meet with me and, demonstrate mastery, and I will go in and correct some of their pre-class question scores without them having to write to me like, we’ll just do this one-on-one because you’re doing what I asked you to do, which is come to office hours, so let’s just let office hours be an example of you’ve demonstrated mastery.

Sharona: I just, everything is spinning in my brain that to kind of figure out how this is gonna look for me, because, I’m going back into the classroom. I’ve been out for about a year and a half, and prior to that I was pretty happy with all of my systems and I’m still pretty happy with all of my systems, but I’m going to be teaching a class that is a very challenging situation. So it’s pre-calculus. So it’s the lowest level math class we teach. It’s the students who are the least prepared. And because it’s in the spring, they’ve most likely failed it already. So it’s a very, very low level of persistence, low level of perseverance, high risk class. And I’d like to find a way to blow it out of the water, which I’ve not successfully done before. So I feel like I need to make some radical changes in some ways to try to really just get extraordinary success. And I feel like this might be a way for them to, especially because you have those personal goals that you put in.

Sharon Stranford: Yeah.

Sharona: I wanna really explore that. So that would sort of lead me to my next question is what do some of those personal goals look like for students, and how do you help them come up with them?

Sharon Stranford: Yeah. First time around when I suggested they make goals, as you might imagine. A lot of them were not measurable, very vague. Oh, I wanna do well in this class, right?

Boz: I wanna get an A

Sharon Stranford: I wanna get an A, right? So I had to then coach them. And again, like, I get my best ideas from my students. A student gave me feedback to say, oh, have you ever heard of SMART goals? I think, you know, so, so I was like, oh yeah, how great that would be. So from then on, I sent them a link to sort of a description of what SMART goals are, which I’m not gonna remember what all the acronyms stand for, but it’s measurable, time bound.

Specific.

Sharona: Yeah. Yeah. Specific. Measurable. Actionable.

Sharon Stranford: Yeah.

Sharona: I forget the R And then time. Timely. Rel relevant, huh?

Boz: Yes. Relevant, relevant, specific. Measurable. Achievable, relevant and time bound.

Sharon Stranford: Good. Good. So, so I, you know, I gave them that. And then, so the, so the next year it went a little better when I gave them a link to the SMART goals. But then I also found some of them were struggling with, well what, what would good goals look like? Like they were a little challenged to think about this in a personal sense. I hadn’t really done this before. So they asked me could I see examples of goals from before. So I was like, wow, what a great idea. So I took the goals that students had given me before and kind of grouped them into categories, things that are like career related like, I wanna take what we’re doing in this class and really use it as a way to explore what my next career move might be. Or there were other students who said, I know that I am like really limited in my ability to participate orally in class and I wanna push myself to do that more ’cause that’s something that I feel like is a challenge for me. Or I had other students who said, I really wanna see the connections between the different STEM classes that I’m taking. And so I I now give them like a list of example goals that students have had before and they really appreciate, ’cause some of them will look at that and be like, oh yeah, that’s me. That that’s the thing I, I should probably do.

Sharona: Well. And I think that’s gonna be really important for the class that I’m gonna be teaching. Because we find that students who don’t pass this class, many, many, many of them don’t stay in college. Like, it’s not even that they just switch their major, they drop out altogether. And so if I can help them somehow connect this class to their goals, hopefully that’ll be motivation for gaining in confidence and perseverance and persistence and things like that. So I have, I have high hopes for this class. We’ll see.

Boz: Well, and we’ve talked about, especially the self goal part being. You know, interesting and, and really motivational for that class particular, because if they’re in a pre-calc class, they could be going to a dozen different majors. I mean, like the, the pre-calc class. Yeah. It’s calculus is a, you know, base for so many STEM and even some non STEM courses that you might have, you know, business majors, engineer majors science, there’s so many different majors that giving them that option of kind of making a self goal that is related to the field that they’re interested in and exploring how the math that you’re doing is actually relevant to, you know, a, a pre-nursing or a business major could help the students see the point and therefore be more motivational to Yeah. A, a group of students that, for one reason or another, are likely that they are struggling with their math.

Sharona: Yeah, exactly.

Sharon Stranford: Yeah. The other thing that I noticed in there, if they really put themselves into it, I noticed that it often revealed things to me that helped me to build their confidence. So sometimes they would tell me things about, like, things that hadn’t gone well or whatnot. So they would, in revealing their goals to me, revealed to me the places where they needed the most help. And so then in my one-on-one conversations, I would often come back to those things and really kind of try and boost them in that area or help them along in that way, or even started adding some supplemental material to my weekly units. That were related to their goals that they told me about. Like, oh, I know I’ve got a dental student in here. Oh, let me find a paper that’s related to that and I’ll pop it on the site so that, you know, after we’ve gone through that unit, that student can be like, oh wow, look at that relationship to my field of interest. So it really helped me to get to know them better as whole human beings as well. When they set these personal goals.

Sharona: And the personal goals do factor into the final grade, right. They, they’ve chosen to put those in that final grade wrap up. So do they do reflections on that? I mean, how do you measure

Sharon Stranford: Yeah.

Sharona: Their achievement of those goals?

Sharon Stranford: Yeah, so each I have three summative assessments, which I renamed because I decided even the word exam and quiz is scary to them. So we have three class questions, which are online quizzes, and we have summative assessments, which are like. So after each of those summative assessments, they answer a set of metacognitive questions where they reflect on how they prepared for assessments and their learning growth. And then I ask them to also talk to me about their personal goals. What were your personal goals? How far have you gotten towards achieving some measure of that.

Sometimes they would set goals that were so far off that like, I had to warn them, you’re gonna have to think of some medium steps because like getting into medical schools like way down the road, you know, what are gonna be the pre-steps that you take to get there. So they had to have something that at the one third, two thirds end of semester, they could tell me, here’s how I’ve gotten further in this, here’s what I did towards that. And occasionally in those that we’d have a one-on-one conversation after each of those summit of assessments, which takes a lot of time. We should talk about that. They would sometimes modify their goals. They would either have like, Hey, I’ve achieved this goal, like this is going a lot better. I’ve now, I’m not gonna expand it or change it a little bit. Or we’d have to reframe it, like, that’s not really looking like it’s working for me. Let’s reset a goal.

Boz: I was gonna ask about that. As the students set these goals, do you allow them to kind of reexamine and readjust? So, you know, like you said, maybe at, you know, at that one third mark, they’re realizing, yeah, this goal, I’ve already accomplished it. Or, oh, maybe, maybe this goal was too much or not quite what I was thinking. And can I tweak it a little bit here? Like, do you allow your students to do that?

Sharon Stranford: I, yeah, absolutely I do. And so I have to keep track of that. So I have a spreadsheet where I keep their goals and then I have to update it occasionally and, and I actually ask them to set two goals at the beginning, because often I find that one of them, they either achieve it so quickly that then it gets kind of boring or it’s unattainable or it changes so often by asking them to set a couple of goals, we end up with at least one good one to sort of track throughout the whole semester. But I would say maybe a third of them end up having to change that at some point, or they’ve hit it and they add something else.

Sharona: I’m really excited to try to do this. Now I’m thinking of doing something else and I wanna see if this is something that you did. ’cause you do mat mention that you look for papers, but, mm-hmm. I’ve been starting to use AI personally to create customized materials. And I read an interesting blog post just this weekend about how it might be unethical or, or detrimental to use AI for learning, but not unethical or detrimental to use it for working. So I consider my use, I’m trying to do my job better. So I was thinking of finding out their interests, their majors, their goals, and then asking the AI to take like a particular math topic and be like, Hey, give me a nursing version of this, giving me a biology version of this. Give me an engineering. Do you try to get that customized or is that not really an area you’ve gone, or what’s your thoughts on something like that?

Sharon Stranford: Oh yeah, that’s interesting. I I try and write so the, the pre-class questions that they answer are, the multiple choice and true false versions of those are pretty formulaic. Like they tend to be, you know, you’re right, you get the right answer or not. But then the essay questions are often applications. They’re always some sort of application or there’s some sort of tweak on a, you know, create a good metaphor for this thing. Right? So, they potentially are AI able, some of them, not all of them, but some of them are. So I worry about a, a little bit, although often when I’m reading them I can kind of tell when they’ve used that and when they haven’t. But I do sometimes go, I go to the literature more than ai. I go to the literature, I go to sort of popular science ideas for ways to add some flavor to those pre-class questions and also the summative assessments that are very current. So like for, I’m an immunologist, this is really easy right now. Like during the pandemic, I feel like immunology just taught itself. It was like, pick up any newspaper, listen to any news report,

Sharona: mRNA vaccines, and like,

Sharon Stranford: right, it’s all in there. They’re all talking about it. I was like, ah, herd immunity was never so easy to teach as it was like, while we were all worrying about it. So I still feel like this topic really lends itself so easily to finding examples. So I will sometimes use what I know their career interests are. So I had some physical therapy students and I was like, oh, I could explain this concept using a physical therapy example. So I will sometimes do that, but I don’t know that I’ve really used AI as much to help me with that.

Sharona: Yeah, I think what I, the difference with my class is like immunology, even though there’s lots of ways to go into immunology, it feels a little bit more contained than like linear and quadratic models, which,

Sharon Stranford: yeah,

Sharona: literally economics, business physics, chemistry stems psychology data, like everything you can use a linear model or quadratic model or you know, and also it’s the place where we teach problem solving. So it’s really interesting ’cause I’ve been doing this, playing with AI for the context and AI starts to learn you if you’re not careful. And so one of my other jobs is I’m in theater so it tends to give me like stage spotlight questions and moving lights questions, which I think is awesome because this is a very, theoretically non-ST stem and yet so much pre-calculus.

So

Sharon Stranford: yeah,

Sharona: I’m kind of enjoying that, I’m not gonna lie.

Sharon Stranford: Yeah, yeah. Sometimes you can get your students to write really good questions. That apply the concept to their area of interest. So get them to actually write exam questions, you know?

Boz: Yeah. So I did wanna ask, and again kind of give Sharona some things to think about, because this was one of the things that came up in our conversation last week when we were looking at that first paper and kind of going through it and, you know, seeing, okay, I kind of already do this I might need to adjust this, but that is those self-assessment conversations that you have with your students. Like, can you describe a little bit about like what does that actually look like? What’s the, you know, the purpose of those?

Sharon Stranford: Yeah. Okay. So I meet with my students three times in the semester for sure. Like, it’s, it’s required that we meet three times in the semester for I set aside a 30 minute block for each student. Usually it’s a 20 minute meeting, you know, with a little bit of extra time, but, you know, I don’t schedule another one for, you know, every 30 minutes. I start, those are intimidating conversations. Like to ask students to come in and talk to their professor can be really intimidating. It’s a little less so at a small college like Pomona where we get to know our students pretty well, and when you have mostly juniors and seniors, so it’s a little less intimidating. I think if I were gonna do this in a more entry level course I would need to force some sort of very informal meetings and conversations in the first couple of weeks.

So I set that up. Like I tell them all that I would like them in the first two weeks to come to office hours and if they can’t make office hours, you know, come meet with me at some point. They could do it in groups just so we can get to know each other, just so we can talk to each other. And that lowers the tension just a little bit when we’re talking after they’ve just done a summative assessment and most of them will do it. But especially with juniors and seniors, I don’t force it. I think I might, if we were talking about first year students so most of them will come and meet with me, so I get a chance to get to the, know them a little bit. Then the next time we schedule a set of meetings, I’ve actually learned that this, it takes a lot of time.

So when I had 24 students, that was an extra 12 hours that I was, you know, meeting with my students. Now I’m not. Grading their assessments, and I think the time I would’ve spent to sit and grade 24, like open-ended, they, they have three really open-ended questions, so they’re writing me a page or more response for each one. It would’ve taken me at least 12 hours to grade all of those. And I’m not grading them. I read through them and kind of have some collective feedback that I provide to the whole class, but I’m not grading each one of them at that point. So I, I see that as a trade off those 12 hours. It’s pretty intense. So it’s exhausting for me. But I learned so much from talking to the students. I can tell in the first five or 10 minutes that we’re talking to each other how well they understand the material. And so I can then pivot and begin to ask them more and more complicated questions. Or tangential, like sometimes we’ll get into these conversations about, Hey, I just read a paper the other day. It kind of relates to something you said. What do you think about this? And we have these great conversations. And then there are other times where I realize, oh, they’ve got a couple of concepts that are, they’re confused in their mind. So I start asking them some questions to sort of dig into it. And occasionally they’ll realize that and be like, oh wait, did I say that in my, when I wrote to you? No, that’s not right. I didn’t mean positive selection, I meant negative selection.

And so actually there are moments when I get to be like, oh, phew, you know, because you said this when you wrote to me, but actually I can tell that you really understand it. And then there are some times when it’s really clear that they still have a lot of concepts confused in their mind. And so I start backing up further and further in our conversation to sort of get to the places where they are on more solid footing. And then we can sort of talk through some of those and at the end we wrap up. And I’ll often tell them, I feel like, here’s where you’re really strong, here’s where you maybe need to go back and rethink some of these things you had these concepts, you know, confused. So the conversations can be really hard for students, especially the first time around. ’cause they’ll walk away feeling like it was an oral exam.

Boz: But, you know, and we’ve talked about this before on the podcast though when it comes to being willing to gather evidence of mastery wherever it comes from, and this idea of having, you know, taking in that oral evidence. Because you really can, not just what they know, but where their holes are, where their misconceptions are, where they’re actually have a better grasp of, whatever the topic is than they were just able to express on that. You know, at that assessment at that time. I don’t understand some of my colleagues’ holdups on, especially in my high school setting, how many times I’ve heard, oh yeah, I know my student can do this. They just bomb the test. And I’m like, okay, so if you know they can do it, why aren’t you giving ’em credit for it? Yeah. Well, they didn’t do well on this test, but you just said you have all this other evidence that you know that they can do this, you know, they can solve quadratics. They just messed it up here, so why not? Take that evidence into account. And

Sharon Stranford: I, I totally agree with you. And the funny thing is, students wanna ding themselves for it. They’re used to this pattern, right? They’re used to, I got it wrong the first time. So when we talk about what grade would you give yourself, the student will sometimes say on the pre-class questions, like, I got 50% on the first three, and like the first time I submitted them to you, like they, I was only at 50%. But like, and then I say to them, well, but then you came back and you read to them and you hit like 95 or a hundred percent. And they’ll be like, yeah. But the first time I was wrong. And I was like, well, do you understand it now? Oh yeah, I totally understand it now. And then I’d say to them, well, why wouldn’t you give yourself an A then? And they’d like, well, because we always get hurt by getting it wrong the first time.

Boz: Yeah. You, you actually wrote that. You actually, talk about that in the blog. I don’t remember if it was in the paper, but Yeah. It, it’s in the blog where you say you have to go as far as convincing students that they deserve a better grade. ‘Cause they’re hung up on this. But I got it wrong the first time. Doesn’t matter if you got it now, it’s, I got it wrong on the first time, so of course my grade must suffer.

Sharona: Yes. That, which actually brings up something really important that I wanted to ask you about because I looked through some of the criteria where they set themselves very high bars for things like attendance. And I don’t know about in your context, but in my context, that could potentially be extremely inequitable. And like my fellow faculty, a lot of them have real struggles with trying to understand equity. And I’ve done a ton of training on it for myself, but my students don’t have equity training, so how much power do we give to them? With understanding that they may be perpetuating inequities that they don’t understand.

Sharon Stranford: Yeah. Okay. So this is the fascinating thing about when you have that discussion about creating the rubric, you have to have those conversations, right? You have to talk about how any number that you might set for yourself might not work for someone else, and why? And then how can you create some sort of a system that allows for both of those things to be true? So so even the, I think this is where we added in a lot of times, or in conversation with Sharon. So for example, I had students who you know, as simple as they were on a sports team and they were traveling and they had to miss a bunch of classes, and so they would have, they, what it says in there is it’s your responsibility to sort of make up whatever you’ve missed. And so they would just come to me ahead of time and be like, I’m not gonna be there. And so occasionally a friend would be there and the friend would record the lecture that day and the student would listen to it or watch it, or they would come talk to me about something or they’d, they’d literally set up ahead of time preparation for, I’m gonna have to miss this class. And then there were other times where students couldn’t be in class for other reasons, and they would then compensate by talking to me about it, like planning with me about it ahead of time. They realized that it was important to be present and important to get the information in a way that was also accessible to them.

So there were certainly conversations that we had about the difference between equality and equity. Then when we built the rubric, we tried to build into it not just an individual person’s life, but like what is life happens to all of us. And so we tried to build in contingency plans for things. And so, it always worked out with even a think they said, what did, they set a 90% attendance think with crazy,

Sharona: something crazy. I was like, my poor student.

Sharon Stranford: I know. I was, was so surprised when they suggested that. And I think then we all talked about, okay, being present here in class is clearly really important. And what that 90% would look like is that only, you’re only missing 10% of like what happens in lecture. So you have to find a way if you’re not gonna physically be there to capture that in some other way. And so they would come up with creative ways to do that and some of that was built into teamwork. And so, the one thing that could be hard, I imagine hard to replicate is the way in which I had them working outside of class in their teams. ’cause we have a residential campus where they’re physically all together.

So it was very easy for them to find times to meet with each other. And still there would be some groups that really struggled with that. And I learned the hard way the first time I did this, that for students who maybe came in with a lot of prior preparation, they could afford to not meet in their group, not come to class and still do really well. And so we had to like go back in and say, no, that’s not allowable. Because it’s important we’re learning from each other. So you can’t check out of your group. So if you are someone who actually has a lot of mastery, your role is gonna be a little bit different. Your role is gonna be, I have to be there for my peers. I have to help them. To, you know, get where they need to be and explain this concept because I’ve already had this other course that they haven’t had, or I had some experience that they didn’t have. And so this sometimes would come up in their personal goals. Their personal goal was going to be to be a better mentor to other students. And, and so that, that seemed to help a lot with some students not saying, I’m gonna take my capital and just walk away with it. But no, you have to invest that in helping your peers and working with your peers.

Boz: And then I know something else Sharona that you brought up when we were doing this last week, is also that concern of this student conversation being kind of dominated by those, you know, just outgoing, you know, more a type personalities. How do you make sure you’re incorporating, more voice than just those that are the most comfortable talking and the loudest voices in the class.

Sharona: I resemble that remark, just saying.

Sharon Stranford: Yeah, yeah. No, I hear you. So I do a number of things. I think, as you mentioned in the previous podcast I have a lot of this process starts with them meeting away from me, in a mentor session, for example, talking mentor session. So they get a chance to talk to, and I have the mentors structure that, so they’re in small groups, so they’re talking within small groups about like, how could we write this criteria or tweak this criteria? And often when they’re in smaller groups with each other, they’ll say things. The shier students, the quieter students will say things that they wouldn’t say in the bigger group. So that’s, that’s one avenue. Another avenue is to prepare for this, some of their pre-class questions. Ask them to tell me things that they think should be in the criteria or changes to the criteria. And then that they get to do very privately. I’m the only one that sees that.

And so now I can use that as a way to kind of drive a conversation in class or add something to the criteria that I think, you know, should be in there because I know it’s a particular concern for an individual student. And then we come into class with, you know, a criteria, a draft in front of us, and again, put them in small groups. And I try and mix up the small groups for that sort of thing, not the same people they’ve been talking to, but put them into different groups. And then I ask some of the groups to report out, like, what did your group talk about? So that we’re not, again, always dominated by the, those students who are more comfortable raising their hand and speaking in class. And I’m pretty confident that between those three different ways, they can just write to me personally. They can talk with each other away from me. They can talk with each other in class, or they can raise their hand in class that I’m getting a diversity of viewpoints. And then we also all agree that the document is a living document and we come back to it again later and we have occasionally had to tweak it like, okay, we’re now a month into the class. This one isn’t feeling right. We have to change that a little bit.

Boz: Okay, I just have to point something out that is topic adjacent, but. I had a pd about a week and a half ago that I was facilitating and someone was challenging one of the activities that we were doing that we call TIPS, think, ink, pair, share, and their reasoning was, we don’t do this kind of squishy crap in college. You just described a very detailed tips process that you used to set up your entire class structure, so, yep.

I’m sure that particular teacher doesn’t listen to this podcast, but I wish I could play that for

Sharon Stranford: Yeah, yeah. I mean, it’s, I don’t wanna waste time with my students in class, but I actually think it’s good to have those conversations, interactions. I wander around the tables, I listen to what they’re saying. I have mentors wander around the tables. I often pick up things that students have said in those conversations that nobody would’ve raised their hand to say. And then I get a chance to say, oh, okay, let’s talk about this. And I don’t have to attribute it to an individual student.

Sharona: I just, I love all of this.

Boz: We are coming up kind of on time and so Sharona, what are some last minute things that you have? ’cause I know we kind of started this journey even though we’re hoping that our listeners are getting a lot outta it, but we really started this journey because this is your personal journey right now, Sharona.

Sharona: So it really is, and I’m just, honestly, it’s a whole illusion, the podcast. It’s just me getting personal coaching, like. The only thing that really I’m thinking about right now, you said a couple things about being high structure and I’m very high structure. The one thing I don’t relate to is the outside of class group work. ‘Cause I don’t do that. Mm-hmm. I don’t believe in the classes that I teach specifically, that group work should be part of your grade other than as a sort of individual participation component. So I’m doing that, but just could you touch on for just, we have a couple minutes at the end of the semester.

Sharon Stranford: Mm-hmm.

Sharona: How do they know or how do they, if they wanna get that A, they wanna get that B because you’re still giving an end of semester grade, right?

Sharon Stranford: Mm-hmm. Yeah.

Sharona: How do you come up with that? Is there like a baseline criteria that’s in the rubric and then you have them suggest something? Or like, how does that work?

Sharon Stranford: Yeah. So I have them suggest a letter grade for themselves at each of those three summit of assessments, so we don’t wait till the end. And that was the lesson I learned from my students who mentioned the bait and switch. You know, like you can’t not talk about grades all the way through the semester and then shock us with, okay, now let’s talk about grades. So I think it is important if you’re gonna have a class that is graded, that you have some conversations before the end of the semester about an individual’s like level of mastery and where they would fall on the grade scheme. ’cause I don’t want them to be shocked. So, so they, so they’ve told me, so we’ve had three conversations now by the end of the semester, this is our third where we’ve been talking about grade. So sometimes it’s really clear there’s been a lot of growth. And so we’ll talk about that. Like, you and I were grading you at like a c plus or a B- , in your first, and now let’s talk about how far you’ve come. And we built growth into the criteria like we built, I’ve made big leaps into, I’m gonna give myself credit for that. And then sometimes we’ll have to talk them into, you’re an A or you’re an A minus or something.

Other times we have to go in the other direction where they really, really want that. A, but I have to say, so looks like you didn’t go to mentor sessions, didn’t meet with your group outside of class, and you know, this was a requirement. And I have them set up times when they’re available to meet. Like, I make sure the groups, sometimes I form the groups based on when they have overlapping time to meet. And that could be by Zoom, you know, then I have to do it in person. But you know, you didn’t do that, you know, and, and we talked along the way about how that was something that was important or you didn’t come to office hours. And I, I tracked that even though you were struggling some of the material. And so we have to talk about how like you had all these opportunities and you didn’t take them. And, it’s the rare student, honestly, who, who doesn’t step up to it because they see the writing on the wall through the semester as they’re we’re talking about their grade. I have to honestly more often talk the students up in grade and, and you, as you might imagine, it falls along the demographic lines that you probably would picture.

And sometimes I even have to say like, we are having conversations that I don’t always have with students. I can throw something at you from the literature I just read and you can have a conversation with me. You’re speaking immunology, you’re doing very well. And so stop docking yourself for all these little things along the way that didn’t go well. You’ve gotten where you need to be. So most of the times they’re really good, joyful conversations where the students walk away feeling affirmed if they’re not getting the grade they hoped for. They usually saw that coming from prior conversations

Boz: .We are coming up on time. I do want to read something directly off of the blog because I think this is such a important message, but in the blog, in your final part, it’s called Final Thoughts you said you spent more time on feedback and gave up complete control over grading. Students spent more time revising responses, reflecting on their learning. They also began to notice that the learning cycle didn’t work if they weren’t allowing themselves to make mistakes. This has been a common thing that’s been, you know, really since we started this podcast, the importance of mistakes and what role they play in learning and in the learning cycle, and. What traditional grading, like you were talking about earlier, the students just refusing to believe that they deserve a higher grade because they made mistakes early. Just what Traditional grading, no matter what you’re saying, no matter what you’re doing in class, how that practice is just making it impossible for students to appreciate and learn from mistakes and the power that mistakes really have. ‘Cause we don’t, we don’t learn without mistakes. I mean, if we’re not making any mistakes, it’s not learning. It’s learn. We, we already knew it.

Sharon Stranford: Yep. And I don’t know about you all, but students, my students often don’t want to reveal to the professor what they don’t know. Right? Like that’s the last thing they want you to see is what I don’t understand. So I have to coach them a lot. Like I can’t help you if I don’t know where you’re confused. So I have to give them a lot of positive feedback early on for revealing to me, like what they really do and don’t understand. Some of that happens in pre-class question feedback, but it happens a lot also when we meet. And I also have to give them opportunities to recover. So as an example of that, like there was a time when I had a student who a summative assessment didn’t go the way that they had hoped, and they had clearly not put enough time into it. And they realized it when they got in the meeting with me and were very confused about a couple of core concepts, like had them completely reversed in their mind and were pretty appalled with themselves at that point, but realized like they had done the first round, they had resubmitted and still hadn’t done the what they needed to do. And said to me, I’m really unhappy with myself at this point. What could I do? And so we came up with another plan, which was, okay, like these are the two concepts. You really have confused. I have some other questions in a database. Do you want me to like, just, do you wanna try again? And they were like, would you let me do that? And I was like, sure, if you wanna put the time into it, why not?

And so they went away, took those questions, came back and met with me. We had this whole big conversation about it and they were like, okay, now I think I really get it. And it was because they didn’t wanna walk away with a lower grade. They wanted the higher grade, but they realized they had not put the effort in during the process of that, I think it was the middle Summit Assessment to get there. So they went the extra distance to, to redo it in a new format and prove to me they understood it. So then I was like, okay, you’re good. You’re good to go.

Boz: Yeah, the learning by the mistake, it doesn’t matter if they didn’t get it the first time, they’ve got it now, so why not? But.

Sharon Stranford: So you have to give them those opportunities. The hard part is at the very end when you can’t do it anymore.

Boz: Yeah.

Sharon Stranford: Right. So as long as you have opportunities to say, okay, what could we do about this? And I brainstorm with my students and we often come up with a list of like, so what could we do at this juncture? The hardest part is at the very end, if they’ve waited until the very end and now they’re trying to recover something and so, I start warning them ahead of time that it’s, we’re starting to get to be too late for that process to happen.

Boz: Yeah. All right. Well I wanna thank you so much, Sharon for coming on. This has been a great conversation. Sharona, any last parting things?

Sharona: Just thank you for sharing the details because this is how we learn from each other and make this work better for all of our students.

Sharon Stranford: Yeah, I love this. The alternative grading community is just so amazing. I’ve learned so much, and I can’t wait to hear how it goes Sharona, so you’ll have to let us know how the process goes.

Sharona: Oh, keep listening to the podcast. You’ll probably get a blow by blow or play by play.

Sharon Stranford: I will. I will. And let me know if you have any questions. I’m happy to talk. Sure.

Boz: All right. Well, I wanna thank our audience. You’ve been listening to the Grading podcast with Boz and Sharona, and we’ll see you next week.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode’s page on our website, http://www.thegradingpod.com. Or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the Contact us form on our website. The Grading podcast is created and produced. By Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.

Boz: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State System or by the Los Angeles Unified School District.