The Grading Podcast

Eden: You know, maybe you get chronic migraines and it just so happens that the testing day was one of those days that you couldn’t get out of bed. So I really like how much flexibility it provides for my disabled students, for my students who have a lot going on at home, who are working at the same time, who would fail I think in almost every other context if they had to sit this exam, even with like drop grades or whatever, right?

Like, it just, life happens. And so I really like that there’s flexibility.

Bosley: Yeah, our students do have lives and responsibilities and things outside of our classes. Welcome to the Grading Podcast, where we’ll take a critical lens to the methods of assessing students learning, from traditional grading to alternative methods of grading.

We’ll look at how grades impact our classrooms and our students success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist, and instructional designer. In the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach and instructional designer.

Whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Bosley: Hello and welcome back to the podcast. I’m one of your two hosts, Robert Bosley. And with me as always, your other host, Shana Krinsky. How are you doing today, Sharona?

Sharona: Well, I’m a little tired this morning. We’ve been here all weekend at CMC South, which previous episode would probably have been. from CMC South, but now we’re going a little further afield and we have another guest that I am extremely excited to talk to. Dr. Eden Tanner is an assistant professor at Ole Miss, right? And her background is amazing. I was introduced to Eden through the intentional teaching podcast. assistant professor of chemistry and biochemistry. with degrees from Oxford and the University of New South Wales, postdocs at Oxford and Harvard.

And her research interests are solving biomedical and bioengineering problems using physical chemistry, and particularly ionic liquids and nanomaterials. However, that’s not why we’re talking to Dr. Tanner today. Because she also has some thoughts to share on grading. So welcome,

Eden: Eden. Thank you so much. I sure do.

I’m so excited to be here and to just take the time to really reflect and share my experiences with alternative grading and with introductory STEM courses.

Sharona: That’s fantastic. So can you share just a little bit more than the two sentences, like what is your research and then what’s going on with your, grading and pedagogy a little bit?

Yeah,

And instead, I really envisage my classroom as a space where everybody can come and appreciate chemistry no matter where they intend to take it. And hopefully inform, because I’m often their first college course, sometimes I’m their very first class, how they engage with STEM and. there at college education more broadly. and so that’s kind of what has brought me to, alternative grading methods as a way or an important part of accomplishing those goals.

Bosley: That’s really interesting that you say that you’re kind of philosophy about your class. Cause I know when I was, in college chemistry definitely was what we called a weeder class.

It was meant to weed people out of college. that was the intention of the class and That class had an incredibly high fail rate, which was kind of the purpose of the class, sadly to say.

Eden: Yeah, and that’s incredibly sad to me, because there’s no reason why it has to be that way, and I think it really gets in the way of people’s learning, firstly. but also, just the appreciation for the beauty of understanding the world in this way.

Bosley: So how did you kind of start this journey of alternative grading, especially since, like you said, you’re a research faculty. So, your pedagogical training was minimal, to say

Eden: the least. Yeah. Slack non existent. I think you could say, so I had taught. Some, but I’d never been an instructor of record. And so I’d never been in charge of assessment before, or really course design before starting this job. and I will say I did start with graduate level physical chemistry courses. And so that gave me a little bit more flexibility because my classes were kind of five to 10 students instead of hundreds right at the beginning to at least experiment with, More exciting, I would say, forms of assessment engaging with the material than just doing five exams and having that be it.

And so that a little bit informed going into kind of teaching general chemistry. but I was just unsatisfied, I think, with firstly the data. So looking at the data from the last 10 years of attainment in general chemistry, it became clear that certain demographics of students were succeeding while others weren’t. and that was regardless of instructor, regardless of, a lot of other factors, and so in particular, our African American and Black students were not being given A’s at the same rate as our white students, and they were being awarded D’s and F’s at much higher rates. rates. And so that was deeply concerning to me, and because it didn’t seem to matter who the instructor was, it became clear to me that if I continued programming as usual, that I would add to this problem.

And that was kind of unconscionable to me. I thought I can’t, even if I try something and I fail, better to have tried some alternatives. And to see if we can move the needle on this, our failure as educators, really, to be able to give everybody equal access to education and to chemistry, and so that’s where this kind of came in.

And as I was engaging with more traditional assessment, the idea of the percentages and the points and the one and done exams, it became clear that I was losing people because they weren’t all coming in at the same stage of being prepared. And so with the traditional assessment models, really, after the first exam or two, you basically lose them forever, and there’s no real chance to get them back, or you have people who aren’t quite prepared for what college entails, and so they kind of fall off the wagon a little bit in the first half of the course, and again, without flexibility and this kind of alternative method.

For them to succeed, we would just lose them, and they would just fail. And so that’s kind of, I guess, my origin story is looking at the data, which was bleak, and then looking at my experiences, in the first semester of teaching, which were also bleak, and thinking something has got to change here.

And then engaging with the literature to say, okay, so if something has to change, what is there available that other people have looked at before that’s been successful? See, and

Bosley: we always ask a new guest that question and more times than not, it boils down to like one specific student or one specific story that just.

Tips them over, but it’s interesting. and I thinking this is maybe the researcher in you, but I think you’re the first one that we’ve had that said they came into this after looking at data and just going, this data is a nightmare. I have to try something different. So that’s, I mean, correct me if I’m wrong, Shona, I don’t think we’ve ever had a guest that came into it

Sharona: that way.

Well, I think that one thing that struck me from what you just said is you not only looked at the data, because a lot of us have seen that data, right? And the first reaction is, well, that’s everybody else. Everyone else just is a terrible teacher and I’m a great teacher and that’s the way it’s going to be.

But you looked at it and said, it didn’t matter who the teacher was. And so your self awareness was, I’m probably going to be the same problem. And that’s where I think the jump was. I suspect for you is you didn’t have that whole, well, I’m going to be fine. Everyone else is the problem. So I love that.

But yeah, no, we haven’t heard someone say that. So you said you looked at the literature, what’d you do then? Like what was your first thing that you decided you were going to do or the first set

Eden: of things? Yeah. So I guess where I started. was with this idea of Amnesty Week, and so the last two weeks of class I started by reopening all of the assessments for everybody, so I had a student who basically didn’t realize that our learning management system existed until week eight, and so he’d done nothing, and he did end up getting an A in that class.

And then he ended up, serving as kind of like a teaching assistant, they call it supplemental instruction. But he ended up doing fantastic, but he just wasn’t oriented. And so, that was a big help, is like saying, okay, well I’m gonna reopen it for everybody, it’s equitable, you all know it’s coming, it’s in the syllabus, as a kind of like a Hail Mary, if you need it.

So Amnesty Week was probably where I first and then I moved into including metacognitive activities as part of the assessment. So reflection activities at the end of every week, and then also including, I mean, this is, I guess, more of a flipped classroom model, but including pre class work, to kind of complement the in class work to try and catch the folks who maybe were a little bit less prepared.

And all of that was for completion rather than for. Like, you just have to show up and, act alive, and then here you go. And then I moved to all of the, assessments that we were doing, basically having unlimited tries within the homework or the quizzes or the in class work, And originally, I kind of kept exams out of it, out of my assessment methods, but I noticed that my students, were very anxious about the final, because so the other piece of context is, we have to give the ACS final exam as part of our accreditation, so I have no flexibility on what the final assessment looks like, and so I learned that I actually did them a big disservice by not having any exams, because when they got to the final, they didn’t know what they were looking at.

And so then I put the exams in, but then I thought, this isn’t really working because they just kind of cram for the exams for three days, everything leaves their brain. And then, maybe they don’t do well on particular concepts. And, but it’s like, I’m never going to talk to her again. Like, I got the stoichiometry problems wrong.

Cool. I guess I’m just not going to know stoichiometry. Cool. And so the biggest leap that I made was last semester when I transitioned to allowing them these kind of unlimited exam retakes, where I would make five or six unique versions of the same exam, testing the same content. And then they could. sit the original exam in class, they would work the corrections with me or with someone else to figure out what they did wrong. And then they would come and take the exam again. and they would do that kind of on repeat until we figured out what are the underlying things that you’re missing that are stopping you from getting this right. and then they would take their exam again and again until they kind of got the score that they wanted. And so that’s kind of where we’re at now. Except, we’ve also transitioned to kind of a fully mastery framework, where you essentially complete a portion of the work to get a certain grade.

It’s somewhere between, I would say, mastery and contract grading, because they essentially just have to show up and complete all of the other activities, but they do have to get a certain score on the exams to get, for example, an A in the course. But they have unlimited, essentially, shots on goal to get that.

Sharona: wait a second. You just said you have 130 to 170 students per. Semester with unlimited assessments. How the heck do you manage that

Eden: workload? Yes. So the last semester was unmanageable. That is for sure. But it was just an experiment, right? So I just wanted to see, if I did this, would it make a difference?

Would it measurably move the needle for these students? Because I think you can kind of, Hobble through 12 weeks, and just see if it makes it makes a difference. And so that was an enormous amount of work because I was doing written printed exams. I was hand grading everything. I was their main point of contact.

They weren’t really going to anybody else to understand what they did wrong. And towards the end of the semester, it got really, really tough. And so I basically talked to the folks at my institution. So Josh Eiler and the other folks. at our center for teaching and learning and kind of brainstormed with them about how I could make this sustainable.

And the answer is really moving the exams online because they’re already multiple choice because I don’t have any other grading support. so I can’t reasonably, even if I only did one exam, grading 130 short answer responses. would not work by myself. and that has really changed the game. I still make unique exams on Blackboard, which is our learning management system, but now I’m not spending hours and hours grading every night, they get immediate feedback on what went wrong, and I can also see as an aggregate the kinds of questions that they’re getting wrong and getting right, which I couldn’t do as easily when I was hand grading and kind of managing all of that by myself.

So it’s become a lot more sustainable since I’ve moved the exams online. They do sit the exams still proctored, so they either come to me, or to, we have this help desk that’s staffed by graduate TAs. or to the testing center, so they’re still doing the exams in person. They still upload a picture of their working, so they’re still working it out on paper. but the actual submission of the answer. And the grading is online.

Bosley: Yes. And, you bring up a good point and a common mistake that we’ve seen with new practitioners is setting things up in a way that sounds great and you want to be as. Open to your students as possible, but there is a limit.

You can drown yourself in grading if you’re not careful. And, we did that, Sean and I made lots of mistakes. our first couple of times, that was one of them, and we talked about this in one of our early episodes, the importance of finding and utilizing, tools that are going to help you.

manage some of this. So you said you’ve gone to online testing, which is one of those tools that I think is underused. I actually tell all of anyone that’ll listen, that’s one of the changes I made because of pandemic, when we, especially out here, I don’t know how long, you were out for pandemic, out where you’re at, but here in California, we were, remote for a year and a half.

I mean, it was a really long time, but finding in ways that I was satisfied with the online testing and was comfortable with it. I’ve never gone back. I’ve gained so much instructional time because I do all of my assessments online and actually don’t do most of them, even in class time.

Like, I’ve gained so many instructional minutes that I used to have to give up because of

Sharona: the structure. But ours are all short answers, so, because of the structure, so they’re not proctored. They don’t have to be because we have extensive versioning. Yes. So. That’s the thing. But I also love what you said about a testing center. can you talk a little bit more about what that is? Because that’s come up several times in

Eden: conversation here. Yeah. So we have this testing center that’s run kind of independently, I guess, from each of the individual departments. And they have this kind of bank of computers, and they’ll essentially provide what the students need to be able to do the exam there. so the And the calculator. And so that was really important for me because if I’m going to move to using online testing, I don’t want to do that and then have students who don’t have access to the technology not be able to engage with the assessment and not be able to engage with the assessment in the same spirit as their peers, which is that they can go and book in and do it as many times as they need to. so that was really important, an important point of equity and students can actually go and book into that testing center to do the original in class exam too. So let’s say they’re traveling. Or they don’t want to be in the lecture theater with 130 of their closest friends crammed into the little seats while they’re taking this exam.

Every single student could choose to go to the testing center and take it there, which I really like. And I still do run the exam in class time, because for me that’s also kind of an equity issue. Because I want to make sure that, if they’ve allocated this hour for chemistry, And they’re otherwise working full time or they are looking after other folks that they still, are able to have the time to take this assessment. but the testing center has been incredibly valuable in terms of me feeling a little bit more relaxed about the equity piece of, okay, but does everybody have a laptop or an iPad that they can actually access this assessment on? Do they have a quiet space to kind of take the exam? So by having this physical location they can go to, I think that really eased my mind there.

Bosley: Yeah, and that’s another one of those tools that is really common to most universities that, again, I think is, Fairly underused and can be a really powerful tool. unfortunately, with my K 12 world, they’re a little bit less accessible just because there’s no time during the day that the students aren’t supposed

Eden: to be somewhere.

Sure, that’s right. Yeah, so it’s for sure. And, you know, thinking about, in this whole piece, thinking about what matters and what is the high value activity that I can spend my time doing, it isn’t really grading multiple choice questions by hand. That’s, the lowest value. my students don’t really get anything out of me grading by hand. and so that preserves more time to engage with them on the actual content of the questions that they missed. And so that was the key thing for me. It’s like thinking about making it sustainable. What can go? Well, the things where I’m not actually engaging. With the students anything that’s like that we can try and energy minimize to make this more

Bosley: sustainable Yeah, and that’s one of those key things, Find the stuff that is most impactful to the students.

That’s where you spend your time Anything else find ways around it or tools to help you with it when people keep talking about the time and the energy that some of this stuff might take. That’s, what they’re not getting. I’m like, no, there’s tools to do a lot of what you’re doing.

Stop making life so hard on yourselves.

Eden: that’s right. And so ultimately I think this version of what I’m doing is really sustainable. so I’m, pleased that I was convinced by my colleagues to let go of the paper assessments because I had this real idea in my head that, without the paper assessments, it wasn’t going to work.

But I will say, looking at the data this semester compared to last semester in terms of, how long it takes students to get, essentially perfect scores, it’s similar. So they’re still taking about three exams, whether they’re doing them on paper or whether they’re doing them online, it’s still about the same, my averages have not skyrocketed since we’ve gone online, it looks like they’re making the same amount of progress, which gives me confidence.

So I don’t necessarily regret doing the paper exams the first time, because now at least I have almost a control group to say, no, being online isn’t hurting the kind of learning process of doing this iterative exam, which I was would have been worried about without having the paper version to compare to.

And also, for my colleagues who who might be a little skeptical. I’m a tenure track faculty member, so I’m still pre tenure. And so I really do have to kind of prove myself as an educator, as well as as a researcher. I think having the control group to compare to is useful, but yes, definitely not sustainable and definitely not required for me to hand grade. at that time I had 170 students in my class. and by the end of the semester, they took 800 retakes. That’s not the original exams, 800 retakes of the exams and every single member of my class retook at least one exam. So,

Bosley: yeah. That is a lot of things

Eden: to get through. And I was furiously grading them and I just wasn’t able to keep up and they were of course frustrated because they wanted feedback quickly.

And I just wasn’t able to, I promised them a 24 hour turnaround, which I made, but if you’re in the last week of class and you’re trying to retake all four of your exams, having to wait 24 hours in between each one, like, that’s tough. And now they don’t have that problem. Instantly, when they leave the exam, they know how it went.

They know

Bosley: exactly how it went. Yeah.

Sharona: So I have, questions based on what you’ve said. So first of all, commenting on the three times, because we’ve seen that now we’ve had about 8, 700 students go through our general education, quantitative reasoning with statistics course, since it’s been redesigned to use mastery grading is what we call it.

And we also have seen pretty distinctly that try number three is the magic one. If they don’t get it by try number three, they need a more intensive. intervention or experience, or they have to do something differently, or they’re not going to get it from three to four. Like they can learn on their own as long as they’re learning, and it could take up to three but past that.

So commenting on that a little more useful. And also, you clearly have not stopped. innovating and iterating this since you started this, is it working as far as your original goal, your challenge to make chemistry a gateway instead of a gatekeeper or weed out? Are you seeing progress? And that’s why you’re continuing to iterate it.

So those two questions three times.

Eden: And is it working? Yes. So, I agree with you that essentially if we go past three, that usually communicates to me that we need to do something different. And usually, thankfully, the students who are on their third or fourth attempt usually will reach out to me and say, This isn’t working. What’s wrong? And then we can sit down and actually go through And it’s not a surprise that it’s the same questions they’re missing on every version, right? So we can go through and I can say you aren’t able at the moment to balance an equation, and that’s what’s going wrong with all of these questions.

So you’re right, that it does take extra intervention after the third attempt, and that’s what I’ve seen too. The highest number of attempts I’ve seen has been seven. Until they got an A, and I think that they were not, stopping and pausing and reflecting, necessarily, between those later attempts.

They were just kind of doing what they needed to get done to re sit the exam again and again and again, and it was the same questions they were getting wrong. and I guess maybe the point of the mastery assessment, is really that it reveals what students don’t understand. Because if you get it wrong on the third attempt, you do not understand something that’s fundamental that’s underlying all of those concepts. and I think maybe that’s the real power of it as an educator, is I get data on what is the missing piece. and I have actually had other people ask, why not just limit it to three then? If you know that most people get it in three attempts, why only just make why not just make three versions? and I think that there’s something about the optimism of knowing that it’s really in your hands as to how hard you want to work. That it’s possible to get there. It’s possible for every single person. Whereas I’ve thought about putting a limit on it. And I think it just destroys some of the, kind of magic of knowing that you could learn it. There’s time for you to learn it. There’s time for you to master it. and to the second point, am I doing this because it’s working? Yes. I think I only failed one person. Out of 170 last semester, and they did not show up to the final, which is why they failed. So I only lost one student to pessimism.

Like I only had one student to give up. That’s,

Bosley: and that was, what did you say, like 130

Eden: semester section? I had one student out of 170 Give.

Bosley: that is in a gen ed first semester chemistry class. that is incredibly

Eden: impressive. And I should also maybe preface this by saying I’m not selecting my students in any particular way, shall we say, that there is no.

We welcome all folks. This is not like a special section of chemistry that is reserved for people who are super prepared almost the opposite. It usually ends up happening and my section is often the biggest of all of the sections. So the next biggest section was, I think, 110. So I’m dealing with a lot more students and that means that a lot of them Are less prepared, shall we say, from their secondary experiences coming in than some of my colleagues sections.

So that’s, to me, that’s the thing. They didn’t give up. They didn’t give up. And so the proof will really be in the pudding as they progress through the rest of their college sequence. and so what I’m really paying attention to is how they do in organic chemistry and importantly how they do in physics because it’s the same kind of problem solving that they’re going to need to use in their physics courses.

Organic chemistry is a little bit different with the way that they have to use their brains, right? So now I’m watching those students who graduated from my section go through this semester of physics and organic chemistry. And so that’s what’s really going to tell me, is this.

Bosley: So is your chemistry class a class for majors or is it a gen ed class?

It

Eden: is a class for majors. but I should say STEM majors rather than chemistry majors. I think maybe I have one or two chemistry majors out of 170 in my class. but it is a class for people who are in a STEM. degree. So very, very rarely I’ll get, for example, business majors or philosophy majors. taking my class.

It’s usually people who are pre med, pre health in some way, so they do need a solid foundation in chemistry, but they’re not necessarily chemistry majors.

Bosley: I’ve pointed this out on as many episodes as I can, but I’m going to continue to point it out. I’ve always said what has sold me on this and why I’ve never gone back, is the conversation I have with my students.

And it sounds like you have similar conversations, before I. transition migrating when it came close to the end of a semester, the conversations were always, how do I get more points? Is there extra credit? how do I get my grade up? But it was always about the point, mongering or the point, chasing, It wasn’t about. The math and it sounds like you’re having those same kind of conversations that it’s about the chemistry. It’s what am I doing wrong? Okay, we’ve missed this question. a similar question multiple times. Let’s look at Oh, it’s it’s the balancing equations like having those conversations about actual chemistry, which as an educator, I love those conversations.

I hate the, okay, let’s see if we can gamify the points. Where do you need a few points to raise the grade? Cause it’s nothing about learning. it’s just about that game of point

Eden: chasing. Yeah. And you’re right. I haven’t thought about this deeply, but you’re right. The kinds of conversations I have with students are much more focused on content, or skill acquisition, and much less focused on like, what do I need to do to pass this class?

I’m like, don’t even worry about that, let’s just focus on figuring out what’s going wrong in this exam, and you’ll be good. Which, it is refreshing actually thinking about that, that I don’t have to, and the other thing that I really love is firstly, I’m not a cop. And so I don’t have to be a cop, so that’s really nice.

And if students need flexibility, I am not the arbiter of whether they deserve that flexibility. So, I’m never gonna be an instructor that asks for a copy of someone’s family member’s obituary. that’s just never gonna happen, because it’s, I can’t, like, I just can’t. I can’t bring myself to do that.

And because of the way the retakes are set up, if they need to leave and they can’t take the exam, who cares? You can get a zero on the in class exam and just retake it next week. And retake version one. Like, nobody, like, you know, your best friend’s cousin’s dog’s sister’s wedding? Enjoy, have a great time, Maybe you get chronic migraines, and it just so happens that the testing day was one of those days that you couldn’t get out of bed. So, I really like how much flexibility it provides for my disabled students, for my students who have a lot going on at home, who, are working at the same time, who would fail, I think, in almost every other context, if they had to.

Sit this exam, even with like drop grades or whatever, right? Like, it just, life happens. And so I really like that there’s flexibility in there.

Bosley: Yeah, our students do have lives and responsibilities and things outside of our classes. Especially. at the higher ed, but even, my older students in the, in the K 12, they’ve got, the population I teach in, they’ve got responsibilities. they have to help with jobs. They have to help us siblings. So, and I think as educators, we have sometimes have so much passion for our subjects that we forget that. Our subject isn’t the be all end all for everybody like we might want it to be. So, our students have other priorities and a lot of times those priorities are real life need priorities and traditional grading, will punish a lot of that.

Eden: Yeah. And I also just like how it shifts the roles, right? I am not the person who is deciding your grade. You are. You decide. I’m the person who can help you get to the grade that you want. I’m like the coach. I’m the person who walks alongside you. I’m not the cop who’s like show me, you know, perform to my exacting standards.

Otherwise, you’re not going to get an A, you know, it’s like, I’m, it just changes our roles. And that’s what I really love because we have very different relationships. Yeah, Sharona,

Bosley: you’ve talked a lot about this, Sharona, about that kind of relationship. how that shifts with this kind of grading. So you want to talk a little more about

Sharona: that?

Yeah, well, I’m second generation in math education. Essentially. My mom was a math educators, had a PhD in math education, but operated out of an actual math department at a time where it was very unusual. so she has a master’s in pure math. originally, I went into college to get a genetics major. and I will tell you, organic chemistry is the reason I am not in a genetics major combined with the fact that I didn’t end up loving lab work. So it really was organic chemistry. which is why, what you said about you’re watching, I am really curious that they have this one great experience of what it can be like.

Is it even worse going back to traditional after that? Or have they learned how to learn enough that they do better? I would love to hear from you to see what happens. But when I did decide to go into into math education, and I went to go get a PhD in pure math, because at the time, you had to have a content.

PhD in order to really get into a discipline department. I didn’t finish because I didn’t really want to do the math. I want to do the math education. So I went away and did a bunch of other things. But even when I came back, I wanted to be that teacher that changed someone’s life. And I couldn’t figure out how to do it. Like it wasn’t working. I mean, I was good and people liked me, but I wasn’t having that sort of, life changing affirmative for me and for the student experience. And I tried it all, again, second generation, my mom was on the cutting edge of active learning.

She was such an inspiration to me. She worked with the K eight school teachers. I wanted to do all this and I’m like, I’m going to take all this on. And I did it and I flipped and I active learning things and I was approachable and nothing worked until I switched my grading. And now on a daily basis, I get to be this, human being who shows up for other human beings.

And I get to recognize them as people who are attempting a difficult task. And I have. thousands of students to show that I can do this and I can help them and I can coach them. So, I love what you It’s

Eden: just such powerful reframing and I really love the conversations. I mean, I don’t love the fact that students come to me upset with how they went.

That’s sad, but I like that we get to have the conversation where I say, Your performance on this one exam does not tell you or me how good you are at chemistry. how much of a scientist you are or could be. or your aptitude for chemistry at all. It tells us about your current level of understanding of these particular concepts at this hour on this day, which is completely changeable.

Sharona: I love saying to students at the end of the semester, when I do have some who aren’t going to make it through and they come back and they’re like, they’re so far behind, they can’t possibly catch up. But I will say to them, you still need to start coming because right now you can start setting up for next semester.

You can learn the end part of the course. And then please come back with me next semester and let’s keep going, you had some big life event, mental health, whatever you were gone for eight weeks. Yeah, that’s pretty much not, I mean, you could still recover the grading system would allow you to, but realistically, all the things in your life haven’t changed and made space.

But okay, fine. So let’s learn the last five weeks together and then come back next semester. And you’re just going to be starting up on a stepstool, and I would love to have you back. And I will look at literally look at them and they’re looking at me like. But I’m failing your class. How could you want me back?

I’m like, because I do, you know, and that’s been incredibly

Eden: affirming as well. Well, because it’s not like you are not valued by a grade or a number, right? You have inherent value as a human being who is, as you said, It’s trying to learn something difficult.

Bosley: Tell Eden how you do your grading and kind of why, because that’s exactly what she was just talking about.

How I what? How you do your proficiency scores.

Sharona: Oh, yeah, so I use emojis. That’s great. I don’t use words. So, You’re successful is a check, you’re done, complete, whatever. I mean, I tie words to them, and then some of my classes I allow revisions. So I have the little hand with the pencil, meaning you’ve got to go rewrite something.

And then the, you’re not there yet is the yellow smiley face with the thinking hand going, hmm. And then if you didn’t turn anything in, you do get a red X because that’s the, you didn’t give me any evidence. That’s not good. You know, you

Eden: need to try to give me evidence.

Sharona: And so that is. I don’t like words. I don’t like numbers.

Because there’s so much value. I mean, first of all, as a mathematician, if you put numbers in front of anyone in the world, they will try to do math on them. Even if they’re like, the only time they don’t is, you know, is a five star restaurant, a four star restaurant. They don’t say that the restaurant’s 1.

25 better than the four star. So, and that’s what grades are, right? Grades are actually categorical data that we misuse as

Eden: quantitative data. That’s right. And can I say my dream is if I were able to figure out a way to support. some grading is really to do the specifications grading with this kind of system, where instead of having multiple choice exams where they get a point if they get the question right, is like each individual component of getting the question right is like, yes, you got this step right.

We need to work on this step, right? Like, so that is the dream for me, is to be able to untangle the four or five different concepts that go into a single question. And at the moment, I’m kind of just stuck with knowing that they have to master all five of those. For them to get the question right. And I have to just intervene in between to see where I lost them, if that makes sense.

And just encourage them to say, look, you got four fifths of the way here. but I love the idea of being able to do that with the emojis, right? Because that’s, you’re absolutely right. If you put a number on there, if you say you got a zero on this question, they don’t see that they got almost all of the question right. they don’t see that.

Sharona: so now I do essentially hand grade, they take the assessments online, but they’re not multiple choice, but I am at the point now. my assessments have really honed in and I have a hundred students, but that’s a class for sections. So. You know, it’s not one class that’s 130, but I use rubrics.

And then it’s like, I can look at a piece of student work and know instantly, is it getting a check, a revision or a cloudy face? And the reality is in the, in the class where I do revisions, everyone gets a revision unless they’re so far off that it’s like, no, we need to sit down and retry. Yeah. but I am also then holding them too.

You have to get it right. Like I didn’t get to do that, but I had to give them part of the way. And now I want to go all the way. Cause I can, cause I kick back those revisions, but I, it’s funny. I have three different classes that I teach. And all three have completely different systems. So I actually use specs in one of my classes, which is history of math. because I don’t care any particular fact that they know. I don’t care, not a single fact. I do want them to get these broad themes of gender identity over the course of history as identifying as a mathematician, but they do these projects. And the projects have to be clearly written, the mathematics in the project has to be correct, there’s all these things, and oh by the way, you have to have one of our thematic things in your project.

And so they get to pick, do they want to do the numeracy, the numerical systems project first, or do they want to do the gender, you know. So they have a lot of freedom. So I do that, but then I do, basically standards based in my other two classes.

Bosley: And I dabbled for the first time in specs grading last semester with one of my high school classes, which was an interesting experience. it. Had a little bumps on the way, but it was my first attempt at specs instead of standard

Eden: space. Yeah, so I would be excited to experiment with that. and I think we are at the moment, hiring a chemistry education colleague in our department. So I’m hoping if there’s interest from them, I could partner with them to at least have a partner in crime to.

Sharona: Do you have any influence over the lab class that goes with the Gen Chem

Eden: course? Yes and no. So not directly. but I work very closely with my colleague who runs the lab classes and he is just as gung ho as I am about equity and about, alternative. So I would say,

Sharona: yeah, so lab reports are perfect for specifications grading.

Yeah, that’s true. That’s my experience. we have, Courtney sober. I don’t remember Courtney up at, was it Rutgers or I don’t remember where she is right now, but she does specs grading and all of her organic chemistry labs. And I mean, she both tears her hair out

Eden: and we’ll never not do it. So, well to me, it’s like I now approach kind of education the same way that I do my research, which is that, there is expertise in this that I can learn about, there are innovations that I can try and test and assess and see if they work. and I’m not really satisfied. With the status quo, The same way I wouldn’t be in a research lab, right? you don’t write a proposal to the NIH and say, I’m pretty satisfied with the status quo, it’s good enough. a bunch of people are dying, but that seems fine. And the education version of that is I’m failing a bunch of my students, a lot of them are coming from similar circumstances, so clearly it’s not them as individuals, right?

It’s us and our structure, but I guess that that’s fine, because I don’t have time. To teach because I’m research faculty. So they’re just going to get what they get. And that is deeply unsatisfying to me as an educator.

Bosley: I think I’ve said it before, but I have to say it again.

I love how research mindset you’re approaching the grading. I don’t know if I’ve talked to someone that has this kind of just research frame around the why and how, what you’re doing. And it’s really fun to listen to.

Sharona: Well, and I mean, we’ve talked to a lot of people who say, show me the research.

I won’t try to change until you show me the research and you’re coming at it from. It’s clear that this doesn’t work. Like I don’t need more research to know that this doesn’t work. Do I know that my intervention is going to work? No, I’m going to test and try. But when you have something so bad, right?

Unless you are blind, you almost can’t do

Eden: worse. Like I’m not magic, right? There’s no way that I As a completely untrained educator, I’m gonna magically show up and, fix this, doing exactly the same thing as everybody else. And I’m just somehow special. No. No way. maybe it’s possible that I care more about it, but I don’t think that’s true.

I think my colleagues deeply care about their students, and deeply want them to do well. they certainly have a lot more experience teaching than I do. so what nerve would I have to come in and say, Don’t worry guys, I’ve got this. I’m not gonna do anything different. But magically it’s all gonna be different.

Somehow. Like, obviously not. So, as you said, like, I’m not sure that this is necessarily going to fix things, but it is so badly broken that I feel like anything that is evidence based by other people at least giving it a go. is worth it. And I’m very upfront with my students, and I actually say to them, if you would prefer to be graded on a traditional scale, I will grade you on a traditional scale, you just have to let me know. if you don’t want to participate in this experiment, that is cool, because you could sit my course. as a traditional course. You would just not retake any of the exams, you would have the first pass through all of the assessments, I would figure out some arbitrary waiting for all of these assessments that doesn’t really mean anything, and then you would get a grade at the end with numbers, right?

So if that’s what you want, we can do it. But nobody has yet taken me up on that offer to surprise. No, I was just

Bosley: about to ask if anyone.

Eden: And so the one kind of sticking point is that they do have to make a minimum score on the final to get an a. So they have to make a D on the final to get an A, uncurved, which is actually tricky, and so they all lose their minds when they realize that they have to make this minimum grade and they only get one shot at the final. but I think I only had maybe three or four students who took the final and didn’t make the grade that they needed to kind of keep, the grade they had going in. But virtually everybody had an A going in. So that’s the other thing you have to understand is they all mastered the content to a level of an A before they went into the final.

Bosley: so not only are your students passing at a much higher rate, they’re actually passing with much higher grades it sounds. Yes. Because you said that, yeah, that is really. Absolutely.

Eden: So that does expose me to a little bit more scrutiny as an educator, which I’m happy to answer to. Because I think you can honestly say if you have a student who has, it doesn’t matter how many times they had to take the assessment, but it’s a unique assessment and they were able to get 90 percent of the questions, right?

I don’t think anyone could argue with the fact that they know this content and they deserve. As much as anyone deserves anything because grades are silly, but they’ve earned this, they have demonstrated mastery. Well, that

Sharona: reminds me of Joe’s class, Boz, because, so one of our repeating guests who’s sort of like a third co host occasionally is a guy by the name of Joe Zicola.

He’s an English language arts high school teacher that Bosley’s worked with and teaches AP, AP Lit. in Los Angeles Unified. So he’s not in an affluent, high performing private school. He’s in Los Angeles Unified and not in one of their affluent schools. He had a hundred percent AP pass rate last year.

Bosley: He, had a hundred percent take rate and a hundred

Eden: percent pass rate. Yeah. Sorry.

Bosley: Yeah. The take rate is as impressive as the

Sharona: pass rate. So how do you say that any of those students don’t deserve an A when they take the AP, which is worth, by the way, AP Lit’s. specifically is usually worth two college courses, not just one. and they not only took it, but passed it every single student. So by all external measures, the same thing you said, your final is the ACS, right? So how do the other classes that are not doing this? Do they hit that D levels? Most of the students, or do many of

Eden: them not? although I will say that it wasn’t so my year on year scores got much higher for the ACS exam, and when I started I was kind of in the bottom of the pack, not quite the bottom, but near to the bottom of the pack of all of our instructors, and this put me equal top of the pack of the non honor sections.

So we’re not doing orders of magnitude better necessarily than the other sections, but for me, if I’m my own control group, we are. And if we look at the levels of preparedness of my students going in, if I could kind of compare what their predicted scores to their actual scores, they’re certainly much, much higher.

Well,

Sharona: and then just the number of students taking it, I suspect, is much higher. So, because not everyone in the other sections is even going to take the final because they’re already locked out of passing

Eden: the class. That’s right. So I guess I would say on average, perhaps the other section had maybe a 10 to 15 percent non attempt of the final.

Right, so that to me is the impressive part, and it’s also the people who did really well were not the most prepared people, necessarily. I had this one student who got a C, which is a very good, like a high C on this national exam, and at the beginning we were working on multiplying decimals together in week one.

So he had so much catching up to do, and in literally any other section he definitely would have failed. There’s no way around it. There’s no way around it. But

Sharona: that just goes to show the power of it. Yeah. The other thing that occurs to me, though, is that, we work with a lot of engineering faculty at Cal State LA, and there was some messaging that we screwed up the first time we implemented.

Because of the retakes, students go, Oh, well, then maybe I just won’t even take the first one. Let me see from other people what it was like, and it’s okay. And, or I can just do it later. And what we’ve had to teach them is no, the reason we do this is because the material is so hard, you’re going to fail it the first time.

So you have to actually take it the first time so that you can engage in the feedback loop so that you know where you personally are not succeeding. So you actually are going to hurt yourself tremendously if you don’t take it. So it doesn’t matter how badly you do. It matters whether or not you take it.

So we’ve had to have some of those conversations to say, don’t think that the fact that you retake it means. You can just wait because then you’ve locked yourself out of the process

Eden: and that happens, right? And so I will say maybe the two weaknesses of this method are one students are generally less prepared for their first Attempt on the exam than they would be in other sections Which I don’t actually care about, because again, like, yeah, so they’re coming in kind of fresh, right?

This is testing their content without, with like maybe, if we’re being generous to them, maybe a day or two of revision, right? Like they’ve tried, but not the way that they would furiously cram for an exam where they only had one shot. and then the other thing is that even though I bracket the exam retakes, that once the next exam comes around, you can’t retake the first exam until the end of the semester, students do kind of get into a hole of mastering that content and not engaging with the content we’re currently learning.

But again, I’m not sure that I really care about that because I teach them all the content and then they have time to engage with all of it before the course is over. So they can retake that final exam which matches, they can retake the final in class exam, exam four, which is the same content as what’s going to be on the final for like two weeks, thereby studying for the final. so maybe those aren’t weaknesses and that’s, a feature and not a bug. But those are the two big weaknesses that I’ve noticed in this method.

Sharona: Well, it depends what you’re measuring. If you’re measuring their chemistry knowledge, yeah, it’s a weakness, but if you’re trying to actually teach them how to learn, right, then maybe it’s not a weakness because. We have found, what’s the purpose of assessment? For me, the purpose of assessment now is to help students learn. Right. Because that’s when they learn. I actually, in some of my classes, have gone to an extreme. And I actually will, on the fly, while they’re taking the exam, if they’re like, can you come check my work?

I will go and check their work. While they’re still working on the exam and I will look at it and be like, oh I think you need to go back to this step here I think there’s something not quite right and they’ll look at it and they’ll look at it and they’ll engage with me and they’ll be like Oh, and then they’ll keep going like they have aha moments literally during the exam

Eden: Yeah, and I will do that too, you know Like I will not the first attempt but certainly like one of the retakes if they need help with something I’m not gonna give them the answer, but I’ll certainly engage with them knowing that The way that the exam’s set up, like, this isn’t gonna be the last time they’re gonna take it, right?

So, if they are asking for help while they’re doing a retake, they’re gonna take it again, because they only have two minutes of question. So there’s no way that they’re gonna get to a score that they’re gonna be satisfied, you know. So it’s not, I’m not being unfair by helping them, because they’re gonna retake it again independently before that’s their final score. and my exam’s a cumulative too, right? So the idea really is that you get to the end of the semester, maybe, And maybe you didn’t have time to retake exam 1 until you were happy with it. But now you’ve been using exam 1’s content for 12 weeks. So now you can go back to exam 1 and be like, Oh man, this is easy, because I’ve been using this for 12 weeks. so that’s also what I really love about it, is it seems like we’re staggering their learning in terms of they’re not doing as well on the first attempt of the in class exams as the other sections are. That’s True. But, I mean, that’s not really what I’m measuring, right? Like, I’m not measuring how much you know on Tuesday, the, whatever, 6th of November, right?

Like, I’m measuring how much you know by the end of the course.

Bosley: Yeah, that’s supposed, at least in our definitions of final term grades, it’s not supposed to be what you knew, when we ended the first exam for the first exam material. It’s what do you know at the end of the semester? How much of this content have you gotten?

By the end of the semester. Don’t care when you get it, if you get it back in October, or if it took you to November, or if it took you to December. As long as you’ve got it by the time you leave the class, your grade should reflect that.

Eden: I agree. and if it took you to week 13 to get stoichiometry, and then it clicked, great.

Fabulous. Our job here is done.

Bosley: It clicked. That’s all that

But, we’ve been doing this a long time and. One of the things we do a lot of what’s wrong with traditional grading education, because we found that even people who sort of amorphously understand that it doesn’t work, they don’t necessarily know what the problems are. So we do a lot of that training.

These were elite institutions that were in the business of ranking and scoring people against each other. But if we’re going to say that that’s not what it’s supposed to be today, I love, and I know Bosley, you love this, what Matt Townsley said, we want to do 21st century grading, not 18th century grading.

Acknowledging that there’s extremes of either super bright or people with real limitations. Everyone else should absolutely be able to come up to a level that will support the type of citizenry and world that we need. And if that’s our goal, then let’s change our grades until we do away with them, because there’s definitely that group of people, but we’re in the sort of, let’s bring the middle 80%, change it to what we want them to mean, or what we’ve been assuming that they mean, that they

Eden: never met.

Yes. Yeah, because I don’t think people are intentionally thinking, I’m going to go in and rank my students. But that’s what they’re doing. And they’re not even ranking them by, Anything meaningful, that’s the horrible part. if we’re ranking students by their ability to do well the first time they show up, that’s not a meaningful metric, actually. we’re handing out grades to measure affluence, to measure preparedness, to measure, Privilege, basically.

Bosley: Especially when you, on top of that, put, other things like homework and compliance stuff. You’re not measuring their knowledge, you’re measuring their ability to spend a couple hours a night doing your work. so, yeah, you’re measuring. Privilege. because not everyone has all that time. Like we talked about earlier, our students have lives. Our students have responsibilities. They don’t always have, a bunch of time to spend on homework to show something that they already know

Eden: how to do.

Right. And so this has been the real, one of the struggles for me is how do I balance creating real structure for my students? Because often I’m the first. college class they have, how do I balance like having deadlines, knowing that they’re helpful to people, having things like the homework and the quizzes that they can engage with because it gives them the extra practice that they need, without accidentally measuring something that I’m not intending to measure.

So

Sharona: you mentioned, we’re getting up on about an hour now, you mentioned that you are pre tenure. and that is definitely a riskier position for someone to be in than someone with full professorship tenure. How have you been either supported or questioned in your department, as we start to wrap up here?

Eden: Yeah, so I think, on the whole, my department’s very supportive but nervous. now they’re less nervous now that they’ve seen the ACS scores from the spring. Now they’re more excited, by seeing that it’s kind of working. but yes, it was nerve wracking for everybody. me being the volunteer to go out and do this. there’s definitely some concern in the organic and upper sections. It’s are students going to come from your class and expect the super high level of flexibility that they’re not going to have. and is that going to set them up to fail? so there’s some concern about that, but I think that really, I am essentially, I’m hoping that what we’re going to see, the data is going to show us that I’m doing the opposite, because now they know how to learn chemistry.

They know how to spot what they’re doing wrong. They know how to ask for help. You know, all of these other attributes that they get out of this, engaging with this mastery process. And so, I’m waiting with kind of bated breath to see at the end of this semester for organic chemistry, how my folks did and whether it’s true that giving them, you know, unlimited access to this assessment, help them or hurt them in this upper level chemistry course.

I’d

Bosley: be curious, because it sounds like since you’ve got, some real supportive colleagues, I’m wondering if any of them are going to make the jump after they start seeing some of this data. And like you said, this longevity data with how they actually do in the organic chemistry.

I’d be curious to have a follow up and hear if anyone else’s jumped onto the alternative bandwagon with

Eden: you. And I’m optimistic about that, to be honest, because my colleagues are all extremely open minded, committed to student success. And so I think that once they see the data, I think that they’ll be.

convinced, and not to say that their students do badly, but I think that there are some demographic differences in who’s succeeding and who isn’t. And I think that if I can show data that bears out, my students are succeeding, not just by demographics, that that will be meaningful. and I have another colleague, Dr.

Clark, who is experimenting with doing oral exams after the written exams. So not quite the same retake, but the same vibes of kind of alternative grading, and that kind of thing. So I think it is going to be a little contagious, I hope, in our department. and, I think me working out how to do this sustainably is an important part of this and collecting all the data which we are doing very fastidiously. but yeah, I really hope so. I’m optimistic about it. let me put it that

Sharona: way. Are there any final thoughts you want to share before we sign off today?

Eden: I guess I would just say to be brave. I know it’s hard, but even little changes are better than the broken system that we operate within. and you don’t have to necessarily commit to redesigning the whole thing, or doing the unlimited grading thing that I’m doing, to see positive change.

Like, any little baby steps are better than The status quo, and it does really move the needle on your relationships with your students, and I think it does so in a way that kind of reduces the risk for burnout, almost, right? Because now, all of the time that I spend teaching is really, really meaningful to me, building personal connections, building people’s knowledge, and it just doesn’t feel soul sucking the way that It did before I started experimenting with this, so just to encourage anyone who might be listening to this out of curiosity, but not having made the jump yet, although I wonder how many of those folks there are.

I’m pretty sure we’re preaching to the choir here, anyone who’s listening to this, but just in case. We

Sharona: are starting with preaching to the choir, but most of that choir wants to be evangelists themselves, and so they need these interviews and these conversations to go share with someone they’re trying to convince.

So I’m hopeful that it’s going to spark a contagion of its own, because, as this grows and also we’re finding that there’s so many pockets, like you didn’t know about us before this conversation. The conversation we had yesterday with someone who didn’t know about us, but we’ve all been doing this for several years.

So we’re also building where our own like neural network of people doing this and doing this in community. Is critical. I agree. So thank you so much for coming on. I have so enjoyed this conversation. Boz, any final thoughts before we go?

Bosley: No, just again, a big thank you. I absolutely love this kind of research framework that you’ve been coming at this from, and I do hope that if you’re available that, you might come check out the grading conference and bring some of your friends and really see how this is being done. And other fields, not just STEM, or not just science, but other stems, but how it is also being done in some of the other bio and chemistry classes. And, let’s work together on moving this needle.

Sharona: So, Yeah. And unusually the conference actually started in STEM, so it’s growing out from STEM as opposed to.

coming into STEM.

Eden: That’s fabulous. And I agree. Community is so critical when we do anything, but I think especially here, let’s put our effort where it’s most useful, learn from each other, build things together for sure. Absolutely.

Sharona: All right. Thank you everyone. Another great episode. And if we don’t have any final thoughts, we’ll see you guys next week.

Please share your thoughts and comments about this episode by commenting on this episode’s page on our website. www. thecreatingpod. com or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the contact us form on our website.

The Grading Podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our

Bosley: website. The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State system or by the Los Angeles Unified School

Eden: District.​