The Grading Podcast

[00:00:00] Jeff: You mentioned earlier, I think that it just like makes you angry or makes you mad to hear this. And like, I think lean into that because like me too, I think that it’s infuriating to see that we, you know, sometimes in these undergraduate classes in good faith are interpreting grades and evaluation a certain way and then folks who, who it most matters to the folks who are looking at those grades later on are interpreting them in an entirely different way.

[00:00:25] Boz: Welcome to the grading podcast where we’ll take a critical lens to the methods of assessing students learning from traditional grading to alternative methods of grading. We’ll look at how grades impact our classrooms and our students success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist, and instructional designer in the Los Angeles Unified School District and with Cal State LA.

[00:00:50] Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach and instructional designer. Whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week you will get the practical detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

[00:01:16] Boz: Hello and welcome back to the podcast. I’m Robert Bosley, one of your two co hosts, and with me as always, Sharona Krinsky. How are you doing today, Sharona?

[00:01:25] Sharona: I am doing very, very well. It is a exciting week in our world because we have just today announced the keynotes for this year’s grading conference. We’re going to have three absolutely fantastic keynotes. One of them is Dr. Laila McCloud from Grand Valley State University. One of them is someone who, it’s a little surprising, we haven’t had her on before, but it’s Dr. Susan D. Blum, who wrote the book, Ungrading. And the third one, we actually have in studio today, Jeff Schinske is with us. Jeff Schinske is a professor of biology at Foothill College, which is a California community college. He conducts education research on inclusive curricula, course content expectations, and science identity. And one of the reasons I’m super excited is Jeff was the co author of the essay titled Teaching More by Grading Less or Differently that our community knows so well and is such a foundational paper. It’s been highly cited in our community. He’s also the co editor in chief of the journal CBE Life Sciences Education, and he co founded the Scientist Spotlights Initiative. . One last little tidbit about Jeff, he was the 2018 recipient of the National Outstanding Undergraduate Science Teaching Award from the Society for College Science Teachers. And when he’s not doing all of that, he spends time with his kids playing basketball and baseball, following it and pondering what black dot work tattoos to get next. So I might have to ask you about that. Welcome Jeff.

[00:03:01] Jeff: Absolutely. It’s such a pleasure to be here. Thanks so much for inviting me.

[00:03:06] Boz: And, you know, in the intro there, Sharona was saying how cited that paper is, but it’s not just cited in our community, Sharona, this is one of the, like, foundations of our literature to practices that we often use. I mean, this is one of our big ones that we personally use in our training. So it is very exciting to actually get to meet the man behind the paper.

[00:03:29] Sharona: Exactly. Cause I don’t think you know this, Jeff, but Bosley and I do a 30 hour training for faculty on doing course redesign. And part of that is what we call literature to practice. It’s actually reading specific articles and discussing their impact within course redesign and yours is one of the first ones that is literally linked in our course. So thank you for that.

[00:03:52] Jeff: Well, thank you. You’re getting, getting more eyeballs on it and increasing the download count and all that. So yeah,

[00:03:57] Sharona: so exactly, exactly. So my first question that we always ask is: what made you get started with this? And what led you to write this paper that has been just so influential on so many people?

[00:04:12] Jeff: Yeah, yeah, thanks so much for asking and it was I think a paper that ended up being really helpful for me personally, as an instructor as well. And so it was , influential for me to to go through that process. And I should also definitely give a shout out to my co author on that paper, Kimberly Tanner from San Francisco State University, who has been my close collaborator for well over a decade at this point. And that was one of our first, maybe our first paper coauthored together. I forget, but that was one of the very first ones, if not the first and there’ve been many since then. So it’s been a, just a lovely collaboration. And I guess in terms of origin stories and I’ve enjoyed, I’ll say, listening to some of the other origin stories on your podcast and, and hearing how other people have come to think about this. But,when I was reflecting on this beforehand, I was thinking that there was sort of three phases for me that I kind of went through and it was sort of before the article and then right around writing that paper and then since then that have been influential for me in terms of where I am now thinking about grading and and also related to where that article came from. So, I would say that even before the article, when I was first teaching in the community colleges, starting in 2007, 2008, I had already really gotten really fascinated by a lot of literature around metacognition and growth mindset. And I felt that all that was quite compelling and of course somebody like me coming out of a research graduate work that was in ecology and evolution, this was relatively new stuff to me. But it was, it just really caught my attention. And I became convinced and I really saw that the cognitive science is telling us that in order to learn, you have to be able to make mistakes and be confused. And that those are really critical steps in the process of learning. And to the point where I’ve now, and since then have always told my students that, well, our whole goal in this class is to find really wonderful, constructive, safe environments in which to be confused and make mistakes. And if we don’t do that, then we’ve not met our goals. And so I think that that’s become a real mantra of mine. And, and I even will survey students throughout the course now. Have you had opportunities yet to make mistakes and be confused? Because if not, I’m not doing my job. And that’s what, that’s what we need to do. Of course, inherent in that is the ability to overcome those confusions and overcome those mistakes. And it became quickly apparent that traditional grading practices simply don’t accommodate that. They don’t allow for that opportunity to do what I think the literature is so compelling and telling us to do, which is to allow these opportunities to change your ideas and to be very aware of the changing of your ideas and when you were confused and when you were not confused. And so, very quickly in my career as a community college instructor, I came upon this barrier from grading that would not allow me to do what I thought was best for my students and best for their learning. So, I think ultimately that led me to some grading strategies that were certainly not to the level of even calling it alternative grading at the time, but really restructuring my exams in ways that they became more frequent quizzes and where everything was open for revision. And I should say that this is all also, you know, completely open ended constructed response kind of things, not, not multiple choice sorts of tests, and so they’re, they’re just having these really rich, I think, essay like questions that were bound to the core learning goals and then allowing people to really grapple with them at multiple times, exact same question, not just one time on a test, but one time where you can be confused, and then a second time where you can be confused, and a third time where you’re a little less confused, and eventually getting to that point where like, yes, we’re on the same page, and we’ve met that goal. And , that felt right.

[00:07:42] And, and I’ll say that that actually led to, I think, my very first education research article that I published as a community college professor back in 2009. And I should have looked up the name. I’ll forget it now, but it was something like Taming the Testing Grading Cycle and Classes Centered around Open Ended Assessment. That was it was sort of telling in some ways that my first article that I published in this realm was, was actually about grading in some ways. And so I think that that was really the beginnings of my origin story. And then what really led from that to this paper, to this essay, was in working with Kimberly Tanner on some professional development efforts where we received an NSF grant to do professional development with community college professors. And we were doing a week long institute and we were doing professional learning communities afterwards and communities of practice and we were seeing these folks get really excited about innovative educational strategies, but we would often get to the end of the week or the end of the term. And in people’s reflections, we would see things like, Well, this is all really great, but the problem is, when am I ever going to grade it? When is this going to get graded? And it became incredibly clear that a barrier to innovation in adopting new techniques and in particular active learning and other innovative strategies was the grading effort.

[00:08:58] And so that’s when Kimberly and I said, it actually took a few years, it was, we were a few years into the project where we said, Okay, I think we finally have to write a paper on this. We have to finally relieve people of this, of this expectation that everything needs to be graded or that grading stuff is even helpful for, for everything. And so that was the motivation where we said, like, we were not experts on grading, we were still not experts on grading, but we said, it’s become such a barrier to our work to drive change and the way that people engage with students in their classes that we want to dive into this and write about it. And, I think that’s what let us down that rabbit hole of more than a century’s worth of, of information about grading systems and how grading systems impact students and learning. So that brings us up to the point of actually writing the article.

[00:09:45] Boz: See, and that’s really interesting. And I’m also so glad that you did because, you know, I, I also started teaching in 2007, granted at the high school level, not at the college level. And, you know, I think I first heard about Dweck’s book, 2008, maybe 2009. So not long after they come out, you know, and I always had this. Yes, math means mistakes allow thinking to happen. And, and, you know, wanted to encourage this and spent tons of time giving feedback. And I never understood why it wasn’t coming together and why the students weren’t taking more chances and, you know, being okay with making these mistakes, and I tried to do all this gimmick stuff. It wasn’t until after I started reading some of the alternative grading stuff that I realized that my traditional grading was what was actually hindering all that, because no matter how many times I said, yes, math stands for mistakes allow thinking to happen.

[00:10:46] If I’m using averages, those mistakes are still punishing the kids .So, the grading, and like I said, it took me years and it wasn’t until I started doing some of these readings, you know, by, you, by Guskey, by Feldman, even that I realized like, well, duh, now I look back at it. I’m like, God, I was an, I was an idiot for, you know, five or six years. The fact that I was thinking about this and not figuring it out. Just like, damn, what was I doing for those six years?

[00:11:19] Sharona: Let’s be clear here. We don’t want other people to feel this shame because I’m listening to the two of you with your "I started teaching in 2007". I’d like to point out that my first in classroom math class that I was teaching was 1992. Okay, so I’ve got both of you beat by 15 years. And on top of that, I’m a second generation math educator. So my mom hit the classroom and she was instrumental in getting active learning into the math education curriculum. But she hit the classroom in the 1960s. Okay, so she actually even predates when Guskey was in the classroom. So I, you know, this stuff can take time. But I agree with you, Boz. Like, I was so frustrated. And I felt like a failure because I was raised on this stuff. Literally, from the time I was 12, I’m listening to the importance of active learning, we called it cooperative learning back then, in the classroom and the importance of not grading students as a group because of the free rider problem. So I’m literally growing up with this at my mother’s knee and I can’t make it work. I cannot make it work. So yeah, that’s amazing.

[00:12:29] Jeff: And I think in some ways that’s not a surprise. And in some ways, like, you know, I don’t think any of us were idiots or any of us are to blame really for that because we are it’s, it’s the system we’re swimming in. Right. I mean, we all to extent or another, grew up in this system of evaluation and grading, and it’s I, I think like, as, as indicated in a lot of the writing about it, including, you know, the paper Kimberly and I wrote, it’s been forever assumed to be like an immutable constant of the universe and that, that’s like what it feels like when you’re going through it as a student. That’s what it feels like. As an as, as a professor or a teacher or an instructor. And even though that’s not the case, that just is a sign of, like, the system that we’re in, and I think it actually gets to a broader point, and we can come back to this in some cases later, because I think it’s a bigger point, but I think that the oppressive system of grading is oppressive on all of us, and I think it’s especially oppressive on certain populations that and sometimes often the ones that we care most about and that we’re most focused on, but it’s, it’s oppressive on faculty and teachers as well. And so anyway, I’m trying to give us all a little bit of credit here and allow u s to feel safe in our perceived naivete, I guess, about the grading system, because it is an oppressive system. And it is something that we have all unfortunately been oppressed by and have been grown up in. And so it’s no surprise that we’re so bound to it.

[00:13:49] Boz: And I think part of it is that, you know, we are, it’s so unexamined by so many people and just accept it. I mean, like I said, I am definitely guilty of this as well. You know, most of us grew up in this kind of grading. If you are making your living as an educator, at some point you had success in that traditional grading system. Like you had to have had a level of success to be where you are as an educator. So for us to have succeeded in that and just completely leave it unexamined, it’s, that’s just the way grades are done. You know, I, I think that is why it’s so hard to see until you really do start examining it and go, wow, this is crazy.

[00:14:36] Sharona: I just think it’s interesting though. That for me, it feels like it came later than I would have expected, given how much work there’s been done on some of the equity issues on some of the, there’ve been people out there smarter than me that have been examining all kinds of systems and like you and I’ve talked about, I mean, to be honest, Dr. Benjamin Bloom was examining this in 1967. So why was it so unexamined for so long? That just is like kind of percolating with me.

[00:15:08] Jeff: Well, I’ll do you one better than that. I was struck by this when I actually returned to the paper, that essay, and I saw that we started with a quote, which says, "when we consider the practically universal use in all education institutions of a system of marks, whether numbers or letters, to indicate scholastic attainment of the pupils or students in these institutions, and we remember how very great stress is laid by teachers and pupils alike upon these marks as real measures or indicators of attainment. We can but be astonished at the blind faith that has been felt in the reliability of the marking system." That’s from 1913. So I’ll do even better than the 1960s. This is over a hundred years ago. This is over a century and, I think that I was struck. I was like, well, I’m glad we started with that quote because that really drives it home. I think that it again shows what a strong sort of set of systems grading is amongst many other systems in our world, right? That that continue to go unexamined. And I think Boz, you started to get to this point, that it’s not by mistake that these continue to go unexamined. It’s because they work pretty well for the people who are in power. And as much as we say we dislike them and, you know, I’ll say that, I’m one of these people in power, right? Like, you know, I made it through that system and I succeeded pretty well in it. And and so in one sense, it’s, it’s, it’s shocking to look back and see like, yeah, over a hundred years, we’ve been, there’ve been people saying almost the exact same things as us except in early 20th century language over a hundred years ago, and we’re still saying these things now. That’s not by mistake. There’s a reason that it stays the same and that’s because it has worked pretty well for the people who have managed to keep, keep it the same.

[00:16:38] Boz: Yeah. That’s a great point.

[00:16:40] Sharona: So that article comes out, do you get any pushback? Do you hear anything about it? Does it just kind of sit dormant? What, what happens at that point?

[00:16:50] Jeff: We got a really lovely reception, actually, I think, and this was before, I don’t, I can’t say this initially, before like Twitter and all that kind of stuff, because that’s probably not true, but it was certainly before I was deeply engaged in, those kinds of platforms. And so I think that in that sense, it was a little bit like, more hidden to me, how it was being received and, and perhaps there would have been more pushback if I’d been more looped in with those kinds of platforms back then. But the reception we did get was, was mostly quite lovely and, we got approached by some, reporters. And so there were like some articles about it and in the more popular press or academic press and, lots of professional developers who were wanting to use it sort of similarly to the ways that you all have spoken about using it. So my recollection having to go back 10 years, which is, tough. I’m thinking, like, how old were my kids then? Wow. It’s hard to think back that far, but my recollection is that it mostly got a really lovely reception and got some attention in ways that we hadn’t expected, that were quite, that was quite good. I’ll say we also then, of course, immediately incorporated it into our workshops, because it, again, this is very much driven from a practical sense of us as workshop facilitators wanting to relieve people of this, and I still keep referring back to Kimberly, but I still remember Kimberly, on the day that we’d introduced this article, she would figuratively wave her magic wand and say, I’ve officially relieved you of the need to grade everything and like, in fact, you may not be doing anybody any favors by trying to grade everything. So consider yourselves relieved right now. And of course, you know, it’s a little tongue in cheek. And in reality, we know that there are administrative and logistical and, and collegial pressures to, to grade things and all that. But we, we immediately started using it in those workshops as a way to say like, Hey, you know, if grading this active learning activity is the thing that’s going to stop you from doing it, then guess what? How about you just not do that? And here’s some, here’s some literature that can maybe back up why.

[00:18:35] Sharona: That’s, really remarkable. And.

[00:18:38] Boz: Actually, Sharona, before you go on to the next one, I did have a question because you said that it had a good reception. Did it gain a lot of traction quickly or was it, you know, did it take a little bit of time before you started seeing some of that traction and some of those, it being used as citations and everything else?

[00:19:00] Jeff: Certainly the citations took a while, of course, which is maybe not surprising in terms of people coming out with new peer reviewed papers and that kind of thing. And I think that the reception initially was more informal, and it was more like reaching out by email or maybe Facebook or those kinds of things and those kinds of interactions. But now of course, you know, I have citations alert on my alerts on my papers like most people do. And now really a week doesn’t go by where I don’t get one for that paper or multiple for that paper. So it’s been interesting to see that it’s a decade old now, which is kind of exciting. I mean, I think it’s in June that it was published and so in June, it’ll be a 10 years old and it’s pretty interesting to see that even now it’s getting cited quite regularly. So so yeah, I think that there was a reception fairly quickly and then the it’s just kind of continued on, which is great.

[00:19:45] Boz: Yeah. I was asking cause we had Dr. Thomas Guskey on a couple weeks back, and he, in part of that interview, he was talking about not only how slow some of his early works were to gain traction, but how slow his mentor Dr. Benjamin Bloom’s taxonomy, how slow that was to start gaining traction and then once it did, it was just this exponential growth. So I’m wondering if that’s some sort of universal truth that, you know, you have this really slow, but then exponential growth with these kinds of you know, educational reform or just reform in general.

[00:20:23] Jeff: Yeah, well, I’ll first say that I really enjoyed that podcast. I think I told Sharona earlier that I was driving in my car and I was wishing I had a pad of paper next to me to jot things down because I thought that that was that discussion was fantastic and I really enjoyed hearing from Dr. Guskey. And yeah, I guess I would, I would say that resonates in some ways that like we received a fairly immediate reception, I think, in terms of people viewing the article and all of that. But speaking to Dr. Guskey’s point, I think, like, it’s not to say that immediate change happened in the education landscape, right? I mean, I think that even now, 10 years later, alternative grading is still called alternative grading because it’s not what most people are doing, right? And so and that goes back again, to Benjamin Bloom’s work and Century’s worth of work that has been trying to push folks on this, but, it’s not to say that the reception we got was necessarily an immediate drive towards institutional change or structural change by any means.

[00:21:20] Sharona: So in other words, Bosley, we have to commit to 10 years of the podcast. This is what I’m hearing.

[00:21:26] Jeff: Apparently it has to be like over a hundred because we’re still waiting on some centuries old things. I don’t know. You might have to, you might have to really stretch it.

[00:21:33] Sharona: Multimedia. It’s, it’s, it’s audio. So it’s going to go faster.

[00:21:37] Jeff: Oh, okay. There you go. There you go. I’m with you there. Okay.

[00:21:41] Sharona: So ,where have you gone? Because it sounds like you are an education researcher. You’re in biology. You’re a biologist. You teach in biology, correct? And what do you teach?

[00:21:53] Jeff: Yeah, I’m a proud community college faculty member. I think that like it’s the best job in the world. And I sometimes look around and I’m like, why doesn’t everybody teach community college? Because it is, there’s literally no better job than doing this, and so and not to say bad about anybody else’s work, but I’m like it is fantastic. And the, and the, my students are the most inspirational, hardest working, brilliant people in the world. And I say all the time that the only downside is it’s frankly a little intimidating to be around them because I’m so impressed by them and all that they do. That it’s, it’s just a little humbling and not a little humbling, a lot humbling. So yes, I’m very proud to be a teacher and in the community colleges, I primarily teach Human Anatomy and Physiology for pre nursing and pre allied health majors.

[00:22:36] Sharona: And do you use some form of alternative grading in that class? Can you maybe tell us what you do?

[00:22:42] Jeff: Yeah, absolutely. I think that it definitely drew off my earlier desire to promote these ideas of metacognition and growth mindset, and to stress the importance of confusion and the importance of failure, that these are all things that we actually aim for and we aspire towards and we actually lean into. And so I think that my grading is similar. Still somewhere around that, though, I was going to say, and it’s actually a very good segway, your question, in the sense that the, the sort of third iteration of what really influenced me beyond like, you know, that those initial forays into thinking about how to allow for revision of ideas over time to writing this paper with Kimberly and having that experience of seeing that, oh my gosh, there’s a, hundreds of years history behind this that, is really fraught actually with a lot of problems about grading systems. And then coming to the community colleges, or the job that I’m at now, where I teach these pre allied health and pre nursing majors and finding a new dilemma that really struck me. And I’d be really curious if, if either of you have encountered this in, your experiences in teaching, which is that I would have quarter after quarter after quarter where my students would beg me, sometimes in tears, to give them an F and to not give them a C. And have you, have you seen that happen?

[00:23:56] Sharona: No, I have not.

[00:23:58] Jeff: So this is sadly not unusual story. Whoever’s listening to this, who happens to be in a similar scholarly position as me, well, this might resonate with, but they would be absolutely distraught that they absolutely cannot get a C sometimes even a B and say like, please, I just need you to give me an F or maybe a D, but an F or a D. And what this got to was this point that I found out that as I think somebody was saying on your podcast recently that, grades are a currency, but the currency of grades, there’s a conversion rate between different actors in the system and that the currency is not always the same. And so it turns out that what these pre nursing students were hearing, and is, I’m not sure it’s exactly this true, is that getting a C in Anatomy and Physiology means you cannot go to nursing school, that your nursing career is done at that point if you get a C in one of your key prerequisite STEM courses. And I don’t think it’s quite that true, but I’ll say, at least in the competitive environment of the Bay Area, if we’re getting into nursing school, it’s very true that if you get a C or multiple Cs in one of your prerequisite science courses, it is substantially more difficult. I’m not willing to go to impossible, but it is substantially more difficult. What’s interesting about that though, is that of course in my own institution, if you get a C, then what we’ve decided that the way that we’ve decided to interpret that grade is that we’ve said, well, you have pass this course. You have attained the minimum requirements to move on to the next stage of this course or to go into your career or whatever you’ve met the learning outcomes to the extent needed to move on. But the nursing schools are interpreting that differently in a way. Whether consciously or not, they’re sending the signal or actually directly saying that a C means you have not accomplished the goals of that course. The problem is that if you get a C in the course, you’ve passed it, you cannot take it again. And so that C is a permanent grade. However, a D or an F is not a permanent grade. That means you did not pass the course according to my institution, and most institutions, and therefore you can retake it, and it would, and whatever you get in that second trial supplants the original grade. And so, you know, if you get a D or an F and then you get an A next time the D or F still shows up on your transcript, but the A is what counts towards your GPA, and that’s what counts towards your nursing school transfer GPA and all that, usually. There’s, it’s different formulas for different schools, of course, but that’s often the case. And so you can see then that getting a C is actually worse than an F. Getting a C means that like you’ve made it substantially more difficult for you to move on to that next stage of your career that you’re very passionate about. And my students are very passionate and they have very good reasons for wanting to serve their community and get out there in these health careers and, move their family forward and economic liberation and all these wonderful things that we can do for them. But if you get a C, that is at least substantially making that much more difficult to move on and so that really made me think hard because I’m like, wow, like again, this system of grades or this currency or whatever these grades are representing are not being used the same way between different actors in that system.

[00:26:58] There needs to be some alignment in that and, and I’m not going to be able to align the nursing schools on that because I can’t go out and change their systems of accepting. The only place that I have agency over control is my classroom. And so, if I feel that a student has absolutely met the learning outcomes and is you know, in terms of my prerequisite content for my course, and as far as I’m able to tell, able to move on to that next stage, then I probably can’t be giving them Cs. I have to, in some way, get Cs kind of out of my grading lexicon. And this, I think, getting back to your point about, like, where I’ve moved since then has maybe moved me a little bit more towards what I understand, and I’m not an expert on these terms, but it’s a little bit more like specifications based learning. Which is like, okay here are the learning outcomes that we really care about in this class that are really critical that you need to move on to that next stage. I just need to get everyone who’s engaged in the class to move on to that next stage and then check it off and then they move on and that’s full credit Right?

[00:27:57] So you, fulfill these goals, you’ve done this case study, you’ve done this group activity, you’ve completed this essay question, whatever it is whatever those tasks are, that you check it off and then you move on. And so I’d say that migrating has changed significantly, because of that, in the sense that there is a lot more sort of binary kinds of rubrics going on and like, yes, you made it through here. Or if you didn’t, then like we’re going to work until you do again, that frequent resubmission of work until you get to that point. And to the extent that now, like I really don’t give out C’s. I give out mostly As, some Bs, and then Fs. And I do give out Fs. I do give out Fs. And I think there’s some good reasons for Fs sometimes. And I think that there’s some cases where students should be actually very proud that they got an F. Because, like, or not proud, but at least understanding of the reasonings behind it because of things going on in life or the world or whatever. So that’s where I kind of am now is moving a little bit more towards this what I think again, if I’m using the terms correctly, more specifications based of here’s, here’s the goals we need to work throughout this course to make sure that everybody who’s engaged in the course reaches those goals on an appropriate timescale and then you’re checked off and move forward. And then I can communicate to the nursing schools with an appropriate grade that allows you to move forward and, not stick you with a grade that my own institution interprets as satisfactory and able to move on, but other institutions have said, well, actually, we’re going to interpret that differently and that’s not satisfactory.

[00:29:14] Boz: All right. So I know Sharona, I know you want to get in here. I actually want to set you up because first, I, don’t think I’ve ever heard of students complaining about C’s, but I completely understand it. Once, as soon as you started saying it and talking about nursing school, I’m like, Oh yeah, D and F they can replace, a C they can’t. So I am curious for other listeners, if you’re at a community college or even a n university and you’re teaching at a prereq class, because Sharona, yours and I classes, at least the stats isn’t. But, that stats class has that same distribution of grades. That almost like inverse bell curve where we have virtually no C’s because the students that typically are in the C range, you either get the students that work really hard but never really get you know, get well on the on the assessments. They just do all the work and then you have the students like kind of student. I was that will scrape by doing well on the assessments and not doing anything else. That group completely goes away with alternative grading, at least any of the forms that we’ve seen. So we we have a lot of days, a lot of bees. Some C’s and then, you know, our 20 to 30 percent F’s. So we have that inverse but that’s really interesting that how the students are looking at it and how the other institutions were looking at it because, Sharona, and it actually really is kind of making me mad right now thinking about it. But we need to have this larger conversation about what exactly grades mean and represent because Sharona, and we won’t say names, but weren’t you just asked a really interesting question about our inverse bell curve distribution.

[00:31:09] Sharona: Yes. So, I was in a conversation with someone who is very interested in our grade distribution. Well, we’ll say that. And we’re looking beyond just our letter grade distribution. We’re actually looking at the distribution of the mastery of the quantity of the 15 learning outcomes in our statistics class. So we have 15 learning outcomes. You need 14 to get an A, 13 to get an A minus, and it just counts down. So I actually, at the end of every semester, know exactly what percentage of the class got 15, what percentage got 14, what percentage got 13. So going all the

[00:31:45] Boz: And which percentage got which ones. How many had S1, how many had S2, etc.

[00:31:52] Sharona: But in particular, we tend to look at the aggregate more than anything else. And what we’ve noticed is not only do we have A’s, B’s, and F’s, but those F’s are all the way down at nothing. Like 20, 25 percent of our class has absolutely nothing. There’s very few that are sort of, we got one, we got three, four. Once they get three or four, if it’s early enough in the semester, they can probably get to seven, which is passing. And if they get to seven, they can get to nine, which is a B like, so, it definitely builds like that. And, this person who is very interested in this distribution looked at me and said, huh, that sounds like an opportunity with those low ones. Can’t we create a normal curve out of this? Wouldn’t it be better to, like, get this to be more like a normal curve? And after I started to explain the racism and eugenics by that, they said, no, no, no, I didn’t mean normal curve. I meant bell curve. And I’m like, no, still not. And I ended up saying, I think what you mean to say is, is there a way to take those zeros and ones and continue to move them up because there’s no reason to increase our C’s, although this person was very interested in the fail rates. So for them, success is getting the zeros and ones to C’s. So what you’re saying is fascinating to me, because we had a student this semester who accidentally was given a no credit and should have been given a C last semester and had re enrolled, never said anything, had re enrolled. And we discovered this and we said, Oh my gosh, we should change the grade and I said, wait a second. We are past the add, drop deadline. If we pass this person, they will have to drop the class because they can no longer repeat it. That could put them below minimum units for the semester. That could impact their housing. That could impact their food. We cannot change this grade without checking with them. And my administration was like, what are you talking about? We have to change the grade. I’m like, no, no, no, stop. So I emailed the student and I said, I’m so sorry, you actually earned enough to get the C. So you have a choice. You can either have us change the grade and here’s the implications, or we will go ahead and transfer what you learned to this semester and you can start there and keep going and get a better grade. And that’s what that student chose. They said, I want a better grade. They said, that’s what I want. But I’m like, you know, I’m like, you can’t just, because a C, not only in this case might have the implication you’re talking about, , but now the student has to try to add another course and they’re three weeks in. So they’re now behind. Like, no, that’s not equitable. I feel bad that this happened to the student. And if we had caught it before the semester started, the student might have made a different choice, but right now you have to give the student agency on this. So, yeah, I, oh, that’s interesting. I wonder if we should look at, Bosley, giving students who have those low, low standards, the option to take the no credit and continue I wonder if we can get away with that.

[00:35:05] Boz: It’s a little bit different with our situation because he’s talking about prereq classes.

[00:35:09] Sharona: No, but we have the biggest number of students that takes our class, Boz, are pre nursing. They are the biggest single major. Yeah. So I just, it’s just interesting things to think about. Wow.

[00:35:22] You just rocks my world.

[00:35:23] Boz: But the idea of the currency that we talked about that, that you were bringing up, Jeff, that one place looking at C is not only acceptable, but C as where most students should be, right? And at the same time, those students having to use those C’s in a way that is very harmful to themselves, if, if not impossible, is putting them behind the eight ball with trying to go further. So what, like we, I think going back to Jesse Stommel’s interview and just the fact that we just need to have a very open, you know, national K through 16 conversation of really about what grades in general. I think this actually brings that point up more than I’ve ever heard before, but that’s really interesting that your students really didn’t like those C’s because of that.

[00:36:20] Jeff: Yeah, well, and I think that it you mentioned earlier, I think that it just like makes you angry or makes you mad to hear this and like, I think lean into that because like me too, I think that it’s it’s infuriating to see that we, sometimes in these undergraduate classes in good faith are, interpreting grades and evaluation a certain way, and then and then folks who it most matters to, the folks who are looking at those grades later on are interpreting them in an entirely different way. And I would say that it’s especially infuriating given the students that I think some of us work with, and including me, like, that these are the people, I’ll just put go to my own context here like pre nursing or whatever pre health kind of fields. These are the people we most need in those fields I mean, there’s You know, there’s a whole other 30 or more years record and research around health disparities and the fact that we have literally people who who experience experience death and disease all the time not because of something about that particular disease in them, but because of their social identities that a mismatch that a lack of concordance between patient and provider identities and cultural competence and cultural humility and cultural responsiveness in healthcare causes death. I mean, like I, I’m really, I don’t want to seem like I’m being hyperbolic. I think I’m being actual real about how the world works that, that if, if my students don’t get to be nurses, there’s the real possibility that will cause unnecessary death because my students have that cultural responsiveness and have that linguistic capital and have all these other sources of cultural capital that they can bring with them and literally save lives just by being who they are and having the experiences they have as individuals. And so, when the grading system or other factors, I’ll say, there’s many other factors, but when when the when arbitrary systems get in the way of them going out there, and doing what they need to do to make our world a more healthy and with less people dying kind of place. I think being mad is a very good reaction and I’m with you.

[00:38:23] Sharona: Well, and think about this opportunity. You teach anatomy and physiology. That is actually something these people really do need to know. So, if a student is given a choice, you’ve gone through this course here’s where you’re at. You’re at this point, you can take the C and move on, or you can take the no credit and repeat. And we give them the agency because repeating does have consequences, whether it’s repeat limits or cost or time or any of those things. But if you have a student who has chosen to continue their journey to repeat in service of this goal. Is that student going to be a better nurse, a better allied health person, a better doctor, potentially? It’s not a given, but if they’ve made the choice, if they’re faced with a crossroads of, you can move on. Like I will tell you right now, my son, my older one, sorry, Nathan, if you’re listening, absolutely would be like, take the C. I’m out of here. Okay. 100%. He is like, he has his goal and his goal does not depend on grades. And he’s like, just get me through. But that’s why he’s never going to be a doctor. Right? But I know so many other people, my sister’s a doctor. She absolutely would have, I think, chosen the no credit or the F or whatever and moved on. And I understand why we have repeat limits that have to do with financial aid and resource limits and things like this, but the community colleges are a great place because of the cost structure here in California. The cost to the student for repeating is much lower than at Cal State LA. And I don’t know if your students have the same implications, but if our students fail to make satisfactory academic progress, or a lot of times I’ll talk to a student, I’ll say, you’ve told me that you’re hospitalized. You’ve told me that your grandmother’s hospitalized. You’ve told me all these things. We’re in week nine. You’re not going to be able to recover. You should withdraw. And they will look at me and say, I will lose my housing. Because they have to be a full time student of 12 units or more to be in the dormitory. I am willing to give an incomplete you know, to me, you’re, in my class? You’re passing until you don’t pass.

[00:40:42] Jeff: Yeah, well, I think that it was just bringing up the point that getting back to Benjamin Bloom and, who we actually quote in that essay that Kimberly and I wrote, Benjamin Bloom was one of the early and main folks, main critics of the grading curve. And we quote him in saying that there’s absolutely no reason that everyone can’t reach mastery given the appropriate resources, time and everything else. And of course, that’s a big "given" especially in this strained world we have of strained resources and faculty time and all these other things. But that there’s that, there should not be a need for C’s, frankly, because given the appropriate resources and instruction, then it should be able to accomplish mastery for all folks. And yeah, I would just note that, you’re getting a little bit more into my wheelhouse here in the sense, and also actually into what we talked about being a focus of my presentation at the, at the grading conference, which is this issue of like the, I’ll frame it as a "fascinating assumption", to test.

[00:41:35] That you said that I teach in a field where nurses really need this. I think that it is actually incredibly difficult to come upon evidence of that statement. And I would go a step further and say that for a intro prerequisite, or just, I’ll just say intro STEM classes. It’s very difficult to find evidence of that for any of those, and the evidence we have is that quite a lot, if not most or all of that knowledge from the prerequisite classes is quickly forgotten anyway, and so this is where the system becomes even more infuriating because the, the grades that we have then are not only themselves a part of an oppressive system that is interpreted differently by different actors in the system, whether these professional schools versus undergraduate institutions versus employers and all this kind of stuff. But at best, they’re perhaps grading people and evaluating people on something that they maybe don’t even need or remember or use. And there’s a deep literature going back to Talking About Leaving, I’ll say, which you might be familiar with, but a really, really influential book. And then Talking About Leaving Revisited, which are both enormous, I mean, they’re books, but they’re enormous studies.

[00:42:38] And one of the key findings of all those is that folks who left STEM fields and frankly, in agreement with the folks who stayed in STEM fields, agree that their intro courses were unmemorable and exclusionary in various ways. And, so here in my, in my own field of anatomy and physiology, I mean, it’s not uncommon to have 40 percent pass rates or even less at institutions. That’s not my classroom, obviously. But that’s not uncommon nationwide and those 60 percent who aren’t getting through are potentially being forced out of the field for reasons that actually do not necessarily even have anything to do with their future profession. So to be clear, I think nurses should have to know something about the human body to be a nurse. That not where I’m going, but what I’m saying is that we’re, what we’re currently offering them in classes like the ones I teach. Everything we can, as far as we can tell, is not what they need to know about the human body. And even if it was, it’s actually not remembered or used anyway. And so again, I think the sense of dissatisfaction or anger or frustration only deepens.

[00:43:40] Boz: So you know, I would love to see some of that because I’m actually in the process, the very beginning process, of trying to take a lot of colleges and high schools together to do this dual enrollment statistics course similar to what you and I did, Sharona, with SLAM. And yeah, I’m at that point right now where I’m looking at all these different college, you know, beginning statistics courses and looking at all of their learning targets or their, their SLOs and seeing all this stuff. And I’m like, okay, how much of this do we really need all of like, and I know why it’s there. It’s there because it’s always been there. Like that’s just a traditional, you know, why is ANOVA test in a beginning statistics course? Because it’s always been there. I mean, that’s just, but like you were saying, I know when I took some of my upper division statistics courses, other than basic correlation and basic hypothesis testing, I didn’t remember ANOVA and multi analysis, I didn’t remember any of that stuff, and I did really great in my stats classes. So, why do we keep, and, and you’re right, and oh God, the amount of just information in most of the science what are they called survey classes. Just that, that intro bio, that intro chem, that, that intro physics, just the amount of stuff in those classes typically is ridiculous.

[00:45:16] Sharona: What’s coming to mind for me is this idea that I think technology has overtaken at the speed at which universities are capable of changing, because thinking back to when I was in college, I am just enough older than Bosley to have had a college experience where we didn’t have the internet, okay? I did not get my first email address until my senior year in college.

[00:45:42] So I just want to say this is, and by the time I was a second year grad student, we had it. So, I was right there. So, I suspect that a lot of this is left over from the time when information retrieval was much harder. It was much, much harder to get the kinds of information in the 60s and 70s. And doctors really did have to have, and nurses, a lot of this stored in their head. Whether or not they were successful at it, they really did, but it’s different now. We have a very different set of tools. We have a very different set of information retrieval systems. I do think human implants are going to be here sooner than we think of it where you’ve got USB drives in your head. Okay.

[00:46:26] So this leads me to two things. One is everyone has to come hear your talk on content quantity at the grading conference. But, I want to go back to something that you said and that Bosley, you almost picked up on and then went a different direction, which is we asked Dr. Guskey, if we want to make this change, if we want to get other people to change, what is the number one thing that we, as change agents, should be doing? And he said, ask the following question. What is the purpose of your grades? Where "your" is defined both as an individual instructor, but it could be to an administrator of your institution. It could be to a program of your program sequence. It could be what is the purpose of your grades? What are they supposed to do? And if we can begin to have that national conversation of understanding and deconstructing what they do, versus what they’re supposed to do, then maybe we can come into alignment somehow.

[00:47:27] Jeff: Yeah, I’ll just go back and I feel like I need to point out that I think that I agree that STEM has a particular problem is related to some of these issues we’re talking about. But I’ll also say that I was a music major. I have a bachelor of arts in saxophone performance and that the initial introductory music theory and music history classes I took for two years, those are the prerequisite courses in music. Look at those course outlines, same thing. Now they don’t have the same kinds of attrition rates because I think that they have maybe smaller classes or they put more effort into student retention than the sciences do, where it’s more acceptable somehow to be a weed out kind of culture, so the implications are not quite the same, but the implications in terms of like actual learning and what the grades mean are the same in those.

[00:48:06] And so I’ll just say as someone who comes from an arts background, I, and that’s maybe very different and distal from a STEM undergraduate background. There are some universal and pervasive parts about this across fields. And yes, and to, your point Sharona, I think that I really appreciated Dr. Guskey’s talk about change and the need for evidence and change theory. And I appreciated how he said, like that there’s some misconceptions about, like, you need to change beliefs and belonging first, and then you can get to outcomes, well, maybe you need outcomes first, and you need, you need tools and outcomes first, and then you can get to changes in beliefs, and I, I thought that that was a really important and great discussion. So I, I agree that these are all things that need to be done. I agree that, I, I also agree, like, Dr. Guskey, mentioned that, like really bringing in evidence because there can be a lot of great innovations and ideas out there that can be well intentioned, but actually not be helpful to students.

[00:48:57] And I’ve had some of those in my own class, right? This is gonna be a great idea. This is gonna help students and then I try to be more systematic about collecting evidence around it, and I realize oh, that didn’t really accomplish the goals, or maybe counteracted the goals. And so Dr. Guskey made that point, too, that, like, that collecting evidence is a critical piece of this. Not that we all should be doing it, but, like, those of us who have the capacity or who have the interest, like being able to collect systematic evidence about our innovations is a really critical piece as well. And I think that the only part I would say to maybe extend some of your point about having that discussion is, I guess I increasingly feel that we need to be having that discussion, not just about what grades mean to us or whatever, what any part of this system means to us, but to the extent that we as educators are passionate about improving the system that we’re all cogs in and that we work in, and to the extent that we’re interested in equity and social justice in our fields, we need to start pushing on the system in general and not just asking about what things mean to us, but what do they mean to the system and, and then critiquing that system.

[00:49:54] And so I think that yeah, I don’t know. I think that it starts, of course, in a local context, and it starts with, like, local reflection and meeting in our own classrooms and in our own practice. But, I don’t know. I would love to see an upswelling from across disciplines of folks from the faculty ranks, not just from education research ranks or from folks like Dr. Guskey, but from the folks who are teachers and educators and professors really as many people are frankly, but just even more so asking those questions of the system and why the system is doing what it’s doing to us and to our students.

[00:50:25] Sharona: Well, and to clarify actually what Guskey said, he did say, ask the question, but what he said was actually get people to write a grading purpose statement. So it wasn’t just asking what they do, it was to create what we want them to do. So it is taking it that step further and as well as examining.. I am not the part of the community that thinks that we’re gonna get rid of grades anytime soon. They serve some critical purposes that we will have to replace if we get rid of grades and whatever we replace it with is gonna suffer from the same things. So that being said, let’s start to get intentional about what we want these things to do and not monolithic, because there may be different places through the system where this communication we need, we have changes.

[00:51:20] My opinion about what my intro class grades should mean and what role and purpose they should serve is going to be different than a senior design class. Those are just different courses, different context, they may get the same grade, but they mean something different. And there’s a lot of opportunity here. I heard someone talk about something called a double clickable transcript, where the transcript comes over electronically, and it’s got the course title, but if you click on the course title, it’ll open and pop up a more robust description, and you can do that with grades too. So I think it’s a, I agree with you, but I think it’s a step further. We need to start crafting the answer to what purpose do your grades serve? Not just what do they do, what do we want to create them to do?

[00:52:08] Jeff: Absolutely. Yes.

[00:52:11] Sharona: So, Boz, I think we’re coming up close to time. Did you have anything else you wanted to?

[00:52:17] Boz: Well, I first wanted to thank you, Jeff, for coming on. I was excited when I heard that you were definitely going to be one of our keynotes. I’m even more excited now. So anyone that’s listening to this that really wants to make sure that they get this keynote Sharona, cause we do have three keynotes and that is because we’re, the conference this year is going to be three days, but Sharona, can you tell everyone exactly which day he’s Jeff’s going to be keynoting.

[00:52:43] Sharona: Yeah. So Jeff will be opening the conference on the Friday, which is the middle day of the conference, June 14th. Keynote that session begins at 10:30 AM Eastern time, 7:30 AM Pacific time on Friday, June 14th. And it is going to be a fabulous talk about content curation and how the quantity of content, which we, carefully sort of stayed away from to not give that away, can become a barrier to the innovations in grading. So really excited to hear you speak about that. And it’s just, yeah, you blew me away with this conversation. So any last comments or thoughts from you, Jeff?

[00:53:22] Jeff: Yeah, no, I just really appreciate the conversation. I realized we kind of went down a bit of a rabbit hole at a couple of points and I hope that was an okay way to go at those times.

[00:53:32] Sharona: The great thing about the podcast is that we go wherever Bosley and I decide to go. So there we go. So yes.

[00:53:39] Jeff: Yeah. But I mean, I do think that everything we talked about has deep implications for grading. I think that all these things we’re talking about are intertwined. I mean, I think the only thing, other thing I wanted to say was that I appreciate all the work that you and others in this community are doing. I think that there’s reasons that this is tough and there’s reasons that there’s pushback sometimes, and there’s reasons that there’s a difficulty and pain, because I think that these systems of grading are often at the center of a lot of, as I mentioned before, oppressive systems in our field of education. And so, and those systems don’t want to change and the, and the people who benefit from them don’t want them to be changed. And so I just wanted to shout out one last note of appreciation that this work is sometimes hard, but I think it’s quite important.

[00:54:27] Sharona: Yeah, I, I think this work is hard. I hope that we are contributing to the work and moving it forward. And I wanted to also thank all the people who are listening from, by the way, I don’t know if you’ve looked Boz, 31 countries around the world now are listening to us or have listened to us at least once. So, 31 countries around the world, over 5, 000 downloads. I am so grateful for this opportunity to speak to people and. I think with that, we will wrap it up and we’ll see you all next week. Please share your thoughts and comments about this episode by commenting on this episode’s page on our website, http://www.thegradingpod.com, or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the contact us form on our website. The Grading Podcast is created and produced By Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.

[00:55:29] Boz: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State System or by the Los Angeles Unified School District.