The Grading Podcast

22 – Interview with Katie Mattaini

===

Katie: I was at the Deep Teaching Residency, the first one, outside of Washington, D. C. So that is kind of the brainchild of Bryan Dewsbury. And I met Drew Lewis there, who’s another one of our conference co organizers, and the whole idea of this Deep Teaching Residency is people who are interested in being more inclusive teachers in STEM, in higher ed.

So we were there for like three or four days, kind of talking about those ideas. And we were at lunch one day, and Drew and I were sitting at the same lunch table. And we didn’t know each other very much at all at this point. And we were talking about grading, the whole table was talking about grading.

And someone said something about a colleague of theirs who didn’t give partial credit on their assignments. And, me being me, I just open my mouth and go “What type of monster doesn’t give partial credit? You know, and Drew is this very, kind of like, quiet you know, not, like very assertive person in the way that I am when I just blurt these things out.

And so he just turns and goes, Well, I don’t give partial credit, and here I am, just like…

Bosley: Welcome to the Grading Podcast, where we’ll take a critical lens to the methods of assessing students’ learning, from traditional grading to alternative methods of grading. We’ll look at how grades impact our classrooms and our students’ success.

I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist, and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach and instructional designer. Whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you.

Each week, you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Bosley: Hello, and welcome back to the podcast. I’m Robert Bosley, one of your two co hosts, and with me as usual, your other host, Sharona Krinsky. How are you doing today, Sharona?

Sharona: I am very excited because it is the end of the semester. I just taught my last class. I’ve got finals coming up. So I am very happy about that.

And with Bosley and me, she is one of the co organizers of the grading conference. She also hosts the biology grading for growth community of practice. As an avid crafter, Katie also especially enjoys creating, and I have seen quite a few of these, snarky embroideries, such as Rigor is for Corpses, and published her first knitting pattern this summer, a shawl with representations of the four classes of biological macromolecules.

Wow, what a bio. Katie, welcome to the pod.

Katie: Thank you so much for having me, Sharona and Boz.

Sharona: So we’ve gotten to work with you for several years and I’m really excited to have you here. So Katie, anything I left out of your bio or got wrong? ,

Katie: I think that’s pretty comprehensive. My name is pronounced Katie Mattaini. That’s like a most common mispronounced name, so no worries there.

Sharona: Well, I should know it by now, and I still keep forgetting, so my apologies.

Katie: No worries.

Bosley: It’s actually just nice that you’re mispronouncing a name and not me. . .

Sharona: That’s why you made me say the bio.

Bosley: So welcome Katie, it’s great to see you again. I haven’t seen each other since our last organizing meeting. Something we always like to do the first time we have a guest on is just kind of ask them about their origin story.

How did you get involved in this crazy world of alternative grading?

Katie: So that’s actually kind of a funny story, which I think. The two of you may have heard before, but I’ll recap here. So I taught intro bio for five years at Roger Williams and especially in the fall semester, you have a lot of students coming in and really taking a college exam for the first time and kind of being shocked sometimes by the low grades that they can get.

And I always thought it was really unfortunate that students got so discouraged from persisting in stem because of this really low first grade, especially when you have a traditional averaging system where that low grade is going to stick with them through the whole semester, essentially.

And the whole idea of this deep teaching residency is people who are interested in being more inclusive teachers in STEM, in higher ed. So we were there for three or four days talking about those ideas. And we were at lunch one day and Drew and I were sitting at the same lunch table and we didn’t know each other very much at all at this point. And first we were talking about grading, the whole table was talking about grading and someone said something about a colleague of theirs who didn’t give partial credit on their assignments and me being me, I just opened my mouth and go what type of monster doesn’t give partial credit?

And Drew is this very kind of quiet, not a very assertive person in the way that I am when I just blurt these things out. And so he just turns and goes well, I don’t give partial credit. And here I am just trying to hide and be, oh my gosh, what did I just do?

And fortunately Drew is very forgiving and we became fast friends after that. And that was when he started to explain to me about alternative grading, including standards based grading and how there it’s not like a partial credit system, but there are repeats without a penalty.

And so I actually, one of my snarky, embroideries that I’ve made has the monsters from Monsters, Inc. And it says, “famous last words, what kind of monster doesn’t give partial credit?” And here I am.

Sharona: Well, and I think wasn’t one of the names of your talks or something at one of the conferences, something like what kind of a monster doesn’t give partial credit.

I think that’s become one of your catchphrases.

Katie: Yeah. I mean, it’s just, it was such a perfect moment, in retrospect.

Sharona: Yeah. That’s awesome. I love that.

Bosley: That’s got to be, out of all the stories we’ve heard so far, that’s got to be the funniest one. I mean, we hear these depressing ones about students that traditional grading just destroys, or we’ve heard one recently about just someone diving into the data and the research and doing it that way.

I don’t think we’ve ever had one that got into it by putting their foot in their mouth.

Katie: Yeah. So now sometimes when I give talks, I show a picture of Drew and I’m like, Drew “not a monster” Lewis.

Sharona: Does he know?

Katie: He does, yeah.

Sharona: That’s awesome. That is fantastic. And so where are you at, I know one of the other things I’m curious about is you just switched universities. So how did that go as far as being able to port the grading system with you?

Katie: Yeah, so that was really interesting. And I have talked to a couple people recently who are on the job market asking for advice about how much to lean into their alternative grading, in their teaching philosophies and things like that.

So actually this semester, I am not doing alternative grading, which feels really weird. I feel like, at least for me, it’s been really important in figuring out what your system is going to be, to know your population. And I felt like I needed to kind of get some time to know my population before I really start burning everything down.

I am teaching a different class next semester on my own. And I think I want to do a standards based grading in that class.

Sharona: That’s interesting. I have gotten to the point where I would find it extremely difficult, if given any chance whatsoever, to go back to points and percentages. And then Boz, you’re having some interesting experiences.

Bosley: Yeah, I’m having a similar experience. I’m not at a new institution, but I am teaching in a new department at Cal State LA So, I’m still in the math department, but I’m also in another department and it’s a coordinated class that uses traditional grading. And although I enjoy the class, the grading has, it has been very difficult going back.

If they continue to ask me back as soon as I establish myself, that’s going to be the first thing I do is we want to talk about how to improve this course? Here’s my suggestions. Let’s look at the grading.

Katie: Yeah, absolutely. And I think I do want to do that in the future for this course.

And I agree with you, Sharona, it is really hard, right? Because now you don’t want to be like, I don’t know, the whole philosophy of alternative grading is so embedded in us, right? That it is really hard to do that. I just, again, for me, the knowing the population really helps me figure out kind of what system is going to make sense.

And I do feel like my population that I’m working with here is very different than the last place I was working. And it’s actually really interesting because we have a ton of very high achieving students at Tufts, and so I really think that some of the benefits are going to shift. I think that the distribution of grades may not actually change that much, but I think that the kind of decrease in anxiety is probably going to be the biggest benefit for the students that I’m working with now.

Sharona: Well, and I’d like to springboard off of that to the other thing I want to talk to you, because you just said twice, it’s really important to know your student population. And one of the things that I know you for is talking about equity and inclusion as it relates to alternative grading. So, can you share some of your thoughts on that and how that’s playing out with this differing populations conversation?

Katie: Yeah, for sure. So, I think one of the things that is really important to start out with is that any particular set of tools or any particular set of practices, you have to both have a mindset of equity and inclusion and also a set of practices. So I don’t think that doing alternative grading automatically makes someone an inclusive teacher.

And I think we’ve definitely seen that before. That you have to have equity as a guiding principle, and that will help you figure out which parts of alternative grading and which kind of specific decisions you make around how to set up your grading architecture. You can do alternative grading and not be equitable.

Overall, I think it’s a lot more equitable than traditional grading, but it’s not an automatic band aid, right, for making things equitable.

Sharona: Can I ask you to define what you mean by equity and inclusion for the people listening so that we’re working from the same space?

Katie: Yeah, for sure. So these are going to be very kind of off the cuff definitions.

But I think about equity in terms of giving everyone a chance to show their own best work, giving everyone a chance to show their learning. And equity is not the same as fairness or equality, right? Because equality means you do the same thing for every person, whereas equity means, every person has an opportunity and different people may need different circumstances to give them that opportunity.

So that’s what I’m thinking about when I’m talking about equity. The other thing we can talk about is equity in actual grades, actual outcomes, right? So there’s been a ton of research on, whether you want to call it the achievement gap or the opportunity gap, right? Showing that people from different groups, whether it be different racial groups, different income brackets have either traditionally different scores on things like SATs or different grades in school. So you can talk about equity and outcomes as well, right? Making sure that there are not differences based on those different identities that people have and then talking about inclusion that mainly, I think, is just trying to say are you doing things in a way that everybody can access?

Are you doing things in a way that as much as possible, make people feel safe, make people feel welcome. So that’s what I think about inclusion as meaning.

Sharona: So then I interrupted you to get those definitions. So how has this been playing out for you and what are some of the, well, first, let’s start with that.

How’s it been playing out for you? And then I have a follow up.

Katie: For sure. So the first class I started using alternative grading in was my intro bio class. And so I have used standards based grading in intro bio for the last three years, I guess. And I mentioned before about especially students coming in and doing that transition to college, right?

And I think that standards based grading, and the fact that you don’t have penalties for earlier attempts, really evens the playing field somewhat for students who come in with different levels of preparation. Be that because they were maybe in a school district that had different resources maybe because they are first gen students and maybe don’t know as much about accessing tutoring or accessing different resources or what is an office hour. And I try to do other things in addition to grading to try to make those things clear.

But a lot of times, as I’m sure you’ve seen, students don’t necessarily take advantage of those resources until after they have a bad first exam. So I think it helps somewhat with the different levels of preparation coming in. It can also help with students who maybe just take longer to learn things for whatever reason.

We are still working within the constraints of like a semester or a quarter or whatever time period your school works on. But I, for instance, right now, have a student who is dyslexic and has other learning differences, and so she just, it’s not how many hours a day she spends on something, she just needs more days to get to the point where she understands at the same level as the other students, and she’s a great student, and so it’s really would be a shame to penalize her for that.

So those are a couple ways that I can think of. I think there’s probably a lot that we could talk about, but those are the most kind of obvious ones that come to mind, especially for a standards based system.

Bosley: Equity of outcomes. We had a interview not too long ago. And that’s how.

And I’m drawing a blank on the name. Help me out, Sharona.

Sharona: Eden Tanner.

Bosley: Thank you. I don’t know why I couldn’t think of that. But yeah, we just had, and that was her origin story into it was, before even getting into the classes or anything, looking at the discrepancy of outcomes amongst different demographics and the fact that it was almost universal, regardless of which instructor that they had and saying, yeah, we can’t keep doing the same thing. It is producing this incredibly unequitable in outcomes, which, Sharona, you and I have talked quite a bit on this podcast about some of the origins of our traditional grading system and the fact that that’s what it was actually designed to do.

So it’s doing what it’s supposed to do. And it’s unfortunate that we don’t take that deep look at where these traditions came from and realizing, yeah, we see some of these equitable gaps across the country and why? Because we’re using a tool that was designed to make them.

Katie: Yeah. And I think it’s easy to look at something like that, those equity gaps in the outcomes and say, oh, well, we know why, so I’m in higher ed, right? And so it would be easy to say, oh, well, there’s an equities in the K through 12 system. So, therefore, it’s not surprising that their equities when they get to higher ed and is that true on one hand?

Yes. But I feel like some people use that to absolve themselves of the responsibility of having to do anything about it. And. I think we need changes at all levels and it’s not you can’t just absolve yourself by saying, Oh, we know that there are reasons for this.

Sharona: Well, and one of the things that’s really bothering me about the whole conversation around equity is I’m not sure that all of us who say we want equity have really examined our own beliefs of what does an equitable outcome mean. And what I mean by that is, I’m in math and we have the opportunity or detriment to be able to say a problem done a certain way is the right way of what I’m looking for as the instructor. And so equity to me seems to mean, for some people, we’re raising everyone to the point where they will be able to do what we want them to do the way we want them to do it.

But is that actually equity? Because why is our way of doing it the thing that we should be measuring? Because if you look at the history of mathematics, it’s way more chaotic than the hierarchical course sequence thing that we have. So I’m really grappling with what does it mean to have an equitable outcome?

I mean, I know that in our multi level grades, it means a grade of a certain level that’s enough to let them progress, but what does it mean for the learning and the knowledge? Is it growth? Is it actually being able to do certain things? I don’t know, I’m really grappling with that. I want to get your thoughts because I know you’ve really thought about some of this stuff.

What does it mean to have success? What does success look like for a student?

Katie: Yeah, I mean that’s a really great point and one of the other examples of this that I’ve heard that I think really spoke to me first about what you’re talking about, what is an equitable outcome, is about writing and grading writing for grammar for instance.

So in academia we all use, I’m going to maybe get this wrong because I’m not an English person, but I think it’s like standard academic English or it has some name. Yeah. No, it’s standard.

Sharona: Yeah. It’s standard academic English. Yeah.

Katie: And of course there are different grammars within English. And so I’ve heard a lot of people in various equity and inclusion communities that I’m in kind of grappling with the, how do you deal with writing?

And how do you teach students good writing and what is good writing? And I think writing is something that probably transcends most areas, regardless of what you’re teaching. Probably your students are going to be doing some writing at some point.

Sharona: Well, and the writing differs from discipline to discipline. What I’m looking for in math for writing is very different from yours in biology. Or from an English professor.

Katie: Right. But, something like, and again, I don’t know all of the different grammars, but there are different grammars within certain cultures, certain ethnic cultures, like Spanglish has its own grammar and certain African American grammars exist.

I’m white, so that’s not my area of expertise by any means. But, who is to say that that should be graded in a lower way or thought of as poor writing? And so I think that’s maybe even a step before the grading. And this is where I’m saying using alternative grading doesn’t automatically make something equitable because you still have to ask yourself what are the outcomes we’re looking for, which is exactly what you’re talking about, Sharona.

And that is a that is a whole conversation that I don’t think any of us have solved.

Sharona: No, but I think we need to have it. We need to, even amongst ourselves that have adopted this, we need to take the next step and be willing to examine those beliefs that we still have, even if we’ve already overcome some of the grading ones.

Katie: Yeah, and in terms of the writing, I know that Asao Inoue has done a lot of work around grading and writing with this issue of kind of different grammars, and I think that his work supports the idea of what’s the word I’m looking for? More of like an effort based grading, like how much time do you spend in your writing?

How many revisions do you do of your writing? Can you identify the audience that you’re writing to? Instead of doing things like are you following the correct grammar of what we in academia would use, type of thing. So that’s a really interesting take. I think that that’s hard to know how to translate that into STEM, right?

Because I don’t think that, just for us, that just effort is going to necessarily produce the outcomes that we want. And so I feel like there needs to be some more thought in how can we bridge that gap in some of his work and what does that look like in STEM? I’m not sure yet.

Sharona: Well, go ahead Boz.

Bosley: But that’s just it. You’re talking about one of the presteps to even setting up your grading architecture when you’re looking at trying to redesign a course for non traditional grading is going back and looking at what is the purpose of your course. And that does have a lot of freedom if you take the time to really do that and you can set up your grading architecture to support that.

But, and this is one of the two biggest mistakes I see new practitioners doing is just try and, they want to jump straight into it without doing some of this looking inside of themselves and their course to see really what is important to them. And what is the purpose of the course?

Because you’re right. If the purpose of my course is one thing, then maybe, and efforts or a contract based type of alternative grading will work really well, whereas if it’s some sort of sequence and this is a foundation of a sequence and you really won’t be successful in the following courses without that foundation, then yeah, maybe it does need to be more specs based or standards based. So that first step in this process really is not something people should be skipping and far too often, I think they do. Especially those new practitioners cause they just like, we’re bought in, we’re fired up.

Let’s do this. Let’s jump into it. That ends up being something that ends up hindering their progress.

Katie: I absolutely agree. I feel like that kind of backward design process is, like you said, a prerequisite almost for figuring out your alternative grading.

Sharona: I’m going to be a little contrarian here.

Because when I started, of course, we didn’t have the communities we have today. So I definitely did the, I’m just going to jump in. And I think it’s okay to jump in and mess it up, because I still think most of our mess ups are better than what we have. And helping people not give up on this and iterate because that’s what that’s what I did, I jumped in, I didn’t do any of this intentional stuff.

I mean, I did a little bit. I came up with the story of my course, but I’ve done so much more since I did that. I think maybe we should be telling people don’t let it stop you that you’re not ready to do this or that you haven’t done this. Go ahead and do it. But be aware, you’re then going to come back to it because these things are going to come up.

So I’m going to push back on that a little bit because yes, it’s a mistake, but I’m kind of coming to believe that you have to make the mistake for yourself or it’s not going to make any sense. I don’t know. I’m willing to be pushed back on though.

Katie: Yeah, I think the only, the thing I worry about there, Sharona, is if, like, there’s a total disaster and then people are like, Oh, well, this alternative grading stuff doesn’t work for me.

Or doesn’t work at all. Right. Right? Because I think we all also know people who have tried it, it goes poorly, and then they’re just, go back to traditional grading. So it’s tough cause there’s no one size fits all.

Bosley: And, quite honestly my first attempt at it, if you took those last two weeks of that semester away, I would have never gone back to alternative grading. Those last two weeks, which is where I had those conversations with my students that have absolutely sold me on this and that that conversations wasn’t about how do I get more points? How do I chase these points? And instead was about the math. But if those two weeks didn’t happen after that first attempt that you and I did I wouldn’t have continued with it.

Sharona: So maybe I can moderate my comments to say, don’t let it stop you, but be aware, your disaster might be worse than you think. So if you’re going to do this without having taken the time to do this work, you have to commit to doing it for more than one semester. Like you’re not allowed to give up.

Katie: That’s fair.

Sharona: Pinky promise. All of you out there considering doing this, if you’re going to do this without doing some of this other work. Pinky promise you won’t give up after one semester.

Katie: Yeah, I think that I actually had a pretty smooth first semester doing this compared to most people, but I spent a lot of time thinking about it before doing it, and I had the luxury of being able to do that and having the community that we have and lots of people who are willing to help.

So like I said, I was just super lucky that my first time went fairly well and had to make relatively small changes.

Sharona: Yeah, no, I jumped off the deep end with three weeks before the semester and no community. And that wasn’t the one that Bosley and I did together. That one we did spend a year preparing and it still was a nightmare because we made some pretty bad mistakes.

But, yeah. So I just, it’s really just striking me as I do more of this work and I hear about the pushback on the DEI initiatives and things like that, that I really feel at some level the issue is the conversation isn’t deep enough, even at the highest levels of DEI, where they’re really, well, and even in some, going back for a moment to that standard academic English, we have fields that have are steeped in white elitism in this country.

I mean, the STEM fields are absolutely steeped in it. Centuries, in math, we have centuries of Eurocentric male white dominance and because I also teach history of math. So that’s really eye opening. So how do we transition? Because the grading gives us, I think, the opportunity to start to have these conversations.

But at least right now, we’re all still working in a structure. where they’ve got to come out of our class and function in an existing system. So what can we do and what can’t we do within the grading itself to facilitate equity and inclusion while understanding the structures that we’re working within that we’re trying to change?

Katie: Yeah, that’s a great question. I’m probably not going to be able to solve that entire problem for you. I have heard a lot of people give this argument, well not argument, but like a concern that they have with alternative grading is, Oh, well, my students or your students are going to have to go on and either have a later class that’s not alternatively graded or for me as a biologist, they’re going to have to go on and take the MCATs.

And that’s again, one of these super traditional exams. Right? And so they’re like, you’re not preparing them for X, Y and Z. And my response to that always has to do with, well, what is their learning? Is their learning better? Right? Because if the learning is deeper, if the learning is more lasting, if they’ve actually had a chance to go back and review things, because I’m sure we’ve all had this experience where in a traditionally graded class, a student gets a 75 on the exam, they look at it, they see that they got a 75 and then they just throw it away. And they maybe never revisit it until the final or maybe never at all.

And one of my favorite comments I got the first semester I did standards based grading was that the grading system almost forced you to go back and review the things you didn’t learn the first time. And I was like, yes, this student gets it. They know what I’m going for. You know, and so my thought is, yeah, they’re maybe not having as much practice in one of those traditional settings, but if the knowledge is there, they can always practice doing more of these traditional things.

You can take practice MCATs, you can do practice problems for other classes, but to me, the most important thing is that they have that foundational. knowledge. I also think that that is a really bad argument. The idea that you’re not preparing them for other people doing things traditionally.

I think that’s a really bad argument for not changing because it’s kind of like saying, Oh, well, your kid might get bullied at school. So you might as well just bully them at home. So they’re prepared for it. It’s just such a weird argument, right?

Sharona: Well, and I keep making the argument that all of those tests are actually the ultimate in mastery graded because, first of all, the scoring is all whack a doodle, right?

Because it’s not based on points of, I think the GRE, if you get 50 percent of the questions correct, that’s considered like top notch. And then also you can retake them.

Katie: Right.

Sharona: You retake them.

Bosley: And they don’t average those.

Sharona: No. So I actually would argue with the alignment there. Yeah.

Bosley: And show me a doctor or an architect or an accountant that has to tell you how many times they took those qualifying tests.

Like, do you know how many times your doctors took? Like I said, I made the analogy, I’ve been driving for a long time and I have been pulled over several times. No cop has ever asked me how many times it took me to pass the driver’s test, which was more than once for me.

Sharona: Right.

Katie: Well, and not to mention that things in medicine are like constantly changing. So if you have a doctor who’s been practicing for like 30 years, half the stuff on their certification tests or whatever it’s going to be different now. The recommendations are different.

Sharona: All that being said though, again, Dr. Eden Tanner, we just recently had an episode with, she teaches gen chem and they have a mandatory final for accreditation. That’s the American chemical society exam. I think that’s what ACS stands for. She had 169 out of 170 students take that final and they all passed it. All of them.

Katie: That’s amazing.

Sharona: And that is a standardized nationwide test that is scored essentially traditionally.

Katie: And I think we need more people collecting data like that, because I know, for instance, so Bryan Dewsbury, who ran the DTR that I did and that Drew Lewis did, he redid his first semester intro bio class to be a lot more inclusive, and in doing that had to cut a lot of content. And so, of course, there were people who were concerned about the students not being prepared, and so he has some data looking at how students from his section did in the next class they took in bio versus more traditional sections.

And he saw that they did just as well in the next class, if not better, regardless of the fact that they cut content to focus on more of these inclusive practices. I think we need a lot more data like that to make these arguments to people who are skeptical.

Bosley: And so if anyone’s listening and needs a education research doctorate idea, here it is for you.

Cause yeah, we’ve heard this argument a ton and I have seen several pockets of data that says, no, you’re absolutely wrong. At worst they do as good as traditional classes and oftentimes they do much better.

In fact, Sharona, you and I met through the slam program through college bridge. And their data, the students that went through that, their success rate in the following college classes and just their persistent rate in colleges were several magnitudes higher than other students from my high school and from that same area.

Sharona: And what’s really fascinating is the ones that failed the class. Right, Boz?

Bosley: Oh yeah, no, I had regardless of if they passed or not, cause about that first year I did this SLAM program we had something like, oh, I want to say 76 or 78 percent pass rate, but more than that ended up going to college. So even my students had failed.

So I’ve seen tons of little patches of this data. No one’s just put it, we’ve got to put it together. Like we’ve got, so if anyone is, well I don’t think I want to get my doctorate.,

Sharona: Well and that’s the thing is some people look at our DFW rates and they’ll say, well, you’re not doing any better than traditional. And it’s like, well, the overall pass rate might not have changed.

Sometimes it does, sometimes it doesn’t. I argue actually it has improved, but that’s a whole different story. It’s really interesting when you look at who doesn’t pass and what they do next. That’s fascinating. And who does? So if you compare our course to a traditionally graded course, our students who are succeeding and our students who are not passing, both groups are persisting longer than both groups in a traditionally graded class.

Yeah. And I think that that is just very powerful.

It would be great if we get to the point where we have enough individual studies to have somebody do a meta analysis like that on alternative grading.

Sharona: I completely agree.

Bosley: That’s interesting. One of the other things I’m a big proponent of is professional learning communities inside education.

And the people that I’ve read the most are the DuFor series, rest both of their souls. But that was one of their points is if you look at the research and if this was a medical clinical trial, if you weren’t teaching in PLCs anymore, you would be fired for malpractice.

I mean, it’s kind of odd how we can ignore data and results and then complain about not having results. I don’t know. I don’t get it.

Katie: Especially in the STEM fields where like data is our whole thing.

Sharona: Well, quite frankly, I think that there are some folks for whom the current system not only does what it was designed to do, but it’s what they actually still believe.

It’s very challenging, I think, to have gone through the level of education that we’ve gone through as STEM faculty and not feel a little bit threatened when we’re told that maybe the skills and the things that we were able to do might not actually be necessary. Because we overcame these systems, right?

We figured out how to succeed in these systems. And some of us are very proud of that. I know that I’m proud of my persistence across having failed some of my qualifying exams in analysis six times and finally passing, that’s a mark of honor. And at the time it was like, wow, how did I get an undergraduate math degree from the University of California, Berkeley, by the way, which is supposed to be the premier undergraduate math institution, and I was not prepared to pass my qualifying exams for a master’s.

So it’s a little bit confronting for faculty.

Katie: Absolutely.

Sharona: And so that I think that’s part of where the equity resistance is coming from. Now I want to switch topics a little bit more because Katie, I know you in a certain way, having worked with you so long and you are what I would call part of the burn it all down group.

Katie: Yes, I would agree.

Sharona: I’ve not heard a lot of burn it all down here. So do you want to share a little bit about how you do your cancer bio class? And what do we mean by burn it all down? Because you’re sounding a little bit mainstream here.

Katie: Sorry, I need to turn it up a notch, apparently. So again, I kind of told the story already about Drew Lewis. He’s another one of our co organizers. Yeah. And he and I, I think, on the organizing team, consider ourselves the burn it all down contingent where we’re like all of these traditional structures have to go like now.

Yeah. And so my cancer bio class that I taught again at Roger Williams was much more probably those more traditional folks would call it very loosey goosey.

And so I used what I’m starting to call collaborative grading. Some people would call it ungrading. I don’t know how much you’ve talked about the differing meanings of the word ungrading.

Sharona: Well, we had Jesse Stommel on the podcast so you can imagine.

Katie: Yep. Okay, cool. Cool. So I think that oh my gosh now I’m in a blank on the name of the person, but some of us have started calling it collaborative because most of our institutions require us to give a semester grade still at the end.

And so in my cancer bio class, this is an upper level class, mostly juniors and seniors. It isn’t a prerequisite for any other class. Really, the two big picture goals are to get students to learn enough basics of cancer biology that they could then go on, and if they have questions about it in the future, know enough of the basics to go research their questions and figure it out because I’m sure in math, you have a similar thing.

Like, you have to know a certain amount about something to even be able to learn more about it. Right? And so I’m trying to give them that kind of base level. So that’s one of my goals. And then the other goal is to really, really focus on reading research papers. And of course there are lots of other classes in that department I was in where they’re reading research papers.

I think a lot of times, at least in places that I’ve been, there’s just kind of this assumption that if you give students enough papers to read that they will figure out how to do it. And I think sometimes there needs to be a little bit more intentionality around teaching students how to read papers. Especially if you’re really digging in and trying to be able to critique what controls they use or think about what would the next experiment be?

And so that was the other goal of mine was to really get students. much more comfortable with reading papers. And my experience, again with this particular population I was working with, was that students were afraid of reading papers, that they really disliked reading papers that, so cancer, it’s a topic where you take a lot of the basic areas of biology that you learn about genetics and cell biology and molecular biology and they all have to kind of come together and so the papers can be quite complicated if you’re not used to it. The students were really intimidated by reading the papers.

And what I found was that using this type of collaborative grading where students don’t get marks at all on individual work, they just get feedback, really let them try to do something that was hard for them without being penalized for that process. And so that I had a ton of success with. In fact the students, their final project was to write articles for this site called SciWorthy, which is a basically a scicom site where the students would choose a journal article that had come out in the last couple of years.

They would read the article, understand the science and then write a short explainer piece for the general public. So they would have to understand all of the deep science in their research article enough to explain it to a non scientist in a short number of words. Which is really quite challenging to do and I always got over 90 percent of the students would get their articles accepted to SciWorthy because they would send in the articles and they would go through a review process with the editors there.

So that was one of those situations where like what Boz was talking about, you have certain goals for your class. And for this class, this type of collaborative grading where they didn’t get any numbers or letters on their individual stuff was really the perfect fit, I think, because I was asking them to do something that was so much outside of their comfort zone, that’s so much like a learning process, and that worked really well for that class.

Sharona: Well, because you’re giving them the space to take risks, right? I mean, they have to take these risks. And if they know that the worst that’s going to happen is they’re going to get feedback on their risk, that’s not a bad result.

Katie: Yeah. And of course, that is kind of burn it all down because this idea of not having any letters or any numbers or even any type of like check no check on your work can be quite scary.

Sharona: Well, and then of course the other component of burn it all down is I think you would be super happy if end of term grades didn’t exist.

Katie: Yes, that would be great. If we could just focus on the learning. I mean, it’s interesting because the system that we work in, the grades are a motivator. Whether we want the grades to be a motivator is a different discussion. I think something that I’ve started to think about more is some students who need more external motivation, for instance, a lot of neurodivergent students, tend to function better when they have more external motivators, and so one of the things I’ve been thinking about recently is how to provide external motivators that are not grade based.

And I know some people who will do things like sticker charts. In my last time I taught this cancer bio class, one of the pieces of feedback I had gotten from an ADHD student was that she had trouble forcing herself to read the papers far enough ahead of time. And so one of the things I incorporated was a students only discussion section that met a couple days before students were doing their paper presentations in class, so that I knew at least they would set aside an hour at least to talk about the papers a couple days before we had the presentations as a way to try to motivate preparing ahead of time without it being grade driven.

The students really seem to like that. I don’t think they viewed it as a motivator in that way. I don’t think they knew that’s why I had them doing it. I think they enjoyed the ability to discuss with each other, which is obviously another benefit of that. But yeah, I’ve been trying to think a lot about external motivators that are not grades.

Bosley: That’s interesting because we came across that same thing, Sharona, with our late policies and our due dates. You and I both believe that when things are due really shouldn’t be that big of a deal, but we found the same thing, that our neurodivergent students needed those deadlines.

They had to have something there. So yeah, that’s something we do. We put due dates and they’re kind of like the points and whose line is it? The due dates are there, but they’re not really due dates because we’ll take things.

Sharona: Well, some of them are in the sense that we’re trying to communicate to students some realities that there are certain things that if you’re going to get the benefit, they need to be done at a certain time. Now, if you don’t want to do that, that’s your choice, but there’s consequences to that, right? So what we have is we have points available for doing preparation work and we don’t accept that late at all. Because once the class happens, because it actually happens in the real world that class happens, the preparation work isn’t any good.

Now what we’ve done to compensate for that though is, yeah, you don’t get those points if you don’t do it. So what you end up having to do is that if you want the grade in this one points accumulation standard, you just have to do more of something else. So, okay, you chose not to prepare. Guess what?

You’re going to have to do more practice. But that’s the real world. If you don’t get something done on time, there are natural consequences to that, which might be more work. It might mean have to admit to your boss that you didn’t get it done. I mean, I don’t want to penalize you just to penalize you, but I also want to try to model the real world.

And so we have this learning outcome. That’s literally intensely like, Hey, we think this is how you learn. Here’s these different categories. If you don’t need any of that, actually, then just don’t do it. You might have to show us that you’ve learned more to get the same grade.

And especially I feel it’s very inclusive because of those neuro divergent students who have said, I have to have these. And let me tell you, those students are religious with those due dates. I know that I don’t work well without deadlines. So we’re trying to both accommodate people, but then you have someone like Bosley who how much preparation homework stuff did you do in math class in college?

Bosley: Prep stuff?

Are you kidding?

Katie: He’s like, what’s that?

Sharona: That’s why I asked you the question.

Bosley: Well, yeah, no, especially in high school, that used to be my favorite thing, those last few days where you would return your books to your teacher, I’d go there with my math books. I would open it in front of them to make the cracking noise to show that it hadn’t been opened all year.

I was such a terrible high school student. I mean, I still talk to some of my teachers and they’re like, You were such a jerk.

Katie: I feel like that’s such a power play, Boz, to be like, look, I didn’t even open it.

Bosley: Oh, that’s exactly what it was. I was an arrogant little prick when I was in high school.

Sharona: Describe your test taking and pen strategies.

Bosley: Oh, yeah.

Katie: Oh my god.

Bosley: I would sit where I knew the teacher was going to pass the test out early so I could get it. And then, in pen, I would try to finish the test before they finished passing it out so I could obnoxiously go, I’m done. I’m done.

Sharona: Now, the thing is in a traditionally graded class. Someone like

Bosley: I was rewarded.

Sharona: No, someone like someone like Bosley, if they didn’t do their homework, because I don’t think you did any of your homework either. It hurt his grade, but yet he knew the math better than everyone. So, and then one of my kids, and apologies to the kid, if they ever listen to this, is the one who’s like, Hey, there’s five minutes left before the deadline.

That gives me four more minutes to procrastinate. And the issue was he wasn’t as good as Bosley. So he would run out of time and not succeed that I would be looking at him like, wow, you just did the maximum amount of effort to still not succeed.

And then my other child is like, they’re not really children anymore, they’re both in college, but my other one’s like, oh my gosh, I was assigned something due in three days. Let me get it done right now because I can’t handle the pressure that there’s a deadline. I just want it done. So, two completely different students.

Katie: Well, I think that’s really interesting what you’re describing in terms of your points accumulation standard.

I do something similar in my standards based graded classes, especially intro bio, because like you said, you want to try to encourage habits that are going to help students succeed. And I’ve heard kind of two different arguments for what is the most equitable way to approach that type of work.

Some proponents of alternative grading say you should only have grades that are based off of your final projects, your final exams, whatever. Why am I blanking on the words for these things? Formative,

Sharona: summative,

Katie: you should only have summative assessments, count for the grade..

Whereas there was a recent book that came out. What is it called? It’s just called like inclusive teaching. By Vijay Sathy and Kelly Hogan. And their, again, it’s all about inclusive teaching and equity, and their argument is that you need to show students what is important and therefore anything that you’re going to ask them to do has to count towards the grade as an incentive.

So those are two diametrically opposite views on how you should grade this kind of formative work. And so I’ve, there’s been some discussion in the kind of grading and equity communities around which one of those is true. And I think what you just described is, in some ways, ideal.

Because you’re saying, you need to do stuff to learn, like learning requires work.

Sharona: So my argument, because I’ve heard both of those, and I think they’re not as diametrically opposed as they think they are. My argument is that if one of your goals is to teach students how to learn, you should have a learning outcome on it.

And your definition of summative might change to quantity. Because if you’re teaching students how to learn and you think quantity matters, then you should be measuring quantity. And that’s why I do points accumulation, and I mean accumulation in that I don’t do percentages. So it’s like I have a big pot of points and you need to get so many.

And if you get to the end of the semester and you don’t have enough, guess what I’m going to do? I’m going to add more points to the pot to let you try to do it. You don’t necessarily get to go back and do things that I don’t think are valuable anymore because the time for that has passed, but Oh, you need another a hundred points.

Well, here’s a reflective essay on your learning that you can complete and get these points. And I do think points for accumulation is a reasonable measurement. It is a reasonable measurement tool, because it’s quantity. So you can use quantitative measures on quantity. It’s a weird thing, so.

Katie: On the other hand, if you’re a high school Bosley, maybe you don’t need to do any of those things. I don’t know. So it’s hard, right?

Sharona: Well, I think the thing is, it depends on your final grading architecture. What do you call an A? Because for me, an A is not all of my learning outcomes. It’s less than all. So if you’re a Bosley, you have to get all the content, but you don’t have to show me, you’ve learned how to learn.

Cause guess what? You showed me the content.

Katie: Yeah, actually that’s a good point because when I’ve done this, that has also been true that you could get an A without any of those. You could get zero learning community credits I think is what I do. And you could get zero of those and still get an A if you’re a high school Bosley.

Bosley: Yeah. So it doesn’t, it’s not preventative. It won’t stop you from getting any grade. It might help you go up a grade. But…

Sharona: So I think where I’m landing and I want to get your take on this, Katie, is that the inclusion and the equity starts not only before the class, as we talked about before, but actually in your learning outcomes too, that it’s not just your disciplinary content, the way it would be in a traditional class where it’s like, can you do this biology thing or that biology thing or this math thing or that math thing, but it’s also, can you do writing the way your discipline needs? Can you do reading the way your discipline needs? Can you do teamwork?

It’s all these 21st century skills, right? So, what’s the process that you would use or that you do use that you would recommend to people to really think broader about their learning outcomes than just their content.

Katie: Yeah, I mean I think that this is kind of something that has been around for a long time, which is like backwards design, right? So you say what would a student coming out of my class be able to do that they couldn’t do before? And in be able to do, like you said, I think it’s just, are you thinking broadly enough?

It’s really interesting if you’ve ever done the process of looking at departmental learning objectives or program level learning objectives. Because I think a lot of times we think quite broadly when we’re doing those because you kind of have to but then the question is do our individual courses actually build up to those program level learning goals. So I mentioned in my cancer bio class I really wanted to give students time, space, the ability to take risks when learning how to read papers, for example.

And you’re asking me about learning outcomes, but I think the other thing that’s really important is not just assuming that those things will happen. So if I want them to become better at reading papers, I have to model how I read papers. So we spent the first couple of weeks of class actually doing some boot camps, both on the methods that were in the papers and really breaking down sections of papers, even though they’ve seen those before. What is really being accomplished in each of these where can you find certain types of information, and then oftentimes I would teach on my iPad, so I would show them like, look, this is the paper that I read, the copy of it that I read and look at all these drawings I’ve made around it and stuff like this.

This is one option for you to kind of keep track of what’s going on. And so I think that this also applies for things like creative thinking, reasoning, things like that. We assume that those things will just happen. And I think we have to be a lot more intentional, not only about including those things in our course outcomes, but then designing learning activities that will result in those outcomes. I think that we’re pretty good usually at designing learning activities for the specific biology tasks or math tasks, but less so for these more big picture skills.

Sharona: I am participating in a program with my Center for Effective Teaching and Learning right now, with my department, called Career Relevant Course Design, and they are trying to train us on how to identify the relevant career skills that you need in our discipline, and then also how to teach them.

And so when we brainstormed as a department, what the top three skills were, we all came up with instantly problem solving was number one for math. And then the drop off from that was very interesting. Like we couldn’t even really agree on two and three.

Because math research, which is not a big focus, we’re not an R1 institution. So our faculty are all PhDs, and they all have research interests, but we top out at a master’s degree. The majority of our students are not math majors, we are definitely a service department as most math departments are.

So when we’re talking about career relevant skills, are we talking about math research skills? Not really. Versus the skills of problem solving and perseverance. And even if we look at our master students, most of our master students are a lot of them are going to go into teaching. So, but we don’t have pedagogy trained faculty.

So it was really interesting to kind of argue about this and then try to translate it down into our classes to say, what would this look like and how do we elevate this skill? And so that level of intentionality is yet another element of this.

Katie: Yeah. I think that group work is another one of those skills that’s highly, highly relevant for any type of career, almost, that we rarely give any instruction on how to work in a group well.

And I think, again you probably both know that many students hate group work because of all the problems with dynamics that occur. And there’s a lot of potential inequities that happen there as well. So that’s another place that I know I need to give a lot more thought to in terms of how to make group work as equitable as possible and how to instruct students on techniques for doing group work well.

Bosley: Yeah, that is definitely one of those things that, especially in my K 12 world, you see group work everywhere. It’s very common now, but doing it effectively, and especially setting up the structures to make sure those groups are actually equitable and cooperative. Because the 21st century skills, the four C’s of the 21st century skills, doesn’t say group work, it says collaboration.

So setting up those groups in a way that is structured to be both equitable and actually encourage collaboration is not always done as well as, for as much group work as I see, it’s could still have some improvement there.

Sharona: So we’re coming up on the end of our session. I do want to ask you one last question, which is if someone is approaching this world of our alternative grading coming from an equity mindset what advice do you have for them about the opportunities and the pitfalls of the intersection between alt grading and equity.

Katie: That’s a great question. I think we’ve already talked about one of the pitfalls, which is just assuming that you can kind of copy and paste someone else’s grading architecture and that somehow magically makes you more equitable.

I think that is definitely a pitfall. Again, for me, one of the most helpful things has been knowing your demographic, but that can be quite hard if you’re like, going to be a new faculty member or you’re an adjunct going into a new institution. So, I think any kind of research that you can do, maybe talking to other faculty who already teach at the place you’re going to, to kind of get a sense of what are the students there like?

I think both in terms of like, so I mentioned now that I’m at Tufts, I just see so, so many students who are extremely high achieving, extremely grade focused in the sense of, especially in biology, the, I want to apply to med school and my GPA is really important and all of that, whereas some other places I’ve taught, students priorities were like, can I pass this class, right?

And I think the way that, you don’t necessarily have to do things differently, but I think for me at least it does make a difference in what I’m going to prioritize. I think finding out as much as you can about your students in terms of like, what is the motivating factor? If you have a bunch of students who are I don’t know what we’re calling them now, but non traditional students in the sense of maybe they’re over 22, 23 years old and have maybe jobs or families.

If the students are commuters, if you know how many of your students are in the lower income brackets, Pell Grant eligible, do your students have access to their textbooks or is affording textbooks a problem? We know that that’s a problem throughout higher ed.

So I think as much information as you can get about your students ahead of time, I think that’s really helpful. I’m trying to think if I have any other kind of big picture things. Yeah. I just think that approaching it with, I think something I haven’t really talked about a lot is just the way that we approach it, approach like teaching and approach students, right?

I kind of think that always giving people the benefit of the doubt, like everybody in some way is there for a reason, whether they want to learn inherently, whether they have some career goal that they’re there. And so I think if you approach everything with a “how can I help this person learn” mindset, that is one of my guiding principles that helps me figure out what to do if a situation comes up that I haven’t considered. And that happens all the time, right? Situations come up that we haven’t considered, and you have to have guiding principles that are going to help you figure out what you want to do in that situation.

Sharona: Well, I think that is an excellent place to leave those words of wisdom there. Boz, any final thoughts before we sign off?

Bosley: No, I just want to thank you, Katie for joining us. I know it’s the end of the semester, which can be a crazy time. And I do want to say that I’d love to have you back. Either right before or right after the grading conference, to have you come on with us and talk some about the conference.

Katie: Absolutely

Sharona: Thank you everyone for joining us for another episode and we will see you next week.

Please share your thoughts and comments about this episode by commenting on this episode’s page on our website http://www.TheGradingPod.Com or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the contact us form on our website.

The Grading Podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.

Bosley: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State system or by the Los Angeles Unified School District.

​