The Grading Podcast

Robert: Don’t wait for somebody to give you permission to try the things that you feel to be right for your students. You are the only thing, only permission that you need. The only thing restraining you from helping your students with fixing, changing your grading system is your own creativity.

That’s the only thing holding you back. Everything else can be worked around. So just kind of take courage. Doing things that’s right for students takes a lot of courage. Make the step and you will be rewarded for it.

Boz: Welcome to the grading podcast, where we’ll take a critical lens to the methods of assessing students’ learning from traditional grading to alternative methods of grading.

We’ll look at how grades impact our classrooms and our students’ success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach and instructional designer.

Whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week, you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Boz: Hello and welcome back to the Grading podcast. I’m Robert Bosley, one of your two co-host, and with me as always, Sharona Krinsky. How are you doing today Sharona?

Sharona: I am doing spectacularly well, I am so excited. We’re coming up on the conference this week, but this episode is going to be so much fun and I don’t even wanna like banter about our lives ’cause I want to get to this episode.

Boz: Well, what’s special about this episode?

Sharona: This is episode 100. Woot woot woot!

Boz: That is right. Episode 100, which you and I both looked up and are both, I think, quite surprised. It’s just over 5% of podcasts that actually make a hundred episodes.

Sharona: Exactly. And I mean, and I still remember, we’ve told this story on the pod before, but when I suggested to you that we do a podcast and you said yes, and then I said, because I like to hear myself talk and I think you almost spit soda all over the table of the restaurant that we were at at the time.

We were in San Pedro for one of the theater shows. Remember that? We were at the Omelet Waffle Shop in San Pedro, California. Any listeners from San Pedro, California, some of the best waffles I’ve ever had, but yeah, so 100 episodes, not quite two years later.

Boz: But so this being the a hundred episode, we have to do something special.

So are, who all do we have in the room with us Sharona?

Sharona: So we’re calling it the episode of the OGs. We have Robert Talbert, Dave Clark and Kate Owens on with us. It’s the band getting back together, the original organizing team for that very first conference. So welcome y’all. Just in case somebody doesn’t know you, I don’t know how that’s possible, let’s go ahead and do brief introductions. Who you are, where you’re at. And then we’re gonna talk about how we all met. So let’s start with Robert.

Robert: Well, thanks guys, and congratulations on a hundred episodes. I’m Robert Talbert. I’m professor at Grand Valley State University in the math department. Coming to you from my practice room in my house in Allendale, Michigan.

Sharona: And Dave?

David: Hey, I am Dave Clark. I’m also at Grand Valley State, although in a different place than Robert. I’m also in the math department here. And coming to you from my home studio in Grand Rapids, Michigan.

Sharona: Well, you’re both in home studios. His is just a music studio. And Kate?

Kate: Hi. Yeah, I’m Kate Owens. I’m from the College of Charleston in Charleston, South Carolina. I don’t have a home studio, so I’m in my dining room, but it’s great to see everybody.

Robert: It’s a food studio.

Kate: It’s a food studio.

Sharona: Exactly. Do you do a lot of social media posting, taking pictures of your dinners with your children?

Kate: No, you know, I haven’t done that. Usually we eat outside and usually it’s a phone free environment. But I should do that. That’s a good idea.

Sharona: Awesome.

Boz: Well want to welcome all three of you. It is so great to see and have all three of you guys on. We’ve had all of you on, I think we’ve even had Robert and Dave together once, but I don’t think we’ve ever had all three of you guys at the same time. So this is really fun.

Sharona: So when Bosley and I were sort of thinking about what we wanted to talk about, so many things are coming together. ’cause this is the 100th episode. It’s going to come out the day before the sixth grading conference. And for all of us, I think the conference sort of kicked off so many things, but I wanna go back a little bit further and figure out what was the path that the five of us ended up coming together and doing this craziness.

So I think it started probably with Robert and Dave. You, I would assume, were the first two that connected.

David: I think so. Yeah. I think so. And I keep really serious email records, so I’ve been looking up data for this. Yeah. Robert.

Robert: That’s such a Dave thing to do too.

Robert: Really? I thought that we were talking about the book first. I thought, that’s surprising to me. I guess. Yeah. Okay. Yeah.

Sharona: Well, so, but before that you guys had already been talking, grading, so when did you guys start talking grading? I guess that’s my question.

Robert: Oh man. That’s undeterminable I think because at the time, David. And I I guess we are still kind of, our offices are still kind of in the same area of the same floor at Grand Valley State. So we, but we’re near an intersection of two hallways, and so we were constantly like literally bumping into each other.

And we have a lot of great people in that department who have been doing alternative grading for much longer than I certainly have. And I think, it’s probably been percolating in our brains. And David had been actually getting it done in the classroom, in your geometry class and some other stuff.

And I think I was trying to pick your brain about, Hey, what is this all about? How do I make this work? And so, the good part about being in a physical location with a bunch of really smart people is you run into each other a lot. And that’s the genesis of that whole idea was just me looking for better ways to do stuff. And I have a lot of great people to talk to.

Sharona: So if you had to guess on a year range?

Robert: Google Plus. Okay. Yeah. Well, so I’m super old, like, my memory is really shot. So whatever you say is correct, Sharona, that’s an axiom that I live by. Whatever Sharona says is right.

And that’s sort of what launched me and my series of blog posts around that time was me trying to think through what my philosophy was and what I thought was working, or just keep a track of, you know, I tried this and here’s what went well and here’s what I’m gonna change it a little bit next time to hope that it goes a little bit better. So for me, it was part of the building process for what I ended up doing.

So I think that we did at least a couple of those for a couple of years.

Sharona: Yeah, and so then I was started, I joined the Google Circle and then I didn’t know that Robert was the keynote at Poly Teach.

So I actually missed his main workshop, which was a flipped classroom workshop. But then

Robert: that was an interesting conference.

Sharona: It was, but then I’m like, oh my God, this is like this superstar guy that I’ve been talking to on Google Plus, and he and I ended up in a, it must have been an hour and a half conversation out on the patio.

Robert: Yeah, I think it was. And first of all, superstar and Google Plus are not usually turned near the same set for those, those of you, young whipper snapper who don’t remember, Google used to have its own social media platform called Google Plus. It was really a great platform, I thought. And that was where kind of the community I think really started coalescing.

I know like, like Sharona, you were in on that and it was very small but very powerful at the time. ’cause it was like the first time, I think a lot of people in the higher ed world really kind of got together, quote unquote online and started really hashing this stuff out. And that ended up metastasizing into the Slack community that is now currently, and still in place, I think after Google Plus shutdown.

Robert: Who is scared of Bosley? He’s like the nicest guy on earth.

Sharona: Well, I was scared because I was coming into a program that he had co-founded with another faculty member.

But I had some like weird expectations that had been set for me as the university professor. So I was in some way supposed to supervise him, but yet he had founded the program. So I literally was like, I have to be super careful around this guy. ’cause he could get mad at me and then I’d be in trouble.

Boz: Which I had no like power or anything to get you in trouble, but.

Boz: No, no. We were really still talking about the purpose of the class. Which is interesting because now in our trainings, that’s one of the first things that we talk about, of how you redesign a course is really examining what is the purpose of the course. But that was, what you and I were talking about because it was set up as an a not traditional statistics class. Like this was, this class was meant to be a critical consumer of statistics instead of a producer of statistics.

So it was different than most entry-level college stat classes. And that was something that your predecessor and I don’t have her permission, but I’m gonna say it anyways ’cause I think she’ll be okay with it, but Dr. Kristen Webster had brought. And something that I had bought into so much that I did not want anything coming in and changing it.

But I do think it’s interesting that that’s where we started. We eventually got to grading, but that’s now where we kind of tell people that’s where you should start.

Sharona: Well, and what’s interesting though is the purpose of the class, yes mathematically, it was critical consumption of data, but the actual purpose of the class was to avoid remediation.

That was literally why the class existed, was to take seniors in high school who were almost guaranteed to place into remediation and get them through a college level math class while they’re still in high school, so that they didn’t have to go into remediation. And so when I was at Poly Teach the next year I was also involved in course redesign for the new math Pathways, the Dana Center Math Pathways, removing remediation.

And Sharon was co-author on one of those papers. Yes. And I believe that paper, which is about the same class you’re talking about, also eventually became the basis for your case study in our book. Yes. So all of all of these things just went together and at some point, I mean, you know. You were a name on the page at that point.

And then I think we met at Math Fest, big summer conference. And I remember having like an enormously long conversation with you while we were waiting in line to get ice cream. That was some super popular ice cream place.

Sharona: I didn’t remember the ice cream part, but I did remember the conversation.

Yes.

Boz: I’m glad you brought that up ’cause I was going to ask, ’cause I’ve heard how all of you guys got together to originally start coming up with the conference and I’ve heard how Sharona and Kate were connected and how Sharon and Robert and Robert and Dave and Dave and Kate, but I’ve never heard how Dave and Sharona were connected. Other than that first conversation about the potential in-person conference, that got kind of blown up.

Sharona: I think that was Denver, wasn’t it?

Kate: Yes.

David: Oh, that sounds likely. Yeah.

Kate: Yeah, that’s right. And because we were all there for those few days to really focus on all of these topics, we really thought this is really great. We got so much from each other. We’d really like to have a conference type thing to be able to do this more regularly outside of Math Fest, or one of these things.

And you said something like, oh, Robert and I have already talked, don’t worry Kate. Like I’ll get that done for you. And I was like, great. I can’t wait to go.

Robert: And you are still waiting to go. Because we haven’t done it yet.

Kate: I’m still waiting for the invite to this conference and I don’t know.

Sharona: Well, we’re talking about doing an in-person research based conference, so maybe we’ll put it at Grand Valley just out of nostalgia.

Robert: how much beer

Robert: That was the plan. Yeah.

David: That sure was.

Robert: David. David, you and I had started laying the groundwork for this and I think we pulled in Sharona and Kate because we quickly realized this was gonna be a bigger deal than we, we really bitten off more than we could chew, because putting together a conference is no joke.

Physical conference or online conference. And we’re like, who’s two people we can really trust are really into this? Who would really not only like, contribute a lot and we know they’re gonna get the job done and get it done right. But also like into the grading part, it was just like instantly Sharona and Kate, like, yeah. So no hesitation. No hesitation.

David: Not a difficult question.

Robert: Mm-hmm. That’s right.

Sharona: And then we have a one page conference description from about the same time.

David: Yeah. I actually have something in there that it appears we had a meeting on Zoom, by the way, if anyone hasn’t heard this story before, the reason I think that all of us here know what Zoom is, is Sharona. Because once we had this group set up, she’s like, oh, I have this great way we can all meet online.

And like, what?

Kate: Oh, come on. That was 19. I mean, that was before anyone else in the, like, I was like, what is a Zoom?

Sharona: Yeah. And I don’t remember how I started with Zoom, but it was in that spring because I was gonna, I think it was, I was going to try to teach. I’d been introduced to to it at my university and Boz remember I did that one, we were trying to find a way to scale the dual enrollment program.

Boz: Yeah, I was gonna say that’s where it really came from, was part of that slam program, where you and I met, was the sustainability and how to expand it beyond. ’cause at, at one point you were the dual enrolled college professor at like five different high schools.

Sharona: I think it was four at a time is the maximum I actually got. Well no, I had four that I was regularly visiting plus Santee plus I think at that point I also had the other one, the

Boz: Alliant.

Sharona: So yeah, so I had like six schools that I had to go to.

Boz: You were driving ungodly amounts of hours and ungodly amounts of mileage. ‘Cause you were going between these different schools. And that was something that we, and Lynn at College Bridge was like, yeah, this is not sustainable. And one of the things that we had already learned is you can’t just put any kind of professor into this college dual enrollment and expect it to have the results that we were having.

So we were looking for ways to expand and that’s kinda where you and I started to really use Zoom a lot and get pretty proficient in it.

Sharona: And then you guys invited me to join and I was like, well, we could just get on this Zoom. It’s a really easy video conferencing thing that’s free. And you guys were like, well, when do you wanna do it?

And I’m thinking, well, why don’t we do it at 5:00 AM and you guys are like, like my time. And you guys are like, 8:00 AM is great for us, are you sure?

Robert: So we first got the look at Sharona’s, the contours of Sharon’s personal life. It’s like, oh yeah, 5:00 AM for our Zoom call. No problem. I’d love that.

Kate: We’re very lucky that, that Sharona is the West Coast person.

And that we’re the East Coast people. ’cause if Sharona were the East Coast person, we would have to get up at like 2:00 AM.

Robert: No, you wouldn’t have to get up at 2:00 AM for anything. I would just say, no, that’s, I put my chat up in front of the TV or something. Oh, there.

Kate: Thank goodness our geography and our scheduling worked out as well as it did for this collaboration that we’ve had over these. You know.

David: So what you’re saying is 2:00 AM either way, no matter what.

Sharona: I’m good at 2:00 AM either way. It’s whenever you all.

Robert: Do you actually sleep Sharona?

That’s what, that’s the question. I think that’s under, that’s the subtext here. I,

Sharona: I, I, I do sleep. Okay. I don’t really cook much. I eat out a lot.

Kate: Okay.

Sharona: And I have a housekeeper to do the cleaning. So I don’t cook, I don’t clean, which is the same answer my mom gave many, many years ago when people asked her how she could be a full-time faculty member with a co-parent who had a full-time job and raised kids , in the seventies and eighties because they’re like, how do you do this? And she’s like, it very easy. I don’t cook, I don’t clean.

Kate: I might take that advice, can somebody take some notes?

Robert: Let this be a lesson to everyone. All you pre-tenure faculty member.

Sharona: Okay. So we know how we met. We’ve done sort of the general conference history. People have heard the stories there. So I wanted to really explore, and Boz, if you have somewhere else you wanted to go now, just interrupt me, but I wanted to get reflections from you guys, not just on the conference, but how the community has changed and grown, and what surprises you and what’s still around that you really like, any of that kind of stuff.

You know, where are we at with the community? And then also as a separate question, where are you at now in your grading practices? So, which one do you wanna do first? Boz?

Sharona: Well actually 50 people almost all in mathematics higher ed. Yeah. To be very clear, like not only higher ed only, not only small, but like specifically in math. And that very first conference was still math, and I think it was over 500 actually.

Robert: That’s right.

Boz: But since then, Dave and Robert, you guys have had a blog that’s over 200 episodes, I think you said earlier, plus a book. The conference itself has grown to last year and this year being almost a thousand people. We’ve actually got a nonprofit now that’s helping organize and run it.

David: No. That that’s an easy answer for, yeah, not really. Not an idea. I mean, and we weren’t even thinking about beyond math really. Right? Like you’re saying about that first conference. We, we were in that world only, or I was.

Kate: Well, I think at that time we were also sort of just, we were all sort of novice practitioners of alternative grading. Mm-hmm. And so we were sort of relying on each other in a small community, in the I’d really like to go out to dinner with these people and pick their brains about whatever issue that I’m having.

And so at that time, it really was a, it was a small group of people where we could go out to dinner and get a couple of tables at a restaurant and really find something productive there. And I think that that’s initially what we sort of thought about the in-person version of the conference, that it would be this productive time for us to all really sit around and focus on what we’re doing, that we’re really liking what’s working really well.

And it wasn’t about, let’s try and educate 950 people about the best practices of alternative grading. It was much more like, Hey, I’m having this problem with these quizzes that I’m doing. Do you have any suggestions?

And so I think that as we’ve grown up in our own practice. The conference itself has sort of changed its focus from being, like a very small, 10 person kind of casual conversation into something that’s a lot more structured and there are different entry points. So there’s entry points for novice level practitioners or those who have been doing it for a couple of years, and more advanced practitioners that are trying to find out new skills or whatever. So I think that we’ve grown and it’s grown concurrently.

Robert: Yeah, we were also novices at conferences too.

I mean, none of us had ever put on a conference. The closest you come was David doing math path, the summer camp that you do, which is not a conference, but you know, it involves all this organizational stuff that goes on behind the scenes. And so there was like a real imperative to keep it small and stick to what you know.

And so it was like, I think the idea was we were gonna start small with an in-person conference, so keep it limited to just math people, just higher ed. And then run it, see what happens and then begin to scale it up over time. Whereas COVID blew all that stuff up. But I, I would say definitely we did not do any sort of like market research to see like, wow, this is a really deep well, there’s like a million people out there that would really love to come for a conference.

They’re super unhappy with traditional grading, but they don’t even know that there are alternatives out there. So you start tapping into that well, it’s an open-ended experience. Like there seems to be like no limit as to how far that could possibly go.

David: I remember having this feeling at one of the math fests early on when we had a session about alternative grading that there’s one person at every institution trying or wanting to try something interesting and like that was the place where we could actually meet each other and go, oh, oh, you are out there. Oh my gosh.

Boz: We’re not alone.

David: Spend so much time. And we had a big breakfast and we’d sit in a bar and we would like just all these things talk, talk, talk about them. It was fantastic.

Kate: That was one of the greatest benefits to me about, I know we’re sort of speaking about community and I don’t know if it’s the Google Plus community or the Twitter group that we had or, or those formal meetings at Math Fest where there was some speaker and we would talk about it, or the informal hallway chats that we would have.

For me, I was one of those solo practitioners in my department trying different types of alternative grading. And so for me, the community was really integral because I needed to find out what other people were doing well and get inspiration and sometimes just like get the you’re doing the right thing, like keep going, like pep talk that everyone needs every once in a while.

Robert: And I remember the first online, the online conference, the first iteration of the conference was online. And y’all were involved with this. So I, I don’t know if you were on board with this Bosley or not, or like.

Kate: Well we thought that, we thought that we would go from a 50 person in-person conference to a 50 person online conference. And so we online Yes.

Robert: Because who would wanna go an online conference. Yeah.

Kate: Right. And then all of a sudden, I have like lots of notes to myself about severe anxiety attacks because, oh my gosh, like we just hit 200 people registered. Oh my gosh. We just hit 350 people registered. And we had no idea what we were doing, but we didn’t know, I mean, we didn’t know how to run a 350 person online Zoom conference and like just what, what mechanically it takes. We needed, we needed some AV tech support just to manage like breakout rooms when you have 500 people all of a sudden.

Robert: Yeah. And we did, I remember that at that conference where it is hard to get metrics. It doesn’t have the same metrics as a traditional in-person conference ’cause people can come and go and that sort of thing. But I remember we hit 500 people in one of the sessions and I was just like, holy crap.

Like what is going on? And then so one of those people was like, I remember distinctly, she was eight months pregnant and lived in like Finland or Sweden or something like that, was halfway around the world. And this person, there’s no way, it’s COVID, you’re pregnant and you live in Finland. And it’s like there’s no way this person would have been able to have come to an in-person conference and get here. This person is contributing and learning and, and doing all this stuff. And I thought that’s, that’s pretty cool. It’s like that was the first time I thought maybe all conferences or a lot more conferences should be this way. Maybe not all, but like, they don’t have to suck.

I mean, they can be really good even though it was like, this is a, a total situation where you gotta run before you can walk sometimes. Yeah. And just, just get it out there and go. And don’t overthink it. Just, just run the zoom, make notes, iterate next time. Higher ed is terrible at this and we weren’t.

We, we, we did. Okay. You know

Kate: I have a screenshot that I saved from that, that just popped up on my time hop. ’cause here we are almost exactly to the day what, five years since, since that was going on. It was a screenshot of Google saying like, we’re sorry you have 500 people trying to view that Google document. We’re not gonna let anyone else log into it.

And it was one of those, I never thought that there was a limit to how many people that could look at a Google document at one moment. I mean, it occurred to me, but we hit that limit and all of a sudden I got this huge like warning message from Google, like, what are you even trying to do right now?

And I was like, I dunno.

Robert: You have the FBI or door. It’s like, are you a terrorist? You’re sharing plans on how to go or something, right?

David: So I think you, you all need to tell me if I’m remembering this right, but I, I think at the end of that first conference where none of us really knew what, what was about to have or what had just happened.

We got on a Zoom and we just sort of stared at each other. That’s what I remember.

Robert: It was like the post, it was like the post credit scene in the Avengers movie where they’re all sitting around the Shwarma restaurant. They’re so tired because they just saved work.

Sharona: I think I messaged you guys, because what happened to me is I shut off the last Zoom and I landed instantly back in my apartment, and I wanted to cry, like, because it was, it was so abrupt.

I think I put in Slack. I’m like, Hey y’all, can we just jump on something? Because for me, that lack of transition. Like when you finish a conference, it’s like, bye bye. You gotta go pack your bags. You gotta check out. You see a few more people in the lobby. Like there’s a whole process to coming down at the end of conference and this was like, I felt like I had bruised my forehead on a brick wall, like coming outta that. And yeah, we literally just got on the zoom and started looking at each other. We’re like, what just happened?

And like all, all your social stuff is not only distance, but usually restricted. And it was kind of like if you had fasted for 40 days in the wilderness and then showed up at a pizza buffet and just stuffed your face, you’re gonna make yourself sick that way. And that’s, in some ways it was a good kind of sickness.

And we had just stuffed our faces full of social interaction. Awesome social interaction with no, like turning the volume down. No fade out at the end of the song. It was just like. That was it. Like you said, Sharona, it’s like you gotta learn a few things about social interaction, generally speaking.

Sharona: Well, and that’s one of the things that’s persisted. We actually have, it’s secret. No, but we have an organizer’s zoom that is the last thing that happens after the last closing session where even if it’s just for five minutes, we get on and we go, wow, we just did a thing. We just did that and usually, I mean, in that five minute transition from the last one to the organizer’s zoom, almost everybody has a beverage.

Robert: I was gonna say there’s usually alcohol involved in that.

Kate: Yes.

Robert: Yep.

Kate: Yes. Well, I’m still devastated because part of my original planning for this in-person conference was I was looking up all of these breweries as like a craft beer fan. I was looking up all these breweries in Grand Valley that David promised he’d take me and like, show me where good beer was up there. And then I still have not had any Grand Valley beer.

David: Oh, we are actually, so, so I feel like somebody owes someone else a beer. So I owe you a beer? Is that how it goes?

Kate: I, we just, it’s gonna be several. I think that I.

Robert: I think we just shall chip in and fly Kate up here and say, okay, join everything is on us for the next day.

Join. Okay.

Sharona: I think we need an in-person board meeting for the Center for Grading Reform. Once a year. Once a year. We need to put that in the bylaws because maybe people don’t understand that you guys are actually the board for the Center for Grading Reform now too.

Kate: Yeah. What we’re official 501(c)3 nonprofit status?

Sharona: Exactly. Now I wanted to, I just looked up a couple of other statistics that I wanted to share, which is the Slack channel officially has over 2,500 members. Most of those are inactive, which means people come on for like the conference and then they’re gone. But we have like 170 active members of the Slack channel.

We also have, on the podcast, our getting started part one episode, which is the second episode we ever put out has over 632 downloads. So really from where we’ve come from the beginning until now, it’s pretty remarkable.

Robert: Yeah. People discover all these resources. I mean, they discover the podcast and they realize, you know, you’ve got not only the podcast, but the nonprofit and you and Bosley are doing some training materials for this.

And there’s our book and there’s, there’s the grad for growth Substack and other grading related substacks, like Emily Pitts Donahue’s Ungrading Substack. And it’s like, if you’re interested in this, you can learn as much as you want. You can give yourself like a total, almost university quality education just by reaching out to people who have been doing this and sharing their work.

Boz: And I think that is a huge message. Like you were saying, Kate when you first were doing this, you’re the lone person at your university doing this and just being able to communicate with Dave and Robert and know that you’re not alone, there really is a much deeper well out there now five, six years later than there was.

And I know from when we first had you on the podcast, you’ve been doing alternative grading now for a decade and a half, almost.

Kate: Mm-hmm. So

Robert: Oh, wow. Longer than, longer than I have. I mean, yeah.

Boz: I think out of the five, you are the one that’s done it or been trying to do it for the longest.

Kate: That’s true.

Boz: It really is. The amount of resources and places to go and communities has grown so much. There really is a wealth.

Kate: I know that even my, my own university, while I’m the only one in, in the Department of Math and Statistics, who’s, been as involved as I am the university itself has offered some professional training through our Teaching and Learning center using David and Robert’s book. And so they’ve done an alternative grading thing with this book. I get this invite, like, Hey, these people have a book. You wanna come join the book club? That’s a good book.

Robert: We’re good.

Boz: Aren’t you in that book?

Robert: No, that book sucks.

David: Yeah. Come on.

Sharona: Yeah, I’m pretty sure she’s mentioned in that book.

Boz: Yeah. I about say, oh yeah. Isn’t your class one of the case studies in the book?

Kate: It is., I actually saw a couple people carrying it around. I’m like, look at page 66 or whatever.

David: You’ve got the page number memorized.

That’s awesome. Nice.

Kate: You know.

David: So there’s another sort of origin thing I wanna point out that connects with Kate here, right? Because Kate, you were a student of George McNulty. Do I have that right? Yeah, that’s right. And, and when I first started getting involved in some of these Math Fest sessions and things, I kept hearing this name.

And there were all of these people who had worked with him or students of his who and he sort of originated this idea. We would now call standards based testing as like a very structured, very usable across a huge range of classes, alternative grading system, and I think that was part of the initial dispersion of these ideas.

Kate: Yeah, so George McNulty was a mathematics professor at the University of South Carolina in Columbia. And he was my PhD advisor. And so I worked with him for five years and he was, he was doing, he definitely never used the lingo of standards based testing or standards based grading, but essentially that’s what he was doing in both his undergraduate and also his graduate courses.

But I think part of the reason that that took off is he had a really well written syllabus for specifically calculus 2. That explained his grading philosophy and what it looked like. And he gave example problems.

I mean, so the whole thing about his course is that his final exam was a buildup. And so the first test had four questions, and the second test had question zero through three, and then another four new ones. And then the third one had 12 questions. And then the final exam eventually had them all. And he actually provided that exam on his syllabus to be really clear with what the students were going to see.

So it wasn’t quite a, a list of standards with verbal descriptions, but, it was like, problem eight will always involve some kind of Taylor series expansion or something. And so because it was so well written and because it was available on his website, I think that it sort of went viral.

Unbeknownst to him, actually, I don’t think that he knew.

Robert: I think Kate, that I might have actually learned about standards based grading, whatever you wanna call it from one of your blog posts about McNulty’s calculus two syllabus. ’cause I remember distinctly thinking like, whoa. And I went and just looked it up and it was available online as API think you may have linked to it.

I, I looked at it and was like, it kind of blew my mind. And that was, I don’t even know if I was at Grand Valley at the time.

Kate: And he was, I mean, and he was, he was using, I guess what, what Sharona maybe has convinced me that the right term is proficiency scale. Like what kind of mark are you going to assign to this particular student artifact?

And he was using a a three-tiered proficiency scale that he called, I mean, he was doing an apprentice, journeyman, master.

Robert: Journeyman master, yeah.

Kate: Right. And so even for my own courses with him, in whatever math stuff, he would say I got a journeyman plus like on that problem. Like what?

Robert: And sounds like a subscription service now doesn’t it?

Kate: Right. Try to unpack what that meant. Like.

Robert: Once a month I have to pay for Journeyman plus.

Kate: Right. Exactly. Trying to unpack what that meant. And his descriptions of it was part of my thinking through like, what did I wanna do or what did I learn from him about what was important or what kind of proficiency scale would I want to use?

And so I was sharing that widely because I found it to be a really helpful resource.

Robert: Yeah. Well it is. It was. I still go back to that every semester I look at it. ’cause like you said, that syllabus is like so clear and it’s just like a model of simplicity. There’s a lot of stuff in that that isn’t in that syllabus too.

I’ve come to learn. Like there’s some things that were, that probably should have been in that syllabus that were left out, some of the devil in the details. But still, I look at that all the time and think like, that’s, that’s like my, my goal, you know, to have something that clear and that simple.

Kate: Now that I’m in the administration, I think that that syllabus violates our policy about syllabi at my institution.

Robert: Yeah, it could, it could well do that. You know, there’s a lot of places where it would not pass.

Kate: There’s nine pieces of information that are not there that should be there. You know, the fire code requires that all exits be properly unlocked or.

Robert: In some places. I know, I know in some places, as I’ve kind of worked with other universities the university policy says like you have to have like x percentage of your grade coming from points from exams.

Like it’s real like presumptuous about there have, everything’s being done with points, everything’s done with tests. And if you try to go a little bit left, turn on that, it’s like it’s just a complete square peg in a round hole.

Sharona: I can hack any of those systems. I just wanna say out there I can hack a required points based percentage system like nobody’s business.

Kate: Our, we.

Robert: Now you’re really gonna get the FBI your door. We have a line on a podcast talking about hacking services.

Kate: We have a policy at my institution about, basically you can’t give your final exam early. Like you can’t give your final exam on the last day of class. It has to be at the time that you’re assigned.

But like how to, how to codify like that idea of a policy is something like, it says something like, in the last week of class, no assignment worth 20% or more of the cumulative final grade can be assigned to the student or something. And of course, I don’t know what 20% of my, like, is that 20% of my standards like are what are you saying?

Robert: 20% of what?

Kate: Yeah, 20, 20% of what? And when I, when I tried to point out to people that I find this to be really weird, like they’re like, oh, so you want it to be like 18% then? Right. You know, like they don’t, they don’t understand that. They don’t,

Robert: lemme back up.

Kate: Understand my, they don’t understand my issue with this question. Saying 20% assumes that this has meaning as a number somehow and that every syllabus has to have this kind of like meaning to it. But even try and convince people that like you could run a course where there are no percentage, like there’s no like weighted percentage average that we’re doing at the end of the thing. This just some people don’t. It blows their mind to think about it.

Boz: Well, but doesn’t that just show how our, our world of grading math is and, and how people think? It just has to be there. I, I know. Mm-hmm. And I forget if it’s was you David, or you Robert, or maybe it was a joint post, but on your blog post, the when is a number, not a number.

Sharona: Not a number, that’s Robert’s, we’ve referenced it so many times.

Boz: That is still one of my favorite and one of the ones that I referred people to the most. ’cause it’s, it’s amazing how much people just math has to be in the grading. Like, it, it’s, you just can’t do it without

Robert: Yeah. If it’s, if it’s not, it’s not objective, quote unquote objective or scientific. And I’m like, really? And I, I recently,

David: And if it is a number, it is objective automatically. Right.

Robert: And if you put more decimal points on that number, it’s much more scientific. Right. So like 99.225% of all statistics are completely made up. Right.

Kate: It is funny to me now that I’ve grown up in my faculty role a little bit from when I was a new faculty member. When I’m thinking about, our tenure and promotion process, or even just the annual reviews that are done for faculty at the end of the year. We get a mark that says, you know, for a service that you haven’t met the expectations, or you’ve met the expectations, or you’ve exceeded the expectations.

And your supervisor writes some kind of paragraph blurb about why they believe this to be true. And all of the faculty members I work with know what this means. And they think that this is a fair way to evaluate a faculty member and like they’re good with this. And then when, I tell them, well, okay, like, why don’t we do that in calculus 2?

They’re like, no, no, you need to have quizzes and they have to be worth 12% of the final. There seems to be this cognitive mismatch between the way they evaluate each other or the way they want to be evaluated by their supervisors and where they want to do that with their students.

And it’s perplexing to me ’cause one would think that if this works really well in figuring out if this person’s ready for tenure or what kind of feedback they would need in order to get them up to the level to be tenure eligible or whatever, that they wouldn’t also think like maybe that’s what somebody needs in an intro to abstract algebra course. Like that same kind of feedback cycle.

Robert: I think it would be hilarious to flip the script on that and like say, okay, if you get into a position where you’re a provost or a chancellor or whatever, and say, okay, we’re gonna change the way we evaluate faculty for tenure. We’re just gonna give you professional tests three tests and a final exam, and each one’s gonna be great on zero to a hundred points.

And if you have an average over 80, then you get tenure and to see how people feel about that. It’s like, well, that sucks. I hate that. That’s not helping my professional development. We’re like, exactly. Now you know how we feel.

Kate: Every paper you submit, we’re gonna grade out of 100 and you have to have a whatever.

Robert: Or journal articles. I mean, if you ran a journal, can you imagine like, okay, so when you somebody submits an article for your journal, we’re just gonna give it a score. Out out of a hundred say, okay. What Your journal article gets an 85.

Sharona: Well, unfortunately, to be honest, a lot of grants are actually done on some of that kind of scoring stuff.

Robert: It’s true.

David: Yeah. And it’s so for, for everyone listening, I just wanna point out that this, the last, I don’t know, five or 10 minutes here, this is like every meeting we ever had for the setting up a conference.

Robert: It’s like, can you believe this is such a stupid thing and we’re so persistent with it. What is going on? Why are people so intransigent in higher ed?

David: Well, because this whole group has a lot of fun people to talk with too. And we’ve all got strong opinions and, and you know, you put us together in the same room and we’re trying to organize a conference, like, no, we wanna talk about the stuff we’re actually doing so.

Sharona: Well, and that’s why, speaking of the grading. This year, I mean, Bosley and I give a talk, grading is the misuse of mathematics. And we have been invited to like four or five different places. My travel has kicked up. Bosley travel would kick up, but he’s got a day job. And but I’ve given that talk now five or six times. And for some reason coming at that talk with the credentials of being a mathematician just lends it some weight.

Because I’m able to say, okay, you wanna use math to do grades? Well, guess what? You’re gonna have to talk math. And I’m going to prove to you mathematically that this stuff is meaningless. And we do that. We literally walk from an individual assignment with its point scoring out of a hundred to weighted categories, to like different, we walk our way up the grading chain and we basically prove that two students in three different classes with different weighted systems can flip grades.

Like in one system, it’s a B and then a D, and then the other system it’s a D and then a B. So we’ve already shown that it’s meaningless. And then we do the parachute packing problem and people are like, oh, well we clearly want this one person, and they all have the same average.

So we tend to get through that the math is terrible. But then the devil’s in the details on what do you replace it with? That’s the hard part. So that was one thing. The other thing I wanted to comment on, because David you just said, it’s the group of us, right? I really love both this podcast and the conference for giving us one of the few times where we can actually come together in community.

And Bosley and I have had this amazing opportunity to talk to some people that we would never have gotten to meet. I mean the, when I surprised Bosley by getting Dr. Thomas Guskey on the podcast, I thought he was going to fall over. I mean, this is a man who’s one of the last PhD students of Dr. Benjamin Bloom of Bloom’s Taxonomy, who’s been around Bosley’s world for like his entire professional career. And I’m like, guess who we’re recording with?

So I would, I think we are gonna do better at getting this to work. We need the books, we need the blog posts, we need all these things, but we gotta be talking to each other. And that, I think is what the fact that the conference is so accessible financially. You know it’s $50 or pay what you can. I had an interaction this week with someone who needed help paying for the conference. I’m like, well, just pick pay what you can. And they’re like, well, but I still can’t pay for it. And I said, zero is an appropriate level pay for pay what you can.

Boz: So well, and, and then even after that, didn’t they thank you for funding it and like Yes. We’re not giving you a scholarship. We’re not funding you. Just, just an option that you can choose.

Sharona: Well, in a sense we are, because this community contributes enough money to keep the conference going for those people that can afford to pay for it. Because we also don’t wanna be abusive of people’s labor.

So, unlike the first year where we were all volunteers, or the first couple years, the organizing team does get a small amount of compensation. And we have a nonprofit that is paying its bills. And that’s only because people who can afford to pay for it choose to pay for it. And those who can’t afford to pay for it, they don’t.

And that’s okay.

David: And that I think is something worth calling out to everybody too, is like, as Sharona said, we were all volunteering and at some point we started kind of giving an honorarium to like keynote speakers. But a huge force in making this sustainable, and moving from the we are just pants on fire, running around you know, trying to make this thing happen into something sustainable are Sharona especially, and Bosley. Like they’ve done a ton of work to turn it into something that can continue and not just be people overextending themselves constantly.

Sharona: Well, and shout out to NSF in its previous incarnations because we got funding. We managed to write a grant and get a conference grant from NSF. So that was what tipped us over into sustainability.

Robert: Yeah. And I think the reason that this has all been successful is because this is a proven concept now. I mean, this is not something that people are just sort of experimenting with. I think the next step, I, and we’ve talked about this in the board meeting, is a ramping up research around around alternative grading.

The next layer is to, and so this is clearly a major going concern with a lot of faculty, and we don’t even tap into K through 12, on the blog. But the next level is to think about getting hard scientific research going about the effectiveness of alternative grading, and that will even beget more NSF type funding even in the state that NSF is in right now.

I still do think that there’s a lot of people who apply for grants that are like, honestly, the idea behind the grant is kind of like, I’m not so sure if this is gonna work, but when you have this many people for this long. Tapping in and, and contributing to a grading conference. This is a known quantity.

Okay. And that’s Sharona and Bosley and, and everybody who’s involved in planning the conference that’s a great credit to you all and your work. I mean, it’s not just one of the best online conferences there is. It’s one of the best conferences period that there are.

Boz: Yeah, we, we really do have a great team of organizing committee members. I know the three of you are no longer on there, but you have transitioned to now being the board for the center. But those that we have right now we have an absolutely amazing team. There’s a total of eight of us. It’s wonderful group of people.

And you were talking about next step with research, Robert, I don’t know if you’ve looked at the schedule for the conference, but I am really impressed and looking forward to just how many of our sessions are presenting research.

Robert: That’s awesome.

Sharona: So, I have a question. We interviewed someone on the podcast recently, the episode’s not out yet, but he said something to me that really struck me, which is having a vision and being able to articulate that vision. So I’m gonna throw this at you. You can decline if you would like, but if you look, say 10 years down the road for the center or for this movement, what are some of the things that you might hope that you see happening?

Five or 10 years. And I mean, you can take a minute to think about it or where do you see yourself personally with this stuff? But I was thinking more for the community.

Robert: 10 years I plan on being retired, so maybe that’s like nothing. I see all these,

Sharona: but if you could, but if you could in your rocking chair, pull up the Grading Conference website, center website, what would you hope the center’s doing at that point?

Robert: You know, I don’t know what Kate and Dave are gonna say about this, but when I was writing my first book on flipped learning, I said something to the effect of, we call it the flipped classroom now, but if we’re doing this right, eventually it’s just gonna become the classroom. It’s just gonna become a normative practice, what we’re doing here.

Okay? It’s, it’s just grading. It’s just what people do. And if you look at a class that uses points and percentages, that’s gonna be seen as a really weird aberration. Like how does that even work? What are you even doing? And to get to that point, one thing I really hope to see is more upper level university leadership getting on board with this.

We have a great grassroots, I mean, it’s a massive, influential, generous, grassroots organization for rank and file faculty members. It’s beginning to sort of get a foothold amongst the provosts, the deans, the presidents of universities and boards of trustees and so forth to kind of make this accessible and doable for other people.

There needs to be a lot more of that, if that’s gonna be any further movement on this. So I really hope that we begin to see like senior leadership from universities getting on board, or the people who are currently grassroots, rank and file faculty members becoming the senior leadership at these universities, and can move that and take that on as their personal mission to make this doable and simple. And nobody’s at risk if they try it. In every university in the United States and beyond as far I’m concerned.

Kate: Yeah, I agree with all of that, Robert. I think that having this not be like, oh, you’re doing something weird. Should you ask permission to be able to do that? And instead becoming something that is accepted and is based in the scholarship of teaching and learning and is supported by all of that would really be great.

Both for new practitioners who maybe wanna try this but are a little bit afraid of what their tenure committee might say, but also for the benefit of our students. Ultimately, like that’s the goal. So part of it is about what the faculty member wants to do.

But part of it is that we all know and believe that this is the best way to benefit student learning. And if that’s ultimately our goal, the more practitioners of alternative grading that we can have and generate and continue to support, the better off our students will be down the road. And thinking about the research that’s there, I mean, there’s definitely research that’s coming out all the time and Sharona just alluded to what’s going on at the grading conference and people presenting their research.

One of the things I would be excited to see is what are the best practices that would support alternative grading in a classroom of whatever flavor you might be. So maybe I have some ideas about what that might look like in a 100 level entry level gen ed math course. But man, it would be great if we could figure out who’s doing it best for an introductory writing course or an upper division chemistry course, or sort of broaden our horizons so that there’s sort of a exemplar models for what is the best way to get this done, or what are people doing who are successful at this? What tips and tricks can we borrow from them? And then their experience. So sort of broadening the research outside of like, what’s gonna go well for a math classroom and go in every possible direction from there is something I’d like to see in 10 years. I also wanna see, my own retirement at some point.

Sharona: Dave, thoughts?

David: So kind of a variation on both Roberts and Kate’s point. So this is coming from a paper I actually just read and also my own experience, which is in another 10 years we’re gonna have had a lot of students who have gone through classes that are alternatively graded, even if that’s relatively sparse across the world still.

And having that experience makes people more willing to try it themselves. And that’s not just those students who go on and get a PhD and become a professor or whatever. It’s people who go on to teach secondary ed. It’s people who are just become parents and have kids in school. They know what that’s like and that it’s okay.

And that goes a long way towards normalizing it. And so I think the more we can build that in from the student perspective of having a positive and just completely normal experience with some kind of alternative grading and then going into the world with the view of, oh yeah, that’s a thing and it’s okay. That’s worth a ton to make it feel natural and normal.

And they do use alternative grading language. They link many of the things that we talk about. They link Susan Blum’s book. They’ve got the Joe Feldman grading for Equity. They’ve got the Schinske tanner that we often refer to teaching more by Grading less. They’ve got Linda b. Nilsen’s book on there.

So they’ve got Becky Supiano’s article. So if Harvard is getting behind it, this is a very good sign. Also, the EMBER project. Through the transforming post-secondary education in math that is funded in part by the Gates Foundation, has adopted looking at grading and assessment as one of their strands that they think is very important and they are targeting the top research institutions. Because we’ve done that mapping, I think it was Chad Topaz has that paper out mapping where all the faculty members come from. And almost all of them come from like the top 20 R1 institutions at university levels, at least in mathematics.

So I’m hopeful that between Project Ember at TPSE and Harvard’s influence that maybe we might be able to get some real, real traction over the next 10 years.

Robert: Yeah, that’s, that’s really cool to hear Sharona. Glad you said that. It’s not just Harvard too. I mean, MIT has been doing some really great stuff with alternative grading.

They introduced a whole system of like, not giving traditional grades for the first year, and are still running with that and having great success. I spoke at Cornell last fall and like the people at Cornell are absolutely on fire to, to get this. I mean, it’s a huge community of alternative graders at Cornell.

And and, and so that’s, this is beginning to sort of. And Georgia Tech too. All the other

Sharona: University of Michigan, Michigan.

Robert: Michigan, U of Michigan, of course. Yeah, U of M is is fantastic. So yeah, it’s, it’s it’s, it’s good to have grassroots people from places like Grand Valley State or call to Charleston, you know, where a lot of the, like the in the trenches work gets done, but you also have to have a little bit of top down happening too.

So they have that swag. It shouldn’t matter, the credibility factor shouldn’t matter, but it does. When you hear that Harvard, MIT, Georgia Tech and all these places are going all in on some of this stuff it makes a difference, especially if you’re an administrator.

Because administrators wanna, that they’re kind of like, they’re looking, just like always they look at the Ivy League. So like, I might is they, whatever they’re doing is like the back of the book, you’re checking your homework assignments. It’s like whatever they’re doing is supposed to be right.

So I’m gonna make sure I’m doing the right thing.

Sharona: So we’re coming up on time. Bosley, did you have anything last that you wanted to ask or?

Boz: Hey, I could go all day with this group, so .

Sharona: I think we have.

Robert: You don’t wanna do that. You don’t wanna do that.

Boz: So, no, I just really wanna thank you guys for coming on here, for thanking you for everything that you’ve done to bring this community together to really get this conference started.

And, I look forward to continuing to work with you guys in every way possible. But yeah, I just really wanna thank you all for taking a little over an hour to outta your time to just come and sit down and have some fun with us.

Sharona: Any last words anything last thing that you wanted to share before we sign off? Kate, you wanna start us off? Any last words?

Kate: Gosh, I’m so grateful to the four of you for keeping the public facing conversations going and seated very well right? That all four of you are, are doing an amazing job at putting ideas into the ether that then can grow into beautiful, beautiful flowers wherever they end up germinating.

And so since I’m doing much less of that, maybe none of that anymore of my own self, thank you to the four of you for keeping the fire going.

Sharona: I’m pretty sure you have almost the youngest set of kids overall, although Bosley’s got one around your age as well. Yeah, right. You’re a different stage.

Kate: My kids are 10, 12 and 14, so we’re headed into odd number grades next year. I’ve got a fifth grade, seventh grade and ninth grader come August. Wish me luck.

Sharona: Dave?

David: Yeah, everybody listening, do your thing. Give it a try. Just try. It’ll be good. Also, Kate, I apparently owe you a beer.

Kate: A few. The dues have been running on that my friend.

Boz: The interest rate, it’s up to more than just one.

Robert: I know it’s compounding.

Kate: We’re at a six pack. At least. I can’t wait.

Sharona: Wait a minute. Didn’t he ship you some beers at one point?

David: No. I think I said I would. No.

Kate: I’ll let him reassess that foible.

Robert: There we go. Good. Good word. Good. Well played Kate.

Sharona: Robert?

Robert: Well, yeah, so I would just echo what, what Dave said. If you’re a listener first of all, you’re doing the right thing by listening to this podcast. Just keep learning. Don’t wait for somebody to give you permission to try the things that you feel to be right for your students. Okay? You are the only thing, only permission that you need.

The only thing restraining you from for helping your students with fixing, changing your, your grading system is your own creativity. That’s the only thing holding you back. Everything else can be worked around. So just kind of take courage. Doing things that’s right for students takes a lot of courage. Make the step and you’ll be rewarded for it.

Everybody, there’s a whole world out there to support you.

Sharona: Well, and I just wanna say thank you not only to the four of you on this call, but to all the listeners, I never thought I would get to be a part of something this big. And this is why I went into teaching is you know, my mom was a math educator.

I lost her recently. I always said that she worked to change teaching in math K through eight. I wanted to do it at the university and this is my stand, this is where I’m doing it, and I just feel so grateful to the community and to the four of you for giving me the opportunity to continue my mom’s legacy in my own way.

Robert: Your mom would be proud of you. Sharona. Absolutely.

Kate: Mm-hmm. I agree.

Sharona: Thank you so much. Bosley, you wanna take us out?

Boz: Yep. So again, wanna thank the four of you, but I also wanna thank all of our listeners. It is hard to believe that this kind of really niche podcast has reached a hundred episodes, and that wouldn’t have happened with all of our listeners. So thank you and we’ll see you next week.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode’s page on our website, http://www.thegradingpod.com. Or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a featured topic for the show or would like to be considered as a potential guest for the show, please use the Contact us form on our website.

The Grading podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.

Boz: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State System or by the Los Angeles Unified School District.