The Grading Podcast

126 – Derek Bruff

===

Derek Bruff: And so this question of why are we teaching what we’re teaching, I think is a really important question. And I think as educators, we need to have better answers to that. I think there’s a lot of pressure in the United States right now, a lot of skepticism of the value of higher ed. And so I do think we need to be thinking about what do our curricula look like given the availability of AI?

Do we need to change some of our goals and our emphasis, right? Are there things that we don’t need to be focusing so much on more because we know that we have some computer tools that can do that work for us?

Boz: Welcome to the Grading podcast, where we’ll take a critical lens to the methods of assessing students’, learning from traditional grading to alternative methods of grading.

We’ll look at how grades impact our classrooms and our student success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach and instructional designer, whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week, you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Boz: Hello and welcome to the Grading podcast. I’m Robert Bosley, one of your two co-hosts, and with me as always, Sharona Krinsky. How you doing today, Sharona?

Sharona: I can see the light at the end of the tunnel and it’s not an oncoming train. The semester is so close to over I can taste it. And I am so happy because I have been writing exams for about the last three or four days. With all of my course coordination that I do, it takes me a long to, even though we have this great tool to write all the problems with, just the formatting and the overleaf and the LaTex, I just am so done.

How about you Boz? How are you doing?

Boz: See, you and I have different reactions to the end of the semester. Because our roles are so different and because being in the classroom really is what energizes me. A couple of days before this recording, I had my last class of the semester. We still have finals next week, but it was my actual last class with my students. Talking to some of them afterwards and some of the things that they said to me, and I don’t get to teach in the spring very often, so I have a much different reaction to the end of the semester than you do.

Sharona: And when I’m in the classroom, then I have a closer tie like you do. But this is my third semester in the row when I haven’t been in the classroom at all and I am going back into the classroom next semester. So I am looking forward to that. But yeah, I completely agree doing this purely administrative slash coaching job has been really hard.

But speaking of coaching and faculty development and things like that, we have a guest on the podcast today. So I’m very excited to introduce Derek Bruff. Derek is an educator, author, a higher ed consultant. He directed the Vanderbilt University Center for teaching for over a decade where he helped faculty and other instructors develop foundational teaching skills and explore new ideas in teaching.

He consults regularly with faculty and administrators across higher ed on issues of teaching and learning, and I actually got to know him first because I’ve been listening to his podcast, Intentional Teaching, off and on for several years. So we actually are a little intimidated that we have a long time experienced podcaster on the pod today. And he is currently serving as the associate director of the Center for Teaching Excellence at the University of Virginia. And we had his boss, Michael Palmer on a couple weeks ago. So he’s also on our organizing team for the Alternative Grading Institute. I’m really excited.

Welcome Derek to the Baby Podcast.

Derek Bruff: Thank you Sharona and Boz. Thank you so much for having me. Yeah I have been podcasting a long time. I don’t know how professional I am, but I’ve been doing it a while, so that counts for something.

Boz: And it’s strange. We have so many, like close encounters and so many crossovers, but the three of us have never actually met other than like Sharona said, listening to some of your podcast episodes.

Derek Bruff: And likewise, I’ve listened to several of yours.

Boz: Oh, thank you. And you we’re really excited to have you on. But one of the things that we always do with a new guest is we’d like to ask just how did you get started with grading reform?

Derek Bruff: I was thinking about this and I think there was like three stages, if that’s okay. So I’ll go all the way back to my time as a grad student. I was getting a PhD in math teaching calculus courses. One of the faculty there in the department at Vanderbilt, Mike Mahalick. I don’t know what prompted this, I forget the context, but we had a conversation about teaching and he shared his, like one trick I think that’s how he described it, I’m not sure.

But he talked about giving his students an exam and then he would grade the exam and then hand it back to the students and then he would allow students to do test corrections. So any question where they miss points, they could write up a full and complete solution. They could turn that in and get a third of the points back or something.

And the rule was they were only allowed to ask him for help. They were on their honor to do that, but it, he said, look, this encourages them to come back and see me, the students who should have been coming to my office hours. It gives them another reason to come to my office hour. Calculus is pretty cumulative if you bottom out on the first exam you need to learn that material for the second exam.

So this gives them a chance to do this. And so that was my one alternative grading trick for a while. I found that great. I thought it worked really well. I was just a little baby teacher as a grad student. And I thought that it, again, it got students in my office hours, which I liked. I like having that interpersonal connection with students, especially the ones who were struggling and needed a little bit more help. So that was my little first foray into doing something a little bit different. That’s just the a little bit of, revise and resubmit.

And then in my first faculty job outta grad school, I was teaching at the Harvard Math Department and part of a coordinated calculus program there. The course was called Math X. Which was the best name for a course I think. It was like a stretch version of Cal one. So it was Cal one with some pre-calculus taught over two semesters. And so these were students who needed math. They needed to take calculus. They were definitely not math majors. They were in the lowest math offering at Harvard, right? So this is Harvard. So all the benchmarks are a little weird, but so these were students who like struggled at math. They were good students, they were at Harvard, but they struggled at math. And so we were trying to create a course structure that would really help them.

And I walked into this structure, a mastery assessment approach. So we had these mastery quizzes that we gave students, and so we thought about our learning objectives in three buckets. There was the kind of basic skills of pre-calc and calc, there was a conceptual understanding, and then there was the kind of applied problem solving.

And so we took the basic skills out of the exams and had these mastery quizzes. Students would take a quiz in class and they had to get seven out of eight questions right, or something, to pass it. And if they didn’t pass it in class, then they could come to office hours or TA hours and they could take second and third and fourth and fifth versions of these quizzes and keep trying until they actually got there. This was new to me. I hadn’t experienced this structure before. But it played well with this. What I liked about the test corrections was that it gave students a chance to relearn material that they hadn’t learned. It gave them a reason to come into my office hours where I could talk with them and work with them a little bit. I remember one young lady, I think it took her like 10 tries to pass one of these quizzes, but she was so proud when she finally pulled that off. I really liked this approach. It felt like it worked, right? And so that was phase two.

And so I had this exam that I was giving students later in the semester after we’d done all the math content at the course and I realized that I had a handful of students, and this was a small class, right? So I had a lot of options here. But I had a handful of students who seemed to be doing well during the semester on the problem sets where they were practicing all these math skills. But then they got to the exam and they just bottomed out. And I thought, something’s not working here. I need a different structure to help them get there. And I think my theory was they were able to collaborate with their peers on the problem sets. And I encouraged that ’cause I think that collaborative social learning is really powerful and it works for a lot of students.

And I wanted to validate that and encourage that. But I think some of them weren’t getting enough learning out of that process. They were leaning a little bit too much on their peers, and they needed another kind of structure to get there. And so I thought I did mastery quizzes like 15 years ago at Harvard, what if we tried that? And that worked like a charm. And so I set up a structure where again, students would take the quiz, we had three or four of these quizzes during the semester. They would take the first one in class. If they didn’t hit a certain threshold, they would take a second or third version of it in office hours.

Again, I had students who sometimes had to take five or six or seven versions of these, but it was a small class so I could manage that. And if they passed all of the mastery quizzes, they wouldn’t have to take the exam at all. And frankly, I was well motivated when I was down to one or two students who hadn’t completed all the mastery quizzes. I’m like, if I can get them to pass, I don’t have to write this exam. This is awesome. And so they did. All right? I worked with them. They worked hard. We figured it out. I actually felt much more confident. That my students were learning the math that I wanted them to learn than had I done the traditional exam route in fact, I knew many of them were not learning the math. That’s what the exam told me was that I had a handful of students who were just not getting it, and I felt bad that they were leaving the course without getting it. And yeah, I had a course grade and it was a weighted average of weighted averages, right? There was something happening there, but I wanted them to learn some of the math. I felt pretty happy with this mastery quiz structure.

Boz: I love the fact that, this ’cause so many people that we talked to, including Sharona, really, for one reason or another, had some sort of experience that just pushed them into the deep end and all at once went to some sort of alternative grading.

But I always enjoy listening to someone that had a similar experience where it wasn’t just, okay, this tragic thing or this traumatic thing happened and jumped into the deep end. I also, when you were talking about your Harvard experience, ’cause I am my background, even though I do teach at higher ed for the last six years. I still am in the high school setting too, and that’s my true background. I’ve been a LAUSD instructor for, 21 years now. Common Core is a huge thing and I was here when we switched over to Common Core. I was here for all the debates and still having the debates, a decade later.

But what you were just talking about with the Harvard and the focus of the class being, these skills, the procedures and the application, that’s the definition of rigor for mathematics in common core and equal balance of conceptual understanding, procedural fluency and application.

Derek Bruff: There you go. Yeah.

Boz: So I love all the people that complain about it. I’m like, yeah, okay. Harvard was doing this 10 years ago too.

temp: The early two thousands. Yes.

Boz: 20 years ago. Okay. 20 years ago. Yeah. Yeah.

Sharona: So I wanna follow up, so you had that experience with that course and then at some point between then and now, you transitioned , or maybe it was mostly then as well, but you haven’t been in the classroom since then. So what has happened with your thoughts on grading and how does that impact your work now? ’cause you’re not in the classroom with students, but you’re helping to organize the alternative grading institute through the Center for grading reform. Yeah. So what are your thoughts now on grading, both your personal practice, but also in the world that you inhabit of faculty development?

Derek Bruff: Sure. And Boz to your point about slow change over time. Like I was teaching math courses in a math department, but my day job was working at the Center for Teaching at Vanderbilt University. And so I was doing faculty development. I was interacting with faculty of all disciplines and learning from them, right? And trying to help them make informed teaching choices, informed effective choices. And so in that environment it makes sense that I would just start tinkering with my courses. ’cause I’m always learning about new approaches and new strategies from either my reading of the literature or going to conferences or talking with my faculty colleagues.

Boz: Yes, love Josh.

Derek Bruff: i, and and now I’m working remotely at the University of Virginia with Michael Palmer. I’m I like to think that when I was a center director, I hired people. I’m pretty confident I find good bosses for myself. ’cause Josh and Michael are both really amazing. So I like working for great people.

Because of the kind of virtual nature of my job, I haven’t been in a classroom in a few years. But I’ve been working a lot with faculty, both at Mississippi and Virginia and elsewhere. The Alternative Grading Institute came around in part because Michael Palmer and Adriana Streifer and a few others at UVA have been doing a lot of work around grading reform and alternative grading practices.

Like you don’t have to sell them as much. On that, they’re much more ready to actually roll up their sleeves and make some changes. And Boz, to your point, a lot of them have tinkered with things around the edges. And now they’re ready to say, okay, I want to actually go all in and restructure this course to fit something like specifications grading or standard based grading. And to adopt a whole package of something.

The first day is helping faculty think about their values and their situational factors for their teaching. Michael and Adriana had developed this I wanna make sure I call it the right thing here. It’s the grading scheme anatomy. Michael loves, yeah, he probably talked to y’all about this. Michael loves to break things down into components. And so we’ll be working with this grading scheme, anatomy to help the participants think more deeply about their own grading. And then in day two, we’re gonna split off into those three different tracks and do a lot of course design and course development collaboratively.

I’m excited about this. I’m really excited to get to know a bunch of faculty who are invested in this and want to make important changes in their teaching and are ready to go. And are coming at this from lots of different disciplines and different teaching contexts. I think it’s gonna be a great event.

Boz: Yeah. You guys really did put together an Allstar team and, full disclosure, Drew is also part of the Center for Grading Reform. He’s one of the organizers of the grading conference. So is Emily. So yeah, you guys really did put a great group of people together.

Sharona: I think the only person of that group that I’ve not directly interacted with, although I’ve heard her name a lot, is Adriana Streifer. So I think we’re just gonna have to get her on, ’cause I’ve talked with Lindsay Masland, I’ve worked with everybody else, and I’m like, okay, Adriana and I must have been orbiting each other for a while. So that’s if she’s listening, which I doubt, but if she is Adriana.

Boz: We gotta get her on to do like a cap, a capstone of the Yeah, exactly. After it’s done.

Sharona: So you said something really that struck me, that faculty who are aware of wanting to be better with their teaching and aware of national conversations. And that in a way hit me like a dagger in my heart just now because I have been seeing the same things you have while simultaneously experiencing a tremendous amount of frustration in my own situation with faculty who are completely unaware of the national conversation.

So do you have any tips or tricks for getting either colleagues or sometimes your administrators, like your department chair or your dean, more aware that these national conversations are going on?

Derek Bruff: Yeah, that’s a good question. So it’s challenging, right? So I think a lot about kind of awareness and implementation in faculty development work. If I’m working with the university campus, faculty aren’t gonna change their teaching practices until they’re aware of new options or better aware of a problem that they’re facing already. And some of our work is trying to help faculty learn about options and problems. And then once they’re aware there’s a kind of second round, like with the institute, that’s about getting the work done.

And so it’s a really good question of how do you create awareness? I think the best approach, it’s a slow approach, but I do think, maybe this comes, ’cause I work probably more with STEM faculty than other areas, so this may be a little bit more STEM focused, but I find that STEM faculty like solving problems. If there’s a problem in their teaching or their curriculum or student success problem, they will be inclined to collaborate, maybe seek some grant funding. That’s another impulse some faculty often have. And so when I think about some of the places where kind of alternative grading practices have spread, it’s often not by leading with the alternative grading, it’s by recognizing, we have a calculus course that’s not working well. And the dean knows this, and the provost knows this, and the department chair knows this. Or we have a college algebra course that’s not working well. Or an intro bio course that is, it’s not doing the things that we’d like it to do.

There are some faculty who don’t care about that. They’re like, whatever. Students will succeed if they succeed. It’s not my problem if they don’t, but I find that most faculty who are teaching at the undergraduate level want their students to do pretty well. And so I think that’s a place where looking for the problems and then being ready to provide some of those grading reform oriented solutions when a department or a team of faculty has recognized that something’s not working here.

Boz: Oh yeah. We saw a lot of the same thing. That is when the grading conference started, even though it wasn’t in response to remote teaching we actually, changed it from not we, Sharona and her team.

Derek Bruff: Oh my. Yeah.

Boz: With the pandemic and going to remote learning, that is when, Sharona, you and I started doing a lot of our stuff outside of our own teaching together through the SLAM program. That is when a lot of people, and we saw the same thing in the K 12 world where it was like, okay, this is not working and this is doing a lot of spotlighting on things that have been an issue for a very long time.

It just really amplified it and put a spotlight on it. And this is when you and I, Sharona, through College Bridge, started to do a lot of our PDs and stuff that have led us on this path of eventually starting, a podcast, a whole Center for Grading reform nonprofit.

Sharona: It really, we both have always dabbled in faculty development. My mom was a math professor, a PhD in math education, but at a math department who worked with primarily K through eight teachers, pre-service and in-service. And then when I was in grad school, she and I did a presentation at a conference about how to teach grad students to do active learning. So I’ve always dabbled and I always said that I wanted to change the way we taught math in higher ed. Yeah. And then Bosley has always said if he were to ever to leave the classroom, it would be to do faculty coaching and work possibly in a college of Ed. So we’ve already always had that, but it became bigger during pandemic.

Derek Bruff: And I gotta say it’s great work for me to get to talk to faculty in all kinds of disciplines and figure out what teaching and learning looks like in that context and help them see what their teaching choices and options are. It’s fascinating work. And I wrote an essay a long time ago about the indirect impact that faculty development can have, whether you’re at a center for teaching or some other kind of structure.

When I was at Vanderbilt, I might teach 15 students in my writing seminar, or a hundred students in my statistics course. But through my work in faculty development, I was working with a hundred faculty right. In a year, and they were working with thousands of students. And so it’s an indirect impact that you’re having, but I think it’s a much, much larger impact that you can have through this kind of work.

I find that very rewarding.

Sharona: Absolutely. And I need to remind myself how rewarding my work has been because I’ve been course coordinating for eight years and the first six and a half were fantastic. And then this last year and a half has been very difficult because I took on a broader role and I got a little bit more exposed in certain ways so that my back was a little more unprotected, unfortunately. And I definitely. I got some daggers, and so I’m gonna retool and try again. But even in this last year and a half and Bosley has been telling me this, I’ve just been so in the weeds that I don’t see it. I feel like I got nowhere. And yet the reality is I did things like introducing exam retakes and introducing corrections and reflection assignments.

And I have some faculty who are not the ones that are yelling at me who are actually taking those on and being like, Hey, are we gonna have a retake? Are we gonna have this? So I think the difference has been more than I feel like. But yeah, so I’m a little worn down right now, but I’ll bounce.

Derek Bruff: But I gotta say for someone working in a center for teaching, if I wanna make change in my department, I need to find people like you who are in departments, who have some positionality that might help them affect change. They’re doing the good work, they’re fighting the good fight. Sometimes with the math department, I can come in with my back, back math background as a bit of an outsider and catalyze some change. But in other departments, I don’t walk in with that same kind of credibility. And so partnering with faculty leaders like you, or it’s really important to making that change happen. And in turn, if you’ve got an ally in a teaching center who can cheer you on and encourage you, and connect you with resources that can help you do that kind of work as well.

Sharona: And I have to say, our head of our Center for Excellence in Teaching is absolutely fantastic. It’s just some of the institutional pressures right now at our institution, primarily budgetary, but also some other things have just created an environment that’s much harder to deal with right now.

Derek Bruff: I get it.

Sharona: So I’m hopeful, but a little brutal. I do wanna pivot right now though, because we’ve talked quite a bit about how the pandemic opened up this thing, but there’s this other big thing that’s happened to higher ed this last couple years, and I know you’ve got a book coming out about this. So let’s talk about the impact of AI on higher education, on education in general. What’s your book about and do you see this as an opportunity for grading reform or what do you think?

Derek Bruff: Yeah. I think it is right, and there are some kind of similarities to the COVID pandemic where we’ve got this kind of technological change happening that kind of comes with a bit of cultural change too, right? It’s not just tech. But it’s how we communicate, how we interact with each other, how we think about and make meaning, whether that’s the pandemic, or generative AI. And it’s, again, shining a light on some parts of higher ed, particularly our assessment structures, like really particularly our assessment structures that were perhaps not working very well before AI.

Lot’s to say. So the book is called The Norton Guide to AI Aware Teaching, and I’m writing this with Annette Vee and Mark Watkins who are amazing co-authors who come from the English and composition world. I’m again there to bring some of the stem to the collaboration. And we’re making the argument that in fact, all instructors need to be AI aware. We need to understand what does AI do, what does it not do? We need to understand how our students are using it and how they’re thinking about it. We need to be really clear on our own course learning goals and objectives and the kind of context in which we teach. The conversation in online courses is a little different than it is in person courses. And putting all that together then make informed intentional choices about our teaching that are responsive to the current moment around generative AI.

And that may mean deciding, Hey, in my course, AI is not gonna help my students learn. So I’m gonna set up some structures and some processes that are gonna direct students away from AI and towards more meaningful learning. And for some folks, it’s gonna be leaning more into AI and say, hey these tools are helpful for my own process as a teacher. They can help my students learn when they’re used thoughtfully and carefully. Or maybe I’m trying to prepare my students for a career where AI is already transforming that work and I need to get them ready for that. So we’re not saying, use AI or not, we’re trying to say, you have to be AI aware and you have to make some smart choices to respond to that.

And, I think that applies to AI. Whether or not we talk about it as plagiarism or cheating. The idea that learning is hard work and students are gonna be in situations sometimes where they want to take the easy way out and, delegate something to AI, outsource their thinking, get too much help. Th this happened in my writing seminar. When students, they sometimes get too much help from each other.

Jim Lang’s argument was though that because some of that kind of short cutting behavior by students is environmental. We, as the designers of learning environments in our courses can actually change some of that. And so he argued for fostering intrinsic motivation, learning for mastery, lowering the stakes of things, instilling self-efficacy. These are course design and assignment design strategies that not only will reduce students’ motivation to cheat, he argued, hey, they’re actually gonna help students learn. And I think they map on really well as a set of practices to alternative grading practices. And that helps us respond to the AI moment. It’s the same set of strategies, I think, that help us respond to this current problem of what do we do about generative AI. Yeah, Boz?

Boz: So what was the name of that book? ’cause that sounds like the next book on my reading list.

Derek Bruff: It’s Cheating lessons.

Boz: Cheating Lessons. Okay.

Derek Bruff: Great title for a book.

Sharona: I already have it in my Amazon cart Boz.

Derek Bruff: Yeah. I was like six months after chat GPT came out. I was like, this is all fe sounding very familiar. And I pulled the book back off the shelf and I reread it. I was like, oh yeah. Jim has a lot to say to this current moment.

Sharona: So we have said in our alt grading world for quite a while, one of the things Robert Talbert is known to say is that, 95% of students will not cheat if given a structure in which learning is rewarded, essentially. When you build those relationships and you’re able to convince your students that you really mean it when their success is what it’s about. But then I was talking to Bosley last week and I was discouraged about something why students are behaving a certain way, where they are so resistant, especially in some of our supportive math classes, to trying anything. Like they’re just so resistant. They’re gonna just freeze. And he pointed out, and I agree, that almost all of them have been berated at some point or another in their educational career by a teacher. And that creates this sort of freeze response.

So it’s like the parallel threats of, I don’t even call AI short cutting. I think what you said about what James Lang said is very critical. It is a completely rational response to a set of environmental circumstances. And we have to expect that in many cases, our students are gonna do what’s best for them and what’s best for them in the moment, especially in a traditionally graded class, is to avoid the absolute destruction of the math, even if they know it’s to their own detriment.

And Boz, who were we talking to recently, was it Michael Palmer? Who overheard some students say, wow, I really like this. I hope to go learn it someday.

Boz: Yeah.

Sharona: When they were studying for a test, was that Palmer? I

Boz: think it might have been, but yeah I

Sharona: that basically there was a conversation between two students who were talking about preparing for an upcoming exam. He was like, I was prepping for this exam. This is so interesting. I hope to be able to go back and actually learn this material at some point. And just all of these things are conflating for me. Yeah. So one thing though that I’m wondering if you’ve thought too much about, because we seem to be very focused on students and the impact of AI on the students. But is there a bigger impact?

Because I feel like we’re seeing, especially in math, that so much of what we try to teach the students is better done by a computer and an AI and that therefore this desire to shift to the conceptual and to the problem solving skills that many of us in the math ed world have been advocating for so long has now reached a critical point, and does that fundamentally change our courses?

And then what about our teaching? Because I’ve actually been using AI extensively recently to write better material, to do the things that we’ve always wanted to do, career engaged, proper applications, more accurate, real world modeling in these pristine mathematical context, I can say more culturally.

Boz: So more culturally relevant.

Sharona: So I don’t know what’s the impact on that side of it. Yeah.

Derek Bruff: So I have a there’s a granular response, which is I’ve been tutoring a the daughter of a friend who is in 11th grade algebra right now. And my gosh, that course, whew. Why is she learning how to do all of these things? All of these, factoring and finding roots and curb sketching and I know that if she’s gonna go into college and take calculus and be an engineer, this is great preparation for that. I know this stuff is gonna show up again, I don’t think that’s her path.

I think she’s gonna be a marketing professional and she’s gonna be great at it, and she’s not gonna need to know 95% of what’s in algebra two. And so this question of why are we teaching what we’re teaching? I think it’s a really important question. And I think as educators we need to have better answers to that.

I think there’s a lot of pressure in the United States right now, a lot of skepticism of the value of higher ed. And so I do think we need to be thinking about what do our curricula look like given the availability of ai? Do we need to change some of our goals and our empathies. Are there things that we don’t need to be focusing so much on more because we know that we have some computer tools that can do that work for us.

And some disciplines don’t have problems with this. Like I have faculty in data science who are like, yeah, of course new tools means we do new things. That’s obvious, right? I think where it gets challenging is that, Sharona, when you’re using AI to help you be a better teacher, you’re already a good teacher, right?

You have a lot of expertise in this area, and so you’re in a much better position to get some value out of ai. One of the things I say all the time is that expert use of AI requires expertise, right? Our students don’t have the expertise. That’s the catch 22 here. How do you develop the expertise if you are leaning too much on AI at the same time?

And so that’s the big problem I think, that we’re trying to solve with AI right now in higher ed.

Sharona: So let me ask you like a slight nuance to that. One of the barriers that I’ve always found to getting faculty to do the stuff that I’ve spent a long time learning how to do is not having really good materials because most faculty do not have the time and the expertise to develop good materials.

I think based on my experience, coordinating the statistics that giving them good materials. Is a stepping stone to letting them enhance their teaching practices because they can follow along if they’re willing and be like, if I hand them good materials to use in class and they’ve got the subject matter expertise.

’cause my faculty have subject matter expertise, what they don’t have is good pedagogy. So if I hand them materials that are designed with good pedagogy in mind, especially materials that don’t require a lot of training. It feels like that would be a way to get them moving in this direction. So is that the first step?

Can those of us that have been trained by centers for teaching and that have this pedagogical expertise can work to develop better materials. I remember I taught in the second edition of the Harvard Calculus textbook in the nineties, right? I was a grad student and I was teaching with that incredible active learning text.

That means that I have 10, 15 years of using active learning, but it’s in part because I had good materials. Those are materials are so much harder to develop. So is that a good step to developing good materials, which means my faculty are gonna get better. And then students experiencing these better materials and better teaching, which is gonna then incentivize them to do more, learn more, be more of an, become more of an expert.

Because I’m thinking of the value of this in class time that we have the students.

Derek Bruff: Yeah. I think materials are key. I think, there’s a set of courses taught in higher ed that are naturally idiosyncratic, and it’s your course and, I was teaching my weirdo cryptography course and it doesn’t have to line up with anything else.

It can be as weird as it needs to be for my purposes. But when I’m teaching statistics or linear algebra or calculus let’s not. Depend on each individual instructor to invent their way to effective teaching those courses. That’s dumb. There’s a coordination problem here.

And even then I see all the time, particularly in math departments, you may have some decent coordination within an institution’s math department where they’ve gotten together, they have course coordination. They’re working together to come up with good lesson plans and assessments. But they’re teaching the same stinking calculus at a thousand other places. So don’t limit your coordination to your local campus. There are national conversations to have. So that’s one of my answers. I think also, I’m going back to what Jim Lang talked about. One of the elements that he talked about was instilling self-efficacy in students.

And I think this is deeply related to the AI conversation, and I think there’s a faculty development piece to this as well. When I think about a student who has an assignment in front of them and it’s, 1:00 AM and they’d like to get it done in the next hour so they can go to bed and they’re not sure they can do it, that’s a prime opportunity to have AI do more of the work than AI should be doing.

Boz: Absolutely.

Derek Bruff: And so I think what I like are the grading and the teaching practices that build student competence and confidence so that when they sit down at 1:00 AM with that assignment, they’re like, oh, we practiced this in class. I did a smaller version of this last week successfully. I can knock this out in half an hour and go to bed.

That’s a whole different feeling. And it’s related to this kind of expert use of AI requires expertise. It’s knowing I know enough to either use AI in this case or that I don’t even need to go to AI. I can do this myself. And so building structures that get us there.

And I think about faculty too sometimes. I’ve seen plenty of faculty who don’t feel confident as teachers, and so they lean on traditional teaching and assessment measures because it doesn’t feel as risky. They don’t have to develop a whole new set of skills. And so I would argue Sharona that yes, good materials matter a lot, but I do think you need some professional development. You need structures to help people build their skills over time.

Sharona: I would agree. I would argue when I say good materials, to me, good materials are in class worksheets with associated teacher lesson plans. And I’m not talking textbooks. Yeah. I’m actually talking about iClicker slides. Yeah. And active learning activities. And to me, the ability, I know that I can write a good scaffolded active learning activity on a particular topic. Trying to do that for a whole course is more than I’m compensated for. So I will absolutely go to AI and be like, Hey, write me a 50 minute lesson on this topic that’s active learning that here’s the target problem I want students to be able to do at the end. Make it culturally relevant. Give me diverse backgrounds. It spits it out. I read it through, I tell it two or three revision prompts, and then I have an entire complete lesson formatted. It’s formatted. ’cause it’s outputting to overleaf. It is type set in LaTex in 10 minutes. Yeah. And the math, and I have an answer key that matches it.

Like the absolute sheer productivity that gives me, which I have my own concerns about that. But that’s what I mean by good materials. And so when I work on projects and people are like we must give our students integer numbers. I’m like, no. No, you’re doing a gravity problem. It’s a decimal.

And you should be using decimals and calculators. So to me, that’s where the grading comes in, because what skill am I actually trying to teach and what am I trying to assess? And I do think, and Bosley and I have talked about this ad nauseum, I do think that basic number fact fluency in your head is critically important.

I want all the single digits and maybe up to the number 20 memorized in your head, being able to do it super fast. But beyond that, you should just be estimating stuff like four digit, four digits, whatever. And the same thing with, our calculus, this factored into integer stuff and find the roots.

Like why are the roots always three and negative two? Why aren’t the roots 2.67 and 1.93? Yes. So anyways, I could go on that forever.

Boz: All right, so this conversation is really interesting. We probably do two or three episodes, alone on this. But I do wanna pivot one last time ’cause we are starting to already come up on time.

This has been such an interesting conversation. It’s flying by, but it would be a shame if we weren’t able to, ’cause you are the first time, our first guest that we’ve really had that is a fellow podcaster. So I really wanted to pivot to some of the podcast discussion. Yeah. And first just kinda like, how did you get into podcasting and what is the goal of your podcast? Which for those that don’t know I think Sharona you mentioned earlier, it’s called the intentional teaching.

Derek Bruff: Yeah. So I think I got into podcasts because in the early two thousands I had a 45 minute commute to Vanderbilt. And it was, back when listening to podcasts was still a bit of work. I had my little iPod nano and I would have to sync it with my, apple iTunes on my laptop before I got in the car. And then I had some weird device that allowed it to play over the FM radio, it was back in those days. But I really enjoyed the medium. I got a lot of it, a lot out of it as a listener.

And that was done with a few other units on campus that, had an interest in that space. We ran that for about six years. That was really a lot of fun, and I realized what it does, what it did is it gave me a chance to reach out to people at Vanderbilt and my own institution and elsewhere. Who were doing really interesting work and spent an hour with them, learning from them.

So when I launched my own with a little bit of software, I could do what I needed to produce the whole podcast myself. And so that’s been fun. Again, I like to make things and I think, it’s called Intentional Teaching. One it does, give me an a, a reason to call up people. And I find most people say yes when I ask them to be interviewed on my podcast. We like talking about ourselves and talking about our stuff, and if I just asked you, Hey, can I like pick your brain for an hour? You would be like, I’m busy, but can I pick your brain for an hour and then share it with hundreds of listeners? Then all of a sudden it feels okay, that’s good. I like to get my work out there. There’s some value. So I think it makes it an easy ask. What I’m trying to do, I called it intentional teaching because I’m trying to find folks who can be really articulate about why and how they teach that they are making intentional choices as teachers, and they can tell me about those choices.

Because I think then other faculty and instructors can hear that and say, oh, my situation is like this, but not like that. But I can learn from the choices you’ve made and the reasons behind your choices, and I can apply that to my own teaching. And so I really do see it as a faculty development effort.

Like for him to say, the magic happens when I teach, doesn’t help other faculty learn how to teach better. I need folks who are aware and intentional and can articulate that. And so these days, every other episode I’m doing is about AI. I feel like that’s where a lot of the conversations in higher ed now are grappling with AI. And so the other thing I like to try to do is figure out what are the problems that a lot of faculty are experiencing. And who are people that are just a couple of steps ahead of everyone else in finding solutions. ‘Cause those are the folks, especially if they can be articulate about what they’re doing, those are the folks we can all learn from.

And so that’s the space I think I often land in the podcast. So I had Eden Tanner on, from the University of Mississippi, y’all have had her on as well. Doing her kind of mastery approach to assessment. And I felt again, there was a lot of conversations about how can we teach these intro science courses better and have greater student success. And Eden was a couple of steps ahead in terms of figuring out a kind of a test retake approach that worked for her. And so I wanted to get that on the podcast so that other faculty can learn from her experiences.

Boz: See, and that’s interesting that you approached her about the mastery assessments. We approached her on how are you doing this with these gigantic classes? That was the problem that we had and, had kept coming up in our one of the biggest questions that kept coming up in the grading conference. Yeah. That’s funny. We both came to the same person for slightly different reasons.

It’s a conversation. And so one of the things that I think is working with the podcast is that we’re building this national community. I think some of this work would not be happening if it weren’t for the podcast and the conference where people are actually talking to other people. So is that an experience that you’ve had? Like what’s your take on that?

Derek Bruff: Yeah, that’s interesting. I do find that many faculty are persuaded to make change or equipped to make change in their own teaching when they hear a really concrete example of someone else making a similar change. And so it’s one thing to look at the literature and say, yes, statistically X works better than Y, but it’s another thing to hear from an individual instructor about their context, their teaching choices, their course design, their assessment choices. And to have it really concrete, that’s one of the things I sometimes struggle with my guests, is to get them to go concrete. And I’m looking, I sometimes I work really hard on my questions to try to orient folks.

’cause some academics like to think very abstractly. And they like to talk very abstractly. But if I’m really trying to help someone adopt a new set of teaching practices, we need to talk very concretely, what was that assignment? What did that look like? Who are your students? Can you give me an example of what a student did on this assignment so we could really put some color to this?

And so I think the personal connection as part of it, but I think of it more as kind of concrete storytelling is what equips a lot of faculty to to make some change in their teaching.

Boz: Yeah. Those concrete examples is exactly why my first book I recommend to anybody, after they’ve made the decision, when you’re ready to make that decision, go get grading for growth from by Robert Tavern and David Clark. ‘Cause that’s all that most of that book is here’s how it was done in this place, in this setting here. How is, how it was done in this completely different setting? And, but yeah the concrete nature of that book is great.

Sharona: But I do think you’re right. The storytelling is the key word. And I do think the human aspect of the storytelling. So hearing a voice, seeing a video, getting it off the page and into a three-dimensional human world. And that’s where I think that grading, hopefully in the age of AI can go to, is we can reprioritize the human connection between the teacher and the student. Even though it’s weird, it’s almost AI could replace the human, but it actually, I think it’s an opportunity to focus on the human.

I don’t know, maybe I’m delusional, but that’s why I wanna do the podcast. That’s why we’ve this one’s gonna be episode, I believe this is gonna be episode 126. Nice. So that’s a good run. Once a week for a couple of years. Yeah. Good.

Derek Bruff: That’s awesome.

Sharona: Couple of, so I think we’re coming up on time. Did you have anything, Derek, that we didn’t touch on that you would really like to mention or share?

Derek Bruff: I think the other thing that I had in my notes was to circle back to the University of Virginia where I’m working right now, and I noted that Michael Palmer and Adriana Streifer are leading lights around alternative grading practices.

One of my roles at UVA is working as an editor for a website we call the Teaching Hub. And so this is a website full of teaching resources. It’s designed around the idea of a collection. So Sharona, if you had a collection that you would curated, you would pick a topic and you would find the, 3, 4, 5 best resources you know of on that topic.

So Teaching Hub isn’t hosting the resources, it’s hosting the curators who are pointing to the really good resources that already exist out there. And so for the listeners of this podcast, we’ve got a pretty nice set of collections around grading and grading reform, and we’re always looking for more curators who want to help us build that out.

And so particularly since the alternative Grading Institute is full, and probably by the time you hear this already over. Knowing that we’ve got te.

Sharona: Actually, believe it or not, this is probably gonna come out in four days.

Derek Bruff: Oh, okay. Alright. You’re on that podcast schedule. I’ve been there. I’ve been there.

Oh man. One time with leading lines, we were like, okay, episodes due Monday. We need to interview someone Friday so we can pull this off. And we did. All that to say, I think the Teaching Hub is a place that I think folks can go to find some resources that they would find helpful. And again, we’re looking for more folks who want to contribute.

Sharona: We will put that in the show notes for sure. I was looking at it just now. Yeah, that’s an amazing thing. And of course the Center for Grading Reform does have repositories of syllabi and course grading schemes for a variety of primarily STEM disciplines at the moment because we did grow out of stem, which is interesting to me that a pedagogical revolution is coming outta stem.

Yeah. But I am actually going to be dipping my toe into collaborative grading this spring. ‘Cause I’ve been very much in these standards and specs, not so much the student power that didn’t bother me so much. As much as the amorphous nature of I’m like, but I still have specific skills that, it’s pre-calculus.

I can’t just let students entirely pick and choose what they’re going to do. So that’s been my discomfort. Yeah. But I’m.

So I’d love to have you or any of the co-authors back on when that gets, a little bit closer to that date or when we have the actual publication date and know when it’s coming out, just to talk a little bit more about that. But yeah, thanks for coming on. We hope you have a great holiday season.

We are coming up on that time of year. And for our listeners, we hope the same for you. You’ve been listening to the Grading podcast with Boz and Sharona, and we’ll see you next week.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode’s page on our website, http://www.TheGradingPod.com.

Or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the Contact us form on our website. The Grading podcast is created and produced by Robert Bosley and Sharon Krinsky.

The full transcript of this episode is available on our website.

Boz: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State System or by the Los Angeles Unified School District.