The Grading Podcast

164 MIT report

===

Sharona: And then they talk about backwards design being a useful framework. So for example, rather than beginning by asking whether AI should be allowed or prohibited, instructors should begin by defining the purpose of the learning experience itself, what students should come to know, be able to do, and learn to value. But they’re saying now do this in an AI-aware manner. So if in the past we were like, we wanted students to be able to write a certain way, do we really need them to write that certain way now that AI is available?

Boz: Welcome to The Grading Podcast, where we’ll take a critical lens to the methods of assessing students’ learning, from traditional grading to alternative methods of grading. We’ll look at how grades impact our classrooms and our students’ success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist, and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach, and instructional designer. Whether you work in higher ed or K-12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week, you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Boz: Hello, and welcome back to The Grading Podcast. I’m Robert Bosley, one of your two co-hosts, and with me as always, Sharona Krinsky. How you doing today, Sharona?

Sharona: I’m functional, barely. Our semester started, and this is the highest in-class teaching load I’ve had probably in eight years, after not having enough units assigned originally.

Boz: Yeah, ’cause y- you picked up, a whole class last minute that you really weren’t expecting, didn’t you?

Sharona: No, I wasn’t. And so this semester I have three lecture classes and two workshops that go with two of those classes. So I picked up a copy of our stats class that we’ve been teaching for the last eight years, that I do get to continue to teach as is, but I found out about it three days before the semester started. And it wasn’t actually determined what version I was teaching until the first day of classes. So I’m exhausted. I’m so tired.

Boz: Yeah so, just a little bit of background in case you haven’t been keeping up with this or haven’t kinda heard us talk about this. You designed and coordinated a gen ed math class called Quantitative Reasoning With Statistics that we both taught in and, you know, worked together for several years. And for many reasons that we’re not gonna get completely into, but that class got changed quite a bit. Not by your choice, but, Cal State math department wanted to take this class in more of a data science direction.

Sharona: Which they are completely allowed to do.

Boz: Oh, absolutely

Sharona: I want to be very clear. They can do it, and if that’s what they think is right for the students, I support it.

Boz: Yes. But it is now a completely different course one that I am not teaching in this semester, which kills me, ’cause ironically, my daughter is taking that class. Yeah. So the class that I have taught for seven years, she is taking that same title class, and I have no idea what that class is going to be like.

Sharona: And to clarify, it… because it’s a data science-focused class, it requires the instructors to come up to speed on a bunch of technology, including things like Jupyter Notebooks, that I’m familiar with the fact that they exist, but it’s a non-trivial onboarding for the instructors.

Boz: Yeah.

Sharona: And they had training on it over the summer and things like that, and since I wasn’t assigned to teach the class, I was not part of that training.

Boz: Yep. So-

Sharona: And so-

Boz: … So yeah … but they had for whatever reason, I don’t know what happened, why they had a class that was untaught, at the very last minute, but they gave it to you. But because you didn’t have any of that training and because it was so last-minute, they’re actually allowing you to teach the original one that you had designed, Instead of this new data science focused one, correct?

Sharona: Yes. And part of that is also because the new version is grant supported, and they’re doing a research study on it, so having an instructor who was not trained come in and teach it is gonna mess with the data. So we’re just carving the class out. It’s not part of the grant. So-

Boz: Yeah, which is the right thing to do- It is because it’s grant supported, and it, that outlier could really screw with some of the data. The… it’s the right thing to do.

Sharona: And it’s the right thing for the students, because since I didn’t have the training, I haven’t brought myself up to speed on the mechanics. Sure, I know most of the math, but there’s R coding that I don’t know and some other things. So I really do think that the institution made the right decision. I’m just shocked at how much teaching I have, because since I’ve been coordinating for eight years, I’ve always had release time to coordinate, which is its own workload.

Boz: Oh, absolutely.

Sharona: But it’s just different, and I’m like.

Boz: It’s a different workload.

Boz: W- welcome to K12.

Sharona: Yeah, I know. You’re so sympathetic. Okay, but the semester started for you, too.

Boz: It, it did, and, unfortunately, because of several reasons, and biggest one being just the, decline in enrollment which is happening to a lot of California institutes, both higher ed and K12 I, I’m not getting nearly as many units as I usually do. In fact, I am only teaching two units, which is a lab class for someone else’s lecture part. So I’m not even actually teaching anything, I’m running this workshop lab class.

Sharona: And then to make things even more interesting, yesterday the chilled water system at the university went down was operating well below capacity, so they flipped the entire campus to remote in the first week of classes. And I was not sure my students were gonna make it to Zoom because these students that are coming in now, they were in middle school during the pandemic. It’s probably been several years since they’ve been on Zoom regularly, ’cause I don’t know how much the high schools are using Zoom anymore for teaching. So I was like, “Do my students even have Zoom on their computers anymore? Do they have updated Zoom? Are they gonna know how to find it?” But yes, most of my students managed to make it to class.

Boz: Okay, good. I didn’t ask you about that before this, so you did have-

Sharona: Yeah …

Boz: pretty good attendance for a Friday class that had been- Yeah

flipped to Zoom.

Sharona: I had about 80% attendance for the first week of classes, and the one thing that was cool that I really liked, and it is something I miss about not teaching on Zoom, is I had a bunch of things I wanted to show students about things to find in Canvas and answers to get, and it’s way easier to do that on Zoom, where they can be up close and personal, and I have the annotation tools that I can circle things. So like probably 15 minutes of my class, I was like, “Here you go, and here’s where you see this, and here’s where I’m gonna do that, and here’s how you find your grades and your comments.” And I thought it’s probably better for them to do that particular content on Zoom- Yeah … than trying to do it all the way across a classroom. So that was funny. So-

Boz: In general, how did your first week, other than the unexpected flip, go?

Sharona: I gotta say, … something about me is I, unlike some people I know, love change. I thrive on change. And so I really struggle if I teach the same class too many times in a row. I like going back to it, but I have a brand new class for me.

Boz: Yeah. We’ve been talking about that.

Sharona: Oh. I am having so much fun. I have, first of all, both of the classes I managed to get into active learning classrooms. So not only do I have my rollie chairs, but I’ve got movable tables and actual, not those tiny little desks that are on an individual seat, but they actually have real tables. So my students can work together in groups, and we’re beginning to discuss and debate, what does it mean to have multiple voting methods, and I’m really engaging with my students. I’m trying some new things. I have a DJ for a day assignment, so you can pick to be the DJ for the day, and you’ll send me three to five songs on a playlist. And when we have downtime or work time, I’ll quietly play it in the background. I have an ask me anything assignment, where students come and ask me in my office hours anything, and I’ve had two of those sessions so far. And the second one was really interesting, ’cause the student’s question was, “What is my favorite type of coffee?” Because they see me carrying around this massive Starbucks cup all the time. And so they’re very curious as to what’s in the Starbucks cup. So I answered, or she asked, how do I like my coffee? And I said, “In the trash can.” ‘Cause I hate coffee. But I was able to talk about the fact that I drink iced tea and that I stopped doing soda. So it just was a really fun conversation, and just got to know each other a little bit more.

Boz: So to make it clear, what’s in the Starbucks cup is the tea.

Sharona: Is the tea. It is a venti, it’s a venti cup, it’s a cold cup, and it’s tea. It’s always tea. Unsweetened, black, lots of ice.

Boz: Yeah. Okay, so I am really jealous, ’cause I have kind of the opposite like physical setup. So I am in this little bitty this is probably the smallest classroom that I’ve ever taught in at Cal State. I’ve got a workspace at one of my high schools that is about the same size, and there’s two of us in there. There is like 29 students in this class, and so far my attendance has not been less than 27. There is no room I am someone who even when I’m lecturing, is usually walking around the room. Not just in front of the board, but there’s no space to do that. And on

Sharona: top of- I’m pretty sure I taught in that classroom.

Boz: Yeah? A- and on top of that, I’ve got three walls of boards, which is great, except they’re blackboards, not whiteboards. I do not know the art of teaching with chalk, because by the end of the second class session, ’cause the first class session, was setting norms and getting to know each other and everything. But at the end of that second one, I had a black shirt that looked like, I don’t know, a, a zebra with spots instead of stripes. I had so much chalk on my shirt.

Sharona: You’re gonna have to be careful what you wear.

Boz: Oh my gosh. That’s ironic. But yeah, that classroom is so freaking small, and it hurts me. I, like I said, I walk around all the time, and especially when we’re working in small groups, I’m constantly walking around, eavesdropping on the conversations, joining in on the conver- I can’t do that. This room is so small and so crowded.

Sharona: And- So I have a question, because this actually came up. I was having a conversation with your daughter this week, and I was explaining my obsession with rollie chairs. And she said that she thought you would hate rollie chairs. And I’m wondering now that you have a place like this where it’s impossible to lift and turn, ’cause I’m assuming there’s no rollie chairs in that particular room.

Boz: No, not only is it not that, but it’s not uniform. So there’s a lot of the individual desk with the chairs actually attached, but they’re different sizes. I’ve got a few desk chairs that are separate, so they’re not attached. But yeah and the non-u- uniformity of it also creates more travel room issues.

Sharona: So if you had rollie chair desks, so they were all the, the desks attached to the chairs, but they rolled, would that be better or worse? Or would… do you like rollie chairs for a classroom?

Boz: So this is the only thing I don’t like about them is when it’s not all of them, because especially in my K12 settings, I had for a couple of years I was in a, what was basically a converted computer lab classroom. So I had, these really long tables. The computers had been taken off of them so it could be used as a classroom, but I had a mix of chairs. Some of them were rollie chairs, and it was always a fight. The first students that would come in would grab them, but they would try to pick them up and carry them around the room to sit where they wanted to sit, but… so that’s the only thing I don’t like about the rollie chairs is when it’s not uniform because it’s distracting and it really was people would rush into class and would run to… and a few times, like- comically but almost dangerously running to, and trying to get to the chair first so they can move it. Especially that last one. Okay.

Sharona: That’s not where I was expecting you to go when you said not uniform, because some of our classrooms have chairs that have wheels on two legs and not wheels on the other two. So basically they’re easy to move. You lift up the back and then you can roll it around, but you can’t scoot roll, right? And I was thinking you were going that direction with non-uniform, ’cause I’m wondering, I told your daughter that I thought you would like the two and two.

Boz: Yeah.

Sharona: Because they’re not … Because I could imagine with rolly chairs and high school students, them just careening out of control. College is not that problem, but I could imagine it in high school if they were four wheel rolls. So I was thinking maybe two wheel rolls and two stationary you might actually like. But anyway.

Boz: I don’t think I’ve ever seen the Heely of chairs.

Sharona: I’ll show you. It’s in Salazar Hall.

Boz: ‘Cause that’s what it sounds like. It’s in Salazar Hall. The old Heely shoes with the- Yes.

Sharona: Yeah …

Boz: wheels that you lift up and you roll.

Yeah.

Sharona: Okay, so this is a new record, though. We are, like, 13 minutes in, and we have not started talking about grading, so we probably should go there now.

Boz: And you’re right, ’cause we have got a really deep topic to go into. There is a lot of meat on this. So yeah let’s jump in. What are we talking about today?

Sharona: I do need to mention that I think we’ve hit a new phase of the podcast, and I’m … I was joking, but I’m serious about this. Three, four years ago, or three years ago when we started the podcast oftentimes we were just looking at research articles or getting people in to interview or talking about our own thoughts. We never were really dealing too much with news. Like- No … current news. Well- ‘Cause there just wasn’t that much …

Boz: there wasn’t, yeah. There wasn’t that much around- Right … grading specific.

Sharona: So then as of since April we talked about Harvard to death at this point, but now we have something that’s just as big, which is MIT has released a massive report on AI in education. And you might think, okay, but this is not an AI podcast. But it turns out that they went deep at including a lot about grades. So we’re gonna dive in to the MIT’s Ad Hoc Committee on AI Use in Teaching, Learning, and Research Training.

Boz: All right. So before we do that, though, I just have to say, because you were talking about how we’ve been doing this almost weekly our current affairs on grading around the world, There is not a chance in hell we’re gonna start a second podcast, you understand? I’m not doing it. I…

Sharona: No, I’m clear. I’m clear. I’m not starting a second one with you. There is someone you know who’s been bugging me to start a second one. But we’re not doing it yet.

Boz: No. Not yet I’m d- I’m out. No.

Sharona: I don’t know. I know what levers to take to get you to do the editing.

Boz: All right.

Sharona: I have a very significant lever that I can convince you to do the editing if I ever start it, but we’ve been talking about it for two years and we haven’t started it yet, so I think you’re safe.

Boz: All right. So let’s get into this. What did this kind of research center around, and what were some of the conclusions about what MIT should be doing?

Sharona: Okay, so I wanna set the groundwork a little bit for what was going on. So they made a committee that com- consisted of undergraduates, graduate students, faculty from every school, staff from all the relevant units, including the libraries and the teaching and learning lab, and they spent five months meeting, research, and outreach across the community to understand the role of generative AI in both life and the educational mi- mission of MIT.

Boz: And they were given three specific charges to look at. Yes. Which was assess the current AI use at MIT, identifying innovations in teaching and student assessment, and and pose an AI use policy.

Sharona: Yes. And I wanna read the paragraph right after those three bullets in this report ‘Cause I think this is the critical thing that anyone who does alt-grading is gonna be completely not shocked about. It says, “However, what we learned as a group quickly convinced us that the institute community, particularly the faculty, must tackle a set of deeper questions about the structure, meaning, and value of an MIT education in an era where AI is one of several factors complicating the institute’s mission.”

Boz: Gee, that kinda sounds like someone I know that has been talking about the fact that higher ed as a whole needs to reevaluate our purpose and our methods since AI.

Sharona: Yeah. I really think, I, not to be alarmist, but, I’m old enough that I lived through the creation of the internet, the creation of the World Wide Web, the invention of social media, and now this, and this feels like a much more existential threat to higher education as an institution than the other ones did. The other ones to me, yes, they were, quote-unquote, “threats” to the way we taught, but they didn’t, to me, threaten the institution itself as a institution. But they went on in this report, and they hit several things. What I’m so excited about is this is a group that did not come to this through an acknowledgement that we needed to change grading, and yet so many of the things they’re saying in this report are directly in line with the things that we say about grading.

Boz: Absolutely.

Sharona: So one of the things a little bit further down when they get into the landscape, they talk about while AI is allowing instructors to develop new learning experiences, there’s been a number of concerning effects on the life on the campus, and there’s a bunch of things about increasing isolation and undermining student mastery, but the one that I really like at the bottom says, “Challenging decades of distinctive MIT community norms and values about rigor, the creative friction required for learning, collaborative problem-solving, and personal integrity.” Because they think they have these principles, and yet a brand-new technology comes in and just upends all of it.

Boz: Yeah. And these are… these are some of the core principles and foundations of MIT. I mean- Yeah … these are not, oh, it’s a little bit of a… No. They’re saying this is a threat to our whole identity as an educational institute. And going along, along with that at one point they’ve got, AI is generating both immediate rapid changes and long-term d- disruptions, and MIT needs to respond, and they’ve got a couple of bullets, but one of them says, “Every subject taught at MIT will likely need to be re-examined and potentially revamped to make sure that how our students are being taught, what they’re learning, and how they are assessed are AI aware.” Not some of them, every subject taught.

Sharona: Absolutely. So I really liked how they structured several parts of this report. So everything that we’ve quoted so far they set out the landscape, and then they defined some principles that they relied on. But one of those principles is lean into learning, and they introduce this section by saying, “Many,” and I’ve actually seen some reports that say most, but they said here, “Many problems and assignments used in MIT classes to reinforce learning and assess student progress can already be accomplished by AI.” Let’s think about this. MIT, which is like the tippy top of our technical school, right? Of a school that, that focuses on technology and is known for how hard and difficult their assessments are- Almost all of them can already be done by AI.

Boz: Yeah. And this was the one and only school I applied to that I did not get into. MIT is, one of the elite when it comes to STEM education.

Sharona: I applied early admission and got wait-listed, and then I accepted Berkeley before I heard back about the wait list.

Boz: So, did you ever hear back if you would’ve gotten in? No.

Sharona: No. ‘Cause I think I notified them before they got a chance- Okay to tell me.

Boz: I thought a brag might be coming along since I just admitted that- No. No … I applied and did not.

Sharona: Caltech took me. Thank God I didn’t go there. But MIT never admitted me because I notified them that I had accepted before they had a chance to tell me anything.

Boz: So but getting back to this section, ’cause this section is extremely interesting, and when I read some of this, like, all I can see or hear in my head is how this, even though they don’t call it out here, how this relates to alternative grading and some of the practices and beliefs we have.

Sharona: They do. Not specifically alternative, but the very next sentence says, “Unrestricted AI use by students can make some traditional assessments less reliable as indicators of individual learning, potentially weakening confidence in grades and credentials.” So they are already identifying that grading is an issue.

Boz: Yeah.

Sharona: But the one I really wanted to look at is some of the stuff that they say in section 2.5, Teach with Intentionality.

Boz: Hold on. Can we go back? Okay. There’s-

Sharona: Sure …

Boz: there’s a lot more here in 2.4 that I really- Okay … wanna be… A- and the title of this section is Lean Into Learning.

Sharona: Yes.

Boz: They have one sentence that’s right after the first paragraph, “But as a community, what should worry us the most is that many use of AI deprive students of the opportunity to learn.” And then a few paragraphs later, they get into the, the need to focus on learning, that so many things that we have done in education, so many of the, the, the GPAs and how we assess and chase. Takes focus away from the actual learning, and that’s what we need to get back to, is that forget the GPAs, forget the diplomas. Let’s get back to learning.

Sharona: And one of the things that they say in, in, towards the end, and I had not phrased it this way, but I think this is what a lot of alt-graders mean, is they say leaning into learning means creating a new social contract between teachers and students. All of us need, who teach at MIT, but I would say all instructors, need to be prepared to help students understand both that the process of education is necessarily a productive struggle, and their most important product of their education is not a GPA or a diploma, but themselves. So I think that’s the language I wanna start to use, is that I wanna create a social contract with my students in every class that says, “The contract is I agree to create an environment in which you are safe to learn, in which learning is celebrated, and you agree to participate in that learning through productive struggle and be an actively engaged participant in that.” So I wanna think about how I would write a social contract.

Boz: Little bit below that. I wanna read this, ’cause this is one of those that when I read it and everything in my head was just like, “Oh, that’s this pillar of alternative grading,” or, “That’s this pillar.” Instilling these attitudes need to become a central task for every educator so that our students know not only what they should learn, but also how they should learn, why learning matters. We need to help them develop metacognitive abilities to think about thinking, to engage in reflective practices, and to enhance their sense of personal agency. This will both dedicate and refresh parts of instructors.

Sharona: Yeah, so that is what is required.

Boz: Yeah, the idea of our clear learning outcomes, like being intentional, which we’ll come back to learning outcomes, ’cause they talk a lot more about that. But the idea of these clear learning outcomes, that the entire grading architecture of a course is built around, instead of build around points, percentages, and averages.

Sharona: And I think what is hitting me so hard with this whole article, it’s not that these are new concepts to those of us in the alt-grading world. We’ve been talking about this till it’s coming out our ears. But the reality is, places like MIT and Harvard and University of Michigan have an outsized influence on the higher education landscape. You can literally name six schools across the country, or maybe 10, that all the other institutions, especially the research institutions, want to follow and say if they do that there, we should think about doing it here.” And MIT is one of them.

Boz: Oh, absolutely.

Sharona: So this feels very, not only validating, but like now if I want to go to a university’s administration and advocate for this change, and they’re like where’s the proof?” I have this document that is 40 pages long that I can hand them and say, “Here, let’s talk about what MIT is doing.”

Boz: All right. So you want me to talk about the next section as well, 2.5, which was teach with intentionality. So what were you… Before I brought us back, what were you-

Sharona: That’s okay. So it’s the second two paragraphs. They say, “It’s time to reflect deeply and extensively on what we teach and how we teach it. Class structures, assessments, and policies must be built with intention and clear set of purpose. Not every course is gonna change in the same way.” And then they talk about backwards design being a useful framework. So for example, “Rather than beginning by asking whether AI should be allowed or prohibited, instructors should begin by defining the purpose of the learning experience itself, what students should come to know, be able to do, and learn to value.” But they’re saying, now do this in an AI aware manner. So if in the past we were like, we wanted students to be able to write a certain way, do we really need them to write that certain way now that AI is available?

Sharona: up. I think

Boz: it was Susan Blum. This is the second time I’ve referenced it – I was gonna say we can’t keep that cat and mouse game up. We will lose. We will lose every single time.

Sharona: So I wanna give a specific example of what we’re talking about here. Okay. Because I have been using AI to design my course for this quantitative reasoning for everyday life. And I’ve been using it intentionally because I’ve been testing how well I can bring disparate pieces of pedagogy that I’ve been learning over the last several years and integrate them into this course, and what does AI do with it? Is it any good at it? And it came up with something that I am shocked about how much I like the way the AI worded it. So this is a quantitative reasoning for everyday life. That is the name of the course. Five of the learning outcomes out of something like 14 are specifically about quantitative reasoning. So I told the AI, I said, “I need to build a portfolio project. Here are my learning outcomes. Here’s this, here’s that.” But I want to give students an option to either do this assignment with generative AI, or without. And what it said is, it said, “I think you should create a forked portfolio.” So there’s two options. The portfolio is exactly the same, but there’s one of two ways you go. One of them is if you choose independent reasoning, do not use generative AI. And here’s a claim, and investigate this claim and you need to do the calculations yourself. You need to do various… That’s called the independent reasoning pathway. But then the AI gave me a AI-assisted reasoning pathway, where instead of doing the calculations yourself, you have the AI do some of that and do some of the research and sourcing, and then your reasoning part is justifying or critiquing or analyzing the AI’s material. So you’re actually doing the same exact skill. You’re doing the same exact reasoning with quantitative evidence skill in two different ways. One is doing it on your own without the assistance of generative AI, one is doing it with. But it’s the same learning outcome. It’s the same skill at the same difficulty level. And I liked that language of independent reasoning or AI-assisted reasoning.

Boz: Unless you wanna talk some more about this one, I the very next section I think is also really important, and I wanna make a point about it.

Sharona: Yeah, I’m happy to go there.

Boz: So the next section, which is titled No One Size Fits All Starts off by writing, “No single approach to AI in education will serve MIT well. A poetry seminar, a mathematical proof course, and an architectural design lab entail different relationships to AI. In some, using the AI to generate work or solve a problem defeats the purpose of the exercise. In others, AI fluency is already a part of the professional practice, and avoiding it would leave students short-changed.” So I’m reading this, and I’m going back, and we often joke, you get two alternative graders together, you’re gonna have at least three different ways of doing it. And again, one of our first things that we talk about when we redesign a course or when we do PDs around this, is you have to really examine what’s the purpose of your course.

And I’m really thinking at this point, the use of AI actually is part of the grading structure. And I’m thinking we need to start incorporating this same conversation when we’re talking about, w- the purpose of the course and thinking about the audience and thinking about where it is in a sequence ’cause that’s gonna affect your grading architecture of decisions. I’m thinking we need to start also including this AI relationship, like what kind of relationship to your learning outcomes and the purpose of your course does AI have?

Sharona: And what’s interesting though also, and I don’t know that I can find the article right now, but I just read an article yesterday about the need to preserve human writing in the face of AI. Because one of the things that I’m grappling with in math is do we still need algebra skills, like handwritten algebra manipulation skills, particularly in calculus? Do we need to spend so much time on this? But I read this article that even looking at human writing, there’s layers upon layers of interpretation, so getting back to original sources can end up if you go all the way back to an original source, that can result in a very different interpretation and understanding of something in a historical context than something that’s been written over and written over. And one of the issues with AI is it’s writing over the original source material at a rapid rate. So where’s the line, right? So I think I completely agree with you. We need to add to this course design piece, where does AI awareness play in? How does it affect your learning outcomes? How does it affect what the students need to know? But then also not being too quick to throw the baby out with the bathwater, and to say that all of these skills that AI can do, we don’t need to teach.

Boz: Yeah.

Sharona: So I think it’s both of those things. I’d like to move on though, if we could, to section three, the recommendations. In particular, I wanted to call out one thing in the overall and then head over to section 3.1.6 if I can, unless you have stuff before that. But in what, in the beginning of this section, they talked about how to adapt educational processes for an AI aware world, and it says, “Campus wide and as soon as possible, review what our students need to learn.” And it talks about the different kinds of things they need to do, and then it says, and it’s such a throwaway line, but I’m like, “Oh, this is so big,” “This will also require an investigation into the nature of grades and grading.”

Boz: Yep.

Sharona: It’s at the end of a paragraph, and then they move on. I’m like, “Ah. Oh.”

Sharona: Yeah, on Inside Higher Ed.

Boz: Yep, on Inside Higher Ed. MIT’s AI report calls for alternative grading and more social learning.” And I’m sorry. I was gonna try not to bring this up, but I couldn’t. In this article, it goes into and bashes on Harvard again, on saying we- Okay. To be clear … the one thing we don’t wanna do is do what Harvard’s done and cap-

Sharona: But that actually is in the MIT report.

Boz: It is. It’s there too.

Sharona: So I wanna get to that, ’cause I think that’s really important. But yeah, no, I do love the fact, and thank you, shout out to Emma Whitford for just saying, “Hey, the AI report calls for alternative grading.” I’m like, “Yes, please. Yes, please.”

Boz: But that’s, that was one of the things so this report that we’ve been talking about is pretty extensive. A- and- Yeah … it’s got several different sections, and in the recommendations, there is one section that talks about grading. But it’s w- it’s one fairly small piece, and yet it was, it’s so important that’s what this author of this article, what Emma actually focuses on, and even puts it in the title, that this whole report, even though it only mentions alternative grading in this one section that we’re probably about to talk about, was so important, that’s actually where she started this focus on this article.

Sharona: Okay, I have a couple more that I definitely wanna get to. So in section 312, which is ensure durable learning through new course policy structures and forms of assessment, the very end it says, “We urge instructors to consider forms of assessment that are less vulnerable to AI.” Gives a bunch of examples, but this is the sentence that I think is very important: “This likely means that resources such as TAs and class time will become more central to evaluation, and also raises the question of whether we should find ways to limit class sizes.”

Yes. Yes, TAs, yes, class time for evaluation, and yes, smaller class sizes.

Boz: Yeah, and going back a little bit before that, I think it has a really powerful sentence in here that when I first read it, it kinda stuck out, but as I go back and reread it, this is a really powerful line.

“AI has both created and revealed a mismatch between established learning objectives,” so what the– we are wanting our students to know, “and familiar forms of assessment.” And again, this kinda goes back to something you’ve been talking a lot about, that with AI and this cat and mouse game that I keep referring to, we can’t keep relying on our same forms of assessment.

Y- like you said, the, it is too frictionless now to misuse AI. But them understanding and, like they said, create, both creating but also just revealing this mismatch between what we say we want our students to come out with and how we actually assess it.

Sharona: Yeah. Yep. Yep. 100% yep

Boz: Yeah. And later on it, in bold, “Rather than simply AI-proof current methods of assessment, instructors need to revisit what they really want students to know and devise assessment that foster or even include the kind of productive struggle that builds durable understanding and mastery.”

Sharona: So I’ve been really thinking about some of this stuff with this, this new class I have, and we do some beginning of the semester just fun activities to get our point across. We have an asset based activity where we read about these three gamers, and then we write some characteristics and things, and then we often do a math autobiography where they describe their history with math.

I told my… Normally those are, we just do them, we’re in class, and then I never look at them again, and students do whatever. I told my students to keep those. I haven’t told them why, but I’m planning on having them include them as evidence in their end of the last portfolio they do for the semester, and to put in a reflection where they go back and look at what they said about themselves in the beginning and how they feel about themselves now in relationship to both math, but to learning and things like that.

So I think that too becomes a part of the assessment structure. Even though this is not a grade, I don’t give them feedback on it because what feedback am I gonna give them on their self-assessment of their character traits? You know- Yeah … that they’re cooperative or that they’re helpful or whatever.

But I do think that it is actually a piece of what I’m going to assess at the end portfolio is have they grown? Have they learned to reflect? I think reflection, learning how to do self-reflection is probably one of the most critical skills that we need to teach students at this point.

Boz: Absolutely. I would agree 100% with that

Sharona: Okay. I do wanna get to, section 3.1.6

Boz: Yes. Let’s

Sharona: do- Can we go there?

Boz: Let’s do

Sharona: it. Okay. We, when we read this section we could … Okay, so the title of the section is Reconsider Grades and Incentives, but I want to read the first paragraph. Now, what’s interesting is MIT itself actually presents this material in two ways.

There’s a PDF, which we’ll link to in the show notes, but then they have a more friendly facing version that is almost the same, but not exactly the same between the two versions of the report. Which was a bit- So I’m gonna read the- …

Boz: interesting. And-

Sharona: Yeah, because it’s their own report, and the links that we’re giving you are linking to MIT websites.

But I’m gonna read the friendly facing one, but then I also wanna maybe read the other one. So it says, “Several peer institutions recently imposed limits on the number of A’s instructors can give in a subject.” Are there several? Are there really several?

Boz: Yeah, there, there are some that have f- that were already established before the Harvard thing came out, and then-

Sharona: Right

Boz: since Har- But I

Sharona: think there’s only one that has done it recently.

Boz: No, I think there was Was there a second one? … one other big one that followed Harvard,

Sharona: Okay. So they say here, “We do not advocate this kind of grade rationing, in part because we expect it to be counterproductive in helping the institute navigate the challenges of AI.”

Period. Mic drop, right?

Boz: Yep. “

Sharona: Responding to an intense societal focus on credentials and grades, students who are set on maximizing their GPA have a strong incentive to use whatever means they feel are most effective to achieve that goal. Rationing top grades would intensify the temptation to cut corners on actual learning by increasing reliance on AI.”

Boz: Did you hear that, Harvard?

Sharona: I’m pretty sure they’re hearing it, but I don’t think they care.

Boz: That okay, come on. We can’t… Don’t go there. There’s a lot

Sharona: of- Oh, the faculty care, I- Okay … some of them. I just, I don’t know, as an institution. Anyway and then they go on to say, “As instructors explore new forms of assessment and experiential learning activities, MIT should take this opportunity to consider what role grades play in our overall system, and if the current approach could be improved.

For instance, if instructors mainly respond by putting more weight on in-class exams, there’s a risk of narrowing what the MIT credential has long signaled: that MIT students are capable of difficult, independent, and thought-intensive problem-solving, not just acing exams on paper.” And then I’m gonna read two more sentences-

Boz: Okay

Sharona: all the way up to the footnote. “The committee encourages MIT to explore alternative systems of grading and assessment, including those that work in other countries, such as the UK system, which uses percentages to express relative mastery against an expert standard, and newer paradigms such as competency-based and mastery-based assessments.”

Boom.

Boz: Yep.

Sharona: Boom. We’ve got MIT’s imprimatur.

Boz: So th- that came up in the article, but I wanna read, not the research, but the article, the next thing they quoted, which I think is from the research somewhere. But the committee discussed the idea that if MIT did not have grades, did not have grades, many of these incentives around AI cheating would disappear

Sharona: Well, and go one more sentence

Boz: Okay.

Al-

Sharona: Note

Boz: that

Sharona: MIT-

Boz: Do what?

Sharona: Go ahead.

Boz: Already- No, go ahead … many employees are less focused on applicants’ grades than on their performance on internal assessments, such as the ability to answer difficult interview questions or demonstrate problem-solving and mastery through custom exercises.

Sharona: Okay. So you were reading a different one, different version.

Boz: So I’m

Sharona: reading- Either of the two that I have.

Boz: Okay. So I- ‘

Sharona: Cause I was thinking-

Boz: Yeah. This was in the, towards the end of the Emma Whitford.

Sharona: Oh, okay. No, I was going for the one in the actual part. It says, “Note that MIT already refrains from the common practice of awarding summa, magna, and cum laude diplomas because we believe that earning an MIT degree stands as distinction enough.”

Boz: I love that sentence Exactly. It, and again, w- and we talked about this a little while back when we were talking about what was the school that’s just now doing- Michigan? Yeah, Michigan. Covering the grades. Doing the, the covering grades, which is something that MIT has been doing for a long time already with their freshmen.

They already do that pass-fail. So the, the idea of if we, MIT had no grades, I don’t think is as far-fetched as a actual possibility as some people might think.

Sharona: I completely

Boz: agree. How incredibly groundbreaking would that be for… ’cause there are some colleges that do it, and we’ve had them at the grading conference that actually have no grades, but can you imagine an institute like MIT going, “You know what?

We’re gonna go completely away from grades. We are not gonna have-” Yeah … interim grades at all.”

Sharona: Yeah. So a lot of the rest of this document does talk about continuing to talk about the social contract and other elements of it. And I would argue that although I don’t know if it’s in this, ’cause I haven’t read I will admit I have not read all 40 pages-

of this report. But one of the things that has been a little bit of a debate is, that I’ve seen out in the sort of the general news media reporting, is students are upset that instructors are using AI and then coming down on students for using AI. So one of the recommendations is, that they give, is encourage instructor disclosure around their own AI use.

And I’ve been thinking about this because I am using AI in developing some of my course materials because my brain is, does not have enough space to both think, to make things individually course relevant, AI aware versus, you know, AI assisted versus independent, bringing in culturally relevant materials.

I only have my culture. I don’t have everybody else’s culture. So if I wanna try to bring that stuff in, I need to rely on resources. But what I think is interesting is if I disclose my AI use and I tell students and give them sort of a model of, “This is what I could do if I didn’t use AI. This is what I can do if I do use AI.”

What do we think? And have a debate about that, right? I think that could be very interesting in helping students figure out their own sense of ethics around AI use.

Boz: It’s an interesting yeah, it’s an interesting idea. And again, depending on what my course was and wha- some of the goals of my course, that might be a really powerful student activity to do at the beginning of a course.

‘Cause again like this is pointed out and like we’ve said so many times, there is no one size fits all, so this might be really appropriate and powerful to do in some classes, and other classes might not be. But I like the idea of it. I like… I- if you do it, I’d like to hear how it goes.

Sharona: Yeah. I did wanna also, I’m feeling a little bit of like a prophet in my own time, but the AI report the report here in, going further down, section 335, provide ongoing training and instructor support.

Boz: Absolutely.

Sharona: If we don’t start doing professional development at the higher ed level, again, I think it’s gonna be an existential failure for us as institutions. A-

Boz: And it’s not- we gotta do this … and I’m sorry, it’s not just higher ed. Th- this is K12, yes. The K12 system isn’t doesn’t face the same threat as higher ed because it is not optional.

But that, that need for ongoing PD and ongoing training around the use of AI, the potential usage, the purposes, the benefits, and the harms, like this is something that needs to be part of education from K to 16.

Sharona: Yeah, absolutely. But I’m just, going back to something we said at the beginning, I am so excited that literally we’re coming on and reacting to stuff that is hitting our educational media this week.

So ha- being able to react this fast, but also understanding that so much of about grading and assessment is now in sort of the cultural zeitgeist of higher education is exciting. And to see, massive kudos to MIT for taking this leadership role in this direction and saying, “I want an MIT degree to mean something.”

Boz: Yeah.

Sharona: And it should mean… And every the way I interpret this is they want every single person who graduates with an MIT degree to have this experience of being of distinction and quality and not pitting their students against each other.

Boz: Yeah. And I just wanna give kudos and thank MIT for understanding that this discussion, this AI discussion, is also about grades, and that so many things in– and you talk about this a- as, that, that blanket that smothers everything out.

Gra- the grading reform and alternative grading isn’t going to f- necessarily fix any of these issues, but it’s gonna get things out of the way so you can address it. And this, and knowing that if you don’t address the grades and the grading practices, that you can do a lot of this stuff, and it still ain’t gonna matter because as long as it’s focused on the end grade and the GPA and the point-chasing, it’s always going to be a factor.

It’s always gonna get in your way. So thank you, MIT, for recognizing that and really calling that out

Sharona: Yeah. The, the leadership is so crucial. Josh Eyler’s one of the people that we saw was posting about this, and he’s quoted in this higher ed article, and he says, “I cannot tell you how long I, along with many who study this subject, have waited for a major university to take a stand and make a clear call for a shift in practices of this size.

Remarkable, and to be honest, I’m still a bit shocked, but I’m excited to see how it plays out.”

Boz: Yeah.

Sharona: So MIT, we’re here. You need some faculty development, we got you covered. Just saying.

Boz: All right. So w- we are coming up on time. Is there anything that you wanted to kinda close out with before we wrap this up?

Sharona: Just wishing everyone who is listening a h- effective and smooth start to whatever semester, quarter, school year, academic year you’re in or end of year if you’re in a completely different calendar. I don’t know, maybe the Southern Hemisphere has a different schedule. But thank you to everyone listening and it’s really been quite a ride.

Boz: Okay. And I kinda wanna close this out with an- one last quote from the article, and I don’t know if this is coming from an interview with some of the committee members or something that they wrote about the research, ’cause I didn’t find this line in the research itself. But I wanna read this.

And again, MIT, thank you for your leadership and your recogniz- recognization

Thank you for your leadership and recognizing the role that this all plays. But I wanna read this. “The work will require thoughtful commitment and a focused effort from the entire MIT community, students, instructors, staff, and leadership. The goal to preserve and enhance the distinct transformational power of an MIT education,” the committee wrote, “it is not too much to say that MIT’s mission depends on it.”

Sharona: Wow.

Boz: Whew. What a way to wrap up. I wanna thank everyone for listening. This has been The Grading Podcast with Boz and Sharona, and we’ll see you next week

Sharona: Please share your thoughts and comments about this episode by commenting on this episode’s page on our website, http://www.thegradingpod.com. Or you can share with us publicly on Twitter, Facebook, or Instagram.

If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the Contact Us form on our website. The Grading Podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.

Boz: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State system or by the Los Angeles Unified School District