The Grading Podcast

49 – emily and AI

===

Emily: I think I wouldn’t be fully fulfilling my responsibility as a writing instructor if I was not honest with students about this writing tool that they are going to encounter in their lives from this point on. And I think, you know, we could, we can have a much, much longer conversation about the ways in which AI might or might not be helpful to them. But one of the things that I also try to convey in those conversations is that students really need to become deeply reflective about their own learning processes so that they can know when AI is helping them and when it’s hindering them, right?

Boz: Welcome to the Grading Podcast, where we’ll take a critical lens to the methods of assessing students learning, from traditional grading to alternative methods of grading. We’ll look at how grades impact our classrooms and our students success. Thanks. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist, and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach, and instructional designer. Whether you work in higher ed or K 12, Whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week, you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Boz: Hello and welcome back to the podcast. I’m Robert Bosley, one of your two co hosts, and with me as always, Sharona Krinsky. How are you doing today, Sharona?

Sharona: I’m doing much better now that the gray of the June gloom of the California skies has given way to the pretty blue of our summers. Gray is not my color, especially when it’s in the sky. So I am much happier today. How are you doing Boz?

Boz: I’m doing good.

Sharona: And I’m excited to let you know we have another guest in our studio this week in our virtual studio. We’d like to welcome Dr. Emily Pitts Donahoe, which I believe, I hope I pronounced correctly.

Emily: Yes, that’s correct.

Sharona: She is the Associate Director of Instructional Support at the Center for Excellence in Teaching and Learning, as well as a lecturer in writing and rhetoric at the University of Mississippi. She is the Substack newsletter that I will admit I read every week called in the grade, as well as a member of the organizing committee for the grading conference. So welcome, Emily.

Emily: Thanks. I’m really excited to be here.

Boz: Welcome, Emily. Even though Sharona and I both kind of know you a little bit this is the first time you’ve been on the podcast. So one of the things we always like to hear from our new guest is just how you get involved in alternative grading. Like, how did you get involved in this crazy world?

And I did. And it worked pretty well. Obviously, I think the first time anyone does this, there’s a lot of things we wish we had done differently, or, you know, a lot of changes that have to be made, or, you know, a lot of refinement that has to happen. But I was really happy with how it went. And I haven’t been back to traditional grading since that time. Every class I’ve taught since then has been on a model that I’m increasingly calling collaborative grading. So that’s my origin story.

Boz: Quick question for you, because you said you were surprised by some of the research. I’m curious, do you remember what surprised you the most about the research you were doing on alternative grading and traditional grading?

Emily: Yeah, that’s a good question. I I wasn’t doing research at that time, but from my memory of kind of reading the article, what I was struck by, I think the most was just the idea that grades, weren’t very motivational for students, right? I was always a student who was very motivated by grades and, you know, it hadn’t, the, the kind of division between being grade motivated and being learning motivated hadn’t really fully hit me at that point. And I think a lot of us, I did as a lot of us do, kind of taught in ways that I thought would be beneficial to me or that were beneficial to me. And so the, the connection between grades and motivation, I think, was a little surprising to me at that time. And I also just hadn’t really fully examined the idea that, that traditional grades were not really good reflections of learning, right? So it’s not that the research was particularly groundbreaking. It’s just that they were a lot, it was a lot of new ideas for me at the time as a graduate student who’d never thought about any of this before. Yeah.

Emily: Yeah, that was certainly the case for me.

Sharona: So I looked up the article. It’s called Grades Can Hinder Learning. What Should Professors Use Instead? And the research that you’re referring to is the stuff done by Ruth Butler about that Boz that we use in our what’s wrong with traditional grading talk about the results of the different tasks given about whether there’s feedback grades and feedback, just grades. So, yes, we utilize that research from 44 years ago, which is some of the “newer” research and it. So yeah, that’s right. Right there with you.

Emily: Yeah. The Ruth Butler study, I feel like is always where I start with people who are in disciplines adjacent to mine or my own, who might be skeptical about this. Do you ever have the experience of giving students a grade and feedback and they look at the grade and then throw the feedback in the trash on the way out the door? Right? That’s yeah, that partially that’s because, you know, this confusion between, you know, grade and feedback for students, you know, the grade can hinder their ability to take in that feedback. So, yeah, I always feel like because that’s the starting place for me. I feel like that, that can be really useful for other instructors as well.

Sharona: So another piece of the origin story is how we got connected. And part of that is. This was before Twitter got nuked badly by its owner. And so there was actually a robust math Twitter and education Twitter going, and you had, I believe started tweeting about some of these experiences and then good old Robert Talbert threw down the gauntlet one day. And if I recall correctly, He basically said to all the people on that are we’re doing extensive tweeting. Come on people. Can you please write long form blog posts because people need to learn from you and my reaction was Oh, hell no. But your reaction was a little bit different. So how’d that happen?

Emily: Yeah, I’ve heard you talk on the podcast before about your resistance to writing books, and that’s why you have a podcast, right? I, I am not as good in podcast form actually, I think I’m much better in, in writing. And one of the things that I have a colleague, Liz Norell, who wrote recently about how, useful reflective writing is for team building and other things. But one of the main ways that I process experience and ideas is, is through writing. So when Robert posted this he posted something on grading for growth, which was a Stop, Start, Continue for the Ungrading Community, and one of the things he wanted people to start doing was to put more experiences about ungrading out there in longer formats, and in particular I think to get down into the details, right? Not kind of high level stuff about the need for it, but like, what are you doing day to day? And I thought, you know what? Like, that sounds like a thing that I could do. I’m not doing a bunch of research. I’m not in a position to do a bunch of research, but I do have practical everyday experiences with this and I like to write, so why not, right?

So I started every week making time to sit down and write a little reflection about what I’m doing. What was happening in my class that week related to grading and some of it was pretty raw and, I decided not to publish it kind of as we were going like real time, partially because I didn’t want my students to feel like I was, you know, you know, like airing our dirty course laundry or like, you know, talking about them on the internet in real time. So I kind of held it back. And, you know, also I was kind of afraid this could be a train wreck and I maybe I won’t want to talk about it on the internet. So, so I held it back at the time, but at week by week I wrote every Friday I sat down for an hour and wrote something about the experience. And then when the semester was over, I kind of publish it week by week as if it were happening in real time. And yeah, that’s, that’s how the blog got started. And I’ve now outrun that by several months. And so now I’m finding other things to write and, and new things to talk about. But that was the, initial idea for the blog.

Boz: So, where can our listeners find your blog post and we’ll make sure we have it in our show notes as well.

Emily: Yeah, absolutely. It’s on Substack. I guess if it’s on Substack, it’s technically a newsletter. I call it a blog. But it’s called Unmaking the Grade.

Emily: Yeah, that’s, I mean, it’s a big topic. We could be here for hours, not to say days. I mean, I think, you know, one of the, the first things that I could say is that I feel like moving to alternative grading and collaborative grading in particular, I think has really, prepared me well, to be honest, for the arrival of a I partially because I think you have to start thinking in terms of authentic assessments. If you choose to move to a system like the system I’m using, because I What I do in my classes is that I don’t give grades on any assignments throughout the semester. I give copious amounts of feedback, and I do have rubrics of a sort where I kind of indicate to students where I think they are along varying levels for the particular goals of that assignment, but I don’t put grades on anything.

And when you take away that, Great incentive, right? One thing that you and students quickly realize is like there’s no kind of control mechanism here for like how to force students to engage if they don’t want to engage, right? There’s no reward that I’m dangling or no kind of incentive reward that I’m withholding if they don’t complete their work or don’t engage, right? And so one of the things that you really have to do is think carefully about how am I going to connect this to students lives? How am I going to make it valuable or relevant to them? going to make it beneficial for their future learning? And then how am I going to like convey to them how valuable and beneficial it is?

Those are actually, I think, two different things. Like, how do you actually make it relevant or valuable and how do you convince them that it’s relevant and valuable? And so I, When I moved to collaborative grading, I had to do a lot of work on assignments to make sure that I was doing that. And so I feel like that has been really good preparation for the arrival of generative AI. Obviously, it’s You know, no matter how good your assignments are or how much you’re thinking of about student motivation, that’s not going to solve every problem that arises, but I think it’s a huge help. And so I mean, that’s the first thing I would say is that I think those alternative grading goes kind of hand in hand with that kind of assignment design. And I think it has well prepared generative AI for the first time.

Sharona: Go a little deeper into some specifics and give an example of an assignment that has changed like, do you give them essays? And if so, how are they relevant? Like, let’s really get down to the practical if we could for a moment.

Emily: Yeah I do still give essays. I’m in a writing class. It’s hardly avoidable. I don’t really love to call them essays though, because I think an essay is really mostly, at least when most people think of an essay, they think of a thing that they encounter in school and they think of a thing that they write to get a grade, right? We think about things like the five paragraph essay. Don’t love that, right? Have. been trying to move away from the idea of a five paragraph essay for a long time. And so, you know, I used to do really, and I think there’s nothing wrong with asking students to do really traditional essays.

But one of the things that I try to cultivate is really like taking samples of writing from the wild and bringing them into the class. So things that I actually encounter on Substack or in online magazines or on social media and bringing them in and using those samples and asking students to kind of launch off of those for their own work. So, a couple of things that I’ve done recently are I’m trying to bring the personal more into student assignments. So one of the things that I did when I taught a writing class that was themed around education was I asked students to think about an educational experience that they’ve had that was unique or that they thought it would be valuable to share with another audience. And to write about that experience, write about their personal story, the assignment was called Share Your Story.

So they wrote about their story, but then they also had to connect that to an issue of concern in education today. And to incorporate research on that issue, right, and make it relevant for somebody else, make an argument about it, etc. So, things like that, that I think ask students to kind of dive into their own experiences and ideas and opinions have been really valuable. I found it really valuable to ask them to think really locally. So you know, asking them to, for example, write a proposal about how to make education better, especially in a local context, right? Like maybe you’re writing to the professors at the University of Mississippi, or maybe you’re writing to future students of the University of Mississippi or something, right? So like thinking about local context and lines. I think just like having them, you know, delve into things that really interest them.

I had an assignment called, Love, Hate, Refute, where they had to pick something that they loved, something that they hated, or something that they wanted to refute. That could be any of those things. One of those three. And then to make a research based case, trying to convince somebody else why they should also love that thing, why they should hate that thing, or why they should agree with somebody’s opinion. Right? And we used, we used real examples. I mean, this is not like, you know I think students are used to picking, like, stock topics to write an essay on. Like, I’m going to write about gun control, or I’m going to write about the minimum wage, or, or whatever it is, right? And then just doing research on that. But I think encouraging them to come up with things that they’re actually really invested in, and really, audiences that are real to them, like their peers, or parents, or their professors, right? That’s the kind of thing that I like to do in my classes. And I, it doesn’t fully I think, eliminate, as I said, the problems with misuse of generative AI. But I, it does help, right, asking students to do things that have real value, I think, can be really beneficial.

Sharona: That leads me to the follow on question is, how will that assignment look now with generative AI, and especially the latest versions? Have you looked at what we’re finding that the new, for example, CHAT GPT 4o can do where you can literally tell it, answer this prompt using the fact that I love sushi and that I’m a first year freshman English student, so use appropriate level language, like, yeah, literally the generative AI can do that. Yes. So where, what, how do we go? What’s the next step?

Emily: I’m fully aware that AI can do virtually anything that I’m asking my students to do. It may not do it at as high a level as my students could but, but I do think like a lot of what, what I’m asking is something that AI could do. And when I first kind of got into this, I thought, I don’t want to ask my students to do anything that a computer can do.

Increasingly, I’m like, okay, actually a computer can do a lot of things. So, I backed off of that slightly, but I think, you know, there’s been this whole progression that we’ve gone through since chat GPT came on the scene. And I think the first thing that we thought was like, Oh, AI detection is going to be the answer, right? Like if I could just figure out if students wrote with AI through some kind of detection software, then that’s going to be the thing that will save us, right? But that, it became really clear really fast that we’re not going to get reliable AI detection. And then it was like assignment design. Okay, well, maybe I can design an assignment that, That will, like, I can design my way out of this, right? If I only just had the right prompts or did the right things then students couldn’t use AI to cheat in the class, right?

And I still think there’s a lot of merit to assignment design. I, you know, you can design good assignments or bad assignments or assignments that are easier or harder to misuse generative AI with and I still do believe in that. But even that, right is not gonna be a foolproof way eliminate misuse of AI. And so I think where I’ve arrived at, and probably where I kind of started at, I hope other people are like coming along to this, is that there is no way out of this problem that does not involve students deeply understanding, not just like, knowing intellectually, but like deeply understanding the purpose of what we’re doing and finding value in it, right? If students don’t know why they’re in the classroom, learning what we’re learning, they don’t know why I’m asking them to do what I’m asking them to do. They don’t see how that’s important to them. They weren’t going to learn to begin with, and they’re definitely not going to learn now that generative AI has an easy way out for that. And so I don’t have control over my students investment in learning, but I think we can do more than we think to create classes that make the purpose for learning clear and that, really help students understand and invest in that. And I think alternative grading can be a big, big part of how we make that happen in our classes, right? So, I mean, I guess where I’m at now is, I can’t control everything, so I’m relinquishing some amount of control, but I’m trying to really invest in in this notion of helping students understand purpose because I don’t think there’s any way out of it. Other than that, I think that’s the only way out.

Boz: Okay, and I absolutely love that point of view because, like Sharona said earlier, in mathematics, we’ve been dealing with this at a lesser level, but dealing with this for a while now. And there’s really two types of responses. Either we stick our head in the sand and go, okay, we’re going to lock everything down. We’re going to, you know, everything’s going to have to be done on pen and pencil in front of me. You know, I’m going to lock all technology away in the room, you know, just trying to ban everything. And unless you’re going to do every bit of work inside the class where you can watch them like a hawk and basically drive. All enjoyment of the class out. There is no way to stop this like there’s no way around it. But

Emily: yeah,

The only way to do it is say, fine, let’s make the class, let’s make the assignments worth, you know, worthwhile enough. And the students want to learn enough that it’s not that they can’t, but that they’re choosing not to cause we can’t make the, we can’t make the can’t happen. Like there’s just no.

Sharona: I’m actually going even more radical than that, to be honest. And I don’t know how this is going to play out in mathematics. It’s gonna be very interesting, but I have one foot in the academic world and one foot not in the academic world. And one of the big parts of my job not in the academic world is grant writing and AI has been a tremendous tool It’s like it’s flipped it up. I can write in five minutes what used to take me 45. And I’m an excellent writer. I’m an excellent grant writer. I get a lot of grants and AI has cut the low level initial pieces way down.

So I’m looking at this, this saying and, and Bosley knows this story. I listened to a podcast where they gave an example of a student who was in a business writing class and the professor’s like, Nope, Nope, absolutely not. Can’t do it. And then she goes to her job and the boss is like, you better figure this out and start using it right away. So I think we have to, and I don’t know that higher ed can do it as fast as is needed. I think we have to retool every single class to be done in the context of AI, because the number one thing we’re going to have to teach our students is how to use it everywhere, because the new AIs can even do math, they can do math, they can do coding, they actually, programmers have done this huge amount of uptake, because they’ll code something in one language, and they’ll dump it into chat GPT to have it translate it from say, Ruby to Python, the impact across every field is humongous. And so I’m wondering how quickly do we need to really rethink? Not only why they should learn it, but what really they should be learning. And I don’t know what conversations, is that happening in the writing classes?

Emily: Yeah. I mean, there’s, yes, there’s lots of conversations happening all over the place and I think people are sometimes. I’m surprised when I say this, for things like grant writing and like emails and office memos and marketing copy and things like that, I could not care less if my students use AI completely for that purpose, right? That’s not really what I’m invested in as a teacher, right? You know, what I really want my students to get out of my writing class is learning how to share their own ideas and opinions and experiences with other people effectively. And so I do think that AI can possibly play a role in that.

AI can’t do that for my students, but it can be, I can see ways in which it could be a useful tool. Now we have to keep in mind all of the kind of ethical concerns around ai, but I do agree, right? One of the things that we start off, that we do now, every time in the first. A couple of weeks of my writing class is we sit down and have a conversation about generative AI. Not just because we need to set some academic integrity ground rules, although we do need to do that, but because I think I wouldn’t be fully fulfilling my responsibility as a writing instructor if I was not honest with students about this writing tool that they are going to encounter in their lives from this point on.

We could have a much, much longer conversation about the ways in which AI might or might not be helpful to them. But one of the things that I also try to convey in those conversations is that students really need to become deeply reflective about their own learning processes so that they can know when AI is helping them and when it’s hindering them. Because, there are a lot of uses of AI I can imagine that are learning focused and the same student or like two different students could use, for example, chat GPT or another tool in the same way. And for one student, it could be a learning experience and for another student, it could be a shortcut.

So I had this last fall when I surveyed my students about their use of AI, I had one student say, when I asked them, how is AI helpful or not helpful to you in your learning process? I had one student say, Well I found it really helpful because it helped me correct my grammar, and I learned from that. And I had another student say It wasn’t really helpful to me because all I used it for was to correct my grammar. And so for one student, it wasn’t the tool, or even the way the tool was used that was the difference. It was the way that they were intellectually encountering it and processing it. For one student, they used it to correct grammar and they looked at it and they said, Oh, I see why I made that mistake and here’s what I will do differently going forward. For another student, they just used it to correct their grammar and they didn’t think about it and they moved on.

So I think gosh, it’s so hard, but I think every student is going to have to suddenly become very expert in understanding how their learning process works and how, how AI is affecting it. And so I think at this stage, I think becoming critical users of AI is even more important to me than trying to set out ground rules for like what, What is a good, or trying to even establish for myself, what is a good and acceptable and appropriate use of AI for learning and what it’s not,. I think we’re all still in a very exploratory stage with this. And students don’t always like to hear that. I don’t always like to say that I wish I had better answers, but.

Sharona: Well, I mean, the reality is, I just said it, I used it for grant writing, and a lot of people are like, oh yeah, I can see how it would be used for that. But I will tell you, grant reviewers are already sick of getting. the same stuff written by AI because it’s not like I’ve heard a lot of people say it’s the equivalent of of a C level of work and grant writers don’t fund C level grants. So for me, I have enough expertise and I’m reflective enough that I can take my idea, dump it into ChatGPT, take what it gets, and go from there and it’s cut out 40 minutes of my process. And it’s just so much better for me.

But all it’s doing is cutting out the grunt work portion of it. Whereas a student who doesn’t understand, they might read what ChatGPT puts out because they are not an expert in this. And they’ll go, Oh, that’s so good. So is it the responsibility of every instructor to help students do this critical use learning in every class?

Emily: Oh, it’s a good question. I mean, I want to say yes. I want to say yes. I think it, you know, it depends a lot on your own comfort level with AI and what you’re doing in the class. When you were talking about the chat GPT grant writing being C level work but how you’re reflective enough to understand what needs to be changed. It is a skill that I think students need to develop, and I think they don’t always know that they need to develop it. So I think we need to at least be kind of responsive to that and able to help them understand. I had a student who in this survey wrote that I found AI really useful because sometimes you have a good idea and you just don’t know how to put it into words.

And on the one hand, I really identify with that because I think the really good teachers that I’ve had in the past, sometimes you go to those teachers and you have this idea and it comes out in a jumble and you say it really badly. And then they say, Oh, and then they repeat it back to you in a way that sounds really smart. And you’re like, yeah, that’s what I meant to say. And so that is really valuable. But on the other hand, I’ve also had the experience of not quite knowing what I wanted to say and trying to write it down. And then seeing it in, in print or on words and paper and thinking, no, no, no, that’s not it. And then refining it again and then refining it again and doing it like four or five times. So I finally like get that idea that I want and it’s, I hadn’t thought it until I tried to write it. And, but once I tried to write it, I was able to refine my thought. And so if you ask AI to take a jumbled up idea and make it sound good, is that performing the service of a teacher who repeats something back to you and you’re like, yes, or is that performing like a shortcut that cuts off your thinking process?

I don’t know that I would be able to tell the difference for myself and so sometimes, I mean, certainly students who are just beginning their writing journey, I think would have a real difficulty with this. So I don’t think that’s a good answer to the question. But

Sharona: I think you, let me do the teacher thing and I’m gonna repeat back what you just said. I think you meant to say it’s both. Sometimes it helps, sometimes it hinders, and I think you are not doing yourself enough a service to realize that, to know that you would actually realize in the moment whether it helped you or hindered you. So then the question becomes, how do we walk our students through that process? Because one of the challenges we’re finding in alternative grading in general is getting students to reflect on the feedback because they don’t know how to be reflective at all. So every single course now going to become a really, we have this topic of math or history or whatever, but what we’re really teaching you to do is to become a learner. And so we’re going to just really help you reflect and learn. I don’t know, Boz, what do you think?

Boz: Well, I don’t know about every class, but to a certain level, yeah, I do think that is. Or should be part of those course descriptions and those course responsibilities on us, the instructors. So if I’m teaching, a three, 4, 000 level abstract algebra class, maybe not. Teaching in my high school or teaching a one or 2, 000 level gen ed math class? Yeah. I mean, that’s, I don’t think that’s unreasonable.

Emily: Yeah. I mean, so because I use collaborative grading where students are collaborating with me in assigning their final grade, that requires a lot of self assessment. Students have to evaluate their own work and that’s one of the hardest things to teach. Students don’t have to self assess a lot. And I don’t have, in my disciplinary background, a lot of experience with having learned to do self assessment or having taught self assessment myself, and I think it’s an essential skill, but I think we’re all trying to feel our way along in terms of how we help students develop and exercise that skill.

Boz: Well, and going back almost 20 . years now to John Hattie’s groundbreaking research that I think was mostly on K 12, but it has applications for higher ed, that self reflection is one of the things that had a high impact that we as instructors can control, or at least setting up the opportunity for students to do it. And now, Sharona, this has come up in a few of our interviews going back to, I think it was Dr. Theresa Gaines when they were talking about the feedback she got from her students was there was too much, of this metacognitive reflection because they’re not used to it. So yeah, when do we need to start really working and teaching that and forcing our students to do it? Because it is a high impact practice. Like I said, and we’ve known that for more than 20 years now, going back to that really huge meta analysis that Hattie did. Do you know which research I’m talking about?

Sharona: I’m not sure what the actual research is. I mean, there’s a million reflections on reflecting with John Hattie kind of stuff.

Boz: Yeah.

Sharona: So I’d have to find the actual article because there’s been so much research leveraging off of that. So what’s really coming to mind right now is that as a course coordinator, I have the ability to develop these types of lessons and put them into the course I coordinate, but I have to teach the teachers how to teach the students. And this is definitely something that I don’t yet feel extremely effective in. So not only teaching my students how to reflect and teaching my students how to utilize generative AI, but then teaching my teachers to teach my students, to reflect and use generative AI.

What was the throughput? It was the prompt engineering. So it’s almost like the most important thing I can teach my students to write now is how to do prompts for AI. And that has been the biggest difference for me in my use of it in grant writing. I’m constantly, but again, I’m already a good writer. I’m already a good writer. And so I can, and I’m also a good mathematician, which means I understand how to rephrase and program. These are sort of real language programming prompts. I know how to do this. My students wouldn’t have a prayer, which is why they get this sort of anodyne level of work.

So, how fast should we, because I do think there’s an opportunity here for alternative graders, especially because we can rework our course learning outcomes. We can sort of turn on a dime to say, I’m going to put a new outcome in that has to do with utilizing generative AI in a process. I could put that in my course for September and I wouldn’t have to change a lot, but so I’m kind of excited, but I’m kind of nervous. Because who am I to be teaching my students to do this? Well, I’m a lot better than they are. Is that good enough? I don’t know.

Emily: Yeah, I have a lot of mixed feelings. I, so I think that the comparison between large language models and like a calculator is sometimes it’s an imperfect comparison, right? But I do think it’s useful in that when I’m thinking about my students writing skills, right? So you were talking about, you can use chat GPT in this way because you already have enough skills to be able to know how to do that, or you know, your writing skills are already advanced enough that you know what is good and what is not, right? And so, I mean, one of the things that we have to really think about is when I ask students to use ChatGPT, am I, like, handing them the calculator before they know how to do basic math,? So, like, for some of my students, I think we’re still at the kind of basic writing stage, where ChatGPT will really kind of get in the way of the basic skill.

Like, you would never, I mean, I don’t know a lot about math. My impression is that you hand somebody a calculator after they already know how to do some, some basic stuff and have the building blocks and then you can teach them how to use the calculator, right, to help. And so, I feel like I’m still at the stage with a lot of students where we still need to work on being able to write without using this tool excessively. For other students, I feel like they’re at a place where they can use it in, in more advanced ways, or they’re more advanced in their writing so they can start using large language models a little differently because I teach first year students, I’m possibly a little skewed here.

So I’m not moving to like teaching prompt engineering anytime soon, especially since. A lot of my focus is just helping students understand the value of writing for expressing themselves, right? But I do think it’s something that we should be thinking seriously about because as you say, right, these, I don’t think that these things will ever replace the need to develop good, strong writing skills. I just fundamentally don’t think that anything will replace that. But I do think that they are tools that students may need to use in their careers or their personal lives after they leave us. And so we should be thinking seriously about how to help them use those tools. But as you say, like, I’m not an expert in generative AI, I barely know how to use it, but we’re still exploring.

Sharona: I think you said something really critical there. Is that you’re trying to get students to write so they can express themselves. And unfortunately, I think a lot of classes in higher ed are really about content delivery of existing facts that are outside of students.

Emily: Right. Yeah. So it’s knowledge making with my students. They’re not consuming knowledge. They’re making knowledge. Yeah. Yeah.

Sharona: Yeah. So, and that analogy of the calculator is actually really interesting because Boz, you know, we are constantly struggling with the numeracy skills of our students and they have access to a calculator and it’s not particularly helpful a lot of the time.

Boz: Well, that and the flip side of that is you were saying Emily, that you might not give the calculator to a student until they had some of the basic arithmetic skills. Well, you could also talk to people in our world that would tell you, you should never give them the calculator. And I think that’s the kind of mentality that I was talking about earlier that, just ain’t going to work now with these AI writing programs. It’s just, there’s no way to keep it out of hands. I mean, like I said, I literally have five devices within my reach of my hands that I could go and get a graphing calculator or, chat GTP or some other writing AI program. So yeah, that we have to learn. And this goes back to the same argument I’ve had with other math teachers my entire 20 years as an educator. We have to learn how to work with the technology instead of against it, because we are never going to win against it.

Emily: Absolutely.

Boz: What you were saying earlier about working with it by making the students understand the need to do what, and learn what you’re trying to teach is going to be the best and one of the few weapons that I think is going to be successful. Like that’s, we have to make the students understand why they shouldn’t or don’t want to use it.

Emily: Yeah. I think helping students understand and invest in purpose is kind of my go to. I think also, you know one of the things that alternative grading asks us to do really carefully, no matter what kind you’re using, is to identify from the outset what do you really want students to come away with, right? Whether that’s in terms of, you know, your standards, or your specifications, or your learning goals, right? What is it that you really want students to leave this class with? And then I think when you start with that, you can say, Okay, here’s where AI might be helpful for students attaining what I want them to attain, and here’s where AI might hinder them.

And as I said, it’s gonna depend to a large extent on the particular student in the particular context, but I do think it’s worth Investing a lot of time going into the fall, especially to think about to go back to what are our goals, right? What is it that we really want students to leave with? How can AI help or hinder in that and how can we help students invest in the purpose for learning those things,

Sharona: The reason, I think that’s exactly right. And the reason that I have said that I feel like this is as cataclysmic as the internet itself getting created is because there’s so many jobs where AI is already doing the job better than a human being who does it poorly. So medical diagnosis, routine medical diagnosis is being done better than AI, than poor doctors, but not as good as good ones, and certainly not as good on non routine stuff. There’s predictions that in the next 10 years, a huge number of lawyers are going to be put out of business because all those junior lawyers do a lot of these base grunt work of reading hundreds of pages of discovery and looking for patterns while the computer’s going to do that better than they can and faster and cheaper.

So a lot of supposedly extremely high level skills like being a lawyer or being a doctor, the low level of those professions is already being impacted. And so when we take a look at what are we supposed to learn, like you teach writing, you’re teaching self expression, I’m supposedly teaching a very specific set of technical skills. When I’m teaching, say pre calculus, like they’re supposed to learn to factor and they’re supposed to learn to do this and they’re supposed to learn to do that. But at the end of the day, they’re supposed to learn to problem solve because all of those. That factoring and all of that computer algebra manipulation, that is totally doable by a computer. And the, the leap that the generative AI models have done is, it used to be you couldn’t just cut and paste your entire problem into one of these, and it would do anything like a calculator can’t take words and Mathematica and needs programming, but I can literally take a screenshot now of a problem and it’ll get solved by generative AI.

So those manipulating algebra things. It’s useless to assess those unless I ban the technology. So what is it that they really are supposed to know from that skill? And how do I convince them if they’re like, well, I want to be an engineer. Okay. What does an engineer do? An engineer looks at a problem and a blank page. And says, what am I going to do about this? Yeah. I mean, I love more blank pages.

Emily: I love that going back to the root of what we’re doing. For you, problem solving. And, you know, I think a lot of writing teachers would see themselves as teaching discrete skills. Like I teach my students how to write strong thesis statements and good topic sentences and to integrate research. And all of those things are really, really important. It’s a huge part of what we do, but you know, fundamentally right in the same way that you are about problem solving, we are about, I don’t know. So I don’t know if self expression is where I want to land. Maybe it is. But like, that’s, that’s for me. That’s what I want my students to be able to do is like convey their thoughts and ideas effectively to other people.

And so yeah, like we could go back to the goals and standards, but yeah, even take one step back. Like, why are we here? Why do students even come to college? Right. It’s for, for this basic stuff. And it’s not just for their vocations, it’s for their lives and for themselves as whole human beings, right? Their ability to encounter the world with agency and critical thought and to create meaningful and satisfactory lives for themselves, right?

Boz: And I love, cause you said this earlier with your experience as a Contract grader or ungraded, whichever you want to call it, that it helped prepare you because you didn’t have the carrot and stick with grades as a motivator, that kind of practice helps you prepare for how do you get students to motivate now in this chat GTP world. I love the fact what you just said, going a step further back, because that is almost identical to the same question you have to ask yourself when you redesign for alternative grading. What is the purpose of this course? What are we supposed to do? And then from there get your learning targets and then from there do everything else. But yeah, there is a lot of similarities between how we design an alternative grading, how we redesign and our process there. And you’re right. I do think that gives us alternative graders a step up or a step forward, or, you know, a headstart, whatever you want to call it. When we’re looking at how do we teach in the, you know, post AI era now.

Emily: Yeah. Purpose is the word of the day.

Sharona: So what things are you going to be doing this summer to, and again as deep into the weeds and practical as we can get them, what are you thinking about for your courses for the fall?

Emily: Yeah, so I as usual I I make trouble for myself every semester by completely redesigning every course that I teach. I I’m thinking again about I’m going back to the question of purpose going back to some of my learning goals that I had for this last class and thinking about what did students struggle with? And in particular this time it was rhetorical analysis. And they struggled with that because I well, my suspicion is that I, I don’t think that they saw it as a, an activity with real world implications.

So I’m trying to think of, of how to help them understand how analysis works in the real world, not just how it works in the classroom, how it works in the real world. And I’m also going back to, on the blog I’ve talked about a progress tracking document that I made for students. That has been really successful because one of the main things that students struggle with in my classes with my grading system is they feel like they don’t know where they are. They get in the middle of the semester like, I don’t know, well, what’s my grade? Well, you don’t have a grade. And so I need a way for students to track progress. So I’m going to be refining that and also adding a little bit more structure to the self assessments that I asked students to do, because as I said, one of the things that students really struggle with is self evaluation and self assessment. And I struggle with teaching that.

Boz: So taking the personal views of especially this next coming election, as statistic educators, Sharona, tell me if you don’t agree with this, but we love elections. Like it provides us with so much just golden stuff either to use as assessments or things like that. Or also it’s so much fun to look at how bad statistics are around us, especially elections. And I’m, again, nothing personal about this election, but just elections in general. There’s so much just gold out there about how statistics are misused that we love this time of year as stat educators.

Boz: Yeah, no, we really are coming up on time. I want to thank you Emily for joining us. This is really our second AI episode already. I believe this is going to be and ongoing discussion. So I’d love to have you back again to talk some more about this, but like you were saying, Sharona, this is probably the single biggest change and will be for this generation, what the internet was for mine and your generation. Cause yeah, you were in college when internet really came. I was ending high school, so we were really on the, cusp of how the Internet affected education and what it did. And this is going to be that moment for our students of this generation. So thank you, and we I hope to have you back. Sharona, did you have any last minute?

Sharona: I actually just want to kick it over to Emily. Did you have any last thoughts that you wanted to make sure we heard before we sign off?

Emily: I just want to say, everybody’s really fatigued in the post COVID landscape and the election’s coming up, students are fatigued, we’re all fatigued, right? But I do think that there is, there’s hope on the horizon. I do think you know, alternative grading is offering us a lot and generative AI, as difficult as it seems right now I do think, you know, there are there are ways that I think it encourages us to go back to fundamentals in our teaching. And my sincere hope for the coming years is that we take this opportunity to reexamine what we’re doing at a fundamental level. And take this opportunity to like transform our classrooms. So, that’s my, I don’t know if it’s going to happen, but that’s what my optimistic heart hopes for.

Boz: And on that note, I think that’s a great note to sign off with. So thank you for listening and we’ll see you next week.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode’s page on our website. www. thegradingpod. com. Or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the contact us form on our website. The Grading Podcast is created and produced by. By Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.

Boz: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State System or by the Los Angeles Unified School District.