The Grading Podcast

93 – AI PD with Jennifer Montes

===

Jennifer: Well, I wanted to mention that originally I wanted to do my project on EGI, and I didn’t switch to artificial intelligence until like October-ish. And that was because I was super interested in alternative grading, but artificial intelligence was just like this alien in the room that I felt like I wanna talk about and I wanted to learn more about. And it was really weird how quickly I saw connections with both of ’em, like with artificial intelligence and alternative grading.

Boz: Welcome to the grading podcast, where we’ll take a critical lens to the methods of assessing students’, learning from traditional grading to alternative methods of grading. We’ll look at how grades impact our classrooms and our students’ success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharon Krinsky, a math instructor at Cal State Los Angeles, faculty coach and instructional designer. Whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week, you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Boz: Hello and welcome back to the podcast. I’m Robert Bosley, one of your two co-hosts, and with me as always, Sharona Krinsky. How you doing today, Sharona?

Sharona: Well, it’s been quite a week. Unfortunately I have decided to share that I lost my mom on Sunday. It was an end of a beautiful life, a glorious life. And as a lot of people who know me know, she is probably my main mentor in my career. So if people will indulge me for a moment, I wanted to say something about her. My mom was Eunice Krinsky. She was a educator in mathematics. She was in the mathematics department at California State University Dominguez Hills for about 25 years. Primary interest was in reforming the way we teach mathematics.

So I think I said on the podcast last week that I learned things at my mama’s knee.

Boz: Yeah.

Sharona: And I very much did from the time I was 12 and she started at Dominguez Hills. And during her first five years at Dominguez Hills, she was completing her PhD at UCLA in higher education with a cognate math. She was a big advocate at the time for what was called back then collaborative learning, which we now fall under the umbrella of active learning. And it’s really interesting to me that one of the things that I never heard from her at all was anything about grading. Nothing at all. So unfortunately, the last few years of her life, she was no longer really engaged with the education world. So I wasn’t able to talk to her about grading as much as I would’ve liked. But she really, she was an inspiration to me. And as we’re starting to let our different communities know about her, it’s amazing the number of people that are coming out of the wood work and talking about what a role model she was professionally. So it’s been a rough week but I’m really glad to get to share a little bit about her. So thank you for indulging me.

Boz: Well, I’m happy to see you here. I actually wasn’t sure if you were gonna be able to make this because of everything and I know how much of an influence she was on you, but also in education. I mean, I know I’ve worked with several people that can trace their lineage back to her, so my deepest sympathies, but I am glad to see you.

Sharona: Thank you.

Boz: Glad that you’re here.

Sharona: Thank you. And, you know, even in the midst of sorrow, this podcast makes me happy. So I am happy to welcome. We actually have a guest today who just met me. So this is the first thing she’s learning about me. So sorry about that. But in the studio with us today, we have Jennifer Martinez Montes. Jennifer is a ninth grade English teacher. She is in her second year of teaching, and she teaches at Santee Education Complex, which again, listeners of the podcast will know well from Bosley’s 20 years at Santee. Jennifer is a graduate of Boston University in English. After that she participated in City Year and Teach for America, received her teaching credential from Loyola Marymount University and is currently getting a master’s degree from LMU. So, welcome Jennifer.

Jennifer: Thank you so much, happy to be here.

Boz: When we have a new guest on we always like to ask them just how did you get involved with alternative grading? Like how, how did you get into this crazy world?

Jennifer: Yes. So I think as I started teaching, grading was one of the hardest parts, like everything else, lesson planning, and it was easy for me to digest, but grading was the part that was really hard and something that came out of that, or a reason why I felt that way is because when I was in school in ninth grade, I had a math teacher, who it was her first year teaching and. It was very hard for me to do well in that course. And I remember thinking, you know, I’m going to tutoring, I’m asking her for help. I’m doing this and I’m still not successful. And it was very frustrating, I think for her too. Like I was trying and just wasn’t able to get the my grade to where I wanted it to.

So I think from that experience, it just kind of taught me that grades are not really real, grades are not, the system that we have is not accurate because how is it that I’m putting in all this work and my teacher sees that and it’s still not coming out to where we want it to be. So I think from that experience with that teacher is kind of what, once I became a teacher, I was like, okay, grading is something that really matters to me that I really want to focus on, that there’s not a lot of information about, to be completely honest. So all of those things.

Boz: You know, that’s interesting. Do you remember what level of math that math teacher was?

Jennifer: So she taught geometry and it was her first time teaching Algebra one. So yeah,

Boz: I know my, when I had Algebra one, you know, something years ago I had a math teacher that it was their first year teaching and had all the same struggles that any first year teacher had, but yet still made such an impression on me. I, I still take a lot from my, my own practice from him and I remember when he found out that I was becoming a teacher, ’cause you know, I’ve actually stayed in touch with him for a while. He looked at me and laughed. He is like, I hope you have a class full of students that were just like you. ’cause I absolutely drove him nuts. But like I said, it still, even though he was a first year teacher and he did have all these struggles, just how much of my practice came from him. So shout out to all those new teachers that make more of an impression than you realize.

Sharona: And your story is interesting because we’ve spoken with a number of people who’ve been studying this for a long time, and there’s so many stories that we all have about a grade, particularly in a middle school year or an early high school year that we all remember. And it’s interesting to me like yours was a, a bad experience in math. Mine was a bad experience in writing. So mine was a sixth grade teacher who gave me a D on a story. And what I remember her saying, although I doubt she said it this way, was that you don’t know how to write a story. And to this day, I feel like I’m a very good writer now. I don’t even attempt to write fiction. I’m an avid reader, but the idea of writing a story, a fictional story, makes me break out in hives.

So it’s interesting you had a math experience and you’re an English teacher. I had an English experience and I’m a math teacher. Yeah. So my question is, so you’re, so you’re struggling with grading. Can you share a little bit about some details? What were the struggles that you were having as you were getting started? Like as you were thinking about grading? Can you gimme a little more detail?

Jennifer: I mean, just the sheer amount of grading was insane. Like 130 something kids, and they all have different writing levels and it’s not like I can just like at the bottom and see, okay, they got this, right. It’s like you have to look at all of it and give feedback on all of it. I thought that was, I, I really have had no clue how people did it. I was like, that is really a lot of work on top of lesson planning and everything else. So giving feedback I think was the biggest challenge for me.

Boz: I don’t know how you English teachers do it. I mean, I honestly, ’cause especially if you’ve got, you know, 11th and 12th graders where you can have, still have up to 35 kids in a class five classes. I would be writing like big circles with ABCs and D’s and just throwing the papers up and seeing where they land. I, I know I’ve, I’ve talked to Mariah and, and Joe quite a bit especially Joe on the podcast, but the amount of grading and the time it takes, especially if you’re trying to grade everything, which is, I know one of the things that Joe likes to emphasize is he doesn’t grade everything. In fact, he’ll even use students that’s part of his students’ grades or how they do peer editing and peer review and peer feedback that I, I don’t understand how you guys do that much work.

Sharona: So I want to distinguish some things though. ’cause you just said feedback when I asked about grading and we actually, in a lot of the work we do, we separate those things. Right? So you were doing all this feedback work. Were your students using it? Were they using it or was it you because you spend so much time on it.

Jennifer: And see that’s the other thing too, like building on the feedback that you give them is another, it, it was just a mess for me all first year. I went to a lot of, so Bosley gave EGI trainings at Santee and I went to all of them cause I was really trying to understand just all parts of grading. And eventually now I’m at, my grade book is at a place where I like where it’s at. I use a rubric for everything. And I’m pretty clear on the expectations, which has helped a lot. But I still think that what you brought up is my biggest area of growth. Like me giving them feedback and them using them, that feedback to get better. That’s also a process on its own, I would argue.

Boz: Absolutely, and I know when I first started off and when I wasn’t doing alternative grading, I would spend hours on a math, you know, on math feedback. I would still spend hours and my students would look at the top of the paper where I had the number and that’s it. And I, I’ve really come to believe that it’s because with traditional grading, the point of assessment is that score. ’cause that’s what makes your grade. It’s not about the learning. Until I started using alternative grading and started allowing students to use that feedback to show better learning, better understanding, and not punishing them for, you know, not knowing how to solve a system of equations in my artificial timeline. If they can prove to me by the end they know it, then why get punished for not knowing it, you know, a month and a half ago.

Jennifer: I think I would’ve just. Like, like done it and like started off like, ’cause I remember I struggled a lot my first year with grading and I just didn’t have, like, I wanted to just throw it off in the air and like, let me start over. Let me do EGI. But I got scared a lot of people and myself, it was like, well, are you sure you want to do this? Like, that’s really hard and it could impact the students. But I actually spoke with Zeccola about this and he was telling me like, you know, your instinct was right because when you do EGI, you’re not gonna do it right the first time. It’s gonna take you work to get to. And that’s where I’m at right now. I’m still working at making it good. But he told me, you know, just, you should have just trusted your instinct and done it. And I wish if I could go back to a year ago from now, like, yeah, just do it. It’s not gonna be perfect. It’s not gonna be good, but. The traditional grading system isn’t perfect and good either, so you might as well try something new. So that’s the advice I would give myself that Zeccola gave me a year ago.

Sharona: I love that advice. Go Joe, our, our unofficial third co-host. Yeah. Okay, so I wanna transition a little bit ’cause I know you’re working on something. I’m gonna have Bosley ask you the question, but for me, you know, in mathematics we’ve been dealing with the impact of computers for a while. Like when I was in school, the impact was calculators and especially graphing calculators. And we’ve been arguing about this for now 40 years because we’re still arguing about calculators and graphing calculators and things like that. But it was kind of contained in a way because you had to learn how to use the tool. So even though we’ve had very powerful tools, computer-wise in mathematics for a while, most students can’t use them. So we’ve been okay. But then something happened in the last couple years. So Boz, do you wanna take it from here?

Boz: Yeah. So I, I know Jennifer, that you are right now studying to get your master’s and you’re doing some research. What are you doing your research on?

Jennifer: I’m doing it on artificial intelligence in an English classroom.

Boz: So we’ve talked, you know, even though this is the grading podcast, because one of our, our founding pillars of alternative grading is the use of feedback and the use of authentic assessment of our students. AI has become a huge topic. We, we’ve done several episodes on this, on AI in general and AI and other specific areas. So AI is very much becoming this huge transformation. Agent and Sharona. I know you and I have both talked that we really believe this is going to be the single biggest change agent in education since the invention of the internet, which you and I are old enough to remember education before internet. We remember well.

Sharona: And I remember it better than you ’cause I got all the way through college without the internet. He only got through high school without the internet.

Boz: Well, but e even even in college, I still remember breaking out the encyclopedias and things like that too.

Sharona: Yeah, this is true. This is true. It didn’t really take off, I mean, honestly, smartphones are probably as much or more of an impact on internet use than just the internet itself. But we didn’t have that conversation another day.

Boz: So what specifically are you looking at before we start talking about what you’ve seen?

Jennifer: Well, I wanted to mention that originally I wanted to do my project on EGI, and I didn’t switch to artificial intelligence until like October-ish. And that was because I was super interested in alternative grading. But artificial intelligence was just like this alien in the room that I felt like I wanna talk about and I wanted to learn more about. And it was really weird how quickly I saw connections with both of ’em, like with artificial intelligence and alternative grading. Do you want me to start speaking on that or did, what was your question? Im sorry.

Boz: Let, let’s, let’s talk a little bit about specifically what it is that you’re researching and then yeah, we’ll get into the connections. ’cause that’s really why we, you know, wanted to have you come on. And again, we’ve talked about AI and AI use a lot on this podcast, and that is because of. The impact and the ties it has with the alternative grading. So, but, but specifically like, give us a little bit of details of what it is that you’re looking at and how you’re looking at it.

Jennifer: Yeah, so for my question, I want it to answer, does giving teachers guidance on AI literacy and AI as a pedagogical tool allow them to integrate it into their classroom, specifically like an English classroom? So what that looked like was me conducting surveys and giving PDs and just one-on-one, like talking to teachers about how they can use AI in their classrooms specifically with English. And I think a lot of it was, yeah, I’ll just leave it there for now.

Boz: And then, and part of your PD is, ’cause I, I was still at Santee when you were doing some of this and. How was, you know, how have you seen teachers’ reaction to ai? Like what has been their feelings about the use of it?

Jennifer: Well, I think it really depends on how you present it to them. And that’s something that I was super intentional about during my PDs because I think AI has a bad rep. I mean, we’re interested in AI probably ’cause we’re seeking it out and we’re learning about it. But for the average person, at least when I started back in last year they’re not really seeking it, it’s just they’re just getting information about it through the media or through the news. And that information is pretty negative. So I think presenting AI as something that could be helpful to us, a tool like you both are saying something that can help revolutionize education. I think that was a lot more powerful than just telling teachers, like, there’s this new technology coming, use it. Good luck. So I think that difference was really important for this project.

Boz: So have your PDs been more focused on how teachers can use AI for themselves whether it’s lesson planning or grading or whatever other aspect of their pedagogy or has it been on how teachers can let students use it in the classroom?

Jennifer: Yeah, so there’s two types of AI in education, student facing, teacher facing, and I just focused on teacher facing because when we talk about student facing ai, there’s a lot of. Issues and privacy and a lot of things that are not, were not there yet, in my opinion. Or I personally, I didn’t, A better way to say that would be, I didn’t feel comfortable putting this tool in front of kids and just me by myself. But what I did feel comfortable with was telling teachers, my colleagues Hey, there’s this tool that you can use in your teaching practice specifically for lesson planning. That is what I focused on, even though it can be used for feedback and grading and all the other things.

Boz: So I know Sharona, that’s one of the things that I. You know, even though we have had several episodes about AI use, it has been more kind of focused around either how you protect against it for students using it, or you know, how you can actually incorporate and encourage students to use it. I know on one episode a while back you actually told a story about a I think it was a business student or a business management student, and a business writing class, and just the disconnect between the expectations of the real world and academia. But, and we’ve talked a little bit about how we, the educator could also use this as a tool to help us, but not extensively. I mean, we haven’t talked a lot about it.

Sharona: We’ve mentioned. I mean the, the main use I’m using right now, because one of the things that we hear a lot about, particularly in alt grading is when we have these repeated attempts, it can be very difficult to come up with good prompts for those assessments. Whether it’s a math applications, whether it’s good prompts in an English course, or, you know, it can be very hard. And I’m finding a tremendous amount of time savings by having AI do those prompt generations for me. Right. Because it comes up with scenarios that just wouldn’t occur to my own brain, right? Because my brain is limited to the information I have in my brain. And so it’s a very useful tool that way. So we have mentioned that, and, and actually I’m giving a talk this Friday specifically on using ai. Without grading and assessment, but, so, yeah. And so I’m interested to, to hear more about what are we telling teachers to do. I myself have serious ethical concerns about the feedback side of it. Like using it for feedback really does not make me comfortable.

Boz: And, and I get that. And I, I think, I mean, I understand that and I, I don’t have as much hesitation on it. I, I’m optimistic, cautious about it. But I know even though we’ve not talked as much on the podcast about it, you and I have both done trainings. In fact, our last time we spoke at the California Math Conference on authentic assessment and the use of ai, we showed a room, a packed room, a process of how we could take a very typical question. I. You know, that is easily googleable, that students could easily you know, do without knowing any of the math at all and run it through some prompts and come up with a much, much richer question, a much richer problem. So, I’m, I’m curious Jennifer, what have, well, what have you personally used AI for when it comes to your practice and how has it worked in your classroom?

Jennifer: Yeah, so I use it a lot to lesson plan. I actually should have mentioned this, but I also teach RSP. So a lot of that, the way I use it is text leveling or differentiation or even like just telling the ai like, Hey, I have this long four page passage. Can you number all the sentences on there, like even simple things like that I think are really helpful. ’cause that’s all things that I would’ve done in the past, like gone through and numbered all those. And also just asking it to break down questions like, give me DOK one, give me DOK four questions and then I can give those two kids. That’s the way I use it. Something else that I’ve been experimenting with lately has been, I mean, there’s just so much. I think if you’re asking me that question, it’s a very big question. I’ve also been using it like there’s a tool called Otter AI where you can, have you heard of it? Yeah, it’s great. Yes. Yeah. I use that in like my regular life now. I also use, there’s a new one called where it generates text to image. And that has been really helpful because if you have really complicated text and you’re able to generate images for that text, like that is a great scaffold for kids, especially kids that really struggle with like understanding the text or they need that visual. That’s been really helpful. So that’s been another way, just some ways that I use it in my classroom personally.

Boz: So what is that one called that does text to image?

Jennifer: There’s a lot of them. The most famous one is probably Dolly, but for teachers specifically, it’s Canva Dream Lab that I’ve been using. And yeah.

Boz: So you, you had mentioned a couple things, and even though this is, you know, I, I would imagine 99.9% of our listeners are educators. We do have educators from around the world in different parts of the country. You said a couple of things that Sharona and I, you know, absolutely know what you meant, but I want you to explain it a little bit more. ’cause I, I wanna go somewhere with one of ’em. So the first one was RSP and the second one was DOK. Can you just real quickly, what, when you said you also teach RSP, what exactly is that?

Jennifer: Yeah, so I co-teach for a couple of periods in my day. So that means we have inclusion classes with special education students and the, yeah, that’s basically what it means.

Sharona: So, so what does RSP stand for? Do you know?

Jennifer: RSP? I would have to, no, I’m sorry, I don’t, off the top of my head. I don’t wanna say the wrong thing. RSP oh, resource specialist teacher. Yeah. So that would be like my co-teacher. She’s the RSP, just, that’s the term that I’ve heard people talk about. Like I teach RSP, so sorry if I’m using that wrong, but Yeah, that’s her.

Boz: No, but it, it, an RSP class describes a makeup of a class where because at Santee we do have full inclusion, which means all of our students that in the past might have been, you know, isolated into a class all by themselves that have individual education plans or IEPs. So these students might have, you know, some sort of learning disability or, or processing that they just need some extra supports with, whether it’s extra time, whether it’s you know, having, having audio or visual aids.

But an RSP class is a class that, and with full inclusion, instead of separating those students because they shouldn’t be separated, they’re in with everyone else. And if you have a class that has enough students that have these unique educational needs, you, you get a co-teacher that specializes in that, that also co-teaches with you.

But I’m bringing it up because that will bring and a little bit of dynamic about Santi because of where we are and where we teach in a typical ninth grade class. And, and call me out if I’m overstating this at all, Jennifer, but in a typical freshman class you might have up to 28 students. ’cause that is contract is average 28 cap 30.

But in that 30. You might have seven or eight RSP students, you might have another additional four or 5, 6, 7, 10 students that English is not their first language. And some of them might have very little English language acquisition so far. And you might even have a few on the gifted side. So in a single classroom of 30, you might have four of these massively different groups.

And then on top of that, every single student comes in with their own strengths and weaknesses. So am I exaggerating with how diverse a freshman English class at Santee might be?

Jennifer: Oh my gosh, not at all. Not at all. And I didn’t even speak about how I use AI for like language acquisition. So yes. Absolutely spot on.

Thank you

Boz: so. One of the things that any educator in the last 20 years has known that an expectation on teachers is that we differentiate our instruction for our students. Think about that for a second. There’s two adults in your room. There’s you and your co-teacher. We’ve got 30 kids with 30 huge range of needs that we’re supposed to differentiate for.

How helpful can AI be with doing, and you’ve already said, taking a single writing prompt and saying, you know what? Change this to. A DOK one or a DLK four, or I guess we should say what DOK is also DK is, well, we could just

Sharona: refer people to the episode 80. Yes. Which was the role of DOK and aligning assessments.

Boz: But yeah, understanding what that, the appropriate level for a student, but taking that single writing prompt that you come up with, that you’ve designed our assessment. How long does it take you to run that through an AI program and change the DOK levels?

Jennifer: Oh my God, it’s so fast, so easy. But then that’s when I come in as like the expert, and I think that’s.

My message here, like a lot of people, I mean, I just read an article by the New York Times on Monday that was like, teachers are using AI to create assignments. Why can’t students use it? And the difference there is that like, we are experts and like, I can look at these questions that AI generates and I can discern what’s good and what’s not.

So to answer your question, yes, I like right away it gives me information and then I have to still sort it, you know? So it’s not a, a magic wand. You still have to put in some work, is what I’m trying to say.

Boz: Yeah. But, but my point is, you put in the work at the beginning, you put the work into create it, but then to do the differentiation, instead of having to redo that for every single different level, every major differentiation that you need to do for that assessment, whether it’s a writing prompt or it’s a simple reading comprehension questions or a math problem in my case, or a biology question. Being able to rewrite the those for different audiences and different DOK levels, literally it takes seconds. And yeah, you might need to, you know, go back and do a little bit of, of ing around and editing some of those, but you do the work to create it.

You don’t have to do that same, ’cause it would take me, before the use of AI and stuff, when I would do my especially my algebra and algebra two modeling questions. I might have two of them on a test where I’d come up with them, but it would take me just as long for every single different differentiated question.

So if I might have two or three different versions of my test with same concept, slightly differentiated questions because of the needs of my students, if it took me 20 minutes to come up with those questions, it took me 20 minutes or more for each of the differentiated levels. Now, same 20 minutes to begin that, that hasn’t changed, but now instead of another 40 minutes to come up with two other levels, it takes four seconds.

Jennifer: Yeah.

Sharona: Well, and I would actually add a, a benefit though that I think that we underestimate in this because not only is it faster, but if you do your prompt intentionally, it can help you avoid some of your own biases of your own interests. Right, because I know when I’m coming up with modeling questions, my brain goes to things that I am personally familiar with or I, I draw inspiration from my life, but there might be other topics that I can tell ai, Hey, make sure to include something from sports or something from the world of entertainment or things that I just wouldn’t naturally, things that are culturally relevant or career engaged.

So I like that part of it too, because I’m limited by my own experience as well. So I think we have an opportunity to get better quality versions,

Boz: well, I don’t know about better quality, but more diverse

Sharona: of quality, or not quality, but more diverse. I guess that’s what I mean by better quality, meaning relating to more people and not limited by my own personal biases and expectations and interests.

Boz: And, and, and that’s another thing, and, and Jennifer, you can contest to this, something that. Kind of, I saw starting to come about. I imagine you being as young of a teacher as you are, probably something that you saw in your actual back, or not background, but education program. Not only are we supposed to differentiate, we’re also supposed to be culturally responsible and, and relevant.

That, like I said, I, I am old enough. I remember where that kind of first started, and I’m like, how are you culturally relevant in a math class? Two plus two is two, regardless of culture. Except two. Plus two is not

Sharona: two, but okay.

Boz: Yes, I know two plus two is four. I, I I was ignorant then I didn’t realize.

But that’s another thing that ai, like you were saying, Sharon can be extremely useful with. Have you ever tried to do that, Jennifer? Where you, you. Take a, a prompt and like, okay, you know, rewrite this with a sports theme or rewrite this with a,

Jennifer: I have, but yeah, I have, but I, I don’t think I’ve done a great job of like tracking how it impacts students or like the engagement with that.

It’s been more about just my own use and like teachers using it. But yeah, that’s absolutely a way, fantastic way to use it. Yeah.

Boz: So, and another thing I was, I was thinking of when you were talking about the text to image, which is why I was asking about it. I’m, I’m going back to my experience as a, as a high school math teacher and knowing one of the ways that I would differentiate what I was doing, some of my assessments is giving visual aids.

Like if I’m doing a modeling question, I might. At the beginning of it, give some visual aids to everyone as a scaffold, but then not do it all the time. But with those students that I know needed, it could try to, you know, do a visual image or a visual aid image. In that assessment, it would take me hours to try to, in fact, I got to the point where I would find the image and then make the math problem instead of the other way around.

But I can just imagine now being able to take some of those modeling questions, throw it into a text to image, and getting that, that visual aid that, like I said, used to take me hours. In fact, I would change my questions ’cause I couldn’t find, can you imagine doing that sharona that. With some of those modeling questions, even with your, the courses that you coordinate, the pre-calc.

Sharona: Yeah. But I guess I wanted to ask Jennifer, so what has been the reception by teachers of the use of this? And you mentioned the fact that you’re an expert. Can you expand a little bit about that, about what you think is needed for effective AI use professionally?

Jennifer: Well I think overall my sense was that people are willing to learn as long as you present it in a way that’s digestible for them is the biggest thing.

But I think to answer your second question it’s just all about AI literacy and how to use ai. So you were talking about how you fed it, different prompts and you had to ask it different things to get what your response was. So like. That prompt engineering, that’s something that you have to teach people how to do and teach people how to interact with these systems.

Like one teacher was like, oh, you have to talk to it like it’s a person. And I was like, well, yeah, it’s, it’s chatting with you like that. So that part of it, and then you also mentioned using it as like a springboard for your thoughts, and that’s another part of like AI literacy is being able to use it as a thought partner and knowing that it’s only as good as you are.

It’s only like a reflection of you and your potential in you, you, as the human is still in the driver’s seat, but you can use this as a tool to really help you improve your teaching practice. Does that answer your question?

Sharona: Yeah, yeah, I think so. I just, you know, again, I wanted to see what the reception has been.

We have a lot of faculty on who listen the podcast in higher ed and. There’s really been a mixed bag in terms of adoption, right? Where there’s been a tremendous amount of focus, first on student use and a feeling that this is a violation of academic conduct, a vi, you know, and, and also what, what we’re finding is students in unsupervised situations are relying on it heavily, but they’re scaffolding and leading into proctored environments, and then the students are shocked that they don’t know what they’re doing.

So because they think they know what they’re doing when they’re using ai and it’s giving them these very easy to understand responses to whatever they ask of it, and then they get into a proctored environment and they’re asked a similar prompt and they don’t know what to do. So in higher ed, the adoption, there’s been a lot of resistance even on the part of the faculty for themselves.

But a lot more focus on the students. And so I don’t know if you guys just have a more controlled environment or, or what that teacher response has been to what you’re doing.

Jennifer: I think, again, just the way I presented it and I really had to lean into the fact that I’m an English teacher, I still care about English and literature and all of like that.

I’m not saying AI will replace us. It’s really, I think you guys spoke about it. You specifically spoke about it Sharona and your when you guys interviewed Dr. Talbert a couple episodes ago, how it’s AI is creating a divide and or it’s gonna create a divide. And that’s absolutely true. And I think that was also a really big part of why I wanted to do this project, who has access to this technology and understanding how to use it versus who doesn’t.

And I think. Again, the way you present it to people has a really big deal because prior to my PDs there were some people trying to get the conversation around AI going in our department and it wasn’t great. It was really not great. And I think it’s because it was student centered first. And how can you expect people and teachers to feel comfortable with students using it when they themselves don’t understand?

It was really the point of my project. And I think once teachers understand it, it demi demystifies AI for us. And ultimately that, that is I think my goal, my call to action. I mean, AI is not just for people that have the mean, like it’s for all of us. It was created on us. Why are we not benefiting from it is my point of view, I would say.

And I really love that interview with Dr. Talbert ’cause he talked about, well now all of education is changing and the things that we teach, we have to really sit with ourselves and consider that. And I think that to me, that’s exciting. I love that because the system right now is not working for a lot of us.

So there’s this thing coming in that can help us shift it, but we have to be we don’t have to just be passive and just let it happen. We can be active. Participators is my whole point. So yeah. There you go.

Boz: Yeah. Be, be active and help shape how exactly we go forward from here. I, well,

Sharona: can I just follow up on one thing you said?

Yeah, go ahead. So, I really love what you said that it’s really important for an instructor to understand it and an instructor to spend time looking at how it could be impacting our jobs. Because we, one of the biggest challenges I face in my department is bridging the gap between the instructors.

Focusing so heavily on the students don’t know the quote unquote basics that we can’t even try to teach them the bigger stuff. And I’m looking at the instructors and saying, okay, but let’s, let’s turn that mirror on us first. Do you understand the basics of teaching? Of teaching in this environment of what students really need, the skills they really need?

I know you think they need this skill. I don’t necessarily agree with you, so I love what you just said about the importance of understanding AI ourselves first. So thank you for that.

Boz: So I did have a question for you, Jennifer, because I was there when this kind of got started, but has the English department actually come up with a AI policy or statement?

Jennifer: No, we have not. We have not, not for students and not for ourselves either. Yeah, no, we haven’t.

Boz: So I, I, I know that actually me, me, you and Mariah actually talked some about this, but I think that is as we’re starting to move forward and as especially heavily writing heavy courses like, like English and, and history and some of the creative electives, the importance of understanding ai, like you were saying, but also it’s here folks.

It’s a, it’s here, it’s a tool. You can’t just stick your head in the sand about it. So let’s start having those conversations. Let’s start coming up with those AI policy. Statements or, or, you know, policies for your department, for your school. But I, I know it’s a conversation, it did not go well because we were still early.

I, I don’t think you had done most of, or any of your, your PDs in your trainings yet. So I’m hoping that that might be something that if the, if your trainings have been going so well, that you guys can revisit and, and look at again. ’cause like you said, it’s here, we can be passive or we can mold it. And I, I hope you guys do take that and as a department, really look at how to go forward with it.

Jennifer: Yeah, and I mean, L-A-U-S-D just gave us guidance on generative ai like. I think last month they just gave us a training on it. Prior to that, students got a training on it, I think in like November. But that was our first one. So I think, I think a lot of it is people have to realize like it’s, nobody’s gonna come and save us.

Nobody’s gonna tell us like, oh, these, this is ai, this is how you should use it. And in fact, if they do that, then we should be critical of that. You know, but I think, yeah, it really just comes down to us teachers. We we’re the ones that are on the floor, on the ground every day working. We know what system, what impact these systems can have.

So it, it really is up to us, I think, is to, to develop those policies.

Boz: So are you still in, like, are you still doing your, your research or, or have you done with your data collection?

Jennifer: I’m done with my data collection. Yeah.

Boz: I’m done. So o overall, how do you think that, how do you think that went? Like, what was your.

Results.

Jennifer: I mean, I found it to be successful. Like I had a good time. I had people from like outside of the English department talking to me about ai. I, I think a lot of people were very hesitant and like, we, we real, I mean, even myself, this is why I started learning about this because I was like, what is this?

What is it gonna do? What’s gonna happen? And I think just having someone on campus that’s, that’s like, oh, like let me teach you about this was just really helpful to our community. I would, I would argue you’re always gonna have people that are not into new technology like that. That’s just.

Yeah. But I would hope that those people would ’cause I showed data, you know, like AI and machine learning jobs are growing and they’re gonna keep growing

Boz: rapidly.

Jennifer: And I think that like a lot, most people that go into education, ’cause they want to help students, so it does require, there’s like a dissonance there that people have to deal with, you know?

But overall I, I felt very confident and very positive. I felt like it was very well received by admin and staff and yeah, faculty. So I felt good about it. I will say, you know, I’m also a student, if I can just talk about this real quick, like, I’m a grad student right now and the way that AI is being implemented in higher education is weird.

Like, it’s just weird. I have some professors that are like, don’t use it at all. Some professors that literally they’re writing it into their assignments, like, use chat GBT for this. And I’m like, that’s not. Like, that’s not how we should be using, that’s not a tool. That’s, that’s not how AI is going to help people learn.

Just like type in a prompt, gimme a response. Like there’s so much potential to ai and when we just do things like that, it it hurts me. Like I, doing those assignments hurt me. But yeah, just wanted to speak on that really quickly,

Boz: but that, that’s funny. You have that, that kind of range, that, that does not shock me one single bit.

And I’m sure it didn’t shock you at all, Sharona.

Sharona: No, not at all. Well, and you know what, this week’s too, I was asked a question, I think I mentioned this on last week’s episode by my dean recently where he said, does AI require us to completely rethink what we teach? And my answer is absolutely. I mean, we’ve needed to do this for a while.

I would say for 20 or 30 years, especially in math, we have been fighting to get faculty to stop focusing on some of the minute skills and be putting everything in context and let students absorb the basic skills through the bigger context. I mean, when I think back to elementary school and we’re teaching students to spell, we don’t make them stop reading because they haven’t learned to spell yet.

Like it’s okay to do both of these things simultaneously, and in fact, they’re likely to improve their spelling by their exposure to reading. Whereas in math, we seem to think, well, we can’t expose them to really beautiful math that’s already worked out while we’re teaching them how to do math. And I think AI is gonna finally force that out into the open because, because at least in mathematics.

From last year to this year, AI has gone from being a second grader to being a master’s degree student in mathematics in one year.

Boz: So I I’ve got one more question for you, Jennifer, about your research and, and I, I don’t know if you’ve like, completed your writeup and things, but is the highlights or the learning modules or the details of the PDs that you presented part of that?

Jennifer: Yes. Yeah, the PDs and stuff, they are part of the research. Yeah.

Boz: So, ’cause I, I would, I would really love to see that out. Just, you know, here, here are some of the PDs that we did. And be able to give that to a model for other people that, you know, might be interested in it and might be wanting to, you know, have that same kind of mentality that you have of.

Either we can passively let it happen to us, or we can take charge and, and take the reins on where this is going so everyone isn’t having to recreate a wheel that you’ve already created. So I, I, I hope that that’s part of it. I would love, once you are, once you are finished with it and you’re ready for it to be out, I’d love to be able to, to post that here, a link to it and kind of get that PD model out to people, because I agree with you that’s, that’s one of the biggest things that needs to happen.

We have to be trained, like people need to know how to use this and not just what they’ve, you know, the, the tidbits that they hear on social media or whatever, but actually be trained. ’cause it can be. A very powerful tool. It can be a very huge time saving tool. And one of the things that we talk a lot about with alternative grading, because there are aspects of it that are more time consuming and, and a little bit difficult to do compared to traditional grading, that you’ve gotta be able to use your tools efficiently and effectively.

And this is another one of those tools that’s the same thing. We, the more efficient and effective we can use it, the more time we can spend with our students, more time we can spend on, you know, creating our lessons, the more time we can spend on what’s actually gonna help our kids.

Jennifer: Yeah. Yeah. I agree. And I think a lot of the things that I was reading where alternative grading and AI overlaps is just creating AI resistant assessments, which is like.

Again, just changing the whole classroom and everything. But I actually had a question for Sharon. Is that okay? Can I ask her something? Absolutely. I was, because you mentioned that you weren’t comfortable with feedback being used as with ai. So I was wondering if you could speak more on that and then I can share, I guess, my vision for that.

Sharona: Right. So this is probably based on my experiences, limited experiences, receiving feedback. So for me, I think feedback is one of the most important relationships between a student and a teacher. So when I receive a piece of student work and I’m going through it for feedback, it it, it gets in my brain where the student is at really.

Right, whether it’s a mathematical feedback or it’s about their writing. So my concern specifically on assessments designed to demonstrate achievement. So not like I am a big fan of automated feedback on like homework systems and things where students can get immediate and effective feedback, but when I’m trying to see have they learned something and give them feedback, I question outsourcing that to an AI because it does two things.

I feel like it damages my relationship with the students. ’cause now I don’t know it anymore. In fact, I do outsource a lot of my feedback. He’s looking at me in this, in on the screen here. And it actually can do. It can reduce my relationship with my students. So that’s my biggest concern about the use of feedback, particularly in alternative grading, is if we’re trying to fi build a feedback loop, particularly on those mastery assessments or the ones where they’re, the, the student trying to show, they’ve actually learned it, not in the mistake generation process.

I worry about not having that personal relationship with my students. So that’s really where I’m like, again, I see a place for automated feedback and for instant feedback. And even that the AI might be better at it in a lot of cases, but I worry about breaking that human relationship.

Jennifer: Okay. Yeah, I, I totally hear that.

And I think something else that was at the center of my work was like not replacing human interactions, but actually like elevating human interactions. And I think the way I envision it, is getting AI to give feedback on tech, like very technical things like citations and did you introduce the quote and did you cite the author and the title?

Did you use that correctly? Having AI use that, like do that and then I, as the expert can go through and look at that and then I can just have a conference with a student and just talk to them about their essay and about their work. And that conversation can go even beyond just like the mechanics, but into the style of their writing and like their word choice.

And it to me, like that potential could be so much. Like that use of AI could be so beneficial for the student and for the teacher. And it could really build a deeper relationship with the teacher and the student. ’cause I don’t, I mean, I’ve only been a teacher for two years, but I’ve never had the time to do conferences in my room where I just sit down with a student.

Like that’s usually something that happens at tutoring and stuff. So yeah, I just wanted to mention that. ’cause that’s a way that I’ve envisioned and I’ve seen some research leaning towards using AI in an English classroom.

Boz: See, and I’m wondering if that might not be, you know, one of the differences between a, a class like English and a class like math.

We don’t need AI to look for those technical errors because usually there is a, a right answer or a range of right answers. I. So I, and we have answer keys for that. We are using technology for that. Sharona, it’s, it’s check it and it’s the, the generating of the answers. But English doesn’t have, you know, for the technical stuff, a single right answer.

So I’m wondering if this, the use of AI and feedback, ’cause I love what you were saying, Jennifer, about doing the technical stuff, but then the, the more humanistic part of it, Sharon, that kind of, the feedback you were talking about, either done in a conference or still done by, you know, by us makes a lot of sense.

And I, I’m wondering, like I said, that just might be a, a, a difference in our subjects of how you might be able to use AI really effectively to do that. We don’t use AI in our feedback. ’cause that same technical part is just. A number or something that an answer key is already generating that we don’t have to do.

Sharona: Right. ’cause I would, I guess, you know, in my experience, especially with our statistics class, literally we give them an answer key and then we give feedback to guide them on how their stuff is different than the answer key. Because that’s where they have trouble is they, they can look at the answer key, they can look at their own answer and go, I don’t understand why I got it wrong.

Because our statistics does use a lot of words, but they’re conceptual problems. They’re not, I mean, obviously if there’s a technical issue they can go, oh, this is what it was, I got the wrong denominator. Like that’s fine and we might point that out. But they get, you’re right, they get the answer key. So that’s where I was struggling is I’m looking at my type of feedback while at the same time understanding that our homework systems have been automated.

With those answers and even the feedback from much longer than AI has existed. Oh yeah. Like, you know, our textbook publishers have built entire systems around, you know, this is wrong. Consider looking at this step of your math that you gave me. Right. So we have a lot of that. So when I hear ai, I think of a layer on top of that, not, not that piece.

’cause we’ve had that.

Boz: Yeah. But that’s just interesting, your, your initial reaction to that Sharona. And then again, based on your personal experiences, your personal biases, and then when we hear you explain it, Jennifer, it’s like, oh, that’s not the kind of feedback we were thinking about. That’s absolutely a great way to use.

So I, I’m wondering how many other teachers, you know, are out there and they hear use of AI and they have, you know, whatever their images that are tainted their. You know their opinions and you’re getting all this, this just initial pushback and resistance. Whereas whether it’s through a conversation, whether it’s through a training, like what you’ve been doing, Jennifer, getting them to see, oh wait, there’s a lot of different ways that this can be done and there’s a lot of ways this can be really useful.

Jennifer: Yeah. And that’s why I was bringing up that New York Times article, ’cause it really bothered me. They were like, like, oh, it was almost like this narrative that teachers were lazy for using ai. And it’s like, no, we are so hardworking. Actually, we deserve this technology that will give us time for our students time to give back to what we love in the classroom.

Boz: So yeah, time. Time to do the things that are actually going to help our students.

Sharona: Yes. Well, and time to actually achieve the crazy. Requirements that have been placed on y’all. ’cause like I don’t have those differentiation requirements at my level. We just don’t, because college is an option. Students choose to go to college as opposed to K 12 being mandatory and, and, and a right for all students.

So I do worry a little bit that instead of being like, okay, great, we’re gonna let teachers finally do what we’ve asked them to do, they’re gonna go, oh, well now that you don’t have to spend time doing this thing that we never gave you time before, we’re gonna layer something else on it.

Boz: Something else

Sharona: that’s another thing. But anyways,

Boz: that’s a problem for another day. But

Sharona: yeah. So we’re coming up on time, Jennifer. Is there anything else you wanted to make sure to mention or talk about that we haven’t asked you about?

Jennifer: Yes, I should probably should have brought it up earlier, but I think you can cut it and put it earlier maybe, but something with AI literacy that’s really important.

I think we talked a lot about the benefits, but there’s also a lot of risks with AI that I think that was very important for me for people to know. So for example, we’re talking about giving feedback on essays. One of the biggest things with AI is I, I kind of ranked it for us and I just picked the one that I thought was the most important to talk about.

And for me it was just data, like data in my opinion, supersedes bias and misinformation. And I really wanted teachers to understand that, like, why is data so important? So that was a big part of my project as well, like the risk of ai. And just to bring it back to like our, what the conversation that we’re just having right now about feedback and essays.

I wanted teachers to understand why putting a student’s narrative that they wrote into AI and saying like, oh, give me feedback on this. Like, that’s not really a great thing to do because of data privacy issues. Like that is a student’s life story that they wrote. And it was, I mean, the student didn’t really give consent for an AI to, to use, you know but now you as the teacher have put it into the system and now the system has that data and that’s like, you should know what that means.

And what that means is whoever’s creating that model can have access to this student’s data and they can, you know, track that data and they can create a profile. And it’s, it’s a lot. And it, that part I think, kind of goes over people’s heads sometimes because. I, I don’t know, I think that was definitely a weak part in my research, like explaining that.

Even now as I’m explaining it, I’m like, does that make sense? Do you want, can I restate it because I don’t know if that made sense.

Boz: Oh no, it, it, it made perfect.

Sharona: It makes sense. Yeah. But then we’re perfect, but then we’re AI literate, so.

Boz: Yeah. But you, you know, Sharona, it might be actually fun ’cause we’ve done several AI issue episodes.

It might be fun to have you back, Jennifer and have someone like Drew Lewis on with her and, ’cause I know that’s one of his kids.

Sharona: Or we have a colleague who I was just talking to, I won’t mention her name ’cause I don’t have permission, but bring her on because there’s a lot of ethical issues that are go far beyond anything that I was aware of.

So I was aware that if you put something into a large language model, a generative model, there’s a potential for them to then use that data without consent. I’m aware of sort of the intellectual property issues. I was not aware of the issues about the global North versus the global South and some of the resource extraction issues that are environmental, but also the labor issues that AI is creating in the global south.

Because I guess, although I have not yet linked and sourced the, the sites for these, that a lot of these language models were trained in developing nations with very low wages for the people doing the work. And they have a huge potential to replace that workforce with automation. So there’s a lot of ethical issues around this stuff.

Boz: Yeah, I I think that would be a fun if, if you’d be willing, Jennifer. I I think that would be a really fun episode to, to do. ’cause we have, I mean, let’s be honest, Sharona our episodes. At least tend to be lean towards the pro ai. So yeah. Having one, let’s, let’s talk about some of those other ethical concerns and some of the other things that are being brought up by, you know, people that are warning and concerned about the use of ai.

Jennifer: Yeah, and I, again, that was just a, it, it’s not just like, yes, I am optimistic and I lean towards the positive, but I do think the risks are important. And I think that once people know the whole picture, that is when people can really be empowered to use these tools. If a lot of books about AI in education, they just talk about the benefits.

So I, you kind of have to go out of your way to find these. Books that talk about the whole picture. So yeah, that’s definitely an important part. That was hard to communicate to people. I think that is really, it was difficult. Yeah. But yeah. Thank you.

Boz: All right, well, we are coming up on time, so I want to thank you, Jennifer.

Thank you all for joining us, and we’ll see you next week.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode’s page on our website, http://www.TheGradingPod.com. Or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the Contact us form on our website.

The Grading podcast is created and produced by Robert Bosley and Sharon Krinsky. The full transcript of this episode is available on our website.

Boz: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State System or by the Los Angeles Unified School District.