The Grading Podcast

Colin: Engage on Twitter. I’ll always call it Twitter. Perhaps in a little bit of mourning for what was, you know, you share an article about, oh, hey, here, new reporting standards coming in into BC. And there are certain elements of crowds on social media where the obvious, the, the only reaction that is reasonable to something like the government saying, Hey, maybe we should do this differently is it’s obviously a communist agenda. And so we must abandon it and burn it all down. I’m like, okay, really, really now.

Boz: Welcome to the grading podcast. Where we’ll take a critical lens to the methods of assessing students learning. From traditional grading to alternative methods of grading. We’ll look at how grades impact our classrooms and our students success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist, and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach, and instructional designer. Whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week you will get the practical detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Boz: Hello and welcome back to the podcast. I’m Robert Bosley, one of your two co hosts, and with me as always, Sharona Krinsky. How are you doing today, Sharona?

Sharona: I’m tired. In my new job, I’m getting an opportunity to kind of go back to the beginning of working with a group of faculty on thinking more intentionally about their grading practices. And I’ve started, I think I mentioned this in a couple of areas, one is on retakes and how you do that. And another one is beginning to try to calibrate your grading practices. So even though we’re using traditional grading systems at the moment, my sort of baby step has been getting teams of instructors to work together. And discuss exam grading and do grade norming, which we wrote about on the Grading for Growth blog a while back. I did that for four hours yesterday across three different courses after having proctored a retake for seven hours. So I’m a little tired.

Boz: And I’ve got to say though, I am still, like, I don’t know if you realize how impressive it is that you were able to pull this off with this many different teams. I mean, I remember the first time doing this with my high school colleagues, you know, and, and we really did, we had done a book study around learning by doing one of the PLC books from the Defours, we had gone to a solution tree PLC conference. Like we did a lot of pre work before we were ready. to kind of do what you just basically took this whole group of instructors. What, like what? 14, 15 different instructors and said, cannonball, here we come.

Sharona: Well, thank you for the compliment, but I’m going to turn it back on you because I’ve had 6 years to do this in statistics and my number one trainer has been you. So you were starting this brand new when you started it, but you were also new to the process. Weren’t you at the time? Yeah.

Boz: At the time that you and I were doing, no, I had been doing it for,

Sharona: No, when you started it with your school.

Boz: Oh yeah, when I started with Santee yeah, we were, it was one of the early times it was even a little bit before Common Core came in. Yeah, it was the first time I’d ever tried to do it. And honestly, it really wasn’t something that I believed in until I started doing some of the readings and started to understand some of the, you know, this idea of equity across different teachers and not wanting the teacher roulette to be the reason people got grades. So that was actually a big shift in my early pedagogy mindset.

Sharona: Right. So my point is, is that. I’m not starting this with this team’s, where my personal skills are at the level where your personal skills were when you started. I have six years of actually doing this with a team, largely coached by you on how to do it right. So I personally, as the facilitator and coming in at a higher skill level, I also have the benefit of knowing a lot of these people and having built trust with many of them over quite a few years. So that’s why it’s working, but it is very, very exhausting. How are you doing, Boz? I remember the last couple of weeks, I’ve never asked you how you’re doing.

Boz: No, I’m, I’m doing good. I’ve got a couple of new irons in the fire that I probably will be talking about on the next couple of episodes, but yeah, interesting time right now, so.

Sharona: Well, and one of those interesting things is a few, oh gosh, months ago, maybe on the pod, you made a request. You asked to speak to some Canadian educators. I think this was coming off of Matt Townsley’s or Thomas Guskey’s interview.

Boz: It was both because it came up in both. When we were talking with Dr. Thomas Guskey, you know, a little while back and he was talking about areas where they looked for people, you know, for districts and schools that were doing grade reporting in a good way. And they said they had to go all the way to Canada to find them. And then, yeah, it came up again. And in our interview with Dr Matt Townsley. So, you know, we put out a plea that Saying, Hey, we would love to hear from some of our neighbors to the North. And we finally got one.

Sharona: Exactly. So joining us in the virtual recording studio today, we have Colin Madland. Colin is a Canadian educator. He is a PhD candidate in educational technology in the department of curriculum and instruction at the university of Victoria. And his research has. It’s gone into a very interesting direction that I’m excited to talk about, which is technology integrated assessment practices in higher ed, and how faculty can ensure that their methods of assessment are are actually measuring student learning in ways that are equitable, valid, and reliable. Kind of sounds like what we’re trying to accomplish. He also, I also resemble, I feel for him because he has a day job, as the manager of online learning and instructional technology at Trinity Western University in Langley, British Columbia. So welcome Colin Madland.

Colin: Thank you very much. It’s great to be here.

Boz: So one of the things, Colin, that we always ask our new guests is just how did you get involved in this crazy thing that we call alternative grading?

And that was it, or so I thought. But shortly thereafter, I had a and I was teaching in a small private school. I had a parent who was also a teacher at the school mention, you know, my daughter hates phys ed, hates with a passion. Tears in the morning on days where, where there’s phys ed. And, I, I guess it was kind of an intuitive sense of putting those two things together. That, you know what, there’s, there’s far too much riding on these grades. And people’s experiences with physical activity are, are set for a lifetime. Based, in large part, by their experiences in high school phys ed. Particularly junior high girls.

It’s a really hard time to be a human being when you’re 12, 13, 14 years old and expected to, you know, all the expectations that are in many ways phys ed classroom. So I, without talking to anybody or, really, realizing what I was doing, I’m like, okay, I’ve got this curriculum that has these outcomes. I’m going to talk with my students after class. And you know, they’re each going to give me a rating out of five on the outcome for that day. And, you know, without realizing it, I stumbled myself into outcomes based assessment. And the difference was astounding. A follow up conversation with this parent whose junior high daughter was in tears every morning before phys ed classes was like, Oh my goodness, my daughter has changed so much. She looks forward to phys phys ed. You know, it’s, it’s become this thing that’s no longer a stress, but it’s okay. I know that I have input on my grade and I know that it’s, it’s not this rating thing of, you know, this student is better than that student.

So that was quite formative in, in my mind. Fast forward a couple years, I’m in a different, different teaching context but still high school phys ed. What I, what I started noticing is the athletes, in phys ed classes, you know, the people who are really comfortable with, with running around and being skillful with a basketball or soccer ball or whatever the complaints started coming from them and their parents. And I’m like, well, let’s talk about this. Shall we? And I can remember sitting in the principal’s office with an angry parent and her really strong athlete son, eh, probably grade nine or 10, you know, the same age. Why isn’t my kid getting straight A’s? You know, he’s on the basketball team. He’s on the soccer team. He’s on the, he’s on all the teams. Why isn’t he getting straight A’s? And I’m just like, well, Let’s look at the outcomes here. You know, the outcomes in the provincial curriculum at that time where things like teamwork and leadership and participating positively and things like that, I’m like, yeah, you’re seeing your son can shoot a basketball real well. But sometimes teamwork is, is a challenge for him, and that’s why he’s not getting those A pluses.

Sharona: So there’s a couple of things I want to respond to real quick because I, was one of those students who wanted to avoid phys ed if I at all could not because I was incompetent. I just didn’t enjoy it. And beginning in junior high I was taking classes at the high school and the high school schedule and the junior high schedule don’t match. So they always placed anyone who was coming back from the high school at the time when they were coming back into phys ed. for the time to allow, because they were like, well, phys ed is the class that it doesn’t matter if you’re half late too, right? So I loved the fact that I only got half of the phys ed class and no one cared. And I got my A because they’re like, Oh, well, but she’s coming back from the high school. We have to accommodate this.

And then when I went to high school, I was in marching band and marching band counts in the state of California as phys ed in the fall semester. So I only had to take. A semester of real phys ed and half the time we were sitting around. So what you said about junior high women and phys ed really resonated. But I also think it, it speaks to me that, I mean, Bosley and I all the time get, well, I can understand how that works in this other discipline, but not in this one. And I would suspect that phys ed would be, I don’t know, Boz from your perspective, are they more likely to adopt an outcomes based assessment? I don’t even know they, because they’re like, yeah, percentages don’t make sense? Or are they resistant? What’s, what’s your experience?

Boz: Honestly, I have not successfully worked with a phys ed teacher or PE department. I have not been able to sway any anyone from the phys ed side to alternative grading yet. So I don’t know.

Sharona: So those were my two points. You had something you wanted to respond to though, Boz.

Boz: Yeah. First, I just wanted to say as a math teacher that gets this all the time, I’m just thrilled. Other subjects get. Oh, I just hate this subject. But I really find that interesting that the student that you were referring to had such a shift in her mindset around phys ed, because one of the things that we talk a lot about here is, you know, the first thing that changed that I noticed with my students was the conversations we were having going from, you know, how do I get more points? How do I get? You know, this C up to a B or can I get extra credit to, what do I need to show you about descriptive statistics that I’m not showing you? Like it, it shifted from chasing points to actual talking about math. And that’s, you know, what sold me, but I’m really interested. Like, what, what do you think was that shift for that, that young female student of, you know, that the grading. allowed to happen that shifted her so dramatically from I hate this class to I am enjoying this class. Cause that’s a, that’s a big shift. And you’re right for people of that age, and especially females of that age is a, can be a really awkward time that phys ed. I can see how that can be. absolutely, you know, multiplies that awkwardness.

Colin: Yeah. I mean, it’s, it’s a time of, of people’s lives where their bodies are just absolutely rebelling against everything that they want to be. So I, I think what changed and kind of alluded to this when I shared the story is that because students had input and it was a conversation, regardless of how brief it was, like these were literally five, 10 seconds maximum at the end of class where students would come by and talk to me. And they, they give me a number and I record it. But just that, that connection that said, Hey, I trust you and I believe that the effort that you put in today, that is reflected in this number that you’re giving me. And I mean, it was a number I didn’t average the numbers. It, you know, it was more of a competency thing. The symbol I used was a number. But just that little brief connection with, Hey, you’re a real person. And I believe the effort that you put in today because Phys Ed is one of those one of those areas where effort really is kind of important.

Boz: Absolutely.

Colin: And it really is remarkably relative to the student. Those students who are natural athletes and love running around, they don’t have to put in any effort to be able to, I remember so many Phys Ed classes, your grade in basketball is based on how many foul shots out of 10 that you can make. Eight out of 10? A – super job. And you can see that already right there, there’s just massive inequity. And for, for a student who’s coming in, who struggles with being active and it doesn’t naturally enjoy that. Their physical fitness is at a point where, you know what? Walking up a hill for some kids is a really high level of relative effort. Where the kids who, the kid who’s got these high objective numbers of physical fitness, running up a hill, running up the same hill five or six times is nothing. And so, being able to listen to a student and say, you know what, walking up this hill today, that was probably a 90 percent effort for you. And that’s way more than these other kids who literally do that for an afternoon of fun. So I think it’s a constellation of things that come into play when, when you trust students.

Boz: Yeah. That’s just, that seems like such a small adjustment to what you were doing and had such a massive impact. You know on your students and this student in particular, but I just love how, like you said, it was a five, 10 second conversation. So it’s not like you’re drastically changing your whole pedagogy or your class management and routines. But that conversation and giving that the students that. You know, self efficacy and that ability to, you know, grade themselves kind of a, almost an ungrading type of mentality had such a massive impact.

And it’s, I think sometimes we forget as educators that, you know, the smallest change that we can make. Can sometimes have just massive impacts on our students. And speaking of small changes or just small things, like you were talking about in the early part of your origin story, going to that principal and saying, you know, Hey, I am not this precise to know the difference between an 88 and an 87. And that’s one of the things that Sharona, you and I talk about a lot is, you know, this false sense of objectivity with percentage grading and accuracy. I mean, I’ve talked to teachers before that, you know, they’re like, oh, yeah at the end of the semester if kids got a, you know, an 88, I might give them, you know, those extra point and a half to get them above. But yeah, if it’s an 87, no, that’s Like, do you really think throughout the entire year, you have been that precise that you really can tell the difference between that 87 and that 88? That idea of precision is comical to me. And it was sad and funny at the same time that you realize that early on and like, Hey, I’m just not that precise. And your principal’s like, no, that’s the way you got to do it.

Sharona: So I want to take the conversation from here into two different directions. So I want to consider like a middle third of this conversation and then the last third. So I’d like to do the last third on what you’re doing now, which is the technology integrated assessment. I think that’s a key piece, but before that, I want to get to the reason you responded to us, which is Canada and what Canada is doing. And I think you have a very interesting perspective of what your experience has been beyond your personal experience with alternative grading in Canada. So can you share a little bit about that?

Colin: Yeah, for sure. I’ll, I’ll preface this with a bit of a disclaimer. I’ve been out of K 12 teaching for 15 years, I guess. So I’m. I’m an interested, a little bit peripheral observer. And you know, that said my, all three of my kids have been through the public and private and all sorts of schools in, in BC and Alberta in, in the last, you know, , 20 some years. So certainly some, some direct experience as a parent. And final thing is that, you know, Canada’s big, you know, that I’ll state the obvious. And, you know, we’ve got our, our provinces and territories and each province and territory manages education independently. So we, we don’t have a federal department, department of education. Those, those are at the provincial level. So in BC. It’s starting well, the curricular reforms came before these assessment reforms, which I think is good because the curricular reforms focused on outcomes.

Sharona: And this is at the provincial level?

Colin: Provincially. Yes.

Sharona: Okay.

Sharona: You got it for K to nine and then you jump off the deep end and fall off a cliff. You’re like, Oh my goodness.

Colin: And arguably when it matters the most, you’re going back to this, this false sense of precision.

Boz: So, so quick clarifying question. So when you say that, you know, for K through nine, that we’re grading on these proficiency scales. So, like, when you get a report card or whatever, I’m not even sure if you guys use that term in Canada, but when schools send it, information back to parents on their students grades at set increments or at the end of a semester or term. Are you getting like, you know, for your math class, here’s the, you know, however many learning outcomes and getting Emerging proficient on all of these different outcomes. Are you getting some overall kind of wrap up of yes, their math eight grade is this?

Colin: Yeah, it’s it’s a little bit of both. And I’ll send you this link. But I’m looking at the government reporting policy. On the web right now, I’m sure we’ll be able to get this into the show notes later. So there are required communications, at least four learning updates. So what they call them, not report cards anymore for learning out updates throughout the year to our informal and to our written you know, informal updates could be phone calls, emails, you know, those types of informal

Sharona: So they still have quarter and semester report cards. They’ve just renamed them.

Colin: Yeah.

Sharona: Okay.

Boz: But I like the rename.

Sharona: Yeah, no, I definitely do but.

Colin: Yeah, so for K 9 the learning updates communication of each area of learning so that’s the content area, in relation to the learning standards, using the proficiency scale. So three keys there, you know, okay, in math, we’ve got these learning standards. Here’s the proficiency scale. So that’s being communicated. And then included in that are descriptive, descriptive feedback. You know, here’s, here’s where your student can grow and, you know, opportunities to, to develop further. There is information about attendance. And then there’s some student self reflection in grade 10 to 12, only difference, you know, everything is the same, communication of each learning area, currently studied in relation to the learning standards, but using percentages and letter grades. Everything else is the same. There’s still descriptive feedback, there’s still information about attendance, and there’s still some student generated self reflection.

Boz: So, as a parent, you don’t get like what we get here, you know, a report card that my, my daughter’s in six classes. So I have six letter grades. You actually get information about what The student is doing in the class and what areas, what are those learning outcomes that they’re excelling in and which ones that they could use some more support with? Yeah, exactly. Man, as a parent, because Sharona, I know both of your, your sons are now in college. I’ve got one in high school and one that’s still in elementary. The elementary ones here do something similar, like the elementary schools, at least here in California and at a lot of places around the country that I’m aware of do, you know, will do that. They, they break it down and it’s

Sharona: Standards-based report cards, outcomes based proficiency scales.

Boz: But yeah, but my daughter, my youngest daughter’s, you know, elementary report card is two pages long. I mean, with all the different learning targets and how they’re doing, but once they leave elementary, I get half a sheet of paper with science, earth science and forensic AP stats English class, and just the letter grade. And it would be so nice as a parent to continue to get that kind of information that that’s i, I, I’m jealous and, and I see why, like I said, Dr. Guskey was talking about how Canada was, had done some reform and, and was coming out with these kind of extended communications to parents about actual learning and not just, here’s some arbitrary overall final summary of the, of the students work in the class as ABCDF.

Yeah.

Colin: So it predictably you know, a five year pilot and rollout and implementation should seem like, okay, this is pretty broad engagement with a wide variety of stakeholders in the issue, you know, teachers, districts, admins parents, students, the, the whole thing like many things these days in scare quotes, if I may you know, now this is a personal anecdote so take it for, for what it is, but it’s probably relatively representative, you know, as I would engage on Twitter, I’ll always call it Twitter, perhaps in a little bit of morning for what was, you know, it, it, it share an article about, Oh, Hey, here, new reporting standards coming in into BC. And there are certain elements of crowds on social media where, you know, the obvious, the, the, the only reaction that is reasonable to something like the government saying, Hey, maybe we should do this differently is it’s obviously a communist agenda. And so we must abandon it and burn it all down.

And I’m like, okay. Really? Really now? So there was some of that element there, and it wasn’t helped by media reporting on the issue and perhaps a little bit of a self own by the government as they’re rolling this out. One of those self owns and I’ll throw out a hat tip to, to the Tom Shipper podcast. , Tom is much more involved in, in In closer touch with, with the BC education system. But he, he released a podcast. Now I’ll send you the link talking about some of these things. And he’s like, you know, one of the things that they really shot themselves in the foot with was the fact that they said on this proficiency scale, not every student should be aiming for extending.

And so, oh, well, the, you know, you, there are multiple ways to interpret that and possibly a less charitable way is you obviously don’t want our students to excel. You’re obviously undermining our student success. And personally, I don’t think that that was the message the government intended to share. But as you know, people pick up on these little sound bites and run with them. And there may be some, some Dunning Kruger effects playing along in there where so yeah, there was there was some reaction. In the last couple of years since this has been implemented, I haven’t heard anything. Like I was just looking before, you know, before I signed in here, I did a quick Google search for the, you know, BC student reporting policy. I think the first three pages of hits are almost exclusively information for parents and students from individual school districts saying, Hey, this is, this is what we’re doing. This is why we’re doing it. And I think that’s probably a good indication that yeah, people are largely getting used to it. There was one, one hit of, of an article from the Globe and Mail which you might say is somewhat parallel ish to, you know, New York Times or, you know, kind of mainstream media. You know, it’s a bit of a, not really a hit piece, but definitely, you know, old people, parents in BC are leery of the new reporting order and, you know talking about some, some half truths and things like that. So the, I guess the lack of, of heat is possibly telling.

Boz: I’m, I’m still just blown away that the fact that they did this as a pilot for five years, like any, anything. I don’t know if that’s typical or atypical, but anything down here in the States, especially with education, if it’s reformed, like it’s got to happen tomorrow, like we, which is part of the, you know, Achilles heel of our education system and any kind of reform that we try to do. It’s, Oh, no training, no nothing. We just got to do it. And then wonder why it doesn’t work.

Sharona: So I’m sure we could go on for several hours about Canada, and that is what brought you to us. But I am particularly interested to hear more about what you’re doing now, because I know you’re a PhD candidate and you have this philosophy of assessment aligning to learning. So what are you trying to accomplish? Because grade tracking is a big thing in our world, but as a mathematician, ed tech is targeting us and in not good ways and ways that are counter to the way I want my students to learn. So I’m really curious to see what are you doing? What is integrated technology into assessment and how does that match your philosophy?

Colin: Yeah. So by way of a little bit of background, I’m working through my PhD is. It’s a, I’m doing a three paper dissertation and, you know, so three, three journal articles on a unified theme but still standalone articles. And I would be incredibly remiss if I didn’t point out my, my committee who have I’ve been exceptionally helpful in shaping and forming what I’m learning. So, you know I’ve got a committee of Chris DeLuca, who’s at Queen’s University in Ontario and is deep in K 12 assessment and research. And Okon Bola at University of Alberta. Who brings a much more psychometric perspective and, you know, the large scale assessment and things like that. And then my supervisor, Dr. Valerie Irvin at UVic who has been extremely influential and supportive in looking at assessment and particular technology integrated assessment and how that comes to play. So nothing I’m about to say gets anywhere with, without their input.

S A M R.

Boz: No, I don’t think I’ve heard of that.

Colin: Modification, that’s it. Substitution, augmentation, modification, and redefinition. You know, those are four different possible impacts of technology on, on education. You know, sometimes you just substitute one mode for another. You know, instead of a handwritten test, you move it to online. It’s keep all the same questions, that type of thing, you know, augmentation, modification and redefinition. So they’re kind of grades of impact on assessment. And what we realized is not only did nobody talk about the SAMR model in the literature. The SAMR model is kind of an inherently got a, it’s got a positivity bias in that there’s this assumption that introducing technology to education or assessment or whatever is necessarily going to be positive and just isn’t the case. You know, if you think about technology as an amplifier rather than a benefit to, to context. You can see the technology can, you know, if you add technology to a horribly designed course, it’s going to make that horribly designed course bigger and louder and it’s horribleness. And that’s not necessarily a good thing.

Sharona: That’s something I’ve mentioned a few times recently about AI is I’m very worried that especially our, My traditionally marginalized communities are going to fall further behind because the better, more elite schools are going to integrate AI faster than the big urban schools who are going to try to wall it away. So I’m worried.

Boz: Yeah. I was also. You speaking about this amplifier role of technology, I was just reading, and Sharona, you and I were talking a while back about some research and some analysis of claimed research about some of the ed techs and their success rates. And one of the conclusions of this article, you know, was also when you take, you know, take these kind of technologies, you know, like iReady and Khan Academy and was the other one they mentioned? And you put them in places, it seems like they were really successful for certain groups of people. And not other groups. And when you put it into a diverse school, what it actually ends up doing is widening the equity gaps between our students, that the groups that it was beneficial for were, you know, Were the groups that were likely to be successful anyways, but now they’re being even more successful. And it was actually hurting the students that, you know, aren’t that, that weren’t the hugely motivated or you know, students that were getting a lot of support and coming with some extra needs. So it was actually worsening the problem. So it was having that exact amplify effect that you were just talking about.

Colin: Yeah. And we, we saw that through. You know, pandemic restrictions, you know the, the students who were already disadvantaged became more disadvantaged. The trend, trend, the panic rush towards using technology to proctor exams and things like that. All of a sudden, students who have darker skin can’t be recognized by these cameras. And they’re getting locked out of their exam, and it’s slightly problematic, to say the least. I feel like that was a little bit of a rabbit trail.

Sharona: Well, no, so, so you, you found that this is going on, and you wrote this paper. How do you see it impacting where you’re trying to go now? So where are you trying to take this, this research, and what do you hope to, achieve with it as far as assessment.

Colin: Right. So where we’re sitting right now, the, my second paper was kind of taking the results of the first paper, that’s literature review and building out a framework, the framework that I’ve currently called Technology Integrated Assessment Framework, which I feel is fairly descriptive. And you know, what, what was missing from other frameworks are things like a duty of care. And I need to thank my supervisor for this one because she’s like what? Assessment can cause harm. Absolutely. It can actively cause harm to people. There are instances in, in the UK where students have taken their lives because of inflexible assessment practices and that is problematic and what we’re suggesting is that instructors have a responsibility to ensure that their assessment practices aren’t actively causing harm. Or at least they’re reducing the chances and making sure that, oh, hey if you’re in a situation where, you know, you need to reach out for help, then you need to reach out for help and we need to help you. So things, things like just avoiding bias in, you know, the end, that technical, recognition of bias where there’s different groups get different grades just because of who they are. And making sure that practices are inclusive of all the whole spectrum of human experience and making sure people can participate. And then something that I think, you know, well, it, it goes back to what I talked about in my, in my origin story of how that little bit of relationship, even if it’s fleeting. It’s really important. You, I don’t know if you’ve noticed, but I’m wearing this orange shirt. Today is, well this weekend is Orange Shirt Day. In, in Canada. I don’t know if that’s a thing down in the U. S.

Sharona: So we’ll explain why Bosley is trying very hard not to vomit at the word orange, but we’ll go there in just a minute.

Colin: Okay, perfect. So Orange Shirt Day is, is recognition of just the historical wrongs and intergenerational traumatic impacts of the indigenous residential school system in, in Canada. And so it’s in recognition of, of those experiences. And in Canada, several years ago, we had the Truth and Reconciliation Commission, which was a large countrywide kind of, I don’t know if it was a reckoning, but it was a process of gathering information from Indigenous people who have been through and been impacted by these residential schools. And one of the things that, I wouldn’t say it came out of that Truth and Reconciliation Commission, but it’s kind of parallel, is this idea of the five R’s of Indigenous education. And I’m going to make sure that I get all of these in order. So I am scrolling through my paper right now.

So five R’s of indigenous education: respect, relevance, responsibility, reciprocity, and relationships. And so there’s been lots of writing around these five R’s, used to be the four R’s, now the five R’s with the added relationship idea. And so we’ve made sure that these five R’s are embedded in our model. So you can see in this duty of care, there’s an aspect of relationship that’s important. And in respect to making sure that your assessments are designed well, there’s that aspect of reciprocity. Hey, do your assignments benefit your students and their communities? Do they promote connections to the student’s home community? To the student’s families? To the land that the student is coming from? Even though this might be a technologically integrated environment and they’re doing it online. So I think that that in building the 5Rs into this model, and in particular, it’s a very human perspective on assessment. And I think that is at great risk in AI integrated environments.

By removing the human relationship, if assessment becomes just this ugly circle of students using chat GPT to produce their work and teachers using chat GPT to assess student work, and I’ve seen people advocating for teachers using chat GPT, or any large language model, to generate assessment feedback. I’m like no. Please, no. Please, please no. Let’s not go there. And now is the time that we have to be able to steer that back into a more human relational perspective on what assessment is.

All of that brings up is, is this idea of faculty instructor workload. And I, I think that’s the, the biggest challenge I’m looking at is, you know, how, how do you incorporate this idea of relationship when you’ve got 400 students in here, you know, undergrad in English class or whatever it is, these large classes. And it comes back to the idea that education, higher education, K 12 education as well is a system. It’s a very complex system with a whole bunch of moving parts. And often at the bottom of it is funding. We’ve got huge classes because that’s a real cheap way to get 3, 000 undergrads through an English 101 or those introductory courses. And that becomes really problematic for grading reform that’s based on the idea that human relationships are important.

Sharona: I, I feel this so much. One of the, this isn’t directly assessment based, but I feel it’s in the same place. I feel that I’m training my instructors that if a student misses an exam, the subject line of the email that goes out is, are you okay?

Colin: Right.

Sharona: Because before we address the fact that you missed the exam, I need to show you that I care about you as a human being, because that’s more important than the exam. And you’d be amazed at the kinds of response rates we get from an, are you okay email. And it’s like two, three sentences. Hey, I noticed you missed the exam. Are you okay? Let’s connect. I want to see how I can help you. And that’s it. And you don’t send them six links to resources. But that then puts that burden onto the instructor. So we should be compensated for being that person. So I feel so much of what you’re saying. And I think it’s going to be very interesting to see because we have a lot of conversations about the use of technology in grading reform and both how it works. Assists or could assist and how it also hinders because electronic grade books are kind of the bane of our existence at this point. I really wish we could just go back to paper and pencil grade books in some ways.

Colin: Yeah, either that or incorporating some spreadsheet, the functionality of that. And I’ve, I’ve got some ideas on around that.

Boz: But, you were talking about, AI and just any large word model, whether it’s chat GTP or something else. And it was making me think of one of the R’s of the five R’s. Cause we were talking not too long ago with Dr. Emily Donahue and this idea of these large word models are mainly done and produce something that’s in the voice of your typical kind of whitewashed English language. And all of the uniqueness to the different dialects or to the different speech habits of individuals and groups is really lost. And looking at how that goes into the representation of the five R’s and how we continue to use this and get this very generic whitewashed version of the English language and losing all of that individuality, whether it’s an indigenous group or any other group in that, that personal style voice that we just don’t see when you’re using chat GTP or anything like it.

Colin: Yeah, I think that that problem is well known in large language model research, known as model collapse. Eventually everything that the model is being trained on was generated by the model.

Sharona: Well, I think we’re coming up on time. It does feel like we could have an entire additional hour. Bosley, any last things you want to ask or say?

Boz: No, I just want to first thank you for actually answering our call out. It was cool to actually get someone that’s listening to it responding. And I would definitely like to have more of our brothers and sisters from the North and kind of doing some more comparisons of our two systems, because I’m really curious to learn more about the similarities and differences between our biggest land neighbor that we have. So thank you for responding. Thank you for coming. And then is there any last, last bits of things that you want to close out before we end it?

Sharona: I just want to, to him or to me?

Boz: I was going to him, but if you had something to.

Sharona: I do have something, but let me, let me direct it over to Colin first. Anything you want to add at the end?

Colin: I, I’m just, I’m grateful for the opportunity to, to jump in and, and share my thoughts with, with your audience. And like you said, there’s so much more that we could talk about, but the next stages of my dissertation are. Are on the horizon and, you know, just digging into more of a quantitative look at this model and see what makes sense from that perspective. And so I’ll be happy to share that as well.

Boz: Love to hear it.

Sharona: And so, yeah, so my last thought was for the listening audience. I absolutely adored the fact that we ended up talking about physical education as a discipline. And I’d really love to have even more diversity of disciplines. So this is my invitation out. If you are in a discipline that you have not heard represented on the podcast, or something that maybe you haven’t heard a lot of, I would very much love to have you write in and potentially get you on the podcast because these these conversations are amazing. So back over you, Boz, to close us out.

Boz: Well, thank you for for listening. Thank you, Colin, for joining us. And we’ll see you guys next week.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode’s page on our website. www. thegradingpod. com, or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show, or would like to be considered as a potential guest for the show, please use the contact us form on our website. The Grading Podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.

Boz: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State system or by the Los Angeles Unified School District.