The Grading Podcast

58 – Authentic assessment

===

Sharona: Yeah, that was History of Math, and we were exploring different number systems and different bases and things, and they actually composed and recorded, like, a jazz piece based on, I think it was Egyptian harmonics or something like that. Yeah. It was really cool.

Boz: Welcome to the Grading Podcast, where we’ll take a critical lens to the methods of assessing students learning, from traditional grading to alternative methods of grading. We’ll look at how grades impact our classrooms and our students success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist, and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach, and instructional designer. Whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator. This podcast is for you each week. You will get the practical detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Boz: Hello, and welcome back to the podcast. I’m Robert Bosley, one of your two co hosts and with me as always, Sharona Krinsky. How are you doing today, Sharona?

Sharona: I am doing quite well, thank you. It is pretty great to have you back in the co host chair. Cause I had to do a few of them on my own this summer and it is just not nearly as much fun without you there. We did definitely miss you last week at MathFest.

Boz: Yeah, that episode and just the stories I’ve heard from MathFest, it sounded like you guys had a really good time there.

Sharona: We did. And I wanted to maybe share, before we get into the topic of this episode, I wanted to share a couple of takeaways that I think came out in last week’s episode, which is that grading reform is really winding its way through the ecosystem of academia, of higher ed, and it’s really getting into a more common usage. So I’m really excited to see the growth from even five, six years ago to where it is today, and I’m really hoping we’re going to see a lot more of it. Going forward, you know, it’s nice to see instructors really starting to think about this stuff.

Boz: Yeah. Although I do think that’s a little bit of a, of a double edged sword. I think it was actually brought up in your live episode that you did with, oh my God, how many people were actually in that room with you when you were doing that?

Sharona: So there were, there were 12, there were eight people who spoke including me I believe, but there were 12 total.

Boz: Yeah, that just, that’s a lot. That sounded like fun. We’re definitely going to have to do that again. But no, one of the things that I don’t remember who brought it up, but it’s also something that’s come up in some of my other readings and things is alternative grading and grading reform is becoming more common now, but as it gets into a larger and larger population, it’s also getting mistreat. It’s getting, there’s. You know, falsehoods, falsehoods about it that are starting to really percolate that,

Sharona: yeah, David Clark brought that up and

Boz: David, yeah,

Sharona: yeah. Well, actually, I think I actually asked him and he agreed. But one of the things that struck me at math fest, there were two, there was a mini course and there was a workshop on how to do alternative grading, but there weren’t any sort of talks about why this matters. And I’ve given that talk a couple times at MathFest. I wasn’t there to talk about that. I was talking about team based inquiry learning, which I mean, I did throw in my grading, of course, but I’m feeling like, okay, maybe I need to give our base talk as often as I possibly can, whether it’s the grading is the misuse of mathematics talk or.

Boz: Yeah, I really do think, especially in a bigger conference or a conference that’s not just specifically about grading, they’re getting this message about what is wrong with traditional grading is still really important. I actually had a conversation today with one of my new colleagues at my high school setting. I mean, he’s not new to teaching, but he’s new to our school. And he was asking me cause he knows I’m the EGI champ at my campus, which I still, I hate that title. I don’t know why, LA USD loves this champ title.

So I’m like the Schoology champ, the EGI, I hate the title. I feel weird saying it like I’m a champion of something. But anyways he was asking me about it and was asking some help with setting up his Schoology and I could just tell from the conversation that he’s wanting to go to EGI without really understanding it’s more than just grading form. Like it really is a deeper philosophy change than just simply changing how you calculate a final grade. So I do think that’s something, as a community going forward, especially some of the community that’s not just going forward, but trying to push it forward. We need to remember that and we need to continue those baseline conversations about the philosophy and not just the mechanics of it.

Sharona: Well, and I want to throw this out to everyone listening, if you know of a conference that you want to go to and you want to present at, I’m more than willing to share our slides, especially the grading is the misuse of mathematics. Slides because we’re just going with common historical information and some math and we’d be happy to, I’d be happy to, share the slides and let someone else present that talk or if there’s a conference you think that we should be going to, let us know. We do have a conference coming up. We’re going to be at the California Math Council South meetings in Palm Springs in November. I think we’re talking about the topic of this podcast episode actually at that one. But yeah, so if someone has a suggestion of where we should go, let us know.

Boz: Yeah, and, and like she said, we are happy to share any of the materials that if, if you want to. Look at it and use some of it. We’re, we’re happy to share. We want to get this information out. We share everything. The only thing we ask is if you do use some of our stuff that maybe on the what’s next step, you throw a plug for the podcast in there as well.

Sharona: Exactly, exactly. So I’ve alluded to it a couple times, but why don’t we jump in? What is our topic for today?

Boz: Well, we’ve actually had a couple of episodes in the past about AI and the impact that it, is potentially going to have, not just on grading, but on education as a whole. And one of these that came up and has come up in a lot of conversations is this idea of Authentic assessment. And what does that look like in the age of AI? So that’s what we’re going to be talking about today. And, how that is related to grading reform and just what can we do with assessment with AI and everything else that’s out there right now?

Sharona: And I want to clarify what we mean by assessment real quick. This is the assessment that is built into the four pillars. So we’re talking specifically about checking for evidence of learning by a student. This is not program level assessment or some of the other places where the word assessment is used. We’re literally talking about. The tools that you would use in a class to gather evidence of learning. Would you say that’s a correct statement?

Boz: It is, and it’s funny because that is definitely one of the divides between the K 12 world and the higher ed world. When you say assessment, and because I come from the K 12 when you say assessment, that’s all we think of. And then, but you’re right, with higher ed, it’s got a lot of different.

Sharona: It’s program assessment. Yeah. We literally have assessment committees that have nothing to do with grading.

Boz: Yeah.

Sharona: Like at the department level, we have an assessment committee of tenure line faculty, and it is about assessing our program, our math major, our master’s degree to see if we are hitting our own benchmarks and it has literally nothing to do with grading students.

Boz: So yes, we are talking about student level assessment here.

Sharona: Yes, so we were asked to develop some professional development around this. So I don’t know where you started, but I started where I always start, which is I go looking for what’s known. I go looking for literature to read. And I thought that was pretty, eye opening because I started just with a Google search on authentic assessment. You know, I’ve been grading my students and assessing them for a long time, but we have these learning outcomes and you and I have spent a lot of time develop what we thought were really good questions, especially in statistics, and they are very answerable by the latest generation of AI. I can literally take my carefully written hundreds of hours paid for by my university questions that I’m so proud of and I’m sophisticated enough. I can put it in chat GPT 4.0 and say, answer this as a freshman level student who, for whom English is a second language and I will get something that I can’t tell is not my students. And it doesn’t matter what versioning I’ve done, it doesn’t get the math wrong anymore. Even a hypothesis test where it has to calculate a p value, it can do that. The difference this year versus last year is tremendous.

Boz: There’s only one thing that I still, I haven’t seen the AI not do yet. And that’s when it comes to sample sizes, but you’re right. Everything else, it has gotten that much better. And you know, it really is that the abilities that it has to write in a certain voice or a certain level, it, it really is quite amazing and scary at the same time.

Sharona: Exactly. And so on the one hand, that’s where I started. I was like, Oh gosh. Yeah. We have this issue because we’re supposed to be checking for progress, right? Marks indicate progress. And I don’t believe in what some people in our community call cop shit, right? I really don’t have an interest in restricting my students. to a paper and a pencil and what’s in their own brains, because that’s not the real world either. It just feels awful. So I started grappling with, well, what is authentic evidence of learning look like in an age where any problem that I write can be answered by an AI at least at a C level, right? So that’s what I was grappling with. And so I went to Google.

Sharona: Exactly. And as I was looking through this, I want to give you one of the definitions we, we found. And as you know, because we’ve talked about this, this really stuck with me. Okay. So I’m going to give you one definition and then we’ll go into a bunch of other stuff. But one definition is the definition of authentic assessment are assessments that use performance tasks that involve real world problem solving that allows learners to demonstrate their skills by applying them to the task. Authentic assessment occurs in the learning environment, but attempts to model a real world application.

So what struck me about this is this is dovetailing with another issue that we’re facing at the university, which is student disengagement, right? If you’re going to do a performance task that quite frankly is a lot more challenging to design as an instructor than just some paper and pencil quiz and then I give this to a student and they don’t even put any effort in? Like that just feels terrible. So I was trying to think about why would my student be willing to do something that seems on the surface, at least to be a lot more complex.

And we were talking one day and I had that aha moment of the story of my younger son. So are you okay if I share that here? I kind of want to use this. So my younger son is in college, both my kids are in college, and my younger son is getting a degree, but he’s not planning on using his degree for his career. He has a couple of outside loves, but the thing he’s planning to do to make his living, to put food on the table, is he has over the last four or five years through high school and into early college, trained himself in video editing. And I was thinking back about how did this happen? No one in my family is a video editor. I didn’t say anything to him about it. Where did this come from? Cause he is really good. Well, he’s making quite a bit of money.

Boz: Let’s say he’s not just good. Like this is not a pipe dream that you’re talking about. He’s already making money doing this. He’s can, can I say that where he actually has already gotten some work from?

Sharona: Yeah

Boz: he’s gotten work from EA. Like one of the biggest sports game franchises there is so like he’s getting real work on this. So this is not like some pipe dream that you’re talking about.

This is, real plan that, that he is going to be able to enact like this.

Sharona: Yeah. I mean, he has this one profession that I’m not going to name that he does work for where he is able to charge a hundred dollars an hour and they think it’s cheap because he’s so fast and so good at what he does. So. I mean, this kid is, you know, 18 years old and able to make this kind of money. But I was asking myself, and I promise this will tie into assessment in a moment, but I’m asking, where did this come from? So I was sort of tracing it back and I’m like, well, he’s a big gamer. And so he got into this time when you streamed. yourself. And he got into this time where you screen recorded yourself playing games and you would cut them and make them into clips. He had one of his clips go, go really viral. It was something called like 45 seconds to when I lost my 99 game winning streak or something like that.

But then I kept tracing it back. And I’m like, well, when did the video game start? And I ended up going all the way back to when he was in the fourth grade. Now we’re in California. And in the fourth grade, the social studies. Curriculum is about missions primarily. It’s, it’s the history of the state of California, but it’s the missions. And those are the Spanish missions. There’s like, I don’t know, 23 of them, 21 of them, something like that, in the state and every fourth grader going back, as long as I can remember, cause I did fourth grade in the state of California, you had to build a mission model and it’s kind of hellacious. They, they sell these expensive styrofoam kits now with balsa wood. Back in my day, you had to try to do it yourself and your finger, you super glued all your fingers together and it looked like crap. I mean, it was terrible. Definitely one of those projects where many of them, the parents are more involved than the kids, right?

That was not me. I’m like, I am not touching your mission project. Y’all do what you want to do. Well, my son asked his teacher if he could build his model of his mission in Minecraft. And she said yes. So he built the entire mission with all of the rooms and all the detail in Minecraft. And then we ran into a problem. Because at the time, and it’s probably still true, you can’t access Minecraft from on a school site. Like on the school Wi Fi, it is one of those blocked sites.

Boz: Yeah, most school Wi Fi’s have a pretty hefty firewall that will block almost any kind of gaming thing like that.

Sharona: Right. So, how was he going to show his teacher that he built this model? So we decided he had to record himself giving a tour. And we also are a theater family, so he wrote himself a script. But, he was maybe eight at the time because he was very young for his grade. He wrote a script and he screen recorded himself. walking the teacher through this mission as a tour guide. So he had animated voicing and stuff. He turned it in, she loved it.

So she asked him to create a video game in Minecraft. So like a game within a game to review the book that they’re reading in class. And so he had a game show and he invited a couple of his friends after school to come onto this game in Minecraft. So they didn’t come over, they did it from their houses. And again, he screen recorded them playing the game. as a review for this book. So he did way more work than he ever would have done on a mission project or on reviewing a book. And through these things, the teacher was able to see, okay, number one, he knows the book. I mean, he wrote questions and the answers and everything and he knows his mission model because he was touring and talking about the cathedral and everything and keep in mind we’re Jewish. So everything he learned about the mission, he did not learn from home. That teacher’s willingness to allow him to demonstrate what he knew through something he cared about has now translated into one of his future careers.

Boz: Yeah, 10 years later and he’s already making money doing this.

Sharona: Exactly. So to me, one of the dangers with authentic assessment is we’ll say real world tasks and a lot of us at higher ed will go straight to work tasks, but I don’t think that’s necessarily necessary. It just needs to be real world, period. It could be hobby related. It could be life related. It could be anything that you would really see. But if it is authentically something that the student can identify with, I think they’re more likely to be willing to engage with it. themselves as opposed to asking the AI to do it for them.

What do you think?

Boz: No, I think that is one of the dimensions, one of the aspects of authentic assessment But then the question becomes, you know, how do you do that? How do you do that? That I can imagine is hard enough at the higher ed world. How do you do that when you’re teaching anywhere from 150 to 180 students like we do in the K 12 world? You know, how do you, so not every assessment is going to be this Minecraft, you know, life altering assessment. It’d be great if they are. Don’t get me wrong, it’d be great if they are, but not all of them have to be that far, like that, that is a very unique experience that was great and was, like you said, it really was life changing, but that’s not necessarily the goal for every single assessment. I mean, it would be great, but.

Sharona: I think I sort of agree with you in the sense that that particular task was very involved.

Boz: Yeah.

Sharona: Right. But I think the kernel of it being something that the student can understand is something they actually have to be able to do. That piece of it, I think we need to strive that every assessment or almost every assessment can be tied in that way with the student. I mean, I’ll give you another example. Let’s, let’s get, let’s bring it to some math. Let’s talk about quadratics, any polynomial that has an x squared in it. We teach this very abstractly, in my opinion. You know, we do a lot with symbol manipulation, both at the high school level and the, and the college level. Okay. The people that I know that have to use this in real life, my number one person who constantly uses this is the guy that builds the sets for the theater that I run, because he is constantly having to figure out the areas of the sets that he’s designing so that he can buy the right pieces of plywood and Luan and sheets and, and lengths of, legs and he has to make them all fit together because it’s too expensive to buy one sheet and just cut one thing out of it.

Boz: Yeah.

Sharona: So he’s constantly literally doing paper and pencil area calculations and he knows that they’re all quadratics. He’s aware because sometimes he makes his decisions. Sometimes he’ll actually do a formula and go backwards into it. He also has to calculate how much paint he needs. Right? So you came up with something.

Do you want to share what you came up with or no?

Boz: Well, let’s not jump right in to. Okay. One of the assessments. And actually, I’m not even sure.

Sharona: But my point is that I think you can take something even as simple as a quadratic and make it make meaning or have meaning to students by being more related to what they could see themselves having to do in their life.

Boz: Yeah.

Sharona: So that’s not, that’s one element. But the reality is, as we jumped into all this. I think we found 14 total things to consider. So we’re probably not going to go through all of them, but do you want to touch on some of the other major elements that we found that can make an assessment a lot more authentic and therefore a little bit more AI proof?

Boz: Well the biggest one for me, in my opinion, especially when looking at AI is things that have a you know, multiple answers, things that have multiple right answers, that there’s lots of ways of doing it, that there’s not one right answer and AI, it is getting better, but AI in any kind of technology struggles with things that, especially in math that has multiple right answers. And I’m not talking about a polynomial with like three answers. No, I’m talking about things set up to where there could be. You know, a maybe a finite number, but a very large finite number of solutions or could actually have an infinite amount of solutions. So having something set up to where there’s not one right answer and having students going in and explaining some of those decisions. To me personally, that’s the, the biggest number one thing. And if you’re going to try to make these kinds of assessments, that’s one of the components that it has to be there. And that ends up being also what really shields it from AI.

Sharona: And I think that the better that we get to know our students. and the more that we encourage them to make the choices on a given assessment based on their personal tastes, the more authentic we can feel confident in their work. So if you know about a student that they like cooking, and you’re doing something with fractions, and you encourage them to do the assessment using their Information about cooking and someone else is into, you know, skateboards or something like, yeah, there’s infinitely many right answers. You might just be assessing fractions.

So the point is, if you’re asking them to authentically tie it to their own interests and knowledge, again, they’re going to get more engaged in it. And it doesn’t have to be that you personally have to come up with all of these different ideas. You’re going to maybe set some parameters and say, you know, pick the hobby that you’ve already told me about based on that hobby, create something that involves these three things and show them to me as you go. Right. Yeah.

Boz: And then, I mean, and you’ve actually already, I don’t know if you called it authentic assessment, but you’ve already done some of this both with your was it your Calc class that you did the, the curve and then your, the project and then your history of math classes. So, you know, you set up some criteria, some things that they have to land on, but then they have a lot of different ways and. But didn’t you have one student that like made a musical piece out of showing fractions?

Sharona: Yes. Yeah. That was history of math. And we were exploring different number systems and different bases and things, and they actually composed and recorded like a jazz piece based on, I think it was Egyptian harmonics or something like that. It was really cool. But again, Now, I didn’t have to give a grade on the quality of the jazz. I did not grade that. I was looking for evidence that they were able to demonstrate to me some knowledge of an ancient numerical system. Well, they did an explanation of these Egyptian fractions or something that they put in these harmonics. And that was enough for me to say, yeah, they know something now they did the research and I got this beautiful piece to listen to. Okay.

It was amazing. I also love. Yeah, I agree. I love the calculus one because I give a project that essentially the total mathematics is not something you can possibly do by hand. So they have to go to Mathematica. They have to do some research on how to put some basic things in, but I’m actually looking for some very simple things as part of that. I am looking in one of the pieces for under, do they understand how to calculate the curvature of a curve at a point? And I get that information so quickly by whether or not they pick the right point. Cause there’s maybe they could put any point on the curve, except usually there’s two points. that you can’t pick because curvature doesn’t exist there. So I get so much of a better understanding of if they get the concept of curvature, do I really care a whole lot if they can hand compute it? Absolutely not. There’s maybe five equations in the whole world that can be hand computed and everything else you need a calculator or numerical integration for it.

So. I really like to create more complicated things that give my students opportunities to grow and really blossom in and explore something where the evidence that I need is actually very small and simple. So I’m not going crazy grading. even though the projects I get are much bigger than the evidence I’m looking for.

Boz: And then something else that I was just thinking as we were talking about this I had a conversation not too long ago with one of the other groups that I’m working with at my school, my high school, and they were talking about some of the, what’s called anchor standards. So in common core states All of the subjects, with the exception of math, have these anchor standards, and then math has these eight mathematical practices that we’ve actually talked about quite a bit.

And, actually, I think Joe Zeccola talks some about this in some of his, but these authentic assessments, if you’re designing something that’s testing more than just, The mechanical ability of doing the skill, you know, of solving the equation, you can, it’s actually a much clearer, better way of assessing a lot of these broader learning targets, like the mathematical practices, like the engineering practices, like the anchor standards, like the 21st century skills, like all that we’ve talked about this I don’t want to go too far into this, but we’ve talked about this on a couple of our learning target episodes, but all of these learning targets that are bigger than a specific course. You know, that are more of an overarching, these, when you’re doing these kind of projects, these kind of authentic assessments, you actually get a much better understanding of what your students know and can do in regards to those.

Sharona: Exactly. There’s a couple of other ones that you sort of mentioned in there that I want to add in. There’s two other ones that I think are really important. actionable and meaningful. So the product that is being produced In order to have an authentic assessment or a more authentic assessment, we should probably talk about the continuum there, but to have something that’s more authentic, the reality is a paper and pencil exam is really not a very authentic thing. It’s, it’s sort of almost never out there in the real world. Whereas if you can think about, a kind of thing like a report or a presentation or even if you did have a paper and pencil exam, but the grading doesn’t happen with you, the instructor sitting there without the student, have the students sitting with you if you have a small enough class or a small enough opportunity.

You know, I, I know that in high school, sometimes people do these grading conferences where they sit with the student. I don’t know that that’s as prevalent in higher ed, but I think that might lend itself really well because a student having to explain themselves is much more authentic than a student writing something on a piece of paper that has to stand on its own. Now, that’s some or most of the actual components that go into the assessment itself, but what about some of the other things that we found and we’re looking at that we’re kind of calling the decisions, the five decisions you have to make as an instructor?

Boz: Yeah, so when you’re trying to, you know, design, redesign your assessments there, there are some things you need to think about, some things you need to do while making these. Okay. And the first one is just what is that assessment task going to be to begin with? What is the actual task that your assessment’s going to be built around?

Sharona: Yeah. Is it a test that that’s a potential? Is it a project that gets turned in a small project, a big project, an essay, a performance, a business plan? You know, what is it that they’re turning in?

Boz: It that even for example one of the things that I knew I, I drew from was oh, resnick . So, one of the things that he points out and is kind of one of the building blocks of authentic assessment is that most school tasks, you know, really involve some sort of memory work, some sort of mechanical use of knowledge where as in the real world outside of school, most of the knowledge that you use isn’t in memory or recollection. It is in actually putting that to use in some sort of activity with all the resources that you could need or want. You know, being able to use calculators, being able to look things up that you might not remember every single detail, but it’s putting all those together to actually accomplish something, which is part of the reason why, you know I know, and

I know she doesn’t listen to this, so I’m going to throw her under the bus. My wife hates math, absolutely hates math. But she can do one thing better than I can and better than most mathematicians. I know she can add fractions and subtract fractions faster than almost anyone I know. Why? Because she’s been doing it authentically since she was like four years old. ’cause she can read sheet music and was playing the piano and you know, like a lot of musicians, because music is nothing but fractions in motion. But because she’s been using that knowledge in an authentic way, putting music together, putting sheet music, reading sheet music, understanding tempo and beat, you know, what three quarter time is. And if you’ve got a half note, how much time that has left, she can do that.

She can’t explain it. I mean, when our daughters were. You know, in second, third grade and looking at fraction, she couldn’t help them do it, but she could definitely tell them if the answers were right, cause she knew how to do it. But that kind of disconnect from the mechanics of doing it the way we would do it in school compared to actually doing it in a task,

Sharona: I tend to try to draw. As much inspiration I can from things in my own life. So, you know, people have heard me talk about, I run a theater company. When I sit down and I have to plan out a class schedule for new classes. And I’m trying to think about how much time, how long I want the classes to be and what is it going to cost. I’m constantly multiplying fractions, an hour and a half times an hourly rate. I’m trying to think about, I have to think about the costs of the employer taxes, which is usually something like 10. 3%. So I have to multiply their wages by 0.103. So these are very authentic ways that I need to do for my job that there’s no reason a high school or college student couldn’t be asked to do.

I can look at them and say hey put together for a budget for a new classroom offering or a new You know something you want to sell and you have to account for all these things and you can say hey go look up What employer taxes are on a w 2 employee? So yes, I’m doing math, but I’m also bringing in employees and payroll taxes and some of the other things that they need to know. And maybe they need to look it up by state or maybe, so there’s some complexity there too.

And that draws me to a couple of the other decisions of authentic assessment that are part of this logistical decisions is the physical context and the social context of where that task is being done. So it’s recommended that you might want to reflect the way that knowledge, skills, and attitudes will be used, reflect the amount of time it will actually take in the real world to do this and the social processes of the assessment. So for example, if you’re like, Hey, build a budget for a project, you might want a team of kids working together because that’s what would happen in the real world. And maybe they each have an individual piece so that you can assess their individual knowledge. But when you’re planning, say, a new theatrical season, you’re not doing it in a vacuum. You’ve got other people working with you and trying to help you make a decision.

Boz: Yeah. That’s one of the 21st century skills, that collaboration and communication are two of the four C’s of the 21st century.

So absolutely.

Sharona: And then one of the thing I love, but I think it’s a little bit obvious, but I’m going to state it anyways for this community, is Authentic assessments, the criteria of how you’re going to check the work has to be transparent beforehand. So for us, students need to know what, what we’re measuring.

Boz: Yeah. Which, you know, falls right in line to, you know, our kind of alternative grading ideology where we’re grading things on proficiency scales, not on. Points or percentages. And it also, I think allows, because one of the other things that we’ve discussed, and one of the other things I know that you and I have had hundreds of discussions with other educators about is how to squeeze in enough time to do reassessments. Well, when you’re doing something like this, this is such a great opportunity for revisions and resubmissions rather than just. So, okay, you didn’t get it here. Now, now I have to go invent this whole new authentic assessment. That’s going to take me forever to design. No, this is, and again, what happens in the real world? Most of the time you put together a budget or a blueprint. Someone gives you some feedback. They don’t like something you don’t start from scratch, you revise. Exactly. So this is another one of those spots where, you know, you could actually utilize some of the ideology and some of those four pillars in a way that even though we’re getting this much more in depth assessment, actually is saving us a little bit of time because of the revisions.

Sharona: Yes, I, exactly. And the nice thing about that, again, is you don’t have to assess. The whole thing, right? You don’t have to grade the whole entire project. You need to measure the things that you intended to measure. And so I really like keeping that in mind as we do this.

Boz: Yeah.

Sharona: So, we’re gonna provide a lot of links to this stuff, but there’s one more set of things that I want to talk about and then talk about where we think we can go with this, cause this is a lot to try to do all this stuff. Right? So there’s four more questions that we found that some scholars use that I think really provide additional hooks for us to think about. So the first question is, is this task worth doing?

Boz: And I think that kind of, that kind of gets back to what you were saying earlier about it. The student motivation and getting that hook into the students, because if the students think it’s worth doing, you’re going to get a much better product and a much better engagement from the students.

Sharona: And this reminds me to when, you know, when we go back to the mission project, most 8, 9, 10 year olds. don’t see a purpose in building a physical model of a mission. They don’t have a connection to a mission. They don’t really care. They don’t yet really understand what history is. So is the task worth doing?

Is it worth it for a fourth grader to build, to buy one of those 40, 50 kits at Michael’s and try to use an X Acto knife and not have to go to the hospital and attempt not to glue their fingers together? Is it worth doing? Well, my son, it was because he was able to put it in a context where he had a blast. That kid probably spent hours and hours building that mission model. And we’re Jewish, like, you know, so because the building in Minecraft was worth it for him, that’s what was worth doing. Do you want to take the next question?

Boz: Well, I don’t know if you need to go through all four of them, but. I wanna jump to the third one actually. Okay. Which is, who are the students becoming in doing this task? And that really takes me back to Dr. Jeff Anderson’s interview, which if you guys haven’t listened to that one, that I, that was this really fun interview that ended up being a two-part one. But what he was talking about with the theory of learning and what exactly learning is, and. That for something to have been learned, there has to be some level of change. And if there isn’t change, then there’s actually no learning. So this idea of how are the students growing in this project? How are they changing? Therefore, how are they learning? Like when I read that, that was the first thing I was thinking. I was like, Oh yes, this, this, even though it wasn’t, wasn’t done by Dr. Jeff Anderson, I’m like, I could just see him going nuts on this part.

Sharona: Yeah. Well, and actually I was going to mention him with the fourth one because. One of the books that he mentioned in his article was Drive by Daniel Pink. And I have had that in my library for a while to read. So I’ve started to read it. And that is the surprising science of, of motivation. Good book. And I I’m just getting into it, but one of the things that they talk about in sort of the newer models of motivation is that. There’s a little bit of a drive to making life better for, for everyone. That’s one of the motivations people can have. So one of the questions that we can ask, as we’re looking at authentic assessment, is what role does this task play in contributing to a better community? And again, I go back to my feeder stuff because we are all about building community and when our teens get a chance to dig in and help to design or build something for one of the shows that we have coming up, there’s no reason for them to do it. They don’t get paid. They are not getting a grade. They don’t get a better role in the future. So why do they do it? Because it makes our community better. And so, if we can leverage that, I think that has a lot of power. Now that’s a lot to take in. And if, I’m pretty sure when we started designing this, you looked at me and said, If you make everybody do all this stuff, no one’s going to listen to you.

Boz: Exactly.

Sharona: So what did we come up with?

Boz: So we took a lot of these different research and, and theories and philosophies about authentic assessment. And we kind of came up with this continuum, this, this rating, or not rating, but the scale from lowest authenticity to higher levels. And some bench points on, what, what you have at each level and what makes it higher or lower.

Sharona: And there’s some that so that the first level, all the subsequent levels include everything from the beginning, like level two includes everything from level one, the way we designed it. And so I think it’s interesting to look specifically at the first level one, because that’s the one that we would recommend. You try to get these on pretty much everything. Yeah. Yeah,

Boz: anything that you’re doing now, there are obviously times that you’re going to want to test a student’s just base skill and there’s nothing wrong with that. I don’t know how much I would use that in my overall evidence of proficiency or mastery in a learning target. Greatly depend on the grade I was teaching and the level of familiarity that the students have. But yeah, there is some times where you’re going to want to just, okay, before we use this, can I make sure you can actually multiply two polynomials?

Okay.

Sharona: Right. But I would argue that you can just tell the students that. You can say, Hey, we’re working up to this big thing. It’s important for you, the student, and for me to know how you’re doing on some of the things you’re going to need. So we’re going to just take this quick hit thing. Let’s see how we’re doing.

Boz: But, but my point was, I didn’t want, if someone’s listening to this, especially a K 12 educator listening to this and going, I have to do this with my weekly quizzes that are just purely, okay. Let me make sure they’re on track for. You know, proficiency or mastery of the skill. So we’re not saying, you know, things like that those quick check for understandings, those quick weekly quizzes, that that’s not what we’re talking about, but,

Sharona: well, and I also think it depends on your context though. So like if you’re in person and a K 12 and you can throw a paper and pencil, quick little assessment at them, that’s fine. But we have a lot of people who, for whom even the quick stuff, Is done in a non proctored environment, potentially online. So some of those quick hits, those are very susceptible to a I use. So I really am going to question their value integrating.

Boz: Yeah, and that’s what that was my point is, I don’t know how much I would use that in my grading, but still the act of doing it. And that’s, you know, I didn’t want people to hear this and think all of these assessments that we do have to be, this level of, of authentic. assessment, because sometimes just the skill practice, but like you were saying, depending on contents and a lot of other things, what part those little just skill based things are used in your grading wrap up could vary quite, quite immensely.

Sharona: So what I would say is if you are trying to get authentic evidence of learning, And you’re concerned because of the context you’re in about something like AI use, then I would say you want to go for at least level one. And the way that we determine level one is we took a little bit of all the things we talked about. So we said, the task has to be worth doing, and that it’s a student focused task. So it’s something the student is going to consider worth doing. If you can get to that, that’s your first defense against the AI. And then. If it has that ambiguity, so some large number, potentially infinite number of choices, and you don’t, if you can avoid it, you don’t want there to be obvious, easy choices, like pick a number they’re going to pick one, or they’re going to pick two, like you got to put a little more interesting things in there. So some sort of an ambiguity where students make a choice. And then you want to have at least one of those other original components. So, you know either real world or some level of complexity and meaningfulness, like you want those things. And then you have to have the clear assessment criteria. Like that’s because how do you defend against AI?

Boz: Yeah. But especially for us in the, the alternative grading world, that’s but that’s a given anyways, because that’s a cornerstone of, of our grading philosophies.

Sharona: That’s true. But I mean, that, that was the one of the sort of logistical pieces that we were like, that’s your base level. Which yes, it’s a given for us, but that’s what we called base level. And then you start walking up from there by adding more things. like adding opportunities for students to demonstrate growth and change adding more complexity or more actions, adding some concept of some of the other things. And you just keep working your way up. And the more that you can add some of these things in, the higher the authenticity will be. So it may be that you want to save your highest, most authentic things for wrap ups, towards the end of a term. You know, portfolios, or, or presentations where students are making an argument for their final set of work, or things like that.

Boz: Yeah, and then, You know, I think something else that these do, and this is really going back to Joe and some of his learning targets, which if you’ve not listened to the episode and I’m sorry, I’m a bad host. I should have had that episode number pulled up, but his episode about talking about his learning targets and how he designed them, these, you know, his assessments, even though he, I don’t know if he’s ever used this, phrasing of authentic assessment, but his are, and that’s part of the way he’s able to do his learning targets the way he does them, because they are so much bigger than the actual state standards that we have to teach, like those become part of his evidence that he collects.

Sharona: I think you’re talking about episode six, the anatomy of a learning outcome. The first one he did with us.

Boz: Actually, I don’t think it was it the first one or was it the second? Anyways, we Well, I’ll link both of them. Yeah, we’ll link both of them. It allows you to do these broader learning targets, so you’re not stuck with, you know, 29 learning targets because you’re trying to do one for every single state standard skill that your students are supposed to have. Writing them the way he does them, and the way that these kinds of assessment really leans itself towards, makes your life a whole lot easier.

Sharona: And I want to clarify this is not something you only do in higher ed or you only do with your seniors. We’re looking at doing this in middle school even. I mean, some of the stuff you’ve designed a sixth grade. We saw an amazing one recently about choices to build a sustainable ecosystem of a city or build a sustainable city where students had to make a lot of choices and they were given a lot of data to do to use that. So, And One of the things that really hit me as we were looking at these authentic assessments is, why are we trying to teach these students something? It’s because we are trying to teach them to be educated, thoughtful, concerned citizens of the world. So that real world component, that, that remembrance that we can have, that rather than trying to create an artificial environment that protects us from outside influences in schools, I’d like to see us embrace those outside influences and bring them in and leverage them so that we can, you know, turn out amazing people.

Boz: Yeah, that’s a great point. And I think that might be a good point to start wrapping this one up on. And you’ve got any kind of closing thoughts or comments?

Sharona: I guess my, just as we’re starting this journey, cause you and I are sort of just at the beginning of this a little bit. I’m excited and hoping that I’m going to have time to really design some of these things. I’ve designed performance tasks in the past. There’s an amazing stand performance assessment bank that was part of a project I did with Stanford, but it was before I had gotten into grading reform. So going back and really looking at how can I really build You know, things that I would enjoy taking as a student, things that I would enjoy doing as part of my class, instead of having to stress out about exams or stress out about papers that I don’t care about. I just, I go back to that one example. I also heard of a student taking a business writing class. Where the professor said, absolutely no AI, and the student literally left the class, went to her job that day, and her boss said, hey, you better figure out this AI thing because I think it’s going to be important for your job. So I don’t want to wall us away. I think we are doing our students a disservice. So the more that we can make this a value, given the constraints we have on our time, the more that I think we’re going to enjoy some of this work. How about you? Other thoughts?

Boz: Just, similarly, the, you know, as we really look at the role technology has played in is going to continue to play and increasingly play in education. And the idea of getting away from this belief that, or this philosophy that, you know, the students have got to do all this from their heads off rope, memory, and just these skills that. Isn’t just traditional grading, but, you know, has a lot of those same traditional roots and those same traditional purposes, like, you know, like Dr. Jeff Henderson and so many of others have talked about with, you know, the purpose of school 50, years ago is very different than it is now because we need different types of citizens yet the school itself has not changed that. So I think this is giving us. Like you’ve said so many other times with grading being the linchpin of what made all this other stuff work for you. I think it’s the same thing here. This use of technology, if we can really get behind some of the philosophies of the grading reform and alternative grading, then allowing. will help allow you to get past some of these, Oh, we’ve got to do it this way. We can’t let technology technology is doing it. It’s cheating. No, it’s not cheating. We just need to change the task.

Sharona: Well, and that’s the thing is that I’m actually concerned given the types of students that you and I teach. It’s going to be harder for them to develop the skills to properly use the technology than some of their more affluent peers. We actually have an obligation to get them up to speed on it. So well,

I hope clearly this is a rabbit hole that we could, we will explore cause it ties in so closely. I mean, it’s that first pillar. Right? Clearly defined learning outcomes and marks indicating progress. Exactly. This is the heart. This is the heart.

Boz: Well, I hope you guys have enjoyed it. It was, like I said, although related, a little bit of a tangent as some of our other AI episodes have been, but I think this is a crucial thing to continue to look at. So I hope you’ve enjoyed it. And we hope we’ll see you guys back next week.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode’s page on our website, www. thegradingpod. com. Or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show, or would like to be considered as a potential guest for the show, please use the contact us form on our website. The Grading Podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.

Boz: The views expressed here are those of the host and our guests. These views are not necessarily endorsed by the Cal State system or by the Los Angeles Unified School District.