The Grading Podcast

Bosley: And see when you switch your mindset to that, I think that does open up what you can define as evidence of, you know, evidence of learning. So when we talk about how are we going to assess these learning targets, if you change your mindset to what you were just talking about, I think it becomes obvious that some of the things that we might have been doing in the class that we wouldn’t consider a formal assessment, now you can.

Welcome to the Grading podcast, where we’ll take a critical lens to the methods of assessing students’, learning from traditional grading to alternative methods of grading. We’ll look at how grades impact our classrooms and our student success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach and instructional designer. Whether you work in higher ed or K- 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week, you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Bosley: So hello and welcome back to the podcast. I’m Boz here with Sharona. Thank you for joining us on today’s episode where we really start to look deeper into the first two decisions of our grading architecture. All right, so how are you doing today, Sharona?

Sharona: I’m doing well Boz. I can’t believe that we’re on episode seven already. This is really exciting and I have to tell you, the level of support from the community has been amazing. Like I know when we originally talked about this, I thought, okay, if we got like 10 downloads the first week, I was going to be ecstatic. And instead we’ve had like over a hundred for most of the episodes.

So that’s really exciting and I want to thank everyone for listening, and I also want to encourage people to share because we really want to get the word out there, so I’m excited.

Bosley: My biggest surprise wasn’t the number, but it was the number of people outside of the US.

Sharona: Yeah, I, I went and looked at those and we have some people in Albania.

Did you know that?

Bosley: Albania, India, canada Argentina, like that’s the part that I’ve been the most shocked by. And, and if you are one of those people, thank you. Thank you for, for listening and I hope you continue.

Sharona: I hope you continue, and I would love to hear from some of those people because I have the perception that a lot of what we’re dealing with is a United States, or United States and Canada specific, challenge with these multilevel grades.

I don’t know that this is done the same way in other countries. I would love to hear from some of you around the world and let’s, you know come on the pod and let’s talk about how the grading systems might differ in other parts of the world. That would be really fun.

Bosley: So if someone’s interested in that, how can they reach out to us and, and let us know.

Sharona: So they can go to thegradingpod.com, our website and use the contact us form and we will get back to you and schedule a a call to discuss the possibility.

Bosley: All right, so thank you for that and let’s kind jump into our episode. Actually, before we jump into it, probably should recap a little bit about what this episode, you know, what led up to this episode.

Sharona: Right. So if you’ve been following along, then this might be a little repetitious, but in episode three, we defined alternative grading to be based on four pillars, which are clearly defined learning outcomes, helpful feedback, marks indicating progress and reattempts without penalty. And then in episode five, we talked about the grading architecture and the four decisions there.

Bosley: Those four decisions were how are you going to assess your learning targets?

Sharona: Right? Where do you get evidence of, of the learning? Exactly.

Bosley: Yeah. The second one was: How are you going to grade those assessments so you know, what kind of proficiency, skill or skills were you going to use?

Sharona: Right? And then the last two are, how will your students show sufficient evidence for a learning target? And how are you going to wrap things up? So we’ve already gone into some of these. So in episode six, we talked about the first pillar , clearly defined learning outcomes with Joe Zeccola. And in episode five was where we talked about all four of these decisions, but now we’re going to talk about the first two specifically, right?

Bosley: Yeah, yeah. We’re gonna get a little bit deeper into those first two, and because they do, you know, one has so much effect on the other. But one thing I always want to make very, very clear, grading architecture comes after the learning targets. Like everything comes after the learning targets. Without at least a solid first draft of those learning targets, your entire system will fall apart. So please and I, I will continue to say this on any episode that we’re talking anything about the pillars or, or the architecture, is you’ve got to do those learning targets first. That’s, don’t skip out on those. That’s one of the biggest mistakes people do.

Sharona: Well, and we had that experience. You know, last week we were teaching an intensive on this stuff. You know, we had a group of 25 faculty members for 35 hours doing this, and I, I remember one of our participants, you know, on, on Monday we said, Hey, learning targets, learning targets, learning targets. And you know, a lot of people are like, yeah, yeah, yeah, yeah, yeah. And then on day five someone said, you know, on Friday, well, I finally understand why you said it’s learning targets and I’m finally ready to do this. And that’s great. It’s, you know, it’s totally okay if you don’t believe us just yet. But when you finally do come around to realizing that everything’s based on that, we’d love to hear from you too.

And, the two I, one of the reasons Boz, that we said we wanted to kind of talk about the decisions right now is because there’s such interplay between the four pillars and the four decisions. Like they, they really almost can’t be separated in a redesign process. Yeah. So the two questions that we’re going to be addressing today specifically relate to the third pillar where marks are indicating progress because essentially, progress towards what? Progress towards showing sufficient evidence, whatever you, the instructor define.

Bosley: Yep. And sufficient evidence of those learning targets.

Sharona: So Boz, what are some of the ways, I mean, how do people gather evidence of learning? What does it mean Marks indicate progress and, and what do we define as progress?

Bosley: So, obviously, you know, some of the more traditional ways, which there’s nothing wrong with traditional ways. I know the word traditional is kind of a, a bad word, taboo sometimes on this, on this podcast. But some of those traditional ways of gathering evidence of learning, like, you know, your traditional paper and pencil quiz or test, there’s nothing wrong with those. And those do still work in this kind of alternative grading system. So those are some of the probably most common used.

Sharona: Well, certainly in our discipline. I’d say in other disciplines, like in English, it might be papers or essays or other kinds of things.

Bosley: But however you are gathering evidence now, in a, if you’re using a traditional grading system, those still work here. So I want to make sure that’s clear that anything that you are doing, as long as it aligns to your learning targets, you can still collect in that way. But there is also a lot of other ways that maybe aren’t as traditional or there are things that we do, but we don’t necessarily, because they’re not "quizzes or tests", so they’re graded in different categories, with this kind of system, we can actually pull some of that stuff in and, and still count it for, you know, evidence towards mastery or proficiency of a learning target.

Sharona: And the thing I think that is a little bit of a shock when you first start to do this is the way you look at all of your assessments changes. So let’s say you have a test, right? In a traditional system, at least when I was doing a traditional points and percentages and averages, I spent a lot of time deciding how many points to take off. So I was really focused on the mistakes. Whereas now, we’re not looking for evidence of mistakes.

We’re looking for evidence of learning. So we’re looking for the positive. What do they know how to do? What does a student know? You know, can a student demonstrate whatever action we’ve put in that learning target? And if they, and, and is it enough? Like is it enough for us to put a check mark, a metaphorical check, mark next to that learning target and…

Bosley: And see when you switch your mindset to that, I think that does open up what you can define as evidence of learning. So when we talk about how are we going to assess these learning targets, if you change your mindset to what you were just talking about, I think it becomes obvious that some of the things that we might have been doing in the class that we wouldn’t consider a formal assessment, now you can. A perfect example of this what was it, about two weeks ago, I was at one of the local colleges doing a little two hour mini session on alternative grading. And we were talking about the role of homework. And one of the people there very rightfully said, you know, they couldn’t go away from homework because there’s things that, in their calculus class, types of problems that just are too intensive and too timely, you know, and takes too much time to do it in a 40 minute timed test. So those kind of problems, they still needed the homework to do. And I’m like, okay, but stop thinking about those as an assessment and homework.

That homework is one of the ways you can assess. And that homework now, instead of being called homework and put into the 20% homework pile, no, that’s another assessment. That is another way of assessing that learning target. Just because one’s homework and one’s a traditional test, doesn’t mean you can’t grade them and look at them in similar fashions in this kind of grading system.

Sharona: Well, and I think about that there’s some content, particularly in calculus two for me, that, yeah, there’s no way that a student can realistically do any problem, except there’s maybe five that have been very carefully curated and designed over the years to be doable in a short timeframe. And therefore, those five are all over the internet with completed solutions. So instead, I do a project that is almost out of reach for the students to actually get it a hundred percent right. It’s just a little bit too hard because the algebra is just horrific. But I can look for evidence of specific things within that project, and they don’t have to get the whole project done or correct or those things because the evidence I’m looking for is very subtle, but it just screams out of this project. Things that students do and do not know how to do.

And the other thing that comes to mind is in our interview with Joe in episode six, he was explaining some situations where a student, you know, late in the semester, just phoned it in on some essays. And he looked at them and said, it’s not going to hurt your grade. You’ve already shown the evidence that you can do this like five times over.

I respect you less for having phoned it in. So you can choose to repair my respect by redoing it. But it didn’t have to do with the progress because the evidence had already been achieved.

Bosley: Yeah. So we kind of talked about that, you know, you can still use traditional test and quizzes, you can use projects like you were saying. You can even use things that might’ve before been "homework". What are some other ways that you can actually assess learning targets?

Sharona: Well, so many ways. Some ways that come to mind that have happened for me is a student conversation. So either with me in the classroom or in my office hours, or even something I overhear in group work in the classroom. If I hear a student explaining a concept to another student, I literally could mark that down and say, that student knows how to do that.

Bosley: And I think that, and that’s the one I want to spend a little bit of time talking about. I was listening to another podcast earlier today, actually someone that I’m hoping to have on this podcast, but they were talking about math anxiety and how socially acceptable it is in this country. And I’m thinking about all of my students that I know can do something and then when it comes to the test, They just, they, they freeze up. They lock up it, it’s, you know, they just, they’re so convinced that they’re bad at math and have such high math anxiety that why can’t we use actual conversations?

Where is it written that if a student can explain it and a student can demonstrate it, you know, with you one-on-one or, or in a group setting or in front of a class, but for whatever reason freezes up on that test, that that student should be penalized to the point of, no, you don’t get to pass this class because you can’t do it on a test, even though you can come back and explain to me how to apply the first derivative or what the second derivative tells you about the motion.

Just because you can’t do it on a test. So there are, and I’ve seen people get really creative with how they do, you know how they assess things.

Sharona: Well, and there’s two things that are coming to my mind when you say that, and one is looping back around to clearly defined learning targets. Are you assessing the learning target? Or are you assessing something else? Are you assessing a student’s ability to overcome anxiety? Are you assessing a student’s ability to do the problem within a given timeframe? Are you assessing the student’s resources? Do they have a tutor? Do they have access to chatGPT?

Bosley: Are you giving bonuses for people that are quick processors and giving penalties to those that are slower processors?

Sharona: If you’re a math instructor or you are assessing their ability to read and follow directions. And then that brings me, I just mentioned chatGPT, which has, of course taken the academic world by storm. I also want to know, am I assessing my student or am I assessing an AI?

Sharona: Or their access to a symbolic algebra system that they happen to know how to use or their access to a mother who is a math professor, for example. Not that I would know anything about that. Yeah. But yeah, so exactly. Like we can get very creative. So, you know, office hours, conversations with students, different types of projects. What other things do you come to your mind?

Bosley: Yeah, those are kind of some of the highlights. Another one is portfolios, the use of portfolio where you give the students some agency and some choice about what they think demonstrates their own level of proficiency or mastery. And I know we’ve shared this resource in some of our trainings and stuff but you have a couple of websites that kind of do those projects or show how those projects can be done.

Sharona: Exactly, and I know that as a math professor, when I hear the word portfolio, I was kind of freaking out because I’m imagining like art class and you know, oversized folders and things like that. So I got involved, a few years ago I was trained on the Tilt Higher Ed project, the Transparency in Learning and Teaching project.

It’s http://www.tilthighered.com, and we’ll link that in the show notes, but they have an example of a mathematics portfolio that is a set of curated problems from a textbook, and students have choice to pick one of these and one of those and one of these and write them up. And there’s an example of what a fully formulated solution looks like.

So it’s a lot more robust thando all the odd problems from section 4.3. But it’s not like an art portfolio. Like there’s a middle ground there.

Bosley: Oh, actually, I, I think the similarities between that and an art portfolio, actually there is a lot of similarities because, you know, working with some of the art teachers at my local high school, having students do a bunch of different kinds of artwork and then letting them display what they think best showcases their art ability is very similar to what we’re saying here with a math portfolio. And what’s interesting with those portfolios, you can learn a lot more about the student’s actual level of mastery or proficiency of a learning target with what they choose to show, even right or wrong, like just with what they’ve, you know, decided to show you and what kind of things did they.

Picked to put in there, what kind of things did they purposely choose to leave out? You know, am I always leaving out something that has fractions just because I’m a fractophobic and, and they’ll scare me right away? Or, you know, am I, am I staying away from one particular type of derivative, trying to stay away from the chain rule or, so those kind of things.

I think the portfolio is really underused and can be very powerful in a math class.

Sharona: And then the other thing that I have done that I like, I think portfolios are amazing. Not just in math, in in all sorts of, I mean, I can imagine in a foreign language class using a portfolio where there’s a poem translation or other kinds of things involved in that portfolio could also provide a lot of evidence.

Mm-hmm. And a lot of our students now have tools, access to tools they didn’t used to have. So now that most of them have been on Zoom, they know how to screen record so you can get amazing presentations and verbal descriptions from students. And that’s another way where a student who may freeze up in class can use several takes to prerecord and have their work looked at by their peers.

So I do think that we have a plethora of assessment styles, and this is an area where I feel like I’d like to see the different disciplines come together. Because it didn’t used to make a lot of sense to try to learn from, to me at least, to learn from the history professor how to assess something or to learn from, you know, a Spanish language professor or a chemistry professor. Our disciplines seem so different, but now if we look at these assessment styles, I mean, maybe there’s a lab report that you could do, you know, do some sort of an experiment in a math class or something and a lab report could get written up. I’d love to learn what the criteria are for a good lab report.

Bosley: Yeah, so I, I think the overall here is we can actually get really creative and there’s a lot of different ways that you can assess your learning targets. Which way is correct? Well, that really goes back to. You know, your style as a educator, the purpose of your class is it in a sequence? Is it not in a sequence?

Your clientele, your students, all of those are very personal decisions, and that’s one of the really nice things about alternative grading is it does give you so much more freedom and it, you can really internalize it and make it yours. And, and you know, you mentioned Joe earlier.

He was talking about how he did that with his learning targets and how it’s changed his pedagogy. Just having a grading system that just mirrors his own beliefs and you know, what he values as an educator. And that’s what alternative grading, when done right, will do for an educator.

Sharona: Now I want to throw in a little bit of a caution here though. We just mentioned there’s a million choices. That’s great. The con is there’s a million choices. So one of the things in our trainings that people start to get to is a point where they’re like, I just don’t know how to choose the right one. And the answer is – You don’t. You just choose one.

You choose one that appeals to you. It doesn’t have to have any rhyme or reason. It just has to be one that you can logistically manage and try it. Try that one first because there are too many choices.

Bosley: Yeah. Kind of like the title of the DuFours’ PLC book, Learn by Doing. Just Yes. Just forget trying to make it perfect. Forget trying to weigh out everything that come up. Pick something that feels okay enough and jump in and see what happens and learn by doing.

Sharona: I mean honestly, if you had three different choices, I would probably use a dartboard. I just as long as there’s three things that you can logistically do, and we’ll definitely be talking about logistics.

Logistics are king here for the instructor, but as long as you can handle the logistics, pick one. If you don’t like it, pick a different one the next semester or the next term. I can almost guarantee that no matter what you do, it’s going to be better than what you were doing before. Like as bad as, as bad as your choices might be, the fundamental philosophies behind the grading system inherently are just usually a lot better. Now, I do have one other caution though. With all these choices, you do have to have at least a little bit of an equity lens, so you have to think about the repercussions if your students either don’t have access to resources, which could include time, it could include money. Also, what about identities and power in the classroom? So if you choose something, it’s like, well, I’m gonna let the students volunteer to present. Well, what happens if you have a student for whom that’s never going to happen? They are never going to raise their hand and volunteer. So you do have to think through some of those things as well.

Bosley: And, and I did have that student a couple of years ago in one of my college classes. Like it, it almost, it almost brought her into an anxiety attack just to have to answer a question in front of the entire class. It was all she could do to muster up the courage to come up and talk to me afterwards.

So I do have a project, a presentation in my class. So yeah, what did I do? I let her record hers and show the recording instead of having to get up, because it was so crippling, terrifying. For her to try to get up in front of a, a group of people. It was, you know, a combination of a high math anxiety and a very high public speaking phobia.

Sharona: And so that speaks to, when you’re looking at your types of assessments, you should probably have at least two or maybe three different styles of assessments for any given learning target. There should be multiple ways that a student, at least as an accommodation, if not just built in from the beginning, can demonstrate that they can do this. Because there’s pros and cons to literally every type of assessment.

Yeah. So we’ve talked a lot about the types of assessments, but I did want to say something here about the alignment of those assessments. So I know that a lot of people, when they start redesigning, they’re like, well, can I use my existing tests? And my answer would be yes and no. I would caution people that you still have to make sure that the stuff that you’ve been using actually provides evidence for those clearly defined learning targets.

Bosley: Yeah, your assessments need to be aligned with your learning targets to make sure you are assessing not just what you want to assess, but also what you intend to assess. That’s one of the things that we have discovered is oftentimes we can write, especially in math, and I’m sure other disciplines, but we can write assessments that actually end up assessing something we didn’t mean to assess.

Sharona: Exactly. And so what I would encourage people to do is, at least in mathematics, but I suspect in a lot of test-based assessment environments, to not consider the test as a whole what you’re trying to reuse, but maybe the questions from the test, because the test as a whole might be designed to gather a bunch of information in conglomeration, but when you start to break it apart and you’re like, oh, I am not assessing this target at all, and I’m assessing this other target 16 times, and I really don’t need 16 proofs that they can do this thing. So even reusing your material, you might need to break it up and restructure it.

Yeah. And I personally have gone very much away from a, like whole class assessment timeframe. I know there’s a lot of other people who still use it. I know our engineering folks and, and you know, Kate Owens, whose interview we’re going to have coming up also I believe still uses whole class test periods. I personally have broken mine down into basically micro assessments.

But it’s just, just be aware of that alignment issue. And you may have to reconstruct or or redistribute what you’re currently using.

Bosley: And this also, when you start to do this, might make you aware of things that you do want to assess that are important to you, that you don’t actually have written out in a learning target.

So I always say your learning targets need to be first, but that doesn’t mean they’re set in stone. And throughout this process you will go back and forth and revamp and, you know, redesign some things. So when you’re looking at your assessments, you might end up realizing, oh, I didn’t realize it, but I do assess this a lot and this is important to me. I think it should be part of the grade. Go back and add that as one of your learning targets. Just because you’ve moved into the architectural design part doesn’t mean you can’t go back to the learning targets, even though you do need that solid, first draft before you go to architecture, you can go back and forth of course.

Sharona: And I know this came up for me, and it’s come up for quite a few of our STEM folks, when they start to define these learning targets, they’re like, well, but I don’t want to give away the problem solving method. And so my argument is, well, there’s two pieces to that. There’s, can they do a specific method and there’s can they choose the right method in a given situation?

So I actually have separate learning outcomes or learning targets that are specific to various problem solving skills. So, for example, in advanced integration techniques, I have a learning outcome that basically says I can pick the most efficient, or most effective, or correct, or whatever it is, strategy, advanced integration technique to solve this problem. And so they don’t necessarily have to solve the problem, they just have to pick the technique and set it up. And what that does is that opens up the universe of problems for me because again, there’s a lot of problems that are just way too tedious to solve in the context of a timed exam.

But I don’t want to be restricted to these super simplified, algebraically, ones. I want to be able to give more complex problems and therefore I can just say, "Hey, set this up, you don’t have to solve it." And it gets me the evidence I need of the problem solving, but I can assess the actual ability to do the thing on a simpler problem.

Bosley: See, and that reminds me, there’s something that at my high school we used to do in an algebra two class where we would actually have "math court" on which is the most efficient way to solve a matrix or a system of equations. You know, is it Gaussian elimination? Is it substitution, is it graphing? And we’d break the different classes into groups and have a court style debate about which one is better. And I mean, you want to talk about really understanding the student’s level of proficiency or mastery? Have them act, forget having them solve the problems and go through the single steps and Oh yeah, they might, you know, two, two times four and get six, so it screws up the whole problem, put ’em in, have that debate. I mean, you talk about really understanding the, the levels of understanding. Like I, I loved doing that. We actually even, we’d take the best teams for each of the different Algebra two classes and have them go against each other in front of like all of the, that grade, that 11th grade class. And it was one of the most fun things that we used to do.

Sharona: Well, and I can imagine doing this in like a foreign language class, right? Trying to have them argue in the language. Yeah. You’re going to get a lot of evidence of their ability, their fluency on some of the language.

And you know, Joe mentioned this, too, when he was looking at his, when he was sharing some of his proficiency scales, he’s actually gone away from actually assessing memory of specific content, specific novels, specific essays, and all of his learning outcomes are, you know, rhetorical argumentation and…

Bosley: Yeah, it’s all those writings, actual writing skills. And it’s not that he doesn’t find that stuff important, you know, the, the details of the texts or whatever, it’s just he doesn’t need the learning targets on it because those come up in the writing. And what he’s really, you know, concerned about is can a student take a piece of literature and synthesize it and make an analysis or an argument or combine. So it’s not that he ignores in that their students, you know, if they were reading, I don’t know Shakespeare or Tom Sawyer or something, it’s not that they don’t know details about it.

Sharona: They have to, yeah, they have to use it.

Bosley: Yeah, they have to use them, but he’s not going to have otherwise, you know, he would have 30 learning targets and then it’s starts to become unmanageable. And that’s not the important stuff to him. He doesn’t really care if someone leaves his class being a lifelong learner of Shakespeare or lifelong lover of Shakespeare. It’s, he wants him to be a lifelong writer. So that’s what his learning targets, that what’s all of his grading architecture is built around that value. And that’s what I meant when I said earlier about you’re able to take alternative grading systems and set them up in a way that it really values what you as the educator values.

Sharona: And, and the content is the vehicle. A lot of times the content is the vehicle by which they demonstrate the ability to problem solve, the ability to write, the ability to be numerically precise, all of these different things.

And I just think all this stuff aligns so much more with my ultimate goal for a student, which is to become a lifelong learner and to be successful in their own personal goals and aspirations. And that is that asset-based thinking coming in. You know, we, we’ve heard a lot about don’t be deficit minded.

And it’s so hard sometimes in mathematics to say, well, if they can’t do algebra, what am I supposed to be looking at? And the asset-based framing, which is what do they have? And what they have are goals and aspirations, and I can’t promise them that this particular mathematical skill is something they need. I can promise them that the problem solving skills they learn in my class is something they need.

Bosley: Absolutely. All right, so we’ve talked about the evidence. Let’s go into that second decision of the grading architecture, which is around how are you going to look at those and grade those. You know, what kind of proficiency skills are, I keep saying proficiency skills instead of pro proficiency scales, what kind of proficiency scales? And just kind of a brief reminder. Like I said, if you want more details about this go back to our episode on grading architecture, which was episode four. But let’s…

Sharona: actually it’s episode five.

Bosley: It’s episode five, yeah. Okay. So let’s kind of review in, in that episode we, we talked specifically about three different types of proficiency scales.

Sharona: And I think what I wanted to mention here, Boz, is we’re going to first probably go a little bit deeper into those three types of scales, but there’s also a distinction between the choice of scale you make and how you actually describe the different levels. And I think we’re going to do both of those things. Yeah. Right.

Bosley: So let’s look at those different proficiency scales. So the first one we talked about was the two level, which was just, did they, they got it or they didn’t.

The second one we looked at was the three level, where there was one level of yes, they got it and then two levels of, no they didn’t, but one of them was, no, you didn’t. The other one was, you’re close. You didn’t, but you’re close. And we also saw a variation of that, that I actually used last semester in my high school class, which was, yes, you got it, no you didn’t, or you are close enough you just need to go back and revise these little mistakes. And then we also talked about the four level, which is probably by far the most common. That’s the two levels above and two levels below. So those are just kind of briefly, those different scales that we talked about in our episode five.

Sharona: Exactly. And I think where the rubber meets the road in these different ones is proficiency scales get used in different ways in the sense of whether you’re doing more of a, a standards based system where each learning target is the goal of the mark. Or you’re in a specifications situation where the entire assessment has a whole series of marks combined, or other options.

You know, in the ungrading world, it may just become strictly communication to the student in the interim. So there’s a lot of different ways to use each of these proficiency scales. And when we spoke to Joe, he also had a lot of descriptors. So each of his proficiency scales was specifically tied to a learning target, whereas in my world, I sort of have one proficiency scale that I use for all the learning targets. So I think it’s interesting to kind of think about those differences.

Bosley: One of the nice things about, and I, I don’t do this, but you know, when you go more descriptive into your levels of, on your scale, then that single mark actually helps give feedback to the students. So when you do something, when you have things set up, like Joe Joe was talking about in his class, the student knows more about you know, has more feedback from that score of a two than, say, in our class, the scale, like we have to give more personalized feedback to explain that to, so there’s payoffs and, and.

Sharona: Well, and you’re calling it a two, which I’m going to push back on anyway. Yeah, I know. That’s what Joe does.

Bosley: And, and I mean, we haven’t talked a lot about the numbering, so most people on a four level understand what a two is. But yes, we, we don’t like to actually use any kind of numbers at all because as soon as you put a number on it, a student is going to try to turn that into a points game.

Sharona: So we’ve done everything from use words, some people use things like exceeds expectations, meets expectations. Some people use. What is it jedi Master and Jedi Knight? Yours?

Bosley: Yeah, it’s youngling, Padawan, Knight and Jedi Master.

Sharona: And I use emojis because I could not find any words that didn’t come laden with meaning. So I use a check, I use a hand with a pencil for my revise one, I use the yellow smiley face with the thinking pointing finger to indicate that they need to continue to learn ,and I use a red X for the lack of evidence. And so there’s sort of, those are people that are using electronic grade books, which is almost everybody. I want to emphasize there’s a distinction between the level as you communicate it to the student and whether or not you have an L M SS system that supports words or emojis, or if you do have to slap a numerical.

Categorical label on it, but I would like to reemphasize. These are categorical labels. They are not numbers that you can do math on.

Bosley: Yeah, and we, and we talked about this before, but there’s a great blog by Robert Talbert about "when is a number, not a number" that actually, you know, I think we brought that up in our problems with traditional grading, our second episode where he, and we’ll definitely link it again in the show notes. But he does a great job from a statistician’s point of view of why averages in grades like this doesn’t make sense because of the type of variable we’re actually using, categorical, even if you’re putting a number on it.

Sharona: Right? I think this also comes into play when you start to look at the structural differences of an assessment. That it is being marked on individual learning targets versus an assessment that’s being marked a little bit more holistically. So for example, in my history of math class where I have these projects, I sort of have a weird hybrid where a student has to get a successful mark on at least one of my content learning targets. But then they also have to be successful on a whole bunch of specifications that are specific to the project and the project, there are things like writing skills, clarity of writing, errors being significant or not, is the mathematics correct? So I have a whole series of specifications.

And so in order to complete the project, you might have eight things that are on a two level scale of complete or not, and they have to get all of those complete. And then I have like nine or 10 possibilities where a student has to get one of them, to a successful mark, at least one on a three level.

So it’s like a whole complex combination of things that ends up on the final overall rubric for the assessment.

Sharona: Right. So what I would encourage people is to start with figuring out your proficiency scale for your learning targets, and then as you begin to build your assessments, think a little bit more holistically about how you are going to combine things to check the assessment. So if the assessment is a test and you’re just doing separate learning targets, then maybe you just use the proficiency scale for that target.

But if you have a project or you have a portfolio, or you have, I’ve seen this a lot in computer science classes when they have programming and they’re like, well, at the end of the day I want all these things in a program, but I want the program to run.

Bosley: Yeah, you want it to do what it’s supposed to do.

Sharona: So, you’ll probably end up revisiting your proficiency scales in a couple of different avenues. One set for the learning targets, but then really looking at it for the assessment as well.

Bosley: Alright, so but you said you should decide what kind of proficiency scales to use for your learning targets.

Is there better or you know, better scales to use for better situations? Or how does one go about choosing which of those proficiency scales to use?

Sharona: So I think it very much depends on what you’re going to do with the information from the scale. Right? So in other words,

Bosley: I, I would actually push back, I would say it actually that you’re not wrong, but it actually starts with that first decision you were making in the architecture. About what kind of things are you using for assessments?

Sharona: Right. I guess that’s what I meant by how are you going to use it? So are you going to use the mark to tell them to revise it because it’s a type of assessment that could be revised, like a project? Or are you going to like, that’s where I guess I was going with the, what are you going to do with it?

Bosley: So, But I, I do think that, Even though there’s no right answer or wrong answer because this can be so customized and personalized to the educator and the need of the class, I do think there’s instances that one serves better than the other. So we kind of already talked about the projects and the specs really pairing up nicely with the two level, the Yes, you got it, no, you didn’t. I also believe that anything that could be revised, that three level is a really good pair. Yes. And anytime you are, there’s a purpose, a need or a reason to distinguish good from great, four level. But one thing I would say is don’t have any more levels on your proficiency scale that you have purposes for them.

Like if I don’t have that meets and exceeds distinguished in my grades, I don’t, like I don’t do anything with that? So it shouldn’t be on my rubric. Like I should have a three level scale. And I do in most of my classes, but the Cal State is obviously different because that’s a coordinated class, we all have agreed upon teaching, you know, teaching in, in a grading the same way. But if I had that control of that class, I would be using a three level. because there’s no purpose of that fourth level. And that’s my point is however many levels you have, there should be a purpose for each level. And if you have lots of purposes and you want to have a five or six level rubric or a proficiency scale, you want to go to that, you know, the old AP nine level scale? If you’ve got a purpose for each of those levels, do it.

Sharona: So, and I would argue, since I am in control of that class that has the four levels, is purposes are not always just for the student, and that’s okay.

There might be political purposes. So in our case, because we are a coordinated class, we are accommodating the personal grading styles of many instructors. And as we said at the, at the beginning of this, of launching this podcast, grading is highly personal, highly relational. So we have the four level scale because we have some instructors for whom that relational need to distinguish between good and great is very important. And so we acknowledge that. While not using it in the final grade wrap up. Yeah.

Bosley: But it still has a purpose. It has a purpose. And that purpose is to make the coordination of this, you know, class with 15 different instructors with all of our different personal, you know personal styles. Work. And that is a legitimate purpose. And if you’re, if anyone that’s listening is doing anything in a coordinated class, that might be the exact same reason you need to use a four level scale is just to make those different personalities able to work together.

Sharona: And now, you know, we lot, oftentimes we talk about the four level in the, in the metaphor of hurdles.

You know, are you getting more hurdles or higher hurdles to accomplish a certain grade thing. I have to admit that until I spoke to Joe, like just my personality and my institutional experience and my sort of my location in the educational world, I just was never a four level person. I was like, you know, look, my class, either you get the material in my class or you don’t, and you know, I teach in math. I didn’t really see how I could distinguish quality of acceptable work. Either it was acceptable or, or it wasn’t acceptable. Joe made me understand that in his context, which is K -12 and English that he needed to track a good versus a great, because a great became his responsibility to differentiate instruction for that student. I don’t do that in a 15 week semester at the university.

Whereas Joe’s was more about the skill of writing that doesn’t begin and end in that one year or that one American Lit.

Sharona: It may not end in a lifetime.

Bosley: Exactly. But you know, it doesn’t start and end American literature and then next year you’re goingonto something else. It no, it’s writing. Period.

So, having that four level, having that need to distinguish good from great ,did have a real purpose. And the purpose for him is, even though it didn’t affect the grade, it’s, that’s his indicator of "I’ve got a student that’s already got it that I need to different differentiate my instruction and push this kid further."

Sharona: Right. Because he has that responsibility, that is part of his mandate. Yeah. Whereas it may not be part of mine.

Bosley: Yeah. And that’s, I think that’s more of a difference between the, the K- 12 world and the higher ed world than a math versus English. Yeah. Although some of it might be there too.

Sharona: It’s probably a combination.

Bosley: Alright, so we really looked at, you know, kind of gone in depth with the different variety of things we can use for assessment, which is that first grading architectural question. And then looked at the different ways that we can assess that level of mastery, which is that second question.

Sharona: And both of these are, In this pillar three. Marks indicate progress.

So ultimately the goal of these two things, I think, is to communicate to a student where they are in the path of completing this particular requirement. Yeah, right. That’s the goal.

Bosley: All right, so in a future episode, we’re going to continue this and we’re going to go deeper into the rest of the grading architecture. So like we paired one and two up, we’re going to have another episode where we look at questions three and four, which is, "How do you know when a student has achieved proficiency or mastery in a specific learning target?" And then "how do you take those collection of learning targets met or not met, and wrap that up into a final grade" since we’re not, you know, doing percentages and points and just can’t put things into a, a spreadsheet and get a 89.3 and call that a B.

So that’s, that’s gonna be one of our further episodes that kind of ties these things, this grading architecture together.

Sharona: And I’d like to put a call out to our community. I would love to hear about how different people describe their proficiency scales, especially across disciplines outside of math.

I think if we got some submissions, I’d love to either read them or have people on as guests, you know, how are you doing your proficiency scales if you’re currently doing alternative grading? Because I think there’s a lot of value to hear the different language that’s used in different disciplines.

Bosley: And, and kind of building on that another thing I would like to call out is even before that, how did you come up with your learning targets?

We had a really, you know, interesting interview with Joe where, you know, he had a very unusual situation where he could take a student teacher, someone that had been observing him for a while and they went together and he basically, you know, bought her a meal and said, let’s sit down and let’s talk about me as a educator. What you see that I value for, you know, looking outside it, that was a very unique way of doing it. I’d like to hear from other people and other disciplines how they came up with their learning targets. Because you know, like you said, we were doing this 35 hour intensive training last week. And we did, we had a few people that, it was Friday, it was the end of the week long process and they were saying, okay, I finally get it now, and now I know where to start going with my learning targets. So I, I’m curious to see how other people have done that.

Sharona: Well, and then I’m going to make that a trifecta because I also want to hear how people are gathering evidence of learning. And again, you know, we’re pretty well connected, especially within the mathematics alternative grading world. But I’d love to hear from other STEM folks with details. I’d love to hear people outside of STEM. How are you doing this? Because what we have found is this works better when we steal the hubcaps.

This works better when we find out someone else is doing something and we adapt it for our purposes. So a, a big part of this podcast is to continue to build this community, and we can’t do that without you. So if you could, you know, send in, use the contact us form on the website, send in some examples, some ideas.

We will of course credit you. We may have you come on the pod and, and discuss your experience, because this is going to get really boring if people are just here listening to us week after week after week. So I’m pretty excited because we’re going to have another interview coming up soon with Kate Owens.

Bosley: Yeah. And another good friend of the podcast you know, one of the original organizers of, of the grading conference, which is kind of where this podcast was born from as well. So yeah, really excited to, to hear from her and to have her on.

Sharona: And then we do have one more guest already slated in the calendar, and that’s David Clark, the co-author of Grading for Growth and the final part of the original organizing committee of the grading conference, and we’ll get to hear a lot from him. And, and he’s had some great writing as well. So we’ve got those. But you know, we want more. We want more. And we’re excited to bring a lot of people on and to, to get the scope, this stuff’s not easy to talk about, there’s just so much. So we want to piece it out there and have, have different parts come together.

And, and if you’ve noticed some of these episodes, some of these first few episodes are labeled getting started, you know, parts 1, 2, 3, 4. I don’t remember which part this one’s going to be. I think this might be part four, but If you know someone who’s considering alternative grading really hasn’t dipped their feet in, you know, recommend those episodes that are labeled getting started, they’re, they’re a good way to kind of process through this, in my opinion.

Bosley: All right, and with that being said, I think one big thank you to everybody and this should bring this episode to wrap, so we’ll see you next time.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode’s page on our website, http://www.thegradingpod.com. Or you can share with us publicly on Twitter, Facebook, or Instagram.

If you would like to suggest a feature topic for the show or would like to be considered as a potential guest for the show, please use the Contact us form on our website. The Grading Podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.