The Grading Podcast

106 – robert-talbert_spring class revisited

===

Sharona: So Robert, I wanna follow up then on feedback. So did your feedback loops work, which is a burden on you? Yeah, it’s very quick and it’s on the students. So how, how’d it go with the feedback loops?

Robert Talbert: Well, you know, first of all, I, I, I firmly believe that most pedagogical problems are really communication problems. And so when you know you’re going into a six week class and you are a student, I mean, I had to really get with my students as much as you can get with quote unquote anybody in an asynchronous course. It’s like, this course is gonna go by at a million miles an hour and you’re not gonna get partial credit on things, but you’re gonna get, this is how the grading system works and explain why, and blah, blah, blah. In a six week format, you’ve gotta iterate so fast and you cannot mess around with this. You have to take advantage of every opportunity that you get to reattempt something that you have been explained to that you haven’t mastered that. So more so than ever, you have to really work on this feedback. Don’t put off the feedback. So, I mean, this is a communication issue. First of all.

Boz: Welcome to the grading podcast, where we’ll take a critical lens to the methods of assessing students’, learning from traditional grading to alternative methods of grading. We’ll look at how grades impact our classrooms and our students’ success. I’m Robert Bosley, a high school math teacher, instructional coach, intervention specialist and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I’m Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach and instructional designer. Whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week, you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Boz: Hello and welcome to the grading podcast. I’m Robert Bosley, one of your two co-host, and with me as always, Sharona Krinsky. How are you doing today, Sharona?

Sharona: Well, overall I’m well. However, I’ve seen a couple of videos on social media of the last week or so about teachers panicking in July, about August coming. And I am feeling that so much because I had to list for someone else what I’m doing for the next. We have about four weeks, a little bit more till the semester starts and I’m giving two talks running, three summer camps, re not redesigning, but working on three pre-calculus courses, traveling twice, going to Math Fest. So yeah, I’m kind of in full blown panic mode when it comes to the fall semester.

Boz: It’s funny you should mention those videos. I came home from going shopping the other day. And I went up to my wife and I’m like, they have back to school stuff already up and sales up. And this was early July and I’m like, this is too early. Since then, she has been sending me videos of people going and educators making jokes about going into stores and seeing back to school sales before July 4th.

Sharona: And having traumatic responses. Yeah. So speaking of the summer, we have someone with us in the studio. So welcome back to the pod, to the virtual studio, Dr. Robert Talbert. How are you doing today, Robert?

Robert Talbert: I’m good. Sharona, good to see you guys again.

Sharona: You don’t have the panic to August thing going on. Right.

Robert Talbert: Yeah. You know, I never panic about summer. ’cause I map it out. I’ve got a whole thing in my other blog called Intentional Academia about how not to waste your summer and that sort of thing. And plus I’m on sabbatical this fall, so like I’m just sitting back watching people panic, thinking like, yeah, that sucks. Oh, oh man, I’m just gonna be writing, I’m gonna be hanging out exactly where you see me now on the video here in my home office, just typing away. , I’ll be working on the second edition of my flipped learning book. But otherwise, you know, it’s all good.

Boz: But, your summer just recently started ’cause you were doing an intercession.

Robert Talbert: Yeah, kind of. I, did a six week we have, so here in Michigan, you know, it snows so often that we don’t actually have a spring semester. We have a winter semester, and that’s what goes from January through April. It’s still winter. And so we have two six week sessions in the summer. One called Spring that runs from May through the middle of June. And the other one called Summer, which runs middle of June through basically the beginning of August. And I did the spring term. So normally we’re done at the end of April because we start really early. But I taught a six week course May to June. So otherwise I would’ve started much earlier than that. It feels like it did just get started.

Boz: Well, and that is in fact why we’ve asked you back to the podcast. So back on episode 87, we had you on talking specifically about the six week course. And some of the adjustments that you’ve made to your course, you know, for both the environment that you were in in this condensed timeframe, but also looking at some of the impacts that AI has had in some of the decisions you’ve made. So we wanted to bring you back on as kind of a recap as to, you know, when we had you on, you were in the class, you were actually like in the middle of it. So we wanted to kind of see how things went and what was funny is independently. We did not realize this when we invited you back, but you had just posted, or you were about to post a blog on your blog Grading for Growth about the exact same thing. So it was perfect timing.

Robert Talbert: Perfect timing as always with you guys.

Boz: So we did talk quite a, quite a bit on episode 87 about, you know, what you had done differently with this course and things. So I don’t wanna get into a ton of detail about that. But if you could do kind of a little bit of a recap of some of the highlights, especially the differences between how you ran this condensed, or what you called hyper speed or hyper drive course, right. Compared to what you would normally have done.

Robert Talbert: Sure. So just a quick recap for those who aren’t familiar. So this is a class that I teach very, very often. It’s a first semester of a two semester sequence on discrete mathematics for computer science majors. It’s the foundational math that all computer science and cybersecurity majors at my university need to take. And I teach this class it’s called Math 3 2 25. And so you’ll hear me talk about 2 25. I teach this class almost every semester, love teaching it. I’ve got a lot of materials developed for it, so I just kind of prefer to stick with it. And you know, ordinarily you, I teach it in a normal semester context, which is 15, 16 weeks in person. This time I had the opportunity to do it in, like you said, the six week format, and it was asynchronous online. And that was a big, big difference. And first of all, of course, asynchronous is its own thing. It comes to pedagogy and course designs a very challenging modality. Online, of course. No class meetings, no nothing synchronously. And it was crunched into a six week format, which a number to keep in mind is the number two and a half, which is the scaling factor to go from a normal semester course to a six week course. And so. A number of things I had to reconsider doing first of all back in the winter semester, which again, for us is what most people call spring semester. I had changed the assessment structure of the class so that all of the major assessments were done through in-class time testing. I think we talked about that the last time I was on Yeah. On the, on the podcast. As a response to try to triage the AI situation, which I felt worked really well, but I knew at the time in an asynchronous format that’s not gonna work. I mean, there’s just, it’s a non-starter. There are no in-class anything. There’s no class. I mean, of course there’s no class. And so I had to think about to what degree am I going to have to redo my entire scheme for assessments here? And so there was that. The, the very tricky problem that every asynchronous class has, regardless of whether you’re doing alternative grading or not, which is to simply how do you foster a sense of belonging and a sense of presence with your students? And how do you get students to come to believe that they are actually participating in a course when there’s no class meetings which is a, a real issue. So between the grading, you know, and the, the entire sort of social learning structure, you know, you know, other than that it was pretty much the same course other than like having to redo everything about it. It was all pretty much what I was used to.

Boz: So I, I’ve got, I’ve got a question because I have taught in a condensed, you know not quite six weeks. Mine was a, a condensed eight week class. And I have taught hybrid, not completely asynchronous, but hybrid, and I have experienced both of those as student as well. Which of those two was actually the thing that you had to change the most for? Was it the, the condensing or was it the asynchronous formatting?

Boz: I really ask that question because and I, I’m gonna admit something that I’ve not said before, but everyone like this is a big thing at every grading conference we ran is how do you do this in large classrooms? My biggest fear is actually doing this in a condensed class. I have taught condensed classes. I have never done it after I’ve transitioned, you know, into alternative grading. And I, quite frankly, unless I am forced to do so, that is one of as an educator, one of my bigger fears is. I don’t understand the way I have my classes set up now, whether it’s my high school or my college classes, it would be a major restructuring of some of my architectural decisions. Like I, I don’t know how I do what I normally do in, in that condensed class, so I am really eager to. Like everyone else is always worried about, you know, how do you do this with 300 people? I’m like, my biggest fear is how would I do this in a six or an eight week course when I’m used to doing it in, you know, 16 to 20?

Robert Talbert: Yeah. And some people do it with 300 people in a six week class, which is like, oh, I don’t know. I don’t even understand how that’s possible. But that’s a great question. I, I think I’m with you on that too. I mean, in my particular job teaching a 300 student class is probably off the table, but I could very well see myself doing this over and over again. When I sat down to think about this. Condensed course. I mean, I guess it, I approached it like any other major project. I wanna first of all think about what isn’t going to change, right? I mean, when I think about the four pillars model that we wrote up, David and I wrote about in the grading for growth book, like you still have clearly defined standards, okay? Having clearly defined standards is independent of class size and it’s independent of class duration. So you still do that. I mean, you could, that that’s something that you just kind of set it and forget it. You do it once before the class starts. You write out the standards and you have them ready to deploy whatever it’s makes sense for your class. And that does, that doesn’t change. Okay? So that’s good. Okay. Having marks that indicate progress, that doesn’t change either. That’s, that’s part of the decision you make before you run the class, whether you’re gonna do it ungraded, so there’s no marks or you, now that’s a big decision. Like how do you, if you’re, I wanna talk about this a little bit later, about the notion of un grading in a, in a condensed course. But you make that decision once at the beginning, even before the semester starts, is you’re writing a syllabus out. So that’s not a big deal. That doesn’t really change anything or is changed by the, by formatting the class, what really kind of, what really hits is in the giving of feedback and especially the re-attempts about penalty. ‘Cause when you’re giving RET attempts about penalty in an alternatively graded course, there’s this sort of underlying assumption that there is sufficient time to engage in that feedback loop. And so six weeks is not a whole lot of time to engage in an entire course’s worth of feedback loops, especially when you think that some of the material is coming in in the sixth week of the class . So the way our spring term was set up, or is set up is there’s six weeks of class meetings, and then there’s this little three day half week at the end that’s supposed to be for, there’s like one day of class that you can do whatever you want with, and then there’s two days you’re supposed to be for the final exam. You know, we were covering significant material in that class right up until the fourth day of the sixth week. Like, how are you supposed to give enough time for a feedback loop, you know, on that material? Because, I mean, you should not privilege any one particular point in the class in terms of the amount of feedback. Like students should have sufficient chances to do reat attempts without penalty at any point in the semester. But the simple fact of a condensed course is you don’t, I mean, you have way more opportunities to practice stuff that happens at the beginning of the class than you do that happens in literally the 11th hour of of, of a six week class. That to me, was biggest pain point for me in, in making that work out. And if you’ve read my stuff before, you’ve probably heard me talk about the 12 week plan. And this is a, this is the idea of if you have a regular 15 week semester, you plan it as if it were going to live within a 12 week timeframe. And then you just have these three extra weeks that you can do whatever you want with. And that, that actually was born out of teaching summer classes where we have some classes at my university that take up both six week sections. It’s like, like calculus for example, is often taught on a 12 week schedule. And I did that a couple of times and liked it and kept it. I said, why don’t I just do all my classes like this? And so normally I have like a week or two weeks at the end of a semester, two weeks at the end of a semester where all the content’s been covered. We’re done covering new material, and now we’re just gonna keep iterating through feedback loops for two whole weeks for whoever needs to do that. In this class, that was like two whole. Feedback and it’s just a, it’s very, very hard to get that done. And I had to think about how do you adapt the 12 week plan into a class that is six weeks long and it wasn’t easy.

Sharona: You know, it’s interesting because I, I was thinking about your condensed schedule quite a bit and some of the things you wrote about in your post. I have two kids in college, so we’re talking about n of N of two, one of whom does very well on a 16 week semester. And one of whom only does well usually in a condensed class, right, because of, of his level of attention. But my experience with his class condensed classes and my other son’s condensed classes, ’cause they take them is, most of them, at least the ones my two boys take, are not heavy lifts intellectually or academically, they’re not in their major. I wonder, but I wonder if even in a major class, maybe there are some students for whom that focused. Shorter feedback works better. I don’t know.

Robert Talbert: It’s absolutely. Yeah. Absolutely. I think that there is Sharona, in fact, the, when I taught this class two years ago in a synchronous format, it was still six weeks long, but it wasn’t asynchronous. It was maybe a little bit easier of a, of a process for students. I, I asked them, it’s like, how are you doing with all this? And they were a, a lot of them said that they prefer the six week format because it, if you have that little time to work with, you cannot procrastinate on things like it’s physically impossible to procrastinate on something when you literally have no time. And I, I don’t. I, I’m not that kind of person myself for sure. I mean, I, I’ve gotta like, take it easy, take it slow, you know, but some people are really like that. I can, I can, I can understand that. And I think this is a, that’s a format that works pretty well for students. Even some of my students, even on this asynchronous format said, as long as we have some structure and we know when things were supposed to be done, and, one of the things that I did in this class that I think is just general best practices for asynchronous classes is I took this approach of you have to do something every day. You have to do something little every day. Okay? It’s not the case where I just dump a bunch of assignments on you and say, do these by Friday. You know, because what’s gonna happen is people do nothing Monday through Thursday and then they do stuff on Friday. It says we’re all human, right? It’s just our nature. But I had little bits of things, like something all just like graded completeness and effort only formative assessments only Monday, Tuesday, Thursday, and Friday and Saturday. And I, I worried at first that this is gonna create like this, this like constant nagging, not a crushing workload anymore than it would’ve been if I hadn’t spread things out. But just like every day I’ve gotta log in and do something. What if I’m taking the class because I’ve got like this crazy schedule and I can’t do something every day. That never happened. I never ran into that problem. And in fact, the opposite happened. Like students said, I could handle the pace of the class because I was doing something basically risk-free every day. And I think those are the alternative grading kind of poking up there too. We’re we’re talking things like posting or, you know, doing a video response to a prompt posting some work onto an online whiteboard or turning in a practice problem or something like that. That’s just like it’s completeness and effort. I’m gonna give you feedback on it, but all you gotta do is do it. And I think, and there, there was no re-attempts on most of these formative things because it’s no longer formative. It’s if it’s re attempted. But the fact that there were no major grade risks here. You could screw up as much as you need to, but as long as you’re putting the practice problem on the online whiteboard, you’re good to go. You’ve done your job for Tuesday. Okay. And so keeping people on their toes and having like some little check-in every day keeps people consistent. And consistency, I think is the key to maintaining, you know, progress in anything.

Boz: See, and that’s, that’s interesting that you said that. And to your point, Sharona, ’cause the environments that I have taught condensed classes in, has always been in my K 12 world. And it has always been for students that have taken a course, a full length year long course and failed it. It’s never the first time that they’re seeing it. So it’s, I’m, I’m teaching, you know, algebra two A or Algebra two B. So, but the students that succeed, and I’ve had several that succeed at high levels and talking to them afterwards, it’s always been the same thing. It’s, I can’t do this over 20 weeks. It’s too easy to get distracted, it’s too easy to do too many other things. But I know I’ve got this eight week that I’m giving up my summer that I’m giving up some of the stuff that I wanna do to do. And it’s only eight weeks. Like it, it’s, it’s that shorter goal line. It’s like, okay, I can do this for that amount of time.

Robert Talbert: Right. You can put up with anything for eight weeks. Yeah. But, but 20 weeks is too much, even though it’s the same amount of stuff.

Boz: Yeah. But I’ve never had to teach new material, you know, to students. Again, it’s always in my context, it’s always been someone that has taken the full course before and failed it and this is their second attempt. But it’s interesting to hear that students in the same time constraint, still appreciate that condensed format because it for forces them to be focused when it’s new material as well.

Robert Talbert: Yeah. You know, it kind of makes me think sometimes that maybe we ought to do this on purpose. So like if, I know some schools actually do this where you’ll have a 16 week semester and some courses are offered on a 16 week schedule and some are offered on an eight week schedule. So you can stack two of them in the same space as one of them and just like deliberately put yourself on an accelerated schedule ’cause that’s how you roll. I mean, I feel like maybe that’s something to look into.

Boz: I mean, isn’t that basically the quarter system? Except you don’t have these, you don’t have the option of the full 16 week, you only 10 weeks I think was, is it a quarter, like 10 weeks?

I don’t know. I’ve never had the quarter system.

Robert Talbert: I had it my first year of college that I, that was like ages ago.

Sharona: Yeah. I hate the quarter system because it’s a weird, it’s still 10 weeks is long enough that you’re gonna get distracted. It’s not really condensed because they change the content. So you’ll do in three 10 week sessions what you do in two 15 week semesters. So even though it’s a shorter timeframe, it’s not actually condensed.

Okay.

Yeah, that’s, that’s correct.

And what’s coming up for me right now is I am currently in the middle of summer camps at the theater company I run, and those also are condensed. We do a show from casting to on the stage in two and a half weeks, or three and a half weeks, where we’re meeting every day. And it’s amazing because these are six year olds, up to 17 year olds, depending on the camp. And normally we’re like, oh my gosh, how do you put a show up in two and a half weeks when you’re casting on day one? And the show is, you know, two and a half weeks later. But yet these kids with no grades, with No, I mean, they actually pay to be in the program. They go home at night after camp and they’re learning their lines and singing their songs and putting together their costume. I mean, our camps are four hours a day and they’re probably spending eight hours a day on camp. So Yeah. Isn’t that kind of the model that, ’cause they want to.

Robert Talbert: So Sharona, I got a question for you then about that. So you’re giving feedback to this, to these kids? Mm-hmm. Obviously. Mm-hmm. I mean, you’re, or the director is, I don’t know if you’re right, you’re serving as like a theater director or not. So do you find that the kids are listening more carefully to feedback and putting it to better use because it’s faster? Or are they still just basically treating it like any sort of feedback that a kid might get?

Sharona: So I have sort of two answers to that. One is that we actually train them on feedback to a much greater degree interest. ’cause we say we need to talk to you about what it means to take a note. That’s the language in theater. And they, there’s an expectation that if we have to give you that note more than once, in the same way that that’s not good, like that means you didn’t listen the first time. And so that expectation is there. But on top of that, they are very focused because notes are something you want to get. If you’re not getting notes from your director, it means you’re either not doing anything memorable enough that they, they wanna comment on it or change it, or you’re fine, but you’re not being paid attention to. So it. It really, the feedback is what people want because they have this desire to have this great product. So the motivations are very different, but feedback is critical and it is very intentionally taught how to take it. How to give it, how to use it.

Robert Talbert: Yeah. Well, I think you just have an entirely new podcast episode topic that we just uncovered here. ‘Cause there’s, there’s, there’s no carry over to that in math at all. No, no, of course not.

Sharona: You know, I’ve, I’ve been thinking about it though, because I am not an ungrader in my math classes. I, I find it very uncomfortable. And yet the theater environment, which is our theater company, is educational. That is sort of the, the core of who we are. And it’s a hundred percent ungraded, like. There are no marks anywhere. They’re kudos sometimes. And of course there’s applause at the end. But well,

Boz: and there’s, and there’s roles. I mean, they do get cast into roles and those aren’t done by random, I mean, it’s.

Sharona: Right. Yeah. So there are, there are a lot of things, but there’s not great. So in my world, I think if I could create my math class to be what my theater class is, that would my, that would be amazing. And you can destroy a theater class by putting it at a university with a grade. Yeah. Anyways, little, little slice it, but I, I, you’re not wrong, Boz, let’s put that on the list of topics that we need to compare.

Boz: Well, we’ve talked, we, we’ve talked about that quite a bit actually. And especially with Becky Peppler a while back is, you know, talking about the why of what we do, why, you know, we have reassessments or why we’re doing alternative grading, why the feedback is important. And you and I have discussed with our, our stats class, like trying to give the other instructors that aren’t you and I task, or not task, but things that they can do with their students to help the students learn how to take our feedback, use our feedback, and why it’s so important, especially on that first attempt.

Sharona: So Robert, I wanna follow up then on feedback. So, so did your feedback loops work, which is a burden on you? Yeah, it’s very quick and it’s on the students. So how, how’d it go with the feedback loops?

Robert Talbert: Well, you know, first of all, I, firmly believe that most pedagogical problems are really communication problems. And so when, when you know you’re going into a six week class and you are a student, I mean, I had to really get with my students as much as you can get with quote unquote anybody in an asynchronous course. It’s like, this course is gonna go by at a million miles an hour and you’re not gonna get partial credit on things, but you’re gonna get, this is how the grading system works and explain why, and blah, blah, blah. The in a six week format, you’ve gotta iterate so fast and you cannot mess around With this. You have to take advantage of every opportunity that you get to retet something that you have been explain to that you haven’t. You haven’t mastered it yet. So more so than ever you, you have to really work on this feedback. Don’t put off the feedback. So, I mean, this is a communication issue. First of all. It’s, if you don’t communicate this explicitly and often to your students about how crucial it is to jump on the opportunity to reattempt something, you’re gonna end up in week six with too much stuff to do, and it’s just gonna be a disaster. And this is, this is true again, for a 15 week class. It would be true for if I had a 52 week class. But you have much less margin for error in a six week class. And, and it’s just important to communicate this to students because unlike your theater kids you know, my students are, I don’t know exactly who is giving them information about how to schedule their courses, but some of them were taking three of these courses simultaneously and working 40 hours a week or more. Like what are you even thinking here? So I, I think actually, I know what people are thinking, and you brought it up that a whole lot of asynchronous online courses are just complete horse crap. I mean, they’re just there, there’s nothing to them. They’re empty calories the whole way. And you can literally just sit back and click on buttons for an hour a day, twice a week, and get an A in some of these courses. I’ve seen them, I’ve been involved in some of them as a, as a student or as a participant. And you know, and, and I think that we, we communicate to students that, oh, asynchronous courses are perfectly designed for busy working adults, and whatever your schedule looks like, it’ll just drop right in and fill in the gaps. And, you know, again, it’s a communication issue that, you know. I use that number two and a half again as a scaling factor. Like in a regular class, as I explained to students you have meetings scheduled three hours a week. Okay? So those have to be scheduled in, you know, for you, you don’t have meetings, so you have to schedule three hours a week times two and a half, you know, for six weeks. And then we normally expect you to put in two hours a week for outside of class for every hour you spend in class. And that’s a conservative estimate. And so when you total it all up, this is like a 20 hour a week commitment for one class. Okay? And part of that, a big part of that is iterating on feedback loops. So again, the there’s only so much you can do from a distance, but you can, you communicate this to students, like anytime you get a chance to retry something, you must take it. Okay. And just don’t put, don’t do this to yourself. I, I think I did a pretty good job of that. ’cause I mean, as students, I didn’t have an issue with students across the board. Some exceptions unfortunately, but I didn’t have a big issue with students who were procrastinating feedback like, oh, I took a checkpoint this week on one of the learning targets and didn’t pass it. I’ll just sit on it for three weeks and try it again later. I, I didn’t get that. Thankfully. I, I did, however, get some students who. Some isolated cases where students didn’t do this and they get down and they didn’t read the syllabus. And so like they on our problem sets, they were only allowed two reattempts a week on problem sets. And they get down to the end of the semester and they have five of them to redo. Well, you can only do two of ’em in a week, and they miss that whole part in the syllabus about. The, the two per week rule. It’s like, oh, oh, now you’re in trouble. You know, whatcha gonna do about that? And so again, it’s a communication issue to get students do the feedback. What worked for me to get feedback loops is to try as much as possible to front load as much work into the first four weeks of class as possible. And this is not good for students because it just, it further condenses a six week condensed course. But on the other hand, if you’re already condensing by a factor of two and a half, why not go for a factor of three? Right? And you’re already there, you’re already in hyper drive. So why don’t just hit the gas a little bit more and give yourself we were able to generate about three extra days at the end of the six weeks for reassessment. Okay. And for a lot of students, that was enough. Because we, it, it, it required a lot of work on my end to create new versions of checkpoints and that sort of thing, but I knew this going into it and I could kind of work ahead a little bit. And by the time week six and the little three day, half week happened at the end I was, I was set up, I was good to go with, with what I needed to do. So it’s all about looking ahead at the calendar and knowing what your constraints are as far as I’m concerned.

Sharona: Well, and and you said it’s not good for students. I think we overestimate how important spacing out new learning is. Mm-hmm. Because students have such massively different processing times that I’m not sure it’s any worse, that new inter new material comes so quickly. Because having that processing time on the end is, is more important. So yeah,

Robert Talbert: I, I, I agree with that. That’s an interesting point that I hadn’t considered of what’s, what really matters is whether you can do space repetition of information. Exactly. That’s what really gets it to stick in your memory. Having space to process it as it comes in is like a nice to have, but not a need to have as far as I’m concerned. I mean, in our real lives we don’t get that. I mean, you know, when, when we’re faced with something, we have to learn really quickly. Sometimes you’ve just gotta learn it really quickly and just try to do what you can in the moment. And then on the back end of that introduction process of new, material you’re supposed to learn. That’s when the learning really happens. It’s like after, after the fact. I mean, that’s the whole, that’s the whole concept behind flip learning in my view. It’s like you, you front load the introduction of material to provide mo lots and lots of space for the backend processing that takes place. And, I agree I mean, I think that’s, that’s much more important. I’m willing to sacrifice a little bit of, of space at the beginning if it will provide more space at the end for reat attempts and, and, and revisions and whatever else.

Boz: So I, I did want to, you know, kind of ask a, a little bit more about the class and, and really what, what your post was about. So with this first time doing this condensed, asynchronous, like what did work well in your opinion, in the class?

Robert Talbert: Yeah. So what really worked well, and I I mentioned this a little earlier, was this approach that I took of having like high frequency, low stakes. Okay. Like something little that has to be done every single day. And this keeps students from disengaging from the course. It’s encouraging because you really can’t fail at it as long as you do it in the first place. That, I mean, my students told me over and over again every week, I had them do a video . This is one of their low stakes, high frequency things they had to do, was do a video at the end of every week just reflecting on what they learned, what their wins for the week were, what their challenges were, where they’re gonna do different next week. And the every week they said like, man, I’m really glad that I’m being forced to check in with the class every day. It’s not always easy, but it keeps me from checking out. And so that was, that was a major win, and I really feel like that’s something that can be done for any class maybe should be whether it’s condensed or not. Having obviously having t attempts without penalty, even though it’s super, super constrained in a, in a six week class the, the time scale that you have is much, much shorter. That is always helpful for students, because. Even though the timescale is constrained and much, much shorter by a factor, two and a half, it’s still there. And turns out you don’t really, my students at least don’t need a lot, a lot of rea attempts to get things right. They just, they can usually, it’s like if you’re playing golf, most of ’em are par three. Like, I mean, you can get it up kind of close to the green on the first hit, relatively close. And if you give ’em a couple extra shots, they’ll get it. I mean, so’s that’s usually how my, my, my folks tend to work. And you know, one thing that I did, I was able to do in this class that doesn’t really have anything to do with grading, but it was so, this very, very successful was a lot of video back and forth. Okay, so this is something again that I’ve learned. I was just trying to study like, how do you even do an asynchronous class? And the having instructor presence is a big issue, right? If the instructor can project their presence into the class. Whether it’s because you don’t have a meeting to do this for you anything you can do to convince students that you’re actually present with them in some form or another is gonna be helpful.

So one of the great things that this class had going for, it was the size. Like I only had nine students in the class. In fact, the class almost wasn’t allowed to run because the, the lower cutoff for green lighting, the class was 10 students. And up until about a week before the class started, I only had seven students. And so I did, did a big marketing blitz and managed to get it up to 10, and the dean let it run. I ended up with nine regular students, plus one student who was auditing. So she wasn’t doing a lot of like graded work because she didn’t have to, but just whenever she felt like it. With only nine stu or 10 students in the class, you can do a lot with individual back and forth being asynchronous, that for us meant video. Okay? So every week I was making lots and lots of video content for students, including every Friday I would sit down and make a personalized feedback video for every student in the class and send it to ’em three minutes long, three to five minutes long using a tool called Loom which is a real easy to use browser integrated video tool. And just sit down and just have their grade book in front of me and the work in front of me and just, just talk for three minutes and then send it to students. And students were sending me two videos a week for responding to prompts and one at the beginning of the week where they would, like, once they go through all the readings of the video for the week on Mondays, their thing to do on Monday was to make a video, putting one of the big concepts into their own words. And then on Friday they send me another that says like, here’s how I went throughout the week.

And so this video back and forth was like, great for building presence and building a relationship with my students. In fact, I felt like I got to know my students much, much better through this asynchronous environment than I did with two and a half times the time and three times a week physically in the same room with my other students. Even though my usual class size is like 25, it’s not hugely greater than, it’s about two and a half times the size of that number two and a half is like fibonacci’s the, like, the like the golden ratio or something. It’s it’s it’s not hugely greater, like you would expect me to have much more familiarity with students that I’m meeting over 15 weeks in a face-to-face environment.

But that wasn’t the case. And so, i, I realized early on that the one thing that I had, and it was a major advantage that I didn’t have in my synchronous face-to-face classes, was the small size. And if I don’t leverage that to the nth degree, that I’m wasting an opportunity. So that, that was like the MVP of my class was all the video back and forth. It was very cool to, to just kind of hear from my students and there was no pressure. We didn’t expect anything fancy. It’s like, just take your phone and record yourself and then send it to me, because I wanna know what you look like. I wanna know what you sound like. Right? And I have to believe that that, ha I don’t have data. I don’t know how even how to collect data about this, but I have to believe that that mitigated some of the AI risk. Somewhere down the line that might’ve saved me like 10 instances of investigating AI misuse just by having a, a sense of belonging and a sense of community. Like if they see me putting this much effort into making videos for them and they’re connecting with me like face-to-face as far as we can, that I, I have to believe that that lessens the possibility of AI cheating.

Boz: And, and I, I would think it would also, because with that kind of you know, recorded oral interactions going back and forth, that you would get a sense of their voice and their work that Yeah. ’cause that, I mean, that’s one of the things that when we’re looking at and, and trying to decide if a student’s work is legitimately theirs or possibly not. Is having that familiarity with the student, like, yeah, you know, I, I’ve had some student work that has been turned in and looked at by other people, and they’re like, yeah, this is too good. And I’m like, no, I can read this and I can hear the student’s voice, like I, this is how they talk. This is how they, so I would imagine not only did it help convince the students either not to, or they didn’t need to or whatever, but it would also help you be able to recognize it easier than, you know, if you didn’t have these videos. And then the possibility of thinking a student might have used it when they really hadn’t, which I’ve, I have done that once or twice in my 20 year career. Talked to a student, suspected cheating, and afterwards really was convinced that they didn’t, and it damaged that relationship. I mean,

Robert Talbert: yeah. Yeah. Yeah, I agree. I mean, it sounded like if you’re, they, they say that when, if you’re worked for the Secret Service and you’re being trained to recognize counterfeit bills, I mean, the way that you learn to recognize counterfeit bills is to really deeply study what a true bill looks like and all, all the little nuances. And so by having that that one-on-one quote unquote person, I mean, it’s, it is personalized. It’s not face-to-face, but it’s like you’re talking to a real person. And I guess it’s possible that, you know, if a student is making a video for a prompt to respond to, they might have used Chat to generate the thing they wanna say and have it up in front of them. But why would you do that? I mean, it’s like so much work. It’s less work to just, I trust in my students’ self-interest. Right. I mean, so what that boils down to is just how hard is something to do, right? If something is like twice as much work for no grade. Right? You’re not even getting graded on this. I’ve been completely stuff anyway. I, I really feel like I’m seeing the real person and by studying their mannerisms and thinking about how this person thinks and talks and responds, that even bleeds through, I feel like, into their writing. If they’re writing up a mathematical solution to something like that, I can tell. I mean, my, my AI spider sense is undefeated. I mean, whether it comes to art, music, or writing, it’s like anytime I see something that I think is AI and I ask about, it turns out to be ai. I mean, I’ve never, I mean, I, I don’t know if I’ve ever failed on that, but I don’t think I have, honestly. I mean, I think I’m pretty good at detecting ai.

Sharona: So my challenge is with this conversation about AI is in going back to my theater side, there have been things I’ve been able to do because I use AI that I cannot do myself. Like, I literally don’t have the skill. And I have no real interest in gaining this skill. And the one that’s coming to mind right now is something that I’ve been asking our theater teachers to do for several years and they’ve not been able to do, is to do in-depth character work with our teens. So we have a camp of 37 teens and tweens right now doing the musical Legally Blonde Junior. And it’s a condensed schedule, right? So our theater teachers know how to do character development, but they’re so focused on getting the show up that they can’t.

And so one of the things I’ve done this year, ’cause I have no training in theater teaching, but I have a lot of training as an educator, is I threw sophisticated prompts into AI to generate character specific character development worksheets for different groups within the cast. So I’m handing them not your generic character development worksheet, but one where I said to the ai, Hey, I have a group of teens doing Legally Blonde Junior, and I want one set for this group of characters and one set for this group of characters. And it was smart enough to not only give the basics of character development, but ask these students specific questions about relating to another character specifically in the show. I can’t do that. I don’t even think my theater teachers could do that in a reasonable amount of time. So one of the things I was wondering about your class is should they be using AI in certain places and circumstances?

Robert Talbert: Oh, absolutely, yes. I mean, I’m not a prohibitionist when it comes to ai and I don’t quite understand the prohibitionist mindset when it comes to ai. I mean, this seems like saying like, I am never going to use a generative AI in my, as an educator is sort of throwing the baby out with the bath water. There’s times in places for it. And I made this explicit in, I have a whole document that, and I can share it with the, in the show notes if you want about here are the rules for academic honesty. And basically if you are learning something, you should be using AI to help you learn things like truly learn things. And so for, I’m, been doing a lot of thinking about, of back to theater and I’m a musician, as you all know about deliberate practice, the concept of deliberate practice and how that plays into your work as a student and being and grading as well. You know, generating things for you to practice with. Like, you don’t have to have a textbook for this anymore. So one of the things we do in the class is like converting numbers from decimal to binary. That’s a very basic early on, sort of a topic, a learning target we have. And we talk about it, there’s a video for it. We do some work. And if you do the basic practice that I give to you as part of the course, and it’s still not clicking for you, no worries. All you gotta do is go to your favorite AI tool and say, please generate like 25 practice problems on base conversion and give me the answers, but don’t show me the answers unless it’s below you. You can specify any amount of parameters you want, and you can generate so much practice that you get to interact with as a human being.

It would be crazy not to use this. I mean, I use it myself, and when I think about the stuff that I need to learn it’s like you Sharona, I mean, I’m not skilled in everything, right? But there’s certain things that I have to do or maybe I just don’t want to do certain things right? Like in one of the bands that I’m in, we wanna advertise our shows, but none of us, all of us suck at copywriting. So we’re for advertisements for shows. So we absolutely go to chat GPT and hit that up and say like, okay, here’s who we are, here’s where we’re playing generate a fun sounding, you know, thing we can post on social media for it. And it’s always great. Students should have access to these things. And as in point of fact, they do. I mean, in an asynchronous environment, the simple fact is you have zero control over the tools that students use. Okay? You can set up the guidelines for ’em and you should. And my guideline for AI use was if you’re learning something, you should be using it, but it has to stop the moment you switch over to something that’s gonna be graded, okay? If you’re gonna have sent turn in something that’s going to be evaluated, whether or not it’s truly graded, I mean, if I were doing this completely un grading, it would kind of be the same policy. If you’re doing something for yourself to build your own understanding of a topic, AI is like the greatest invention of all time. You should be absolutely using this in your own time. And here are some tools. Here are some things you can do. Here’s NotebookLM. It’s like a great tool for building your own little personal knowledge base.

But the moment you switch over to something that I am going to evaluate, it’s gotta be you. No ai. Okay? Just keep it real simple like that. And the awful truth that we’re all trying to navigate here right now in education is the fact that you, it’s very difficult to tell when students have not followed that policy. Okay? Soon, just turn something in. It literally sounds like a human being. You can, my spider sense still works, but it’s getting worse every day. I feel like it’s, it gets, it’s getting more and more authentic the more training data it gets. Eventually, this is just not an arms race. We can win. Okay? So we have to start saying like, okay, what I’m gonna do instead is I’m going to tell students the benefits of how to use it in certain appropriate cases. Make it clear when those cases end, and then set up an environment that is high trust and then just hope for the best. And I will, I get burned by this. Will I get abused by cheating? Yeah, I will it won’t be the first time and it won’t be the last. This, this predates AI for sure. But my job as an educator is to put students in the best position to gain and demonstrate an understanding on the topics that I’m teaching. And if I, I feel like AI is a, is a tool for doing that the, I’m gonna do what I can and just try not to worry about the rest of it.

Sharona: So you’re gonna teach this class again, not in the fall because you’re on sabbatical. Are you on right, not in the fall, first semester or full year sabbatical?

Robert Talbert: Just one semester, unfortunately. Yeah, the plan is I’ll be teaching two sections of it along with a section of it, second semester counterpart when I get back in January. Face to face Synchron assault.

Sharona: So what are you thinking, because when we got, I had you on for episode 87, you mentioned this class, but we really had you on because you were suffering from AI use this semester or this, this course you talk about a little bit. But where do you see yourself going with this course and the AI risk and all of these? Things.

Robert Talbert: Yeah. So back in the winter, I was teaching the 225, the first semester of the sequence. And I had made all these changes to, I taught the second semester of the class back in the fall, and I had no plan for ai and I got absolutely hosed because of it. I mean, students didn’t know what the rules were and how, what was I expecting? I don’t know. But I, I made some adjustments to, again, it was in-person 15 weeks. No, no weird stuff. And I shifted all of my major assessments to in-class testing, and I didn’t like that at the time. I felt like that was a major step backward like it was gonna disadvantage a lot of students and create a huge burden for me. None of that happened, actually, it was actually pretty good. And I never had to worry about AI use because look, you can use AI all you want in the formative stuff. You’re only hurting yourself if you do that. It’s just completeness and effort anyway, so who cares, right? And you’re getting feedback. It was almost like a mini un grading set up. Like it was giving a lot of formative stuff.

But I would have these periodic in-class exams to kind of put things to the test literally speaking. And that worked out really well. Actually. It’s it was, had a lot of advantages to it. I got some things to change as the getting into the weeds of the logistics, but I think the overall idea of you give a lot of formative assessments. This, again, is something that worked really well in the asynchronous course. The high frequency low stakes type of thing. You do a lot of high frequency low stakes stuff that is either not graded at all or it’s graded for completeness and effort only. And then you use that as an opportunity to kick some feedback loops off. And then your job as a student is by such and such a date, you need to close that feedback loop on such and such topics. And you’re gonna be tested on this and you’ll get re attempts on the test, but you know, it’s greatly to your advantage to kind of get things wrapped up by this particular date. That is probably some iteration on that is where I will be landing in January for sure. Because I, I surprisingly liked how that worked out.

Boz: So looking at both that new iteration that you did in the winter and what you’ve done in the spring with this asynchronous it, it are there, it sounds like there’s a couple lessons learned that you are going to definitely keep going forward with the high frequency, low consequence or low risk. Is there anything. Else that you’ve learned that you’re either going to be continuing or that you’ve learned, Hey, this is not working. I, I need to stop doing this in, you know, my further iterations and teaching this course.

Robert Talbert: Yeah. So one thing that I learned from the asynchronous course is we still had, we couldn’t do in-class testing ’cause we didn’t have classes, right? So there was no, there was not a thing that we could do. So my tests, I mean the low level learning targets, the basic skills aspect of the course were still done through time testing. But they were all located on our LMS, which we use Blackboard at my campus monitored by Respondus lockdown browser respondents lockdown browser, in case you haven’t heard, it’s a web browser that when you launch it, it’s integrated into our LMS. When you launch it, it shuts down all processes on your computer that can access the internet. So it’s literally a lockdown browser.

That worked. Okay. We had some ongoing glitches with Respondus’s lockdown browser that were super problematic. It would tell students when you fired up the browser the browser would fire up on its own when you open up a quiz to take. And about half the time, towards the end of the semester, it was telling students that you can’t access the quiz because the instructor hasn’t activated the lockdown browser. So guess how many emails I was getting per day from students? And I’m just like, oh, this sucks. Why can’t you just work? I mean, we never figured out what was causing this glitch, and it was a, it was a real problem. So I’d have to watch out for the technological tools and just like everything, I mean, when you introduce a technological tool into your teaching, it’s just another point of failure. A potential point of failure.

So you just gotta really be careful about this. And I think overall using a Respondus lockdown browser was a win, but I would not shift my quizzing to online, like a timed online setting, unless I absolutely, positively had to, I would still do ’em in class. And other assessments in the course were still done, sort of quote unquote take home everything. A take home in a asynchronous class, by which I mean, you have, we had weekly problem sets. And so you have these problems and you work on them, and you write them up and you turn them in. And those are still super, super exposed to AI misuse. I don’t know for, I never prosecuted an ai academic dishonesty case in that six week class. So this wasn’t like a big issue, but I could see it becoming a big issue very easily. Right. Yeah. I mean, it’s still out there. There’s nothing out there other than the janky and creepy and Orwellian sort of AI detection software that is built into a LMS is like Blackboard totally untrustworthy.

And so I don’t use them. My spider sense is also totally untrustworthy, but at least I can act on it. So giving take home assignments that are significantly graded in the course, i’m just about to the point where I say that is history at this point. Like, that’s been the bread and butter for math classes for a thousand years. But now, today it’s like, now really, I think the model that I’m leaning toward is like, you can give those problem sets and it’s like, okay, you do these, you turn them in, and I may not grade them except my completeness and effort, but I’ll give you feedback. But then eventually I’m gonna create a timed in-class test with no technology. The whole experience is completely air gapped.

Like, there’s no computers or internet or anything involved where you have to replicate that solution or something similar to it. So do all the AI you want, but you gotta understand what you’re doing and I’m gonna test you on it in a live setting. That’s what I did back in the winter, and that’s what I felt like worked better than, a lot better than I expected it to, and I never had to worry about the ai. So these are, these take home assignments are perhaps permanently fatally exposed to AI risk? Yes. The, the gold standard for me moving forward is either like in class testing or oral testing. Honestly, oral exams which I could have done, I could have done this in the asynchronous class with only nine students getting grades for the class. Totally could have done like a comprehensive oral exam for each student. 45 minutes, spread over three days. Would’ve been no problem to do that, honestly.

Boz: Yeah. ’cause you, you had already established the environment and the expectation of these recordings. Turning that into a oral exam if, if that environment’s already set up in the, you know, first four or five weeks of the course, you’re right. I, I don’t think that would’ve been a, a difficult thing to do or try.

Robert Talbert: Not but nine students. I mean, if I had like, like 90 students of like, no way, unless I had a army of teaching assistants to kind of do two thirds of them for me. But with only nine students, yeah. I mean, I could have probably even done this truly live, like on a Zoom meeting. I mean, most students probably could have found 45 minutes somewhere. I’d, I’d get up at one in the morning if I had to if it really mattered. And I, that’s one of the things that I think that if I were gonna go back and do this class again, that’s how I would do it. I would have, like, everything that happens other than this comprehensive oral exam at the end is merely formative. And you just, there’s all graded completeness and effort and you gotta accumulate X number of completeness to get an A and whatever X minus three to get a, B and X minus six to get a C. But your main grade is done collaboratively at the end based on your performance on a comprehensive or exam and a discussion with me. So, sort of ungradedish, but not quite. I think if I did this again with a sufficiently small class size, which sufficiently small to do this, I would think is probably like 12 and under for me at least. I mean, some people may have a greater appetite for work than that, but not me. That that’s, I I would, I would totally probably redo the class under that kind of structure.

Boz: Yeah. Well, I, I wanna thank you for coming back on. It’s always a pleasure having you come back on, but we are coming up on time. If you want to know more details about the, you know, what happened in this course and stuff, go check out the Grading for Growth blog if you aren’t already. It’s an amazing, I mean, that long predates this podcast. We have stolen so many hubcaps off of that blog to make our episodes. So if, if you haven’t already. Go, go subscribe. And this particular blog post came out a couple of weeks ago on July 7th. It’s full title is alternative graded discrete Math and a Hyper Drive: a Reflection. So there’s a lot more details there. But again, Robert, always a pleasure to have you on and I wanna thank you for coming back and kind of putting a cap to this series of episodes we’ve had with you where we really got to look at how you were changing things and how they ended up working out. So thank you so much for coming back.

Robert Talbert: Thanks for having me. It’s always a pleasure.

Boz: And to our listeners, thank you and we’ll see you next week.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode’s page on our website, http://www.thegradingpod.com. Or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the Contact us form on our website. The Grading podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.

Boz: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State System or by the Los Angeles Unified School District.