SBG in AP Stat – The melding of two assessment styles

I know that there are plenty of teachers using standards based grading in AP classes, but despite that I didn’t really know how I was going to implement it in my classroom. But I decided to give it a whirl, inspired by the technology at my disposal: the Ultimate Gamified Learning Goal Assessment Spreadsheet that I’ve been working on with the help of many others for close to a year now. But the tech was, of course, only the initial inspiration; the idea of keeping track of what they actually know in a much better way is my real inspiration and goal.

I decided that I would not try to change my assessment style, at least for now. I do take up occasional classwork assignments and warmups and use those to inform the scores on my standards (which I call Learning Goals), but the majority of my input is based on big summative tests that mimic the AP style. This is for three reasons:

1. Selfish, but realistic: keeping track of lots of small assessments is difficult and time consuming, and I struggle with the organization it requires. I do better with a big pile and a strict deadline, which traditional tests allow me.
2. Pedagogical: I do believe it is important for students to be able to show mastery of many different learning goals at once, and combine them together in complicated ways, which a large assessment covering many goals allows more easily than smaller ones designed to cover only one or two.
3. Practical:  this is an AP course, and part of how I am judged as a teacher (not formally, but informally) is how well prepared they are for the Big Day. The students need practice on exams that mimic the AP exam. Thus, my assessments remain in that style; several multiple choice questions (usually 10 or 12) with 3 or 4 multi-part free response questions.

We just completed our first unit on experimental design, so I now can report out on my first experience with SBG on a big AP-format test. It was an ordeal.

First, I had to align each question with the appropriate learning goals. I covered 14 different goals in this test (mostly taken directly from my textbook). Some were very straightforward alignments, but some were much more complicated; the multiple choice questions in particular are tricky, because often you needed to have a solid grasp of two or three of my goals to be able to ferret out the correct answer. I will not take the multiple choice away, but I may find a way for them to explain their answers in the future, so I can be more certain that I’m grading those goals correctly. This was an extremely valuable experience, however, because I was able to see which learning goals were getting the short end of the assessment stick and which were the big stars; I ended up having to switch out a couple of questions in this process to improve the balance of the assessment.

Actually grading the test was tricky. I made a table, organized by learning goal, that listed the connected problems and helped me keep track of scores as I graded the test. Here is an example.

This worked pretty well; next time I will probably put the actual wording of the learning goals on this if possible (to aid in reference and thinking). The hardest part was dealing with multiple choice questions; I ended recording all missed multiple choices as 1’s (out of 4) in the standards, but in a couple of cases where a LG was only assessed by one MC question, that ended up being kind of harsh. One of the free response questions also ended up connecting to something like 4 different learning goals, so one student  who otherwise did well but REALLY struggled with that one ended up losing points in lots of places. I need to find a better, more fair and accurate way of dealing with these big multi-goal questions.

As I went, I ALSO graded the test in the style of the AP exam. Each MC question was counted as simply right or wrong, and the FR were graded using a rubric similar to the ones the AP uses, each question out of 4. This allowed me to create a raw AP score and estimate (very roughly) their scaled AP score. A curved version of this grade is essentially what I used last year, so this gave me an interesting value to compare to to see if my new system was giving at least somewhat similar results to what the AP exam might give me.

This is the scatterplot of my learning goal score average vs the calculated AP raw score. R^2 =.71. Not bad! The high outlier is the student I mentioned who struggled with the big multi-standard problem, and if you remove her data the R^2 rises to 0.9. So that’s good! My system differentiates sufficiently to be a reasonable approximator of AP scores.

It definitely added time and work to go through this process, but I think it will be worth it. The students mostly did very well on this test, but a few of them who need to work on some specific knowledge will now be armed to do that. And I will be better prepared to help them. A win for all.

I would love suggestions for other, easier ways to combine the AP Style with a standards-based system – please reach out to me if you have suggestions!