Well, my intro to formal logic (Logic I) course is in the can. I think it was a success! I could not have done it without Graham Leach-Krouse’s Carnap system, and my excellent team (Husna Farooqui, Sarah Hatcher, Hannah O’Riain, and Dvij Raval).
A while back I wrote about the plan to implement specification based grading (Grading for Mastery in Introductory Logic) and before that on how to use Carnap (Adding online exercises with automated grading to any logic course with Carnap). I ended up simplifying the grading approach significantly from that first plan (thanks, past me!) and almost everything the student had to do was either done on Carnap or on the LMS (Brightspace aka D2L). Here’s what I did:
- Split up the material into weekly units, e.g., “Translations in TFL” or “Interpretations for nested quantifiers and identity”.
- Recorded introductory lectures on each unit (typically under an hour’s worth, separated into 10-15 minute chunks). Nothing fancy, just me talking through some beamer slides.
- Of the three scheduled hours for lecture, we used two for synchronous Zoom sessions where the students and I went through exercises and I answered questions.
- Tutorial sections were split between more practice problems as a group with the TAs, and small groups (breakout rooms) of about 5 students working on their weekly problem sets (with the option of calling for help from the TA).
Assessment was on the basis of three weekly activities:
- A problem set (collaboration allowed, all done on Carnap)
- A quiz (multiple choice on the LMS, testing some concepts and applications that Carnap isn’t set up to do).
- A “challenge problem,” also done on Carnap: a timed exercise where collaboration was not allowed.
Quizzes and challenge problems replaced exams: in fact, I used the same kinds of questions I would have asked on an exam. The idea is that they work on the problem set during the week, which prepares them for the challenge problem, then they basically take an exam consisting of 10 multiple choice questions plus one typical, not-too-hard exercise.
Problem sets and quizzes had about 8 questions I would expect a B student to be able to do, and 2 harder questions which they have to complete if they want an A. They were graded incomplete/complete/complete+. Challenge problems are just incomplete/complete. To get a B, you have to complete 10 of the 12 units. For an A you have to complete all of them, and on some number of problem sets and quizzes you need complete+. (I had set that number at 6/12, and I got a lot of As. So next time it’ll be 10/12 for an A.)
This system worked extremely well to keep the students on track. Every week had the same structure, they didn’t really have to keep track of any deadlines: every Monday they had three things due and basically 5 days to do it.
In keeping with the idea that what counts is achieving proficiency (“mastering”) the material, but not how long it takes them to do it, I allowed them to re-do things that they couldn’t get done on the first try. Every student had 6 “tokens” that would buy them a do-over of one of the activities. Every week they got to tell us what they wanted to re-do, then we gave them another shot, and replaced their mark when they scored better.
If you want to see the outline for next term, which includes the list of topics, and more detail on all of the above (and including some tweaks to the grade scheme in response to this term’s experience) see Logic I (Phil 279).