A while back I wrote about the plan to implement specification based grading (Grading for Mastery in Introductory Logic) and before that on how to use Carnap (Adding online exercises with automated grading to any logic course with Carnap). I ended up simplifying the grading approach significantly from that first plan (thanks, past me!) and almost everything the student had to do was either done on Carnap or on the LMS (Brightspace aka D2L). Here’s what I did:

- Split up the material into weekly units, e.g., “Translations in TFL” or “Interpretations for nested quantifiers and identity”.
- Recorded introductory lectures on each unit (typically under an hour’s worth, separated into 10-15 minute chunks). Nothing fancy, just me talking through some beamer slides.
- Of the three scheduled hours for lecture, we used two for synchronous Zoom sessions where the students and I went through exercises and I answered questions.
- Tutorial sections were split between more practice problems as a group with the TAs, and small groups (breakout rooms) of about 5 students working on their weekly problem sets (with the option of calling for help from the TA).

Assessment was on the basis of three weekly activities:

- A problem set (collaboration allowed, all done on Carnap)
- A quiz (multiple choice on the LMS, testing some concepts and applications that Carnap isn’t set up to do).
- A “challenge problem,” also done on Carnap: a timed exercise where collaboration was not allowed.

Quizzes and challenge problems replaced exams: in fact, I used the same kinds of questions I would have asked on an exam. The idea is that they work on the problem set during the week, which prepares them for the challenge problem, then they basically take an exam consisting of 10 multiple choice questions plus one typical, not-too-hard exercise.

Problem sets and quizzes had about 8 questions I would expect a B student to be able to do, and 2 harder questions which they have to complete if they want an A. They were graded incomplete/complete/complete+. Challenge problems are just incomplete/complete. To get a B, you have to complete 10 of the 12 units. For an A you have to complete all of them, and on some number of problem sets and quizzes you need complete+. (I had set that number at 6/12, and I got a **lot** of As. So next time it’ll be 10/12 for an A.)

This system worked extremely well to keep the students on track. Every week had the same structure, they didn’t really have to keep track of any deadlines: every Monday they had three things due and basically 5 days to do it.

In keeping with the idea that what counts is achieving proficiency (“mastering”) the material, but not how long it takes them to do it, I allowed them to re-do things that they couldn’t get done on the first try. Every student had 6 “tokens” that would buy them a do-over of one of the activities. Every week they got to tell us what they wanted to re-do, then we gave them another shot, and replaced their mark when they scored better.

If you want to see the outline for next term, which includes the list of topics, and more detail on all of the above (and including some tweaks to the grade scheme in response to this term’s experience) see Logic I (Phil 279).

]]>We investigate a recent proposal for modal hypersequent calculi. The interpretation of relational hypersequents incorporates an accessibility relation along the hypersequent. These systems give the same interpretation of hypersequents as Lellman’s linear nested sequents, but were developed independently by Restall for S5 and extended to other normal modal logics by Parisi. The resulting systems obey Došen’s principle: the modal rules are the same across different modal logics. Different modal systems only differ in the presence or absence of external structural rules. With the exception of S5, the systems are modular in the sense that different structural rules capture different properties of the accessibility relation. We provide the first direct semantical cut-free completeness proofs for K, T, and D, and show how this method fails in the case of B and S4.

rhsmlt]]>

Any set of truth-functional connectives has sequent calculus rules that can be generated systematically from the truth tables of the connectives. Such a sequent calculus gives rise to a multi-conclusion natural deduction system and to a version of Parigot’s free deduction. The elimination rules are “general,” but can be systematically simplified. Cut-elimination and normalization hold. Restriction to a single formula in the succedent yields intuitionistic versions of these systems. The rules also yield generalized lambda calculi providing proof terms for natural deduction proofs as in the Curry-Howard isomorphism. Addition of an indirect proof rule yields classical single-conclusion versions of these systems. Gentzen’s standard systems arise as special cases.

DOI: 10.1017/S1755020320000015

gencalcs]]>

Mastery Grading aka Standards-based Grading is an approach that’s become increasingly popular in university math courses. In points-based grading, you assign points on all your assignments and exams, and then assign grades on the basis of these points. The system relies heavily on partial credit. Students will revolt if you don’t give it, because so much can hang on fractions of a percentage point. In mastery grading, you define the learning outcomes you want students to achieve, and grade based on how many many they have achieved (and perhaps at what level they have achieved them). Big perk for the instructor: you don’t have to worry about partial credit.

In intro logic of course a great many problems are of the kind that we ordinarily think students must be able to do (so get high point values on tests) but are terribly hard to award partial credit for. If a student doesn’t get a formal proof right, do you dock points for incorrect steps? Grade “holistically” according to how far along they are? If they are asked to show that A doesn’t entail B, is an interpretation that makes A and B both true worth 50%? In mastery grading, instead it makes sense to only count correct solutions. Of course you’ll want to help students get to being able to correctly solve the problems, with a series of problems of increasing difficulty on problem sets before they are tested on a timed, proctored exam, for instance, and with opportunities to “try again” if they don’t get it right the first time.

Now for an online logic course, especially one with high enrollment like mine, academic honesty is going to be a bigger issue than if I had the ability to do proctored in-class tests. Evaluation that discourages cheating will be extra important, and one of the best ways to do that is to lower the stakes on exams. If I can have many short exams instead of a midterm and a final, I’ll have to worry about cheating less. That works out nicely if I want each exam to test for a specific learning objective. More exams also means more grading, and I have limited resources to do this by hand. Luckily most of the objectives in a formal logic course can be computer graded. I’ve already made heavy use of the Carnap system in my logic courses. One drawback is that Carnap can only tell if a solution is completely correct or not. Although partial credit functionality has been added since COVID hit, not having to manually go through a hundred half-done proofs every week will be crucial in the Fall. So mastery grading is a win-win on this front.

Assigning letter grades and incentivizing various behaviors (such as helping other students in online discussion boards) is, however, a lot harder than in a points-based approach. For this, I’m planning to use specification grading: You decide at the outset what should count as performance worthy of a specific letter grade (e.g., completing all problem all problem sets, passing 90% of quizzes an exams for an A) and then use these specifications to convert many individual all-or-nothing data points to a letter grade. To encourage a “growth mindset” (practice makes perfect) I’ll allow students to revise or repeat assignments and tests (within limits). This would be a nightmare with 200 students and 10 tests, but if they are computer graded, I just need to have two versions of each (short!) test — about the same effort as having makeup versions of two or three longer exams.

I’ve already used specifications grading in Logic II (our metatheory course), where I just copied what Nicole Wyatt had pioneered. That, I think, has worked pretty well. The challenge is to implement it in the much larger Logic I.

I have a preliminary plan (learning outcomes, activities, grade scheme, token system). That’s a google doc with commenting turned on. Please let me know what you think!

If you want more info on mastery & specs grading especially for math-y courses, check out the website for the Mastery Grading Conference just completed, especially the pre-conference assignments and the resource page. Recordings of sessions to come soon, I hear.

]]>A. Tarski-style: use variable assignments \(s\colon V \to D\) (where \(V\) is the set of variables and \(D\) the domain), then define \[\mathfrak{M}, s \models \forall x \, A \iff \mathfrak{M}, s’ \models A \text{ for all $x$-variants $s’$ of $s$}.\] This requires a definition of “\(x\)-variant” but is otherwise very clean. Its drawback is that it obscures how we let \(x\) range over all elements of \(D\), and my students have a hard time understanding it and an even harder time working with it.

B. Alternative Tarski-style: we use variable assignments as above, but avoid talking about \(x\)-variants. Instead, we define the notation \(s[m/x]\), the variable assignment just like \(s\) except it assigns \(m \in D\) to \(x\). Then we have \[\mathfrak{M}, s \models \forall x \, A \iff \mathfrak{M}, s[m/x] \models A \text{ for all } m \in D.\]

C. Model theory style: instead of introducing variable assignments that provide the interpretation for variables, we define directly when a formula is satisfied by a sequence of objects: if the variables of \(A\) are among \(y_1, \dots, y_k\) then \(\mathfrak{M} \models A[n_1,\dots, n_k]\) means what \(\mathfrak{M}, s \models A\) means Tarski-style for the assignment \(s\) that maps each \(y_i\) to \(n_i\). Then the clause for the universal quantifier becomes \[\mathfrak{M} \models \forall x \, A[n_1, \dots, n_k] \iff \mathfrak{M} \models A[m, n_1, \dots, n_k] \text{ for all } m \in D.\] This is simple in that it avoids an intermediary function, but can easily be confusing for beginning students because it is neither clean nor precise. We have to understand that \(A\) is a formula with the free variables \(x, y_1\, \dots, y_k\). But what determines the order? Or, put another way, which object interprets which variable?

D. In logic textbooks for philosophers you often see semantics developed for sentences only (i.e., formulas with free variables are avoided). Given a structure \(\mathfrak{M}\) we can define \(\mathfrak{M}[m/a]\) as the structure that’s just like \(\mathfrak{M}\) except the constant \(a\) is interpreted by \(m\in D\). Then we can define truth (not satisfaction) using \[\mathfrak{M} \models \forall x \, A \iff \mathfrak{M}[m/a] \models A[a/x] \text{ for all } m \in D,\] where \(A[a/x]\) is the substitution operation and \(a\) is a constant not already occurring in \(A\).

E. Finally, there’s Robinson-style: we treat every \(m\in D\) as a constant symbol that names itself. Then substituting \(m\) for \(x\) in \(A\) is possible, since \(m\) belongs to both the domain of the structure and to the language, and we can write \[\mathfrak{M} \models \forall x \, A \iff \mathfrak{M} \models A[m/x] \text{ for all } m \in D.\] Naturally, this is not something philosophers like to do because it just seems confused to allow domain elements to function as linguistic symbols naming themselves.

Maybe I’ll find the time to write a paper tracing the origins of all of these at some point. But for now, I wonder: which way is best, pedagogically? Specifically, the Open Logic Project uses Tarski-style, but I’d like to replace it with a definition that is easier to understand and avoids the use of \(x\)-variants. Which would you prefer for the OLP? Which do you prefer in your own teaching or research and why?

]]>For a), you will need a Grade Scheme (see “Schemes” tab on the “Grades” screen) that contains all the letter grades you want to assign. For each grade scale, you need a separate scheme. You might have one scheme for all letter grades (including + and – grades), or one for letter grades and slash grades (A, A/B, B, etc.), or even one with just ✓, ✓+, and ✓-. Once you have such a scheme, you can make the grade item corresponding to the assignment you want to receive, assess, and return in the D2L dropbox of type “Selectbox”. For a selectbox grade item, you have to pick a grade scheme. When you then associate this grade item with a dropbox folder, the assessment pane will give you a drop-down menu rather than a box to enter a numerical score in:

To set up the grade scheme in the first place, and to solve problem b), we have to do some math. D2L does grade schemes in percentages, but we think more naturally of grade point values: an A+ is 4.3, and A is 4, and A- is 3.7, etc. To convert these into percentages, just divide by the maximum score (i.e., 4.3): A+ is 100%, A is 4/4.3 = 93.02%, A- is 3.7/4.3 = 86.04% etc. Do the same for any other grade scheme you want to use. E.g., if you want slash grades, you’d assign 3.5 to A/B, and 3.5/4.3 = 81.50%.

Note that whatever the maximum score is in your grade scheme here should also be the “maximum score” in any individual grade item that uses this score.

The grade scheme in D2L asks for not just an “assigned value %” for each letter grade (or other scale item), but also for a “start %”. If you average the grades (or rather, their “assigned %” values), you may get a value that does not correspond precisely to a letter grade. You have to decide where to start to “round up”. Say, you have three papers. If you want a student to get an A overall only if they get three As, then the “start %” should be the same as the “assigned value %” for an A (93.02%). But maybe you want to give them an A if they turn in two As and an A- (or some other threshold, e.g., 3 As and 2 A- on 5 papers, etc.). If so, compute the average of the grade point values of the threshold pattern, e.g., if A/A/A- earns an A, (4.0+4.0+3.7)/3 = 3.9. Then convert that to a percentage: 3.9/4.3 = 90.70. You probably want to be careful with the start % value of a D: If you have three papers and want to pass a student with a D if they have turned in 2/3 papers with Ds, the start % is (1+1+0)/3/4.3, i.e., 15.50%. But if you have many assignments, a low percentage like that will make it possible to earn a D with an A on a single assignment (if you have 6 papers, and a student gets an A on the first and then never submits another paper, they will earn 15.5% overall. But probably you don’t want to pass that student). For this reason, I like to make the start value of a D the percentage equivalent of 0.9/4.3, or 20.93).

If you’re using some other LMS, you will have to figure out how to do all this there. E.g., Moodle has grading scales (corresponding to D2L’s “assigned %” scale) and also a course-wide system of converting percentages to letter grades, which corresponds to D2L’s “start %” scale.

You can use this Letter Grade Scheme together with other schemes. For instance, you may use it only as the grade scheme for the final grade, but assess other assignments on a more coarse-grained basis, such as slash grades. Some items may even just get a pass/fail or turned-in/not-turned-in assessment. A popular scale for such assignments is ✓, ✓+, and ✓-. You’ll need to decide to assign letter grade or grade point equivalents to such grades for the purpose of calculation (perhaps A, A+, B+) and use these to compute the “assigned %” for your ✓+/- grade scheme. (Unless you want to display a category average using these alternative grade schemes, you can set the “start %” equal to the “assigned %”.) For a scale like ✓+/- remember to add a grade value corresponding to “not turned in” (with assigned value 0%), or else you won’t be able to distinguish between an assignment that’s received a ✓- and one that’s missing.

A final tip: if you’re using grade schemes, having D2L show you the grade symbol and also the percentages will clutter your grade sheet view. So when you create your selectbox grade items, check “override display options” and only leave “scheme symbol” (and maybe “scheme color”) checked. This will also keep students from being confused.

Here is my own letter grade scheme:

Letter grade | Start % | Assigned value | |

F | 0 | 0.00% | 0.00% |

D | 1 | 20.93% | 23.26% |

D+ | 1.3 | 27.91% | 30.23% |

C- | 1.7 | 36.43% | 39.53% |

C | 2 | 44.19% | 46.51% |

C+ | 2.3 | 51.16% | 53.49% |

B- | 2.7 | 59.69% | 62.79% |

B | 3 | 67.44% | 69.77% |

B+ | 3.3 | 74.42% | 76.74% |

A- | 3.7 | 82.95% | 86.05% |

A | 4 | 90.70% | 93.02% |

A+ | 4.3 | 97.67% | 100.00% |

I just developed and tried out a course on formal logic for a 13-week semester. It has:

- a free online textbook:
*forall x: Calgary* - beamer slides for lectures (or screencasts)
- problem sets, which are mostly completed online on Carnap and graded automatically (see here if you want to use Carnap with a different textbook)
- practice problems for Carnap (accessible on carnap.io as well)
- 3 tests (only one converted to online/carnap so far)

Here are the beamer slides. If you’re an instructor and want the sources, drop me an email at rzach@ucalgary.ca. (Of course you’ll also get the sources to the problem sets etc.)

lecturesIf you can bear it, here are screencasts of my talking through the stuff on identity in these lecture slides, and doing some proofs on Carnap.

https://ucalgary.yuja.com/V/PlayList?node=261149&a=1258679219&autoplay=1

]]>My laptop runs Linux (Ubuntu 19.10 to be specific). So there are few options. If you’re on a Mac or Windows machine, there’s lots and you probably don’t need any help. Maybe your University even has a preferred solution for screencasting that integrates with your LMS.

For screencast *recording* on Linux I find Kazam works fine. It’s super-simple, all it does is record the microphone (or computer speaker output) together with whatever happens on your screen (or in a window). So if you want to show your students how to work Overleaf or Carnap or whatever, or if you want to show them a beamer presentation and talk over it, that’s all you need. (Well, you might want to invest in a decent microphone.)

But what if your lecture is chalk-and-talk? You need a way to let yourself “write on the board” while you talk through your proof or whatever. For that you need a handwriting/sketching app and a way to write comfortably (touchscreen/tablet and stylus). I did get a stylus and an Android tablet and tried out a few handwriting apps, but I couldn’t get the palm rejection to work on any of them. (If you rest your palm on the screen, the tablet won’t recognize what your stylus is doing, so you need an app or a stylus that can isolate the stylus from your hand. I’m told iPads are better for that and/or there are active styluses that have palm rejection built in. Not going to buy an iPad just to try that out though.)

I also have a Wacom Intuos writing tablet I got last week in panicked anticipation ($70 US/CAD 90). It works with Linux (plug-and-play USB), just takes a little getting used to. For a handwriting app, I discovered StylusLabs Write. It works really nice. I just fire it up, hit record on Kazam, start writing. It can easily add a new page/whiteboard area, you can scroll back to a previous one easily, and in the end you can save the whiteboard as a PDF. Here’s an example of me talking through the truth lemma in the completeness proof. {Update: See comment below for a vote for xournal++.} {Update 2: OpenBoard now runs on Ubuntu 20.04 — full-feature whiteboard with PDF import functionality and built-in screencast support!}

What is your solution? I made a Google spreadsheet where you can record your solution; maybe it’ll help other instructors who are struggling right now to adapt in the great COVID-19 rush online.

I would prefer to use my ReMarkable for all of this: it has a desktop app for Mac & Windows that shows what you’re drawing on it. So if you have one, try that out! I was hoping to make it work in Linux using srvfb, but have to wait until ReMarkable fixes a bug that turned off ssh access to the tablet. Will let you know what I find out.

]]>