AP/IB Infrastructure Update

[benjamin-shen]

Reference

Some of the AP exams used as examples in this issue are presented below, for quick reference.

Subject	Credits	Placement
Chemistry	4	Placement out of CHEM 2070 or CHEM 2090. A student taking CHEM 1560, CHEM 2070, or 2090 will forfeit AP credit.  Students taking CHEM 2150 will retain AP credit. A student may also receive credit for CHEM 2070 or CHEM 2090 by passing an exam given during Fall orientation. A distinct exam is given before the start of the Spring semester that, if passed, will allow a student to receive credit for CHEM 2080. See chemistry.cornell.edu/course-information for further information.
Computer Science A	4	Placement out of CS 1110. Department also offers placement exam during fall orientation.
Statistics (excluding engineering students)	4	Placement out of AEM 2100, BTRY 3010, BTRY 6010, ILRST 2100, ILRST 6100, MATH 1710, PAM 2100, PAM 2101, PSYCH 3500, SOC 3010, STSCI 2100, STSCI 2150, STSCI 2200. Students in the College of Agriculture and Life Sciences will find credit and placement information at cals.cornell.edu/academics/registrar. Students in Hotel Administration will find credit and placement information in the SHA Student Handbook.
Mathematics BC	8	Placement out of Calculus I (MATH 1106 or MATH 1110) and Calculus II (MATH 1120, MATH 1220, or MATH 1910).
Microeconomics	3	Placement out of ECON 1110 and HADM 1410.
Spanish Literature	3	Department of Romance Studies determines placement. Students should take the LPS and afterwards the CASE to obtain appropriate placement.

Current Infra

Exams can be represented by a set of equivalent courses (aka course equivalents), which are represented as CourseTakens outside of a user's semester courses. They are computed based on the user's college/major, taken exams, and those exams' scores. The equivalent courses are then added to the user's taken courses to be used in the requirements algorithm.

For example, say the user took APCS. Our current algorithm, at the graph computation, converts that to a CS 1110 equivalent course. This is simply a CourseTaken with the same course ID as CS 1110 -- it has a different code ('AP Computer Science A' instead of 'CS 1110') and, in other cases, can have a different amount of credits. Unlike regular courses, the stable unique ID is a string ('AP Computer Science A') rather than a number; see PR #532. This exam to course ID mapping is data we manually collected and is stored in data/exams/ExamCredit.ts.

We need to account for a few special considerations. If an exam can place out of two or more courses, the equivalent courses get split into an additional CREDITS_COURSE_ID special course (i.e. a CourseTaken with a custom course ID). This is because we don't necessarily know how to split up the exam's "credit" (used the same way as in course credit) between the multiple equivalent courses. If an exam doesn't have any equivalent courses, we give the user an equivalent NO_EQUIVALENT_COURSES_COURSE_ID special course. This is because the user might want to opt in/out to a requirement using the equivalent course (eg. foreign language).

Rationale for Change

So, while the algorithm is relatively straightforward when there is a one-to-one mapping from requirements to exams to courses, it gets more complex if there is a one-to-many or one-to-zero mapping. Despite the complexity, this is manageable and not an infrastructure-breaking issue.

The bigger issue is that having a stable unique ID for equivalent courses creates potential problems related to user choice. Specifically, if an exam has multiple equivalent courses, any user choice involving the exam will involve every equivalent course (since they all have the same unique ID). The tradeoff of having a stable unique ID is that the "unique" ID is not actually unique. I think this issue currently happens to be user-invisible but it is still very problematic for the requirements algorithm, which has a uniqueness invariant throughout the code. It will likely break things in the future if not fixed now.

In hindsight, exams with many equivalent courses (eg. AP Statistics) may also be problematic. It is theoretically possible for two equivalent courses to be able to fulfill a single requirement. This introduces unaccounted edge cases in user choice and slot fulfillments, where a single entity is acting as multiple courses. The whole approach just seems unideal and, at times, counteractive to the rest of the requirements algorithm.

Alternative Solutions We Discussed

(optional reading)

1. Allow users to choose which equivalent course to use for each exam

Our first instinct should be to force a one-to-one mapping from exam to equivalent course. In this solution, all the equivalent courses will be listed in a dropdown / radio group and the user will be prompted to pick one. This dropdown can be displayed in the Onboarding modal (which is getting pretty crowded), or another tab like a future User Choice tab (which doesn't seem like it'll exist any time soon).

Despite forcing a one-to-many mapping, there are a couple of issues caused by the nuances of AP/IB exams:

• Many of the OR relations on the website should already be implicitly determined by the user's college (eg. AP Chemistry).
• There is no way to represent AND logic (eg. AP Microeconomics), or more complex AND/OR logic (eg. AP Mathematics BC). If we make the user choice less strict (i.e. checkbox UI), then the one-to-many mapping issue comes back.
• For some exams, it is ambiguous whether equivalent courses can be applied in an AND or OR relation (eg. AP Statistics). These can't even be stored in the data.

Thinking back, this is reminiscent of the very first proposal for storing AP/IB exam data. This solution is great for exams with equivalent courses in an OR relation. Unfortunately, it is not a viable solution for all exams. This exploration tells us that we cannot use equivalent courses directly in the requirements computation.

2. Give each exam a custom course ID and update all the checkers

Instead, let's see if we can develop a one-to-one mapping from exam to CourseTaken. I will call these CourseTakens "exam courses" to avoid confusion with equivalent courses. An exam's corresponding exam course should somehow capture all the granularity and complexity of all the exam's equivalent courses. We can do this by manually updating all the requirement checkers that we know can be fulfilled by an AP/IB exam.

Besides fixing the issues mentioned in solution 1, it would also become much more robust to check if a course is AP/IB: we can look at the course ID instead of the course subject. Sadly, there are also some really big downsides:

• There is an enormous amount of data that would need to be updated. New majors/minors would always need to be checked for AP/IB exam fulfillment.
• The data on the website does not reliably tell us the equivalent courses, let alone which requirement checkers to update. The best we could do is to write a script that tells us which requirements are satisfied by the equivalent courses, but updating the checkers from there is a manual process.

I think the downsides significantly outweigh the upsides. This solution is probably not even feasible.

3. Scrap the data and make AP/IB exams exclusively opt-in

What if we gave up on trying to represent AP/IB credit? For instance, we could let the requirement fulfillments be completely determined by user choice (arbitrary opt in).

This solution essentially scraps all of the data and code in the old infrastructure. It's nice because it solves our perpetual battle with inaccurate/outdated/ambiguous data, and conveniently fixes the foreign language edge cases (eg. AP Spanish Literature). The main downside is that it provides no structure for AP/IB fulfillment:

• This solution makes no distinction between, say, applying AP Computer Science A to the Computing requirement and applying it to the FWS requirement. Ideally, they should be different kinds of opt in.
• We do have a lot of good data. At least for the exams we can represent, we should try to help the user as much as we can.

This is a great solution because it's feasible and doesn't really have any infrastructure to break. Plus the downsides aren't horrible! But, as a team, we have brainstormed a best-of-worlds solution that addresses all of the aforementioned downsides. This solution is presented below.

Proposed Infra

The final proposed solution is very similar to alternative solution 2. However, instead of manually updating the checkers, we programmatically update the requirements json.

In this new infrastructure, exams directly fulfill requirements based on their equivalent courses. Instead of populating the user's taken courses with equivalent courses, we populate them with exam courses. Each exam always has an exact one-to-one correspondence with an exam course. Each exam course has a custom course ID, such as 100, 101, .... Ultimately, these don't really matter because they will not stored in the database. If a dev changes these course IDs, they would just need to re-generate the requirements-courses mapping (i.e. npm run req-gen); no migration script is necessary.

We essentially are cutting out the equivalent courses middleman and replacing it with an exam course. In the old infrastructure, exams can fulfill certain requirements through their equivalent courses. In the new infrastructure, exams are associated with exam courses, which can directly fulfill requirements based on their equivalent courses. The fundamental one-to-one relation of exams to exam courses means that exams become much closer to being an analog of semester courses.

This simplifies and fixes a lot of the current issues. It solves all the downsides of alternatives solutions 1, 2, and 3. AP credit is no longer split between equivalent courses, and user choice accurately represents the user's intention behind their choice. In hindsight, we should have started out with this infrastructure.

However, there are a considerations and potential downsides we should be aware of.

• This infrastructure is highly dependent on the new opt in/out system working -- but to be fair, so was the old infrastructure.
• Edge cases still need to be represented. We currently have one way of broadly representing edge cases: the disallowTransferCredit flag (I'm not sure how accurate it is at this point). However, other edge cases like disallowed majors may no longer be accurate and we would need to put them in as additional checker/requirement data. I don't think this is worth the LOE, since users can just opt out.
• There is no support for choosing which equivalent courses the user wants in an exam with an OR relation. In this case, the user must opt out of everything they don't want the exam to be applied to.
• Issues related to AP/IB fulfillment are arguably more difficult to debug than the old infrastructure because of the indirection of equivalent courses. However, we don't have a debugging system in place right now, and so far we haven't really ran into such bugs.

Tasks

The implementation of the new infrastructure can be broken down into the following steps.

• give each exam a custom course ID (aka "exam ID")
• write script to generate mapping from equivalent course ID to exam ID
• apply mapping to the requirements json (so when you run npm run req-gen, exam IDs show up under the appropriate requirements' courses)
• add exam courses and remove equivalent courses from the user's taken courses
• remove old code and tests

Other Notes

While AP/IB exams are heavily dependent on user choice, I don't think this infrastructure migration is dependent on any of the current opt in/out development. If the migration is somehow implemented before opt in/out, any user choices stored in the database should just be able to be migrated normally with the regular courses.

[hahnbeelee]

@benjamin-shen How will the infra look for the statistics course which can fulfill multiple classes? Does it address that problem? Or is the goal that eventually the user will opt out of the ones that the AP exam fulfills?