IR, bytecode->IR compiler, and optimizations

.githooks/pre-commit.sh

@@ -13,7 +13,6 @@ if [ -n "$no_partial_stages" ]; then
     echo "Reformatted a partially-staged file. Re-interactively-stage and commit again."
     exit 1
   }
-  git add -u
 elif [ -n "$everything_staged" ]; then
   # Format only staged changes. We must re-add them because the formats aren't committed.
   mvn spotless:apply "$format_only_staged"

.github/workflows/maven.yml

@@ -20,10 +20,10 @@ jobs:
 
     steps:
       - uses: actions/checkout@v3
-      - name: Set up JDK 21
+      - name: Set up JDK 22
         uses: actions/setup-java@v3
         with:
-          java-version: "21"
+          java-version: "22"
           distribution: "temurin"
           cache: maven
       - name: Set up GNU-R
@@ -47,10 +47,10 @@ jobs:
 
     steps:
       - uses: actions/checkout@v3
-      - name: Set up JDK 21
+      - name: Set up JDK 22
         uses: actions/setup-java@v3
         with:
-          java-version: "21"
+          java-version: "22"
           distribution: "temurin"
           cache: maven
       - name: Set up GNU-R
@@ -65,10 +65,10 @@ jobs:
 
     steps:
       - uses: actions/checkout@v3
-      - name: Set up JDK 21
+      - name: Set up JDK 22
         uses: actions/setup-java@v3
         with:
-          java-version: "21"
+          java-version: "22"
           distribution: "temurin"
           cache: maven
       - name: Set up GNU-R
@@ -83,10 +83,10 @@ jobs:
 
     steps:
       - uses: actions/checkout@v3
-      - name: Set up JDK 21
+      - name: Set up JDK 22
         uses: actions/setup-java@v3
         with:
-          java-version: "21"
+          java-version: "22"
           distribution: "temurin"
           cache: maven
       - name: Set up GNU-R

.gitignore

@@ -1,4 +1,5 @@
 **/src/test/snapshots/**/*.fail*
+!src/test/resources/**/*.log
 
 ### Build ###
 target/

.pmd-rules.xml

@@ -20,7 +20,7 @@ under the License.
 <ruleset name="Default Maven PMD Plugin Ruleset"
          xmlns="http://pmd.sourceforge.net/ruleset/2.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
-         xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 http://pmd.sourceforge.net/ruleset_2_0_0.xsd">
+         xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 https://pmd.sourceforge.io/ruleset_2_0_0.xsd">
 
     <description>
         This is the default pmd ruleset with the following changes:
@@ -40,12 +40,15 @@ under the License.
     <rule ref="category/java/bestpractices.xml/UnusedFormalParameter" />
     <rule ref="category/java/bestpractices.xml/UnusedLocalVariable" />
     <rule ref="category/java/bestpractices.xml/UnusedPrivateField" />
-    <rule ref="category/java/bestpractices.xml/UnusedPrivateMethod" />
+    <rule ref="category/java/bestpractices.xml/UnusedPrivateMethod">
+        <properties>
+            <property name="ignoredAnnotations" value="org.prlprg.parseprint.ParseMethod,org.prlprg.parseprint.PrintMethod" />
+        </properties>
+    </rule>
 
     <rule ref="category/java/codestyle.xml/EmptyControlStatement" />
     <rule ref="category/java/codestyle.xml/ExtendsObject" />
     <rule ref="category/java/codestyle.xml/ForLoopShouldBeWhileLoop" />
-    <rule ref="category/java/codestyle.xml/TooManyStaticImports" />
     <rule ref="category/java/codestyle.xml/UnnecessaryFullyQualifiedName" />
     <rule ref="category/java/codestyle.xml/UnnecessaryModifier" />
     <rule ref="category/java/codestyle.xml/UnnecessaryReturn" />

doc/README.md

@@ -0,0 +1,3 @@
+# Misc documentation
+
+See the Javadoc for better documentation. It can be viewed in IntelliJ or generated with `mvn javadoc:javadoc` (in `site/apidocs/index.html`). In the future, this documentation will probably be moved into Javadoc.

doc/ir.md

@@ -0,0 +1,39 @@
+# Intermediate Representation
+
+Some properties:
+
+- SSA form.
+- Instructions can return multiple values.
+
+## Mutability
+
+`CFG` and `BB`, and `InstrOrPhi`s are mutable by the user. `Instr`s store their content in `InstrData` which is an immutable record for easy construction and pattern matching; the instruction itself must be mutable so its data can be replaced without affecting other instructions' data (which would recursively affect other instructions and so on, making constructing a cyclic dependency impossible besides having ridiculous time complexity).
+
+## Package structure
+
+`BB` has to access internal `CFG` functions to update exits, track/untrack nodes, and split/merge without adding recording operations (because we record the split or merge entirely, we don't want to record adding or removing the BB which is part of it). `BB` also has to access an internal function in `InstrData` to create instructions, so unfortunately we also must put `Instr` in the same package. From there, we put `Node` in the same package too because why not (instructions comprise the majority of nodes, and other nodes are simply auxillary so their constructor must be internal and visible to the instruction).
+
+A lot of other classes are in the `cfg` package.
+
+`CFG` and `BB` are contain all compound operations, because something like `bb.inline(...)` is a lot nicer to use than `BBUtils.inline(BB, ...)` or `BBInliner.inline(BB, ...)`. However, all these methods implemented inline would create a very large file which is bad for the IDE, so they are instead implemented in package-private interfaces (e.g. `CFGCleanup`, `BBInline`) that `CFG` and `BB` implement. This also ensures that the more complex operations don't use internal `BB` methods or methods that they shouldn't need, enforcing looser coupling.
+
+## Terminology
+
+**Replace vs subst**: if an instruction is "replaced", it means occurrences aren't replaced (other instructions that refer to it aren't updated to refer to the instruction that replaced it). If an instruction is "substituted", it means they are.
+
+## Time complexity
+
+Various ways to improve efficiency:
+
+- When possible, substitution is implemented by mutating the instruction (chaging it's internal data), so we don't have to find and mutate the instructions that reference it, which is expensive. However, this isn't possible when subsititing an instruction with an existing instruction, a new instruction which has a different # of return values, or a new instruction whose superclass is a different one of `Phi | Statement | Jump`. Since these cases are expensive, we delay and batch substitute them in `BatchSubst`. The time complexity of batching many of these substitutions is `O(#instructions)`, as opposed to `O(#instructions * #toSubst)` if we substituted immediately.
+
+- We also batch inserting or removing multiple instructions when possible, unless we're inserting or removing from the end of a basic block. Basic blocks store instructions in an array, so inserting or removing instructions from a basic block with a large list of them may be expensive.
+
+General collection data structures:
+
+- The `CFG` class contains a `Map` of basic block ids to the blocks it contains, a `Map` of node ids to nodes the blocks contain, and a `Set` of basic block exits.
+- The `BB` class contains an auto-sorted `SequencedSet` of predecessors, a `SequencedSet` of phis, and an `ArrayList` of statements. Successors are stored inside of the jump, which is `@Nullable Jump`.
+- The `Phi` class (and subclasses) contain an auto-sorted `SequencedSet` of inputs.
+- Instruction data is stored in Java records. Node arguments and jump targets are memoized every time the instruction's data changes.
+
+`BB` and `Node` store a pointer to their `CFG`, but nodes don't store a pointer to their `BB`, except jumps.

doc/rtype.md

@@ -0,0 +1,36 @@
+# `RType` design and debug representation
+
+**TODO:** This is not ideal and maybe we should change it once we actually use types. It seems that the types we need to represent in instructions aren't represented well in this implementation.
+
+`RType` represents the type of SEXPs in the IR. An `RType` consists of an `RValueType`, `RPromiseType`, and `missing: Troolean`. `RValueType` is the type of the value ignoring whether or not it's in a (potentially-lazy) promise or missing, `RPromiseType` determines whether said value may be or is inside a promise and promise details (laziness, known exact promise), and `missing: Troolean` determines whether the value may be or is missing.
+
+Nothing (`⊥`, subtype of everything else) consists of a null `RValueType`, `RPromiseType`, and `missing: Troolean`; in every other `RType`, `RPromiseType` and `missing: Troolean` are both non-null, and the type of the missing value is the only other type where `RValueType` is null Any (`⊤`, supertype of everything else) consists of an "any" `RGenericValueType` that encompasses "any" value that is a supertype of every other `RValueType`, a "maybe lazy, maybe promise-wrapped" `RPromiseType` that is also a supertype of every other `RPromiseType`, and `missing = MAYBE`.
+
+## `RGenericValueType` and `BaseRType`
+
+Each `RValueType` subclass represents a specialized type, such as a primitive function or vector, *except* `RGenericValueType` represents an "any-typed" value as well as other types such as S4 objects. `BaseRType` distinguishes the former "any-typed" `RGenericValueType`s (which can be supertypes of specific `RValueTypes`) from the latter "other-typed" ones (which cannot).
+
+## Common `RType` properties
+
+These are common properties in every `RValueType` which propagate to the `RType`.
+
+- `base: BaseRType`: This is a simplified version of `RType` which only encodes some properties: the SEXPTYPE (or a subset of SEXPTYPEs for functions and non-promises); if the type is a vector, its element type (if static/known); and if the type is a promise, its inner type and whether it's maybe lazy.
+- `exactValue: @Nullable SEXP`: Non-null iff this type has one static, known instance, it's this. For example, the type of an `LdConst` instruction. Of course, any `RType` that is a union of two or more `RValueType`s can't have an exact value.
+- `attributes: AttributesType`: Refines the SEXP's attributes (associated key-value pairs) in the same way `RType` refines the SEXP itself. Some types of SEXPs don't have attributes, in which case this is always `AttributesTypes.NONE`. Other, object SEXPs are guaranteed to have `class` set to the object type.
+- `referenceCount: MaybeNat`: If all instances have a known reference count, this may be set. TODO on its usefulness because there's no case where this is known so far; `MaybeNat` may change to something more specific (e.g. to determine the reference count is "less than X" without knowing it exactly, currently `MaybeNat` can't represent that).
+
+## Specific `RValueType`s
+
+These can be accessed via methods like `function` and `primVec` in `RType`, which return the `RValueType` casted to subclass iff it's an instance *and* the `RPromiseType` and `missing: Troolean` are `VALUE` and `NO` respectively (otherwise the projection methods return `null`).
+
+### Functions
+
+`RFunctionType` is an `RValueType` which contains a set of overloads for the function. Each overload has known parameter types, effects, and a return type. Each parameter has a name, optionality (a parameter is optional iff it has a default value), and required type.
+
+To permit "generic" (parametric polymorphism), the effects and return type are functions of the supplied argument types, e.g. a function can be defined that returns the type of its first argument. However, the parameter types can't be functions of earlier supplied arguments, so a function that takes two arguments of any type, as long as they are the same type, can't be represented.
+
+### Primitive vectors
+
+`RPrimVectorType` encodes a "primitive vector", which is a vector of a number, logical, or string (every vector except generic or expression). It contains (each if static/known) the type of the vector's elements, length, whether the vector (if numeric) has any NA or NaN elements.
+
+TODO