Add more supported compilers (#423)

* Add all compilers supported by spimdisasm

* docs and such

* black

* bump spimdisasm version

* mor docs

* Bump Python version

.github/workflows/black.yml

@@ -10,10 +10,10 @@ jobs:
     name: black
     steps:
     - uses: actions/checkout@v4
-    - name: Set up Python 3.8
+    - name: Set up Python 3.9
       uses: actions/setup-python@v5
       with:
-        python-version: 3.8
+        python-version: 3.9
     - name: Install Dependencies
       run: |
         pip install black

.github/workflows/mypy.yml

@@ -12,10 +12,10 @@ jobs:
     - name: Checkout repo
       uses: actions/checkout@v4
 
-    - name: Set up Python 3.8
+    - name: Set up Python 3.9
       uses: actions/setup-python@v5
       with:
-        python-version: 3.8
+        python-version: 3.9
 
     - name: Install Dependencies
       run: |
@@ -33,10 +33,10 @@ jobs:
     - name: Checkout repo
       uses: actions/checkout@v4
 
-    - name: Set up Python 3.8
+    - name: Set up Python 3.9
       uses: actions/setup-python@v5
       with:
-        python-version: 3.8
+        python-version: 3.9
 
     - name: Install Dependencies
       run: |

CHANGELOG.md

@@ -7,6 +7,11 @@
   * Useful for dealing with symbols with local visibility.
 * Cleanup some redundant code regarding duplicated `asm` segments.
 * The global option `add_set_gp_64` now defaults to `False` on psx and ps2 platforms.
+* Add compiler options supported by spimdisasm.
+  * New compilers: `KMC` (n64), `EGCS` (iQue), `PSYQ` (PS1) and `MWCCPS2` (PS2)
+  * It is highly recommended to use the specific compiler that the game uses (i.e. `KMC`) than just a general option like `GCC` because splat won't be able to adjust as best as it can.
+* `spimdisasm` 1.31.0 or above is now required.
+* Python 3.9 or above is now required.
 
 ### 0.29.0
 

docs/Configuration.md

@@ -54,18 +54,31 @@ platform: psx
 
 Compiler used to build the binary.
 
-splat recognizes the following compilers, and it will adapt it behavior accordingly for them, but unknown compilers can be passed as well:
+splat recognizes the following compilers, and it will adapt it behavior accordingly for them:
+
 - GCC
 - SN64
 - IDO
+- KMC
+- EGCS
+- PSYQ
+- MWCCPS2
+- EEGCC
+
+In general it is better to use a specific disassembler instead of the general `GCC` option, since splat will be able to better adapt to the specific compiler's codegen.
+For example, most N64 games that do not use `IDO` will want to select `KMC` instead of `GCC`, even if `KMC` is just an specific gcc build.
+
+An unknown compiler may be passed as well, but the internal disassembler may complain about it.
 
 #### Usage
+
 ```yaml
 compiler: IDO
 ```
 
 #### Default
-`ido`
+
+`IDO`
 
 
 ### endianness

docs/Quickstart.md

@@ -14,13 +14,13 @@ Copy the `baserom.z64` file into the `mygame` directory inside your home directo
 
 ## System packages
 
-### Python 3.8
+### Python 3.9
 
-Ensure you are have **Python 3.8** or higher installed:
+Ensure you are have **Python 3.9** or higher installed:
 
 ```sh
 python3 --version
-Python 3.8.10
+Python 3.9.10
 ```
 
 If you get `bash: python3: command not found` install it with the following command:

docs/Segments.md

@@ -474,13 +474,13 @@ splat will try to disassemble all the data from this segment as individual doubl
 
 ### `ctor`
 
-`ctor` is used by certain compilers (like MWCC) to store pointers to functions that initialize C++ global data objects.
+`ctor` is used by certain compilers (like MWCCPS2) to store pointers to functions that initialize C++ global data objects.
 
 The disassembly of this section is tweaked to avoid confusing its data with other types of data, this is because the disassembler can sometimes get confused and disassemble a pointer as a float, string, etc.
 
 ### `vtables`
 
-`vtables` is used by certain compilers (like MWCC) to store the virtual tables of C++ classes
+`vtables` is used by certain compilers (like MWCCPS2) to store the virtual tables of C++ classes
 
 The disassembly of this section is tweaked to avoid confusing its data with other types of data, this is because the disassembler can sometimes get confused and disassemble a pointer as a float, string, etc.
 

pyproject.toml

@@ -5,7 +5,7 @@ version = "0.30.0"
 description = "A binary splitting tool to assist with decompilation and modding projects"
 readme = "README.md"
 license = {file = "LICENSE"}
-requires-python = ">=3.8"
+requires-python = ">=3.9"
 classifiers = [
     "Programming Language :: Python :: 3",
     "License :: OSI Approved :: MIT License",
@@ -20,7 +20,7 @@ dependencies = [
 
 [project.optional-dependencies]
 mips = [
-    "spimdisasm>=1.29.0,<2.0.0", # This value should be keep in sync with the version listed on disassembler/spimdisasm_disassembler.py
+    "spimdisasm>=1.31.0,<2.0.0", # This value should be keep in sync with the version listed on disassembler/spimdisasm_disassembler.py
     "rabbitizer>=1.12.0,<2.0.0",
     "pygfxd",
     "n64img>=0.3.3",

requirements.txt

@@ -4,7 +4,7 @@ tqdm
 intervaltree
 colorama
 # This value should be keep in sync with the version listed on disassembler/spimdisasm_disassembler.py and pyproject.toml
-spimdisasm>=1.29.0
+spimdisasm>=1.31.0
 rabbitizer>=1.10.0
 pygfxd
 n64img>=0.1.4

src/splat/disassembler/spimdisasm_disassembler.py

@@ -7,7 +7,7 @@
 
 class SpimdisasmDisassembler(disassembler.Disassembler):
     # This value should be kept in sync with the version listed on requirements.txt and pyproject.toml
-    SPIMDISASM_MIN = (1, 29, 0)
+    SPIMDISASM_MIN = (1, 31, 0)
 
     def configure(self):
         # Configure spimdisasm
@@ -57,22 +57,26 @@ def configure(self):
         rabbitizer.config.pseudos_pseudoMove = False
 
         selected_compiler = options.opts.compiler
+        spimdisasm_compiler = spimdisasm.common.Compiler.fromStr(selected_compiler.name)
+        if spimdisasm_compiler is None:
+            log.write(
+                f"Unsupported selected compiler for spimdisasm: {selected_compiler.name}",
+                status="error",
+            )
+            log.error(
+                f"The following options are supported: {list(spimdisasm.common.compilerOptions.keys())}"
+            )
+        spimdisasm.common.GlobalConfig.COMPILER = spimdisasm_compiler
         if selected_compiler == compiler.SN64:
             rabbitizer.config.regNames_namedRegisters = False
             rabbitizer.config.toolchainTweaks_sn64DivFix = True
-            rabbitizer.config.toolchainTweaks_treatJAsUnconditionalBranch = True
             spimdisasm.common.GlobalConfig.ASM_COMMENT = False
             spimdisasm.common.GlobalConfig.SYMBOL_FINDER_FILTERED_ADDRESSES_AS_HILO = (
                 False
             )
-            spimdisasm.common.GlobalConfig.COMPILER = spimdisasm.common.Compiler.SN64
-        elif selected_compiler == compiler.GCC:
-            rabbitizer.config.toolchainTweaks_treatJAsUnconditionalBranch = True
-            spimdisasm.common.GlobalConfig.COMPILER = spimdisasm.common.Compiler.GCC
-        elif selected_compiler == compiler.IDO:
-            spimdisasm.common.GlobalConfig.COMPILER = spimdisasm.common.Compiler.IDO
-        elif selected_compiler == compiler.EEGCC:
-            spimdisasm.common.GlobalConfig.COMPILER = spimdisasm.common.Compiler.EEGCC
+        rabbitizer.config.toolchainTweaks_treatJAsUnconditionalBranch = (
+            selected_compiler.j_as_branch
+        )
 
         spimdisasm.common.GlobalConfig.DETECT_REDUNDANT_FUNCTION_END = (
             options.opts.detect_redundant_function_end

src/splat/scripts/create_config.py

@@ -185,7 +185,7 @@ def create_psx_config(exe_path: Path, exe_bytes: bytes):
   target_path: {exe_path}
   base_path: .
   platform: psx
-  compiler: GCC
+  compiler: PSYQ
 
   # asm_path: asm
   # src_path: src

src/splat/segtypes/common/codesubsegment.py

@@ -162,9 +162,6 @@ def print_file_boundaries(self):
         assert isinstance(self.rom_start, int)
 
         for in_file_offset in self.spim_section.get_section().fileBoundaries:
-            if (in_file_offset % 16) != 0:
-                continue
-
             if not self.parent.reported_file_split:
                 self.parent.reported_file_split = True
 

src/splat/util/compiler.py

@@ -1,5 +1,5 @@
 from dataclasses import dataclass
-from typing import Optional
+from typing import Optional, Dict
 
 
 @dataclass
@@ -14,11 +14,13 @@ class Compiler:
     c_newline: str = "\n"
     asm_inc_header: str = ""
     asm_emit_size_directive: Optional[bool] = None
+    j_as_branch: bool = False
 
 
 GCC = Compiler(
     "GCC",
     asm_inc_header=".set noat      /* allow manual use of $at */\n.set noreorder /* don't insert nops after branches */\n\n",
+    j_as_branch=True,
 )
 
 SN64 = Compiler(
@@ -29,17 +31,44 @@ class Compiler:
     asm_end_label=".end",
     c_newline="\r\n",
     asm_emit_size_directive=False,
+    j_as_branch=True,
 )
 
 IDO = Compiler("IDO", asm_emit_size_directive=False)
 
+KMC = Compiler(
+    "KMC",
+    j_as_branch=True,
+)
+
+# iQue
+EGCS = Compiler(
+    "EGCS",
+    j_as_branch=True,
+)
+
+# PS1
+PSYQ = Compiler("PSYQ")
+
+# PS2
+MWCCPS2 = Compiler("MWCCPS2")
 EEGCC = Compiler("EEGCC")
 
-compiler_for_name = {"GCC": GCC, "SN64": SN64, "IDO": IDO, "EEGCC": EEGCC}
+compiler_for_name: Dict[str, Compiler] = {
+    x.name: x
+    for x in [
+        GCC,
+        SN64,
+        IDO,
+        KMC,
+        EGCS,
+        PSYQ,
+        MWCCPS2,
+        EEGCC,
+    ]
+}
 
 
 def for_name(name: str) -> Compiler:
     name = name.upper()
-    if name in compiler_for_name:
-        return compiler_for_name[name]
-    return Compiler(name)
+    return compiler_for_name.get(name, Compiler(name))

src/splat/util/n64/rominfo.py

@@ -434,7 +434,7 @@ def get_compiler_info(rom_bytes, entry_point, print_result=True):
         elif insn.uniqueId == rabbitizer.InstrId.cpu_b:
             branches += 1
 
-    compiler = "IDO" if branches > jumps else "GCC"
+    compiler = "IDO" if branches > jumps else "KMC"
     if print_result:
         print(
             f"{branches} branches and {jumps} jumps detected in the first code segment."