88 Commits

Author SHA1 Message Date
Rainchus
989a86b369
Make osPiReadIo no longer ignored (#133) 2025-02-27 16:58:59 -05:00
Wiseguy
d660733116
Implement reloc pairing GNU extension (#131) 2025-02-13 18:20:48 -05:00
Wiseguy
8781eb44ac
Add a mechanism to provide original section indices for jump table regeneration (#130) 2025-02-11 22:36:33 -05:00
Wiseguy
be65d37760
Added config option parsing for mod toml and populate them in the mod manifest (#129) 2025-01-31 02:36:33 -05:00
Wiseguy
2af6f2d161
Implement shim function generation (#128) 2025-01-30 23:48:20 -05:00
Wiseguy
198de1b5cf
Move handling of HI16/LO16 relocs for non-relocatable reference sections into elf parsing by patching the output binary (fixes patch regeneration) (#127) 2025-01-30 02:54:27 -05:00
Wiseguy
b18e0ca2dd
Fix function signature for RSP microcodes that don't have overlays (#126) 2025-01-26 22:32:15 -05:00
Wiseguy
b2d07ecd5a
Renamed mod manifest to mod.json, added display_name, description, short_description fields (#125) 2025-01-26 21:57:00 -05:00
Wiseguy
38df8e3ddc
Mod function hooking (#124)
* Add function hooks to mod symbol format

* Add function sizes to section function tables

* Add support for function hooks in live generator

* Add an option to the context to force function lookup for all non-relocated function calls

* Include relocs in overlay data

* Include R_MIPS_26 relocs in symbol file dumping/parsing

* Add manual patch symbols (syms.ld) to the output overlay file and relocs

* Fix which relocs were being emitted for patch sections

* Fix sign extension issue with mfc1, add TODO for banker's rounding
2025-01-26 21:52:46 -05:00
Ethan Lafrenais
36b5d9ae33
PIC Jump Table Support (#120)
* Support for $gp relative jump table calls
2025-01-16 00:40:50 -05:00
Ethan Lafrenais
916d16417e
RSPRecomp overlay support (#118)
* RSPRecomp overlay support

* Change overlay_slot.offset config to text_address
2025-01-16 00:32:29 -05:00
Ethan Lafrenais
53ffee96fd
Add ldl, ldr, sdl, sdr implementations (#119) 2025-01-12 22:43:46 -05:00
LittleCube
49bf144b0d
Add TRACE_RETURN (#117) 2025-01-04 22:10:29 -05:00
LittleCube
351482e9c6
Fix TRACE_ENTRY and move function_sizes (#112) 2025-01-04 21:49:31 -05:00
Wiseguy
6dafc108f3
Skip internal symbols when dumping context (#116) 2025-01-02 20:50:46 -05:00
Wiseguy
fc696046da
Fix some calling convention issues with the live recompiler (#115) 2024-12-31 19:12:54 -05:00
Wiseguy
66062a06e9
Implement live recompiler (#114)
This commit implements the "live recompiler", which is another backend for the recompiler that generates platform-specific assembly at runtime. This is still static recompilation as opposed to dynamic recompilation, as it still requires information about the binary to recompile and leverages the same static analysis that the C recompiler uses. However, similarly to dynamic recompilation it's aimed at recompiling binaries at runtime, mainly for modding purposes.

The live recompiler leverages a library called sljit to generate platform-specific code. This library provides an API that's implemented on several platforms, including the main targets of this component: x86_64 and ARM64.

Performance is expected to be slower than the C recompiler, but should still be plenty fast enough for running large amounts of recompiled code without an issue. Considering these ROMs can often be run through an interpreter and still hit their full speed, performance should not be a concern for running native code even if it's less optimal than the C recompiler's codegen.

As mentioned earlier, the main use of the live recompiler will be for loading mods in the N64Recomp runtime. This makes it so that modders don't need to ship platform-specific binaries for their mods, and allows fixing bugs with recompilation down the line without requiring modders to update their binaries.

This PR also includes a utility for testing the live recompiler. It accepts binaries in a custom format which contain the instructions, input data, and target data. Documentation for the test format as well as most of the tests that were used to validate the live recompiler can be found here. The few remaining tests were hacked together binaries that I put together very hastily, so they need to be cleaned up and will probably be uploaded at a later date. The only test in that suite that doesn't currently succeed is the div test, due to unknown behavior when the two operands aren't properly sign extended to 64 bits. This has no bearing on practical usage, since the inputs will always be sign extended as expected.
2024-12-31 16:11:40 -05:00
LittleCube
0d0e93e979
Check if mod context is good in mod tool (#113) 2024-12-26 18:49:22 -05:00
LittleCube
17438755a1
Implement nrm filename toml input, renaming list, trace mode, and context dumping flag (#111)
* implement nrm filename toml input

* change name of mod toml setting to 'mod_filename'

* add renaming and re mode

* fix --dump-context arg, fix entrypoint detection

* refactor re_mode to function_trace_mode

* adjust trace mode to use a general TRACE_ENTRY() macro

* fix some renaming and trace mode comments, revert no toml entrypoint code, add TODO to broken block

* fix arg2 check and usage string
2024-12-24 02:10:26 -05:00
LittleCube
d33d381617
Show error when zip command is not found by linux (#94)
* Show error when zip command is not found by linux

* Exit on linux if zip command is not found
2024-09-15 17:59:19 -04:00
Wiseguy
d5ab74220d
Various mod fixes (#95)
* Terminate offline mod recompilation if any functions fail to recompile

* Fixed edge case with switch case jump table detection when lo16 immediate is exactly 0

* Prevent emitting duplicate reference symbol defines in offline mod recompilation

* Fix function calls and add missing runtime function pointers in offline mod recompiler
2024-09-12 18:54:08 -04:00
Wiseguy
cc71b31b09
Modding Support PR 2 (Finished mod tool base feature set and improvements for use in N64ModernRuntime) (#93)
* Remove reference context from parse_mod_symbols argument

* Add support for special dependency names (self and base recomp), fix non-compliant offline mod recompiler output

* Fix export names not being set on functions when parsing mod syms, add missing returns to mod parsing

* Switch offline mod recompilation to use a base global event index instead of per-event global indices

* Add support for creating events in normal recompilation

* Output recomp API version in offline mod recompiler

* Removed dependency version from mod symbols (moved to manifest)

* Added mod manifest generation to mod tool

* Implement mod file creation in Windows

* Fixed some error prints not using stderr

* Implement mod file creation on posix systems

* De-hardcode symbol file path for offline mod recompiler

* Fix duplicate import symbols issue and prevent emitting unused imports
2024-09-09 22:49:57 -04:00
Wiseguy
5b17bf8bb5
Modding Support PR 1 (Instruction tables, modding support, mod symbol format, library conversion) (#89)
* Initial implementation of binary operation table

* Initial implementation of unary operation table

* More binary op types, moved binary expression string generation into separate function

* Added and implemented conditional branch instruction table

* Fixed likely swap on bgezal, fixed extra indent branch close and missing
indent on branch statement

* Add operands for other uses of float registers

* Added CHECK_FR generation to binary operation processing, moved float comparison instructions to binary op table

* Finished moving float arithmetic instructions to operation tables

* Added store instruction operation table

* Created Generator interface, separated operation types and tables and C generation code into new files

* Fix mov.d using the wrong input operand

* Move recompiler core logic into a core library and make the existing CLI consume the core library

* Removed unnecessary config input to recompilation functions

* Moved parts of recomp_port.h into new internal headers in src folder

* Changed recomp port naming to N64Recomp

* Remove some unused code and document which Context fields are actually required for recompilation

* Implement mod symbol parsing

* Restructure mod symbols to make replacements global instead of per-section

* Refactor elf parsing into static Context method for reusability

* Move elf parsing into a separate library

* WIP elf to mod tool, currently working without relocations or API exports/imports

* Make mod tool emit relocs and patch binary for non-relocatable symbol references as needed

* Implemented writing import and exports in the mod tool

* Add dependencies to the mod symbol format, finish exporting and importing of mod symbols

* Add first pass offline mod recompiler (generates C from mods that can be compiled and linked into a dynamic library)

* Add strict mode and ability to generate exports for normal recompilation (for patches)

* Move mod context fields into base context, move import symbols into separate vector, misc cleanup

* Some cleanup by making some Context members private

* Add events (from dependencies and exported) and callbacks to the mod symbol format and add support to them in elf parsing

* Add runtime-driven fields to offline mod recompiler, fix event symbol relocs using the wrong section in the mod tool

* Move file header writing outside of function recompilation

* Allow cross-section relocations, encode exact target section in mod relocations, add way to tag reference symbol relocations

* Add local symbol addresses array to offline mod recompiler output and rename original one to reference section addresses

* Add more comments to the offline mod recompiler's output

* Fix handling of section load addresses to match objcopy behavior, added event parsing to dependency tomls, minor cleanup

* Fixed incorrect size used for finding section segments

* Add missing includes for libstdc++

* Rework callbacks and imports to use the section name for identifying the dependency instead of relying on per-dependency tomls
2024-08-26 23:06:34 -04:00
Wiseguy
f8d439aeee
Add option to output multiple functions per file, defaults to 50 (#88) 2024-08-15 00:17:09 -04:00
Mr-Wiseguy
424a509b22 Allow zero-sized symbols in a specific range for patches, fixed cases where errors didn't terminate the recompiler 2024-08-15 00:00:25 -04:00
Mr-Wiseguy
4161ef68cc Made recompilation header include configurable 2024-08-15 00:00:25 -04:00
Mr-Wiseguy
dfd90057c9 Rewrote jal resolution to improve handling for overlapping overlays 2024-08-15 00:00:25 -04:00
Wiseguy
ba4aede49c
Add symbol reference file mechanism for elf recompilation (#82)
* Consolidate context dumping toggle into a single bool, begin work on data symbol context dumping
* Added data symbol context dumping
* Fix mthi/mtlo implementation
* Add option to control unpaired LO16 warnings
2024-07-02 21:42:22 -04:00
Anghelo Carvajal
16819a0515
Add libgultra_d functions to ignored_funcs (#60) 2024-06-21 08:38:15 -04:00
Scott Brenner
ecc9dc49e5
Bump actions/checkout to v4 (#76) 2024-06-10 12:31:50 +02:00
dcvz
ca096fd078 CI: Fix Windows CI after image updates 2024-06-07 10:42:47 +02:00
David Chavez
2a2df89349
Remove disable_warnings header include (#75) 2024-06-05 00:02:37 +02:00
David Chavez
8dfed04919
Update headers to use new namespace in N64ModernRuntime (#74) 2024-06-03 21:46:42 +02:00
Gilles Siberlin
6eb7d5bd3e
Implement hook insertion (#73)
* Implement function hook insertion

* Fix recompiled code indentation

* Add _matherr to renamed_funcs

* Replace after_vram by before_vram

* Emit dummy value if relocatable_sections_ordered is empty
2024-05-31 23:31:50 -04:00
David Chavez
5c687ee962
CI: Add Linux arm64 (#69) 2024-05-31 22:48:42 +02:00
Anghelo Carvajal
94b59d56f7
Mask the lower 3 bits of the de RSP operand (#55) 2024-05-17 22:38:19 -04:00
Mr-Wiseguy
e0e52d1fc3
Symbol file toml update (#52)
* Symbol input file mechanism

* Migration to new toml lib

---------

Co-authored-by: dcvz <david@dcvz.io>
2024-05-16 22:33:08 -04:00
Anghelo Carvajal
26c5c2cbb8
Implement tail calls in the middle of functions and allow recomping libgcc math routines (#43) 2024-05-14 18:56:23 -04:00
Anghelo Carvajal
5d46de6de0
Add more renamed_funcs (#35) 2024-05-14 15:06:47 -04:00
Tharo
d4fab15fcc
RSPRecomp: Add some features to help debug unhandled indirect jump targets (file+line of instruction and register dump) (#33) 2024-05-13 21:00:00 -04:00
David Chavez
706e7c5069
Add Initializers for Structs - Fix issue with Apple Clang (#31)
Fixes #30 also adds CI
2024-05-13 20:55:43 -04:00
Anghelo Carvajal
d7b223fde5
Implement bgezal/bgezall (#23) 2024-05-12 22:07:46 -04:00
Mr-Wiseguy
3ab0edf18a Changed RSPRecomp to take a toml config file instead of using hardcoded options 2024-05-12 20:30:02 -04:00
Tharo
dbf0e623c8
Implement doubleword multiply and divide, and conversions between doubleword and single/double precision floats (#16) 2024-05-11 14:08:15 -04:00
Mr-Wiseguy
5838e4223a Add license and readme 2024-05-10 17:19:30 -04:00
Mr-Wiseguy
0f813247a1 Added CMake build system, fixed warnings on clang, removed VS solution/projects 2024-04-20 21:05:50 -04:00
Mr-Wiseguy
32841e85b4 Changed RSP recompiler to emit different macros for loads and stores to handle unaligned access 2024-04-20 20:02:28 -04:00
Mr-Wiseguy
47c9033b61 Improved reloc pairing 2024-04-20 20:00:48 -04:00
Mr-Wiseguy
50d55bd171 Added manual sections input option, fixed bug with multiplications and added mthi/lo instructions 2024-04-20 20:00:29 -04:00
Mr-Wiseguy
72fe4ed79c Added some missing float comparison instructions and fixed functions incorrectly getting detected as the entrypoint 2024-03-11 00:15:31 -04:00