Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Palworld + Voices of the Void support, initial Unreal engine support #1079

Merged
merged 24 commits into from
Feb 9, 2024

Conversation

ethangreen-dev
Copy link
Contributor

@ethangreen-dev ethangreen-dev commented Sep 23, 2023

Voices of the Void is a game about hunting aliens and the wacky / spooky events that happen as a result.

UE4SS (https://github.com/UE4SS-RE/RE-UE4SS) is the standard modloader for a variety of Unreal engine games.

Shimloader is my WIP modloader-loader. It shims the game and UE4SS (the actual modloader) to allow Unreal to load blueprint (.pak files) and UE4SS (.lua) mods from external directories, thus enabling us to use the R2MM "linker" method of mod management.

Blueprint and eu4ss mods are respectively installed into:

  • PROFILE/shimloader/bp_mods (mapped to VotV/Content/Paks/LogicMods)
  • PROFILE/shimloader/ue4ss_mods (mapped to VotV/Binaries/Win64/Mods)

Config files are of .modconf extension and are installed into PROFILE/shimloader/config, which is then mapped to VotV/Content at runtime.

These paths are then injected into the game via the --ue4ss-mods, --bp-mods and --config-dir CLI args, which are then captured by the shimloader binary and used to setup win32 filesystem redirection logic.

The shimloader package can be found here:
https://thunderstore.io/c/voices-of-the-void/p/0xFFF7/votv_shimloader/

@CLAassistant
Copy link

CLAassistant commented Sep 23, 2023

CLA assistant check
All committers have signed the CLA.

@MythicManiac
Copy link
Collaborator

Nice! I skimmed this through briefly and noticed uninstallation seems to not be implemented for the shimloader at least. Whether or not the mods themselves get uninstalled properly is another question that would be good to double-check, since IIRC the way uninstalling works isn't the most reliable at the moment (I seem to remember that the uninstaller didn't necessarily have enough metadata available about what it's uninstalling to really be intelligent about it, but can't remember anymore)

Another suggestion I'd have is to set up a templated way to configure games to use this pattern (specifically the install rules), e.g. akin to how it's done with bepinex now.

I'll do a more thorough review once you've checked the uninstaller aspect, but could also be it's already handled & I just glanced over it so let me know if that's the case.

@ethangreen-dev
Copy link
Contributor Author

@MythicManiac

I've been working through uninstalling for the better part of this morning and haven't yet found a good solution. Top-level binaries are cleaned up a-ok, but shimloader/ue4ss_mods and its contents are not removed. Blueprint mods (which contain just .pak files) can be uninstalled just fine though.

Similarly, ue4ss lua mods are also not currently installable. These are mods which contain script files (usually) in the following directory structure:

Author-ExampleVotVMod:
- VotV/Binaries/Win64/Mods/ExampleVotVMod/
- - Scripts/
- - - main.lua
- - - secondary_script.lua
- - - ...

I have yet to investigate implementing a custom install rule, but the goal here would be to filter by the .lua extension, resolve the second parent of the script file's path, and then move that directory into shimloader/ue4ss_mods. This would result in the following dir structure:

Author-ExampleVotVMod:
- Profile/Default/shimloader/ue4ss_mods/ExampleVotVMod
- - Scripts/
- - - main.lua
- - - secondary_script.lua
- - - ...

Shouldn't be awful to implement.

@ethangreen-dev
Copy link
Contributor Author

ethangreen-dev commented Sep 28, 2023

Made further tweaks to get complete shimloader uninstallation working. This necessitated the following changes:

  • Added a mod: ManifestV2 argument to the ProfileInstallerProvider::installModLoader abstract function. This makes it convenient for modloader install scripts to interact directly with state files.
  • Added state file tracking support to the shimloader install script. This is done by instantiating a new GenericProfilerInstaller and using its addToStateFile function with the provided mod argument.
  • Reworked GenericProfileInstaller::uninstallState to recursively remove directories whose contents were removed as a part of the uninstall process. This will not remove directories with contents not targeted by the state file.

@ethangreen-dev ethangreen-dev marked this pull request as ready for review September 28, 2023 19:09
@MythicManiac MythicManiac self-requested a review October 21, 2023 07:18
@ethangreen-dev
Copy link
Contributor Author

Updates -

  • Made the Shimloader installer completely generic
  • Added Palworld support

These changes are entirely dependent on the following:

  1. The creation of a palworld community.
  2. Me submitting unreal_shimloader to the newly created community.

@ethangreen-dev ethangreen-dev changed the title Add Voices of the Void support, initial Unreal engine support Add Palworld + Voices of the Void support, initial Unreal engine support Jan 23, 2024
@ethangreen-dev ethangreen-dev force-pushed the votv-and-shimloader branch 3 times, most recently from 966cc25 to a7a5acd Compare January 28, 2024 20:00
Copy link
Contributor

@Oksamies Oksamies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed together with the author.

@ethangreen-dev ethangreen-dev force-pushed the votv-and-shimloader branch 5 times, most recently from 489de3b to 6061c6b Compare January 30, 2024 21:51
@ethangreen-dev ethangreen-dev marked this pull request as draft February 3, 2024 19:01
implement VotV + Palworld support via unreal_shimloader

Voices of the Void is a game about hunting aliens and the wacky /
spooky events that happen as a result.

Palworld is, well, Palworld. You know what that is by now.

Shimloader is my WIP modloader-loader which shims UE and the UE4SS
loader to provide R2MM manager support. It allows Unreal to load
blueprint mods (.pak) files and ue4ss mods (.lua et al) from external
directories.

Blueprint and UE4SS mods are installed into the following directories:
- PROFILE/shimloader/pak_mods
- PROFILE/shimloader/lua_mods
which are then injected into the game at runtime by shimloader.

CHANGES:
- add Palworld and VotV assets + rule registration
- add support for unreal_shimloader
- add package mappings for metherul-unreal_shimloader and
  0xFFF7-votv_shimloader.
- add UE4SS_LUA and SUBDIR tracked installation rules
- add InstallRules_Shimloader
- add shimloader game launch instructions
- add .modconf and .ini config file extensions
CHANGES:
- fix UE4SS-settings.ini content reset when mods are disabled and
  re-enabled.
- fix an invalid shimloader package reference. This changes it from
  `metherul-unreal_shimloader` to `Thunderstore-unreal_shimloader`.
- fix invalid shimloader install target. This changes it from `dxgi.dll`
  to `dwmapi.dll`.
@ethangreen-dev ethangreen-dev marked this pull request as ready for review February 7, 2024 18:05
Improve the test data populator to actually create a file structure
which matches what is expected of the current tests as opposed to having
to manually build the structure
Delete the now unnecessary SUBDIR_TRACKED install rule type
Revert the changes done to the uninstallState function to reduce the
surface area of potentially impacted code/logic of these changes. The
changes that were done are no longer necessary after all.
As the stat tracking isn't generally used for the core mod loader
installers, remove it for shimloader as well. This also eliminates the
need for a handful of the workarounds that were in place to exempt a
specific file from being treated as any other file normally would for
state tracking.

This does not yet solve the initial "config should be placed in the game
dir" problem; solution for that upcoming as a followup.
Adjust the mod linker to copy the shimloader config file to the correct
location (game root directory) instead of the binary directory.
This was done in a separate commit to reduce diff noise in the previous
one
@MythicManiac MythicManiac merged commit fcb5151 into ebkr:develop Feb 9, 2024
1 check passed
@Buckminsterfullerene02
Copy link

Buckminsterfullerene02 commented Feb 18, 2024

So what about supporting the automatic installation/uninstallation of basic asset replacement pak mods? I.e. not UE4SS ones that go into Content/Paks/ folder and are "patch paks" by appending the pak name with _P (though usually in order to prioritise load order with games with pakchunks they can be installed into Content/Paks/~mods/). However with this support comes some additional complications:

  • What if the game is using IoStore container files (which has .pak, utoc, .ucas)?
  • What if the game is using pak signing (which requires .sig to be included alongside the mod pak)?

I also have some additional questions regarding game compatibility through UE4SS:

  • What if patternslueth scans fail and custom sigs are required?
    • How are you going to support installing/uninstalling UE4SS_Signatures for different games?
    • What if the game updates and the sigs need to be changed for all mods to work?
    • Who will be responsible for updating the sigs?
  • Will you support games that require custom VTables or UE4SS-settings.ini files (as this is the case for some)?

Some extra questions:

  • For C++ mods, will you run the .dll files through VirusTotal to check for malware? It's a plausible risk that C++ mods may contain malicious code and it is vital that there are checks in place to do this when handling automatic mod installing.
  • Unity mods (I haven't modded Unity before so I don't know exactly what handles it) on r2modman handle configs through the GUI, I'm assuming due to a common config library (probably Bepinx?). I'm wondering if the same could be done for UE4SS-settings.ini?
    • This is a bit of a stretch, but possibility this could be done for lua mod configs. While technically lua mod configs isn't an "official" method (through UE4SS platform), a lot of lua mods have started adding configs through a seperate config.lua file in the mod's root directory. But I think this is definately a thing for me to discuss with the rest of the UE4SS team, if we should support this type of thing ourselves.
  • I noticed Palserver isn't added as a supported game, but r2modman has a "game" and "server" section, and UE4SS works exactly the same way on it (on windows version at least, but native linux support is coming soon). Is there a reason for this?
  • Do you plan on supporting the BP mod loader mod load order system? Or supporting load order at all for that matter? Load order can be controlled through naming of pak patches, and through lua/C++ mods by changing the order of the mods inside of mods.txt file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants