RAF Rebuilding/Modification
vs49688 opened this issue · comments
Relatively simple to do, most of the code is already there. However, there are a few things that need to be taken into account.
When (re)building an archive, how do we determine what exactly goes in that archive? The most straightforward and logical way would be do dump the entire contents of the currently-loaded VFS to a RAF file.
- Could potentially build one massive RAF of the entire game.
- Would need to add filesystem manipulation commands (rm, cp, etc.)
Thoughts?
Perhaps editing in place? Or keeping a log of changed files could work.
What do you think about "Partition Magic"-way?
List of operations that would be applied one-by-one only to the latest versions of files.
Damnit, I wish Github would email me about comments. Sorry I didn't get to this earlier.
That's actually not a bad idea. Wouldn't be that difficult to implement either.
My main issue at the moment is the VFS code. I never designed it for modification, it was originally only meant to be build-once, read-forever.
Honestly, if much more needs to be done to it, I would prefer to ditch it and use an already-completed, tried-and-true filesystem library (possibly https://github.com/google/jimfs). This would make so many more things possible than with the current implementation.
Thoughts?
As of 1e854c3, RAFTools now uses jimfs. What does this mean? Other than a significantly faster loading time, RAF rebuilding is now feasible and relatively simple to implement.
considering this would be the only way for OSX to finally add/modify existing skins that we own, would you be able to implement this sooner than later? the traction of your tool could immensely pick up
I do actually want to implement this, I'm just unsure as to the best way to do it.
How do the existing tools do it? Do they edit an existing patch? Do they create a new one? How do they stop the game from detecting that its files have been changed?
Then there's other issues:
- What is RAFTools expected to do?
- Simply package up the files?
- Or will it be expected to be a fully-fledged 3D modelling+animation software? (Not happening, go away).
- Will the user's know that if they install invalid skins, that the game will most likely crash, and that it's their problem, not mine? The last thing I want is a bunch of script-kiddies flooding these issues with "halp i cant install skinz your program suks" when they install an invalid skin and the game dies.
- I don't want to be held responsible for misuse/abuse of RAFTools.
- This is the safe thing about RAFTools being extraction-only - it can't cause any damage.
Creating a raf/raf.dat pair from the currently-loaded VFS would be the simplest and most straightforward method IMO. If creating a new patch is the way-to-go, users would layout everything on their filesystem, "load" the directory tree into RAFTools (I'd implement this), and then just spit out a new raf/raf.dat pair.
If modifying an existing one is the better method, they'd have to load the existing patch first before applying the changes above. They would then have to manually overwrite the existing RAF files.
How's this sound? I'm open to ideas here.
Unfortunately I don't know the best way to implement or remain undetected. Tho as tools like woofy and others are allowed as they only modify bought skins they are allowed... for the time being. Really Riot only cares if you're modifying the base skin. RAFTools should only extract and update/add files. If the user adds a broken skin that's their trial and error. I recommend keeping this simple so you're just a tool for modifying. That way Riot will have no reason to ban/detect this tool.
TL;DR
- Only extract & update/add
- User deals with broken skins
- I'm not sure the best way to upload/modify the current raf/patch
Simple and reliable tool to edit RAF-archives. Extracting, replacing, rebuilding. That's all we need =)
I think, there are two good options for replacement implementation:
- Replace old data from .dat-file with new one and rebuild raf-manifest; (few files to replace)
- Extract all the files into a single directory and build from it a new archive. (many files to replace)
The first option should cause no problems with the patcher. But the second option is much easier to use.
The problem with editing an existing file is that the .dat files are packed quite tightly. To replace a file, you'd have to essentially shift everything after it. Repeat for each file. Potentially slow, very inefficient, and just plain horrible.
I'm tempted to just dump the current VFS to a .raf/.raf.dat pair. The user loads an existing patch (if needed), can apply their changes, and then RAFTools would build a RAF from that.
The main issue is the "apply their changes" part. The only way this could be done nicely is via file-manipulation commands. I could implement the standard Unix cp, mv, etc. but, most OS' already have these. If I could somehow expose the VFS to the host OS as a drive (or something), the user could make whatever changes they need in their native environment. On Linux/OSX, I could always use FUSE, but I'm not too sure about Windows. Either way, I don't really want multiple implementations.
Regardless, nothing's going to get done in the next few weeks anyway, I've simply too much more-important stuff (uni) to do. So I'll have some time to think about it.
https://github.com/dokan-dev/dokany
Have you seen it?
Last I looked at Dokan it was dead. I'll take another look. If it's being maintained again, I'll consider it
Just had another thought, what about an inbuilt SMB (Windows File Share) server? Most platforms have an implementation of this, so this would work with pretty much everything. Would also allow network users to edit them (if you really wanted to do that).
I could use https://github.com/hierynomus/smbj, it'd be relatively simple to set up.
Also, there's something else that should probably be done. Currently, I'm only using jimfs to represent the file hierarchy; the file data (and the multiple versions) are still handled manually. i.e. Files.readAllBytes(path) will return an empty array.
Ideally, this needs to be fixed. The question there is: How?
Option 1: Read all the bytes from the RAF and write it to the Path - You'd end up with every version that ever existed of every game file all loaded in memory, which is bad.
Option 2: Modify jimfs and hook file reads. Really don't want to do this, high chance for bugs.
Option 3: This will only work if I use a SMB server - Any file changes are stored in the jimfs file system (in memory). These can then be exported as a new "patch".
This is a rather significant change, so any thoughts would be appreciated.
EDIT: Sorry for the massive delay, been working on Thesis. Exams finish in two weeks, so I'll be able to start properly working on RAFTools again then.
Just a quick update: Uni has been running overtime, had to defer an exam due to illness -- hence why there's been no proper activity.
RAF rebuilding functionality will be released within the next two weeks. I also have model viewing (skn/skl) in the works too, so that's something to look forward to, however archive rebuilding is the main priority at this point.
EDIT: I know how this is going to work now. https://github.com/hierynomus/smbj is a client-side library, so it's no good. I'm just going to use Tomcat Embedded and load the WebDAV servlet, which can be mounted as a drive on Windows and Linux (not sure about OSX). The embedded server can be controlled via the console.
EDIT2: Progress!
EDIT3: WebDAV is working well, just doing the GUI configuration. Rebuilding is the next step, and it's proving more finicky than anticipated. I'm working on it...