Dirt Rally 2.0 - Support for game.dat

Question

Dirt Rally 2.0 - Support for game.dat

opcod3 opened this issue 5 years ago · comments

In the Dirt Rally 2.0 the files game.dat and game_1.dat are actually NEFS archives with the header information stripped. The headers can be found inside the executable and once reconstructed the files become valid NEFS archives.

The only caveat is that the chunks are encrypted. The encryption key is the hex string stored in the NEFS header at offset 0x24. The chunks are encrypted using AES in ECB mode with a 256bit key.

I have attempted to add this decryption ability to nefsedit myself, but I got stuck when deflated the correctly decrypted stream as it is padded with random bytes which make the DeflateStream error out. I'm submitting this issue to relay my findings about encryption and in case you are interested in adding this capability to the program.

Feel free to email me for the reconstructed game.dat and game_1.dat and more information on how to obtain them.

victorbush · Answer 1 · Sat May 25 2019 22:02:42 GMT+0800 (China Standard Time)

Great work, thanks for sharing this. I will take a closer look when I get time.

opcod3 · Answer 2 · Sun May 26 2019 04:34:18 GMT+0800 (China Standard Time)

I managed to get everything to extract but my code is super wonky so far, i'll make a PR once i clean it up.

The main issue is offsets are very weird as the headers are divided into two chunks and the game file is separate. And the offsets in the header are all relative to the beginning to the header chunk/data file.

opcod3 · Answer 3 · Mon Jun 24 2019 08:16:11 GMT+0800 (China Standard Time)

Sorry i have not posted a PR for this issue yet. It's just that I got my code working in an extremely hacky way to be able to extract the dat files but breaking compatibility for everything else in the process.

How would you suggest I handle the dat files? There are two issues: finding the headers in the exe, and the offsets in the header. Since the dat headers and data are separate the header offsets are al based on different locations, either the forst, second or data beginning. I found this relatively difficult to handle with nefsedit. How would you handle this in a stable way that doesn't break compatibility with everything else?

victorbush · Answer 4 · Sat Jun 29 2019 08:41:05 GMT+0800 (China Standard Time)

Good questions. I'm playing with some refactoring of the library in general. I was thinking about having the option to specify header location/stream separate from the item data file/stream. I think finding the headers should be done in the application side (NefsEdit) and the library (NefsLib) should just be feed in a stream to process. Hopefully that makes it easier to integrate your changes.

peterboy77 · Answer 5 · Thu Feb 27 2020 06:27:32 GMT+0800 (China Standard Time)

Hello all, i edit a DLC car and in game i have this error. Any help?

peterboy77 · Answer 6 · Thu Feb 27 2020 06:27:53 GMT+0800 (China Standard Time)

Car liveri

victorbush · Answer 7 · Thu Feb 27 2020 06:33:42 GMT+0800 (China Standard Time)

Editing encrypted nefs files (i.e., DLC cars) is not supported. You would have to manually edit the game exe or have access to the private key. Unfortunately the game does not appear to allow custom liveries the same way the non-DLC cars do. You would need to convince Codemasters to enable that support.

peterboy77 · Answer 8 · Wed Mar 04 2020 03:53:41 GMT+0800 (China Standard Time)

Thanks :) :)

victorbush · Answer 9 · Sun Mar 29 2020 05:01:11 GMT+0800 (China Standard Time)

I've got some initial support for this integrated into the dev branch. The app will scan the game exe for header files. It assumes there order of the header files in the game executable matches the order of the game.dat files in the game directory. I'm not sure of a better way to do this. The app has a custom option for specifying header offsets manually if all else fails.

I'm able to open the game.dat files and extract some files. There's still some issues though. For instance, game_1.dat doesn't read correctly.

Unfortunately after refactoring the nefslib, saving nefs archives no longer produces a valid archive that the game can read. This is unrelated to game.dat support (saving game.dat files won't be supported), but is holding up progress while I try to figure out saving nefs files correctly.

opcod3 · Answer 10 · Sun Mar 29 2020 05:47:01 GMT+0800 (China Standard Time)

Try using the file size field for determining the correct header. I think that should work.

…

On Sat, Mar 28, 2020, 10:01 PM victorbush ***@***.***> wrote: I've got some initial support for this integrated into the dev branch. The app will scan the game exe for header files. It assumes there order of the header files in the game executable matches the order of the game.dat files in the game directory. I'm not sure of a better way to do this. The app has a custom option for specifying header offsets manually if all else fails. I'm able to open the game.dat files and extract some files. There's still some issues though. For instance, game_1.dat doesn't read correctly. Unfortunately after refactoring the nefslib, saving nefs archives no longer produces a valid archive that the game can read. This is unrelated to game.dat support (saving game.dat files won't be supported), but is holding up progress while I try to figure out saving nefs files correctly. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLU6EO6YHP46UCPNY23YQ3RJZQSHANCNFSM4HPKWC3A> .

victorbush · Answer 11 · Sun Mar 29 2020 05:58:30 GMT+0800 (China Standard Time)

Ah good idea, I will do that.

opcod3 · Answer 12 · Sun Mar 29 2020 07:12:12 GMT+0800 (China Standard Time)

Also beware of false positives, depending on how you're detecting headers, the string "NeFs" appears multiple times in the exe in addition to the headers.

victorbush · Answer 13 · Sun Mar 29 2020 07:15:31 GMT+0800 (China Standard Time)

Yeah I'm currently looking for a known constant that is some many bytes after the NeFS. All the headers so far have a "zlib" constant in them so I'm using that for now to confirm its actually a header.

opcod3 · Answer 14 · Sun Mar 29 2020 09:09:51 GMT+0800 (China Standard Time)

F1 files don't have zlib (and also have a bit of a different format) not sure if nefsedit works with them. I would do one of two things, either check for the aes, which has to be a string of valid ASCII hex characters (relatively easy to check for), or actually check the hash of the header to be correct.

victorbush · Answer 15 · Sun Mar 29 2020 09:16:42 GMT+0800 (China Standard Time)

Ok interesting, I wasn't aware of the F1 use. I had thought about pattern matching the aes key. Checking the header hash is a good idea as well.

opcod3 · Answer 16 · Sun Mar 29 2020 09:44:06 GMT+0800 (China Standard Time)

Yeah I figured the f1 use after taking a look at the leaked F1 2018 and 2017 debug executables. They are a trove of information, I was hoping to apply it to dirt but the header format seems to be slightly different sadly. You can also check field 0x68 for the 0x20000 constant (it's different for f1 through). This last thing is something the actual game does to check the validity of headers

…

victorbush · Answer 17 · Sun Apr 05 2020 03:02:21 GMT+0800 (China Standard Time)

Temporarily going to keep using the zlib constant since it works for dr2 and d4 right now. For matching the files to the headers, the header contains the file name in part 5, so I used that to match them up.