development build error

[Talnex]

I want develop some new features based on .mcpr file, I clone the repo and use IEDA to build the project.
When I run ./gradlew build I got:

A problem occurred configuring project ':1.10.2'.
> Could not resolve all files for configuration ':1.10.2:classpath'.
   > Could not resolve gg.essential:essential-gradle-toolkit:0.1.10.
     Required by:
         project :1.10.2
      > No matching variant of gg.essential:essential-gradle-toolkit:0.1.10 was found. The consumer was configured to find a runtime of a library compatible with Java 8, packaged as a jar, and its dependencies declared externally, as well as attribute 'org.gradle.plugin.api-version' with value '7.3' but:
          - Variant 'apiElements' capability gg.essential:essential-gradle-toolkit:0.1.10 declares a library, packaged as a jar, and its dependencies declared externally:
              - Incompatible because this component declares an API of a component compatible with Java 16 and the consumer needed a runtime of a component compatible with Java 8
              - Other compatible attribute:
                  - Doesn't say anything about org.gradle.plugin.api-version (required '7.3')
          - Variant 'runtimeElements' capability gg.essential:essential-gradle-toolkit:0.1.10 declares a runtime of a library, packaged as a jar, and its dependencies declared externally:
              - Incompatible because this component declares a component compatible with Java 16 and the consumer needed a component compatible with Java 8
              - Other compatible attribute:
                  - Doesn't say anything about org.gradle.plugin.api-version (required '7.3')
          - Variant 'sourcesElements' capability gg.essential:essential-gradle-toolkit:0.1.10 declares a runtime of a component, and its dependencies declared externally:
              - Incompatible because this component declares documentation and the consumer needed a library
              - Other compatible attributes:
                  - Doesn't say anything about its target Java version (required compatibility with Java 8)
                  - Doesn't say anything about its elements (required them packaged as a jar)
                  - Doesn't say anything about org.gradle.plugin.api-version (required '7.3')

I tried JDK 8 and JDK 16, but it is not solved.

AND ....
In some folder, it seems that some class are not exist.
com.replaymod.gradle.preprocess.PreprocessTask which I only see 'com.replaymod.gradle.remap' package
and
'com.replaymod.replaystudio'
Then I look up replaystudio project, some packages are not exist.

import com.replaymod.replaystudio.lib.guava.base.Charsets;
import com.replaymod.replaystudio.lib.guava.base.Optional;
import com.replaymod.replaystudio.lib.guava.io.Closeables;
import static com.replaymod.replaystudio.lib.guava.io.Files.*;

Any advice would be greatly appreciated.

[Jonas Herzig]

You need to run Gradle with at least Java 16 (maybe 17, don't remember, you do need a JDK 17 for building in any case). The error says you're using Java 8.

Though if you only want to use the mcpr and not the entire mod, you only need to use https://github.com/ReplayMod/ReplayStudio (and if you don't care about more complex stuff like loading old replays, you could probably also just read the file yourself, the format is fairly simple).

The source code for the PreprocessTask class can be found in the "preprocessor" repo: https://github.com/ReplayMod/preprocessor/blob/master/src/main/kotlin/com/replaymod/gradle/preprocess/PreprocessTask.kt

The source code for the com.replaymod.replaystudio package can be found in the "ReplayStudio" repo: https://github.com/ReplayMod/ReplayStudio/tree/master/src/main/java/com/replaymod/replaystudio

The stuff in com.replaymod.replaystudio.lib. is generated by the build script of ReplayStudio. The code which does that is quite old and bad; if your IDE doesn't automatically detect it, you'll have to add the build/preshadow/ReplayStudio-PRESHADOW.jar file to the project dependencies in your IDE manually.

[Talnex]

Thanks for your reply, I have run through the Replaystudio project by adding build/preshadow/ReplayStudio-PRESHADOW.jar manually.
I simply implemented reading the .tmcpr file in .mcpr by referring to the code in the project,

public class testReadMCPR {

    public static void main(String[] args) {
        try {
            File mcprFile = new File("D:\\projects\\ReplayStudio\\src\\test\\java\\2024_04_23_20_03_02.mcpr");
            ReplayStudio studio = new ReplayStudio();

            ZipReplayFile replayFile = new ZipReplayFile(studio, mcprFile);

            ReplayMetaData meta = replayFile.getMetaData();
            ProtocolVersion inputVersion = meta.getProtocolVersion();
            ReplayInputStream stream = replayFile.getPacketData(PacketTypeRegistry.get(inputVersion, State.LOGIN));

            PacketData data;
            while((data = stream.readPacket()) != null) {
                Packet  pkt = data.getPacket();
                System.out.println(pkt.getBuf());
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

and found that it contained byte-encoded data packets for communication with the server.

PooledUnsafeDirectByteBuf(ridx: 1, widx: 62, cap: 62)
PooledUnsafeDirectByteBuf(ridx: 1, widx: 11, cap: 11)
PooledUnsafeDirectByteBuf(ridx: 1, widx: 8, cap: 8)
PooledUnsafeDirectByteBuf(ridx: 1, widx: 5, cap: 5)
PooledUnsafeDirectByteBuf(ridx: 1, widx: 9, cap: 9)
PooledUnsafeDirectByteBuf(ridx: 1, widx: 9, cap: 9)
PooledUnsafeDirectByteBuf(ridx: 1, widx: 29, cap: 29)
PooledUnsafeDirectByteBuf(ridx: 1, widx: 15, cap: 15)
PooledUnsafeDirectByteBuf(ridx: 1, widx: 70, cap: 70)
PooledUnsafeDirectByteBuf(ridx: 1, widx: 9, cap: 9)
PooledUnsafeDirectByteBuf(ridx: 1, widx: 15, cap: 15)
PooledUnsafeDirectByteBuf(ridx: 1, widx: 45, cap: 45)
PooledUnsafeDirectByteBuf(ridx: 1, widx: 5, cap: 5)

This prevents me from directly parsing the .tmcpr file.

I want to save the player's actions (mouse and keyboard) and system events at the same time during the video rendering process, so as to further analyze the player's behavior. At present, it seems that I still need to modify ReplayMod. Do you have any suggestions?

[Talnex]

Can I also achieve the desired purpose by parsing this byte-encoded data packet, for example, according to the definition in this link
https://wiki.vg/Pocket_Minecraft_Protocol#MoveEntityPacket_.280x90.29

[Jonas Herzig]

and found that it contained byte-encoded data packets for communication with the server.

Yes, that is in effect all a replay is: A recording of all the stuff the server sent (plus a few extra bits about the local player it didn't send).

This prevents me from directly parsing the .tmcpr file.

How so? Yes, it doesn't do it all the way for you, but it shouldn't prevent you from doing it either?

Side note: there are some utilities in ReplayStudio that may help parsing some of the data, though beware that they have a very crude API (because they're only meant for internal use) and may not be implemented for all versions or all cases, only as far as required to do whatever it needs.

I want to save the player's actions (mouse and keyboard) and system events

These are not included in the replay. It only contains regular packets describing how the player entity moved, it's not remotely detailed enough for actual mouse/keyboard/system events.

Can I also achieve the desired purpose by parsing this byte-encoded data packet, for example, according to the definition in this link

You haven't actually stated what your desired purpose is. So far you've only said that you want some data about some players for some purpose.

The link is for the Pocket Edition of Minecraft. The one for the Java Edition is https://wiki.vg/Protocol#Update_Entity_Position, and yes, parsing the byte-encoded packet data like that would be one way to extract some data from the replay.

[Talnex]

Thank you for your suggestion. I'm looking to compile a set of video-action-target data pairs to create a dataset aimed at training an agent to mimic human behavior, similar to MineDojo. For instance, if the system event is "Got wood," the video would show the collection process and the action would be tagged as "Chop a tree and get a wood." To achieve this, I need to gather the input actions from human players, along with the corresponding videos and system events to generate pseudo labels. From the link you provided, I noticed that the data calculates the entity's angle of view and movement using yaw and pitch. I'm considering using this information to deduce mouse and keyboard movements.

[Jonas Herzig]

In that case just beware that the information about the recording player in a replay will not exactly match to that during recording, e.g. it is usually only updated 20 times a second, yaw/pitch are clamped to 256 discrete values, no inventory interactions, etc.
If you're using RM only to search for events, then that's probably fine; deducing accurate mouse and keyboard movement from that would probably add another layer of complexity though. You may be better off creating a separate mod that directly records such input information.

[Talnex]

I tried to parse and visualize the EntityPositionRotation packet and found that the action can be restored. If mouse dpi mapping is added, the mouse movement should be restored :)

But I encountered two problems:

1. The packet id of EntityPositionRotation in replay studio is 0x17, but in the web page you provided, 0x17 is https://wiki.vg/Protocol#Set_Player_Position and https://wiki.vg/Protocol#Chat_Suggestions (Wait...two???)
   The packet recording Entity's Position and Rotation in the web page is https://wiki.vg/Protocol#Update_Entity_Position_and_Rotation. Its packet id is 0x2D, and in replay studio 0x2D is OpenWindow. This makes me very confused. My ProtocolVersion is 1.20.3/765
2. After jumping twice, the displacement did not return to the origin. I guess it may be that the entity still had residual speed when the data packet was sent.