Competitive play
kostmo opened this issue · comments
Discussed in #1691
Originally posted by kostmo December 15, 2023
For reference, some ideas pertaining to competition were presented in #967.
I'll describe a style of multiplayer participation that could be fun.
Submitting scenarios and solutions
In competitive play, there should be a way to submit a scenario (a .yaml file along with any other resource dependencies such as .sw or .world files) that is not inspectable by other players.
Likewise, there should be a way to submit solutions that remain private until all participants have submitted their own.
A competition should target a specific Git revision SHA1 of Swarm itself.
Honor-system vs. secure competitions
In an honor-system competition, it may be sufficient for a locally-running Swarm program to fetch a scenario from a server at runtime, so that its only representation is "in memory". This avoids forms of "casual" or "accidental" cheating. However, it will nevertheless be possible to run a locally-modified swarm binary that could dump the fetched scenario data to file on demand. Therefore, a "secure" competition would require the swarm binary to run on a remote server.
Supporting remote execution would require protections against intentional or accidental infinite loops in swarm-lang code. In competition, the "gamemaster" might specify a maximum number of ticks that a supplied solution is allowed to run before being terminated. Distinguishing between an inefficient solution or adversarial input is not necessary.
An external wall-clock timeout would serve as a fallback protection against bugs in the swarm interpreter itself.
Local execution pros and cons
Pros:
- may be perfectly fine in a casual setting among a familiar group of competitors
- less complex in terms of infrastructure
- no significant compute cost or guardrails required
- retains the TUI interface
- supports interactive play
Cons:
- still entails some extra infrastructure, e.g.:
- a central server to exchange scenarios and solutions
- a remote swarm binary to evaluate submitted solutions
- not sufficient for secure/serious competitions
Security mitigations for local execution
Most importantly for local execution, competitors should not have access to the solution program text for a scenario they have fetched from the server. This can be easily mitigated by having the server simply blank the solution field (after having tested the official solution and obtained its metrics) before transmitting the serialization of the scenario to the player.
Program code of system robots, other NPCs, or goal conditions should also not be exposed. The CESK representation may be sufficiently obfuscated for the purpose of competition; so long as the original program text is not transmitted, a player would have to have written a "decompiler" to glean any helpful information from the in-memory programs.
Finally, the scenario world itself will be fully exposed to a locally-executed swarm instance. This is less likely to be of material value in solving a scenario; one can already scroll any amount in the world. However, any scenario goal that entails "finding" some entity stashed in the world could be "cheesed" by implementing the search in a modified swarm binary.
Gamemaster and player roles
One style of play would involve N >= 3 players and N rounds. In each round, one player serves as the "gamemaster" (GM), who submits a scenario for the other players to play.
Scoring
The GM gets a "point" for that round if at least one player submits a "winning" solution (i.e. demonstrates that the scenario is actually "fair"). A player gets one point for submitting a valid solution. The player with the fastest solution receives an additional point.
Teams
The N above could represent "teams", each of which consist of multiple players. They could collaborate ahead of time in developing scenarios as well as work together in real time to develop solutions to the scenarios submitted by other GMs.
Duration
A game (or "tournament") can be asynchronous and span any amount of time. For example, one "round" may be played each of a number of successive weekends. Or three rounds could be played in 3 hours.
Constraints on submitted scenarios
- A working solution must be supplied with any scenario
- The solution's code size and time in ticks should be published
a. These stats may be derived automatically on the server side
b. This demonstrates that a solution actually exists, and gives a sense to the other players of the magnitude of effort they are signing up for
c. A scenario's author may intentionally provide a less efficient/inflated solution to obfuscate the existence of "short cuts", so long as the solution "works"
Interactive vs non-interactive play
Upon first encounter with a new scenario, typically the player will want to, or need to, engage in some manual exploration. So it is a requirement to support interactive play. A submitted solution, though, will be fully non-interactive.