Auto-Import Website Maps
Noordfrees opened this issue · comments
The add-ons server should be able to autogenerate an add-on for each map on the website's Maps section, to allow downloading website-uploaded maps in the add-ons manager.
This could be handled by frequently (every day or so) running a function from the sync thread that checks for any not-yet-imported entries in the maps database and generates an add-on for each of them.
Data about all maps and all their details is readily available in the website database. A map's files (the .wmf for the map, .png for the minimap) are publicly available and can be downloaded with a curl request.
Complications:
- The zipped maps need to be extracted first to allow localization. We may need to add i18n markup to older maps' config files since writing translatable strings to a profile is a fairly new feature.
- Download counter, votings, and comments should best be kept separate between the add-ons server and website Maps section, to avoid "cross-contamination" of the respective databases which could give rise to all sorts of ugly problems.
- Uploader: We could assign all maps to the original uploader, but perhaps it would be better to hand them all to a bot account first, and grant the real author access only if they request it – most of the time they shouldn't need it, unless they want to provide screenshots.
- Deleting a map from the website should also delete the add-on.
- The add-on description should be the concatenation of the map's description and upload comment, with a note that the add-on was autogenerated. For the icon we should use the downscaled minimap. The author name should always be the map's author.
- If the website map has a minimum Widelands version of 1.0 or newer set, use this also for the add-on. (A min version of build 21 or older can be ignored, since add-ons are not available there anyway.)
- Related: widelands/widelands-website#196
Currently there is a category Map sets, will there be an additional category Single Maps?
Can't help myself but having maps as add-on(s) doesn't work well yet. The reasons are:
- The website shows much more information about a single map (size, description, number of players, image of the map). Adding such information to the Add-ons screen makes the whole screen a lot more fuzzy than it is right now.
- For both uploading on the website and uploading as addon one has to register on the website anyway. So the workflow for uploading as addon will be:
- Register on the website
- Start widelands and start the addons packager
- Log in to the addons packager
- Understand what is needed to pack a map
- Understand what is needed too upload the packed map
- maybe upload a screenshot
- Instead uploading on the homepage:
- Register on the website
- Upload map
Maybe the workflow for packing a map as addon can be simplified. E.g. add a menu entry in the editors Main menu Publish this map as add-on. But the downsides of the the first point above still remain.
I try to look at all stuff from a players point of view, that's all...
I am thinking about having best of both worlds.
Installing maps via the game is a nice feature. But displaying (single) maps with its information isn't nice (yet). Displaying maps is better on the website. Also it is much easier to upload a map on the website.
How about adding a new UI "Maps" in the game which shows maps with its information, gathered from the website Database? Including a button for easy download (without a player has to know where to save the map).
Uploading a new map is done via the website like before, or, don't know if it is feasible, add a button for uploading a map inside the game, which uploads the map and stores all information in the website database.
The thing is: I have no clue about where to put a probably new UI "Maps" :-D
- I agree that the add-ons manager UI is not that useful to display a map. So how about adding this "website maps" as another tab next to the Browse tab? Then the "Browse" tab would show non-map add-ons, and the "Maps" tab would show maps with a more map-friendly UI similar to the website's maps UI. Map Set add-ons would be mixed among the website maps (the difference between the two types of map sharing shouldn't matter to the end-user), and Map Sets that contain only 1 map could be displayed just like website maps.
- Uploading a map to the website database from inside Widelands might be… a maintenance nightmare. Would Django notice that the database has a new entry, and send notice e-mails etc out to subscribers? Not to mention that the map files need to be stored somewhere, and the add-ons server runs in a container with no access to the website server's filesystem.
I guess the website could implement some kind of webhook that allows uploading maps via API requests, but unless we have this already (?) adding this seems overkill to me. - Automatically packing and publishing a map as an add-on from inside the editor is a very good idea, +1
Assigned to Nordfriese
On Sun Jan 28 10:41:32 CET 2024, Benedikt Straub (Nordfriese) wrote:
I now have a functional WL branch locally and a corresponding server branch running on alpha that allows you to view and download website maps.
I would like to provide more integration so the download counter is incremented and you can also vote and comment from the add-ons client as you would on the website. So two questions:
- Is it acceptable to give the add-ons server write permission to the website database? (currently it is read-only)
- Will Django pick up changes automatically?
I am especially concerned about maps ratings, where we have one table which links a map ID to a rating ID and another table that links rating IDs to multiple individual votes. Is it enough to change the individual user vote entry, or do we need to recompute the redundant entries for number of votes and average vote as well?
<@>frankystone <@>janus
On Sun Jan 28 16:45:16 CET 2024, ** (frankystone)* wrote:*
I would like to provide more integration so the download counter is incremented and you can also vote and comment from the add-ons client as you would on the website. So two questions:
- Is it acceptable to give the add-ons server write permission to the website database? (currently it is read-only)
I think we should only have one program which have write access to the website database. I fear otherwise there might be two processes writing at the same time which can lead to misbehavior. On the other side i think maria-db should handle such things correctly, but i am not sure though.
- Will Django pick up changes automatically?
I am especially concerned about maps ratings, where we have one table which links a map ID to a rating ID and another table that links rating IDs to multiple individual votes. Is it enough to change the individual user vote entry, or do we need to recompute the redundant entries for number of votes and average vote as well?
On the website we use a third party app called django.star-ratings which does the average calculation after a rating has been saved to the database. This is done by sending a django post-save signal, which is sent by django-star-ratings. I think just adding a vote directly in the corresponding database table wouldn't trigger the signal and no average calculating is executed.
I'll try to contact janus over IRC.