Overwiriting a JOSM or OSC file does not truncate existing file

[Kidhanis De Jesus]

Vespucci Version

19.3.4.0

Download source

F-Droid

Device (Manufacturer and Model)

Samsung Galaxy S10e

Android Version

11

Behaviour/Symptoms

Some files that were saved with Transfer -> File -> Save to JOSM file can't be read back. Same issue with files saved with Transfer -> File -> Export changes to OSC file.

Expected Behaviour

Files should be read without any issues

How to recreate

Example with JOSM file

1. Clear OSM data with Transfer -> Clear data, if data is present. This step isn't necessary, but the rest of the description assumes data was cleared.
2. Save a file using Transfer -> File -> Save to JOSM file.
3. Open file in a text editor. In my case, the file has 104 bytes and looks like

<?xml version='1.0' encoding='UTF-8' ?><osm generator="Vespucci/19.3.4.0" version="0.6" upload="true" />

4. Add bytes to the file with the text editor. In my file, I changed version='1.0' to version='1.00'. Save to same file. Size of my file is now 105 bytes.
5. Back in Vespucci, again Save to JOSM file, and choose the same file. When saving, the file manager asks to overwrite, so accept to overwrite. Size is still 105 bytes.
6. Try to read file with Transfer -> File -> Read from JOSM file, but an error pops up:

Invalid data read
Please consult the documentation on supported file formats and required data fields.

At line 1, column 104: junk after document element

Crash dump submitted (no or yes + date)

No crash dump

Any other potentially relevant information

The file after step 5 looks like

<?xml version='1.0' encoding='UTF-8' ?><osm generator="Vespucci/19.3.4.0" version="0.6" upload="true" />>

It has the same content from step 3 but with an extra character '>' at the end.
Warnings may vary based on the number of bytes added in step 4, but the outcome is the same.

Same behavior using Safe mode on the app. Behavior verified on another Samsung device with a freshly installed app running Android 13.

However, the behavior is correct on an ASUS device running Marshmallow, where step 6 results in successfully reading the file, but on this device the file manager writes to file without asking for permission to overwrite in step 5.

[Simon Poole]

I suspect there are (at least) two things going on here:

• if you save empty data you can't read that back in Vespucci because requisite elements are missing, for example bounding boxes. This is an edge case that can't be easily fixed. Just saving and re-reading files with content to the Vespucci directory works just as expected in my testing here (actually it will read it, but still complain).

• you are assuming UNIX semantics for file system permissions/access, which however is no longer the case since Android 10. You didn't mention where you are saving the files and from where you are reading them? And which editor are you using?

[Kidhanis De Jesus]

# Context I first encountered this issue while sharing a data layer with over 100 elements between Vespucci and JOSM through a Syncthing folder. I created the file in JOSM and then used it in Vespucci where I made small additions and saved them to the same file. I found that JOSM saves XML with newlines while Vespucci doesn't, so a correct overwrite in Vespucci would have resulted in a file smaller in size due to not storing newlines. This reduction in size, however, didn't happen. When I tried to work with the file again in JOSM, the file couldn't be read due to the error in the overwrite. # Steps with empty file The steps I listed are instead done with Vespucci locally (no sharing with Syncthing) in the Documents directory of the phone because the behavior is the same, regardless of what OSM data is in the file.  I think I see what you mean by 'UNIX semantics'. You do have to grant Vespucci permission to access Storage. If not, then saving JOSM or OSC files is disabled in the app. I also forgot to add that if you read the file with Vespucci right after steps 2 and 4, a "No connection to server" warning is displayed saying "Couldn't construct a bounding box from data", which means the file is at least correctly formatted. It's something done in step 5 that results in the error seen in step 6. ## Editor used I think any editor can be used, but the editor doesn't matter. As explained above, the error first showed up without any manual editing of the file. For editing the empty file, I tried vim through Termux, a hex editor app, and a text editor app, and all three methods gave the same results. Mar 6, 2024 05:07:55 Simon Poole ***@***.***>:

…

I suspect there are (at least) two things going on here: * if you save empty data you can't read that back in Vespucci because requisite elements are missing, for example bounding boxes. This is an edge case that can't be easily fixed. Just saving and re-reading files with content to the Vespucci directory works just as expected in my testing here. * you are assuming UNIX semantics for file system permissions/access, which however is no longer the case since Android 10. You didn't mention where you are saving the files and from where you are reading them? And which editor are you using? — Reply to this email directly, view it on GitHub[#2517 (comment)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/ABKXCWV6RYY6I7BMSFDX643YW3MGXAVCNFSM6AAAAABEIK4ZYGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBQGM4TSNRXHA]. You are receiving this because you authored the thread. [Tracking image][https://github.com/notifications/beacon/ABKXCWTPYO6F7LJKCYZJPB3YW3MGXA5CNFSM6AAAAABEIK4ZYGWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTWBKAD4.gif]

[Simon Poole]

On Android 10 and later apps do not have access to files owned by other apps except in very limited circumstances. The old access to external files permission essentially doesn't do anything any more.

We don't support files in the Document folder and it is unclear what exactly happens with 'shared' files there to start with.

Further reading: https://www.openstreetmap.org/user/SimonPoole/diary/156819

But at least with that information we might be abke to suggest a workaround.

Note the issue with the wrong modal being displayed was fixed here: 10640a7

[Simon Poole]

So it seems as if the non-truncating aspect of the behaviour is simply an Android bug see https://issuetracker.google.com/issues/180526528