Publishing to ria store
dickscheid opened this issue · comments
What is the problem?
I created a remote RIA store using ria+ssh
protocol.
Creation of the remote works without errors and results in the appropriate structure on the ssh remote.
When actually publishing to the ria store however, datalad fails due to invalid permission settings on the remote side:
Unable to remove <my-remote-path>/datastore/bbc/62b4a-cdad-4b17-ba6e-47bb04bb12f0/ora-remote-ec45211b-96b8-48ce-8386-907725f6fac1/transfer/MD5E-s305407--4ed88684b9e0d8db36e433fc771ff7e3.png or to obtain write permission in parent directory. -caused by- stat -f%Dp <my-remote-path>/datastore/bbc/62b4a-cdad-4b17-ba6e-47bb04bb12f0/ora-remote-ec45211b-96b8-48ce-8386-907725f6fac1/transfer failed:
copy: 1 failed
Indeed, the png file in the transfer folder on the target, as well as its parent folder transfer
have permissions -r--r--r--
. The ore-remote- ...
folder has permissions drwxrwxr-x
. I have no influence on those permissions since the complete tree is auto-created by datalad.
Using DataLad version 0.19.3 and git version 2.43.0 on a MacBook M1.
I tried different ssh remotes, including the JUDAC system in Jülich and a local ubuntu box.
Done the same with ria+file
to a local folder works fine.
What steps will reproduce the problem?
-
Create test datalad repository
datalad create test
-
Add a test file to the repository and
datalad save
-
Create ria remote on ssh machine
datalad create-sibling-ria -s datastore --new-store-ok "ria+ssh://<user>@<server>/path/to/datastore"
-
Try to push
datalad push --to datastore
DataLad information
0.19.3
darwin/23.1.0 14.1.1/arm64
Additional context
No response
Have you had any success using DataLad before?
No response
Just to add, I also tried -f checkdatapresent
for the push, even after generating everything from scratch (including the local repo and the remote datastore folder), with the same result.
I think this is indeed a bug in DataLad, where the format of the remotely executed stat
command gets chosen based on the local OS:
Inside SSHRemoteIO
there's this (scroll to see the conditional):
datalad/datalad/distributed/ora_remote.py
Lines 463 to 492 in 08315c1
Note that the "Unable to remove" message contains:
-caused by- stat -f%Dp
Digging a bit deeper, I see that it has been reported previously in a slightly more involved context (issue since transferred to another repo) datalad/datalad-ria#40 and discussed in psychoinformatics-de/knowledge-base#5
Thinking loud about a fix or workaround: I think the simplest way would be to only keep the "else" part of the statement, at the cost of breaking the (arguably rare) cause of mac-to-mac. A general solution requires a method to detect the remote OS.