`/utilities/stat` `mode` return value has been broken
chrisjsewell opened this issue Β· comments
In 8fa4044, @aledabin changed the code I added (#133) from returning %f
to %a
for the mode, this has broken its use π
Let me explain, for a simple (empty) file:
$ stat -c '%a' example.txt
644
$ stat -c '%f' example.txt
81a4
Here 81a4
is the hexidecimal encoding of BOTH the permissions and the file type.
For example, using Python you can do:
import stat
# recover the permissions as octahedral
oct(int("81a4", 16) & 0o777)[2:] == "644"
# recover the permissions as string
stat.filemode(int("81a4", 16)) == "-rw-r--r--"
# check the file type
stat.S_ISREG(int("81a4", 16)) is True
stat.S_ISDIR(int("81a4", 16)) is False
There is no longer any way to get the file type
as it says at:
firecrest/src/utilities/utilities.py
Line 438 in 028edf3
what we actually want is enough information that can be used to directly initialise https://docs.python.org/3/library/os.html#os.stat_result, you can do this with
%f
but not %a
I would also note, that /utilities/file
is kind of useless, because for this example, it simply returns empty
.
So no actual information on what the file type is π¬
I would note also, it would be great if you could construct stat_result
from the return items of utilities/ls
:
Currently, you can at least create the mode
from the function:
def convert_st_mode(data: dict) -> int:
# created from `ls -l -A --time-style=+%Y-%m-%dT%H:%M:%S`, e.g.
# -rw------- 1 username usergroup 57 2021-12-02T10:42:00 file.txt
# type, permissions, # of links (not recorded), user, group, size, last_modified, name
ftype = {
"b": "0060", # block device
"c": "0020", # character device
"d": "0040", # directory
"l": "0120", # Symbolic link
"s": "0140", # Socket.
"p": "0010", # FIFO
"-": "0100", # Regular file
}
p = item["permissions"]
r = lambda x: 4 if x == "r" else 0 # noqa: E731
w = lambda x: 2 if x == "w" else 0 # noqa: E731
x = lambda x: 1 if x == "x" else 0 # noqa: E731
p_int = (
((r(p[0]) + w(p[1]) + x(p[2])) * 100)
+ ((r(p[3]) + w(p[4]) + x(p[5])) * 10)
+ ((r(p[6]) + w(p[7]) + x(p[8])) * 1)
)
mode = int(ftype[item["type"]] + str(p_int), 8)
return mode
Ah I see it was actually mentioned by @aledabin in the original PR: #133 (comment)
but yeh, unfortunately its wrong for the reasons mentioned above; it needs to encode both permissions and file type
FYI, this is the actual use case I had in mind for this endpoint π eth-cscs/pyfirecrest#43
Hi @chrisjsewell , sorry, I misunderstood the whole issue. It will be published and deployed soon.
Regarding the file
endpoint it returns the output of the file command, so if the file is empty it returns that. Could that be the reason?
sorry, I misunderstood the whole issue. It will be published and deployed soon.
Thanks π and yeh no worries, I didn't realise the implication of the change until trying to use it
Regarding the file endpoint it returns the output of the file command, so if the file is empty it returns that. Could that be the reason?
Well it just seems that thΓ© file
command is generally not very useful, from a "programatic" standpoint, because it doesn't really give you any standardised return value to work with
Hi @chrisjsewell, the fix was included in v1.8.3 and deployed at CSCS last month, I forgot to update here π
Regarding file
endpoint, I understand your point, maybe it is not that useful to be exposed.
I'll close this issue.
Best,
No worries, cheers!