jaraco / path

Object-oriented file system path manipulation

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Some tests can't be run on filesystems that have strict utf-8 mode

kirillrdy opened this issue · comments

Ubuntu installed on ZFS filesystem have utf8only enabled ( this can not be disabled withouth recreating filesystem )

So code such as

filename = 'r\xe9\xf1emi'.encode('latin-1')
        pathname = os.path.join(tmpdir_bytes, filename)
        with open(pathname, 'wb'):

would fail with something like this

    def test_listdir_other_encoding(self, tmpdir):                     
        """                                                                                                                                                                                                                                                                                                                     
        Some filesystems allow non-character sequences in path names.                                                                                                                                                                                                                                                           
        ``.listdir`` should still function in this case.                                                                                                                                                                                                                                                                        
        See issue #61 for details.                                                                                                                                                                                                                                                                                              
        """                                                                                                                                                                                                                                                                                                                     
        assert Path(tmpdir).listdir() == []                                                                                                                                                                                                                                                                                     
        tmpdir_bytes = str(tmpdir).encode('ascii')                                                                                                                                                                                                                                                                              
                                                                                                                                                                
        filename = 'r\xe9\xf1emi'.encode('latin-1')                                                                                                             
        pathname = os.path.join(tmpdir_bytes, filename)                                                                                                                                                                                                                                                                         
>       with open(pathname, 'wb'):                                                                                                                                                                                                                                                                                              
E       OSError: [Errno 84] Invalid or incomplete multibyte or wide character: b'/build/pytest-of-nixbld/pytest-0/test_listdir_other_encoding_Pa0/r\xe9\xf1emi'

Not really sure whats the good solution is, at least some people can find this it might help them,

depending on whats the purpose of that test:

  • we could either just change it to use valid utf8 filename
  • dont run that test if we find utf8 only filesystem ( dont know how you would do that)

That test references #61. Historically, only Windows rejected bad encodings. Later macOS rejected bad encodings. Glad to see some Linux filesystems are getting onboard. I'm happy to see that test degrade into oblivion.