On Windows, renaming a directory that was just created can fail with `PermissionError: [WinError 5] Access denied`

[Russell Martin]

Describe the bug

Using os.rename() (or pathlib.Path.rename()) can fail on Windows if Briefcase does not have exclusive access to all files in the directory.

A common pattern Briefcase uses is to run shutil.unpack_archive() and rename the unpacked directory; for instance, Android cmdline-tools and Java's JDK. In between these two operations, a background process may acquire a handle on a file; for instance, a Windows file indexer or an antivirus scanner.

Modern Linux file systems are much more resilient to these types of conflicts; so, I would only expect to see this on Windows.

Steps to reproduce

import os
from pathlib import Path

Path("orig-dir-1").mkdir()
Path("orig-dir-1/orig-file").touch()
Path("orig-dir-1").rename("new-dir-1")
os.listdir("new-dir-1")

Path("orig-dir-2").mkdir()
Path("orig-dir-2/orig-file").touch()
Path("orig-dir-2/orig-file").open()
Path("orig-dir-2").rename("new-dir-2")

❯ python
Python 3.11.3 (tags/v3.11.3:f3909b8, Apr  4 2023, 23:49:59) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> from pathlib import Path
>>>
>>> Path("orig-dir-1").mkdir()
>>> Path("orig-dir-1/orig-file").touch()
>>> Path("orig-dir-1").rename("new-dir-1")
WindowsPath('new-dir-1')
>>> os.listdir("new-dir-1")
['orig-file']
>>>
>>> Path("orig-dir-2").mkdir()
>>> Path("orig-dir-2/orig-file").touch()
>>> Path("orig-dir-2/orig-file").open()
<_io.TextIOWrapper name='orig-dir-2\\orig-file' mode='r' encoding='cp1252'>
>>> Path("orig-dir-2").rename("new-dir-2")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\user\.pyenv\pyenv-win\versions\3.11.3\Lib\pathlib.py", line 1175, in rename
    os.rename(self, target)
PermissionError: [WinError 5] Access is denied: 'orig-dir-2' -> 'new-dir-2'
>>>

Expected behavior

Renaming these directories in the Briefcase cache should be successful.

Screenshots

No response

Environment

• Operating System: Windows 11
• Python version: 3.11.3
• Software versions:
  • Briefcase: 0.3.18

Logs

No response

Additional context

Origin: beeware/beeware#348

[Russell Keith-Magee]

See also #394 for a tangentially related issue.

[Malcolm Smith]

Unfortunately this is a common issue on WIndows, and I don't think there's any workaround except putting all renames and deletions in a retry loop. There are packages such as tenacity which make this easier.

[spa51273]

pycon24, taking a look at this. Thanks!

[Deepak6546kumar]

Path("orig-dir-1").mkdir()
Path("orig-dir-1/orig-file").touch()
Path("orig-dir-1").rename("new-dir-1")
os.listdir("new-dir-1")

Path("orig-dir-2").mkdir()
Path("orig-dir-2/orig-file").touch()
Path("orig-dir-2/orig-file").open()
Path("orig-dir-2").rename("new-dir-2")

Path("orig-dir-2").mkdir()
Path("orig-dir-2/orig-file").touch()
Path("orig-dir-2/orig-file").open()
Path("orig-dir-2").rename("new-dir-3")

Path("orig-dir-2").mkdir()
Path("orig-dir-2/orig-file").touch()
Path("orig-dir-2/orig-file").open()
Path("orig-dir-2").rename("new-dir-4")

[Russell Keith-Magee]

@Deepak6546kumar It's not clear what you're trying to communicate with this code sample. We already had a replication case; what extra information are you providing here?