Privacy plugin doesn't fetch Google fonts on Windows

[Lars Wilhelmer]

Contribution guidelines

• I've read the contribution guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the problem doesn't occur with the mkdocs or readthedocs themes
• ... the problem persists when all overrides are removed, i.e. custom_dir, extra_javascript and extra_css
• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

On Windows, the privacy plugin doesn't properly fetch the Google webfonts.

Here's a stripped-down test project to reproduce the error:
test-privacy-bug.zip

If you build this project under Windows, the output looks like this:
test-privacy-bug-output.zip

You can see that the Google fonts are missing in /site/assets/externals.

In our main project, we couldn't even build. Instead, an exception was raised:

File "...\Local\Programs\Python\Python310\lib\site-packages\material\plugins\privacy\plugin.py", line 121, in on_post_page
value.replace(raw, self._fetch(url, page)),
File "...\AppData\Local\Programs\Python\Python310\lib\site-packages\material\plugins\privacy\plugin.py", line 217, in _fetch
os.symlink(os.path.basename(file), path)
OSError: [WinError 1314] Dem Client fehlt ein erforderliches Recht: 'css.css' -> '.cache\\fonts.googleapis.com/css'
The terminal process "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -Command mkdocs build" terminated with exit code: 1.

I assume it's one of the usual problems related to different path syntax on Windows and Unix (slash vs. backslash).

Expected behaviour

See description

Actual behaviour

See description

Steps to reproduce

See description

Package versions

• Python: 3.10
• MkDocs: 1.3.0
• Material: 8.3.2+insiders.4.17.2

Configuration

See description

System information

• Operating system: Windows 10
• Browser: Chrome or FF, latest

[Martin Donath]

Thanks for reporting. Unfortunately, I don't have Windows at my disposal, but I agree that it's very likely related to path problems. I'll see how I can troubleshoot. If it's okay for you, we do this in lockstep together. Any help is appreciated!

[Martin Donath]

Fixed in a195668. I haven't tested this on Windows, but checked all places where we write to the filesystem for the use of os.path.sep. I'm sorry again for the inconvenience this was causing you – I really need to get my hands on a Windows machine in the future – probably a sentence that wasn't said that often 😅

[Martin Donath]

The fix was just released as part of 8.3.4+insiders.4.18.0. Please reopen when the issue persists.

[Lars Wilhelmer]

Sorry, I think we have to reopen this issue or create a new one.

Your commit did fix the issue regarding Google webfonts not being included in the output.

However, there's a second problem which causes the exception given above. It's caused by os.symlink() requiring admin rights on Windows.

So if you're working on a Windows machine without admin rights, include the privacy plugin, have assets that require symlinking, and want to build locally, the exception will be raised.

Is symlinking really required here? Can we find something else for Windows projects?

(Or at least display a warning if the OS is Windows and you don't have admin rights. And skip symlinking.)

[Martin Donath]

Symlinking is necessary to cache files without file extensions. For example, GitHub serves avatars under the following URL:
https://avatars.githubusercontent.com/u/932156

If we use this URL in a Markdown file or HTML template, which then gets compiled to HTML that is run through the privacy plugin, the plugin will download the file and save it under assets/externals/avatars.githubusercontent.com/u/932156. So far so good. However, if we don't modify the URL and append the file extension, the documentation won't be browsable offline, because the file system doesn't set appropriate content-type headers. We'd need image/png, but since no content type is given and cannot be inferred from the file extension, the image will not render. Example:

<!-- won't work -->
<img src="./assets/externals/avatars.githubusercontent.com/u/932156" />

<!-- works -->
<img src="./assets/externals/avatars.githubusercontent.com/u/932156.png" />

For this reason, we need to rename the file and append the file extension before replacing it. The problem is that now we can't use the cache anymore, as the privacy plugins will again receive the extension-free URL on repeated compilation and doesn't know that we already downloaded it. For this reason, we create a symlink to notify the plugin of "we already downloaded it, but we needed to modify the filename, so it works without serving it through a web browser".

Edge cases all the way down. I hope that the explanation sheds some light on why this is necessary. I understand that Python symlinking on Windows doesn't work for non-admin users, which is unfortunate. I don't have Windows at my disposal, so I can only guess. What we could do for Windows is catch any symlinking errors and just swallow them, circumventing the cache, thus downloading the image on every build. That would need some hackery, but it's certainly possible. I'm not aware of another solution to this problem, but I'm happy to discuss if somebody has a better idea.

[Martin Donath]

All in all, please create a new issue, as it's only loosely related to this one.