Create a custom 404 page with absolute URLs hardcoded.
pip install sphinx-notfound-page
Add this extension in your conf.py
file as:
Check out the full documentation at https://sphinx-notfound-page.readthedocs.io/
Strongly based on @ericholscher's solution from readthedocs/readthedocs.org#353