"Initializing search" appears repeatedly even with navigation.instant

[ms2art]

Contribution guidelines

• I've read the contribution guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the problem doesn't occur with the mkdocs or readthedocs themes
• ... the problem persists when all overrides are removed, i.e. custom_dir, extra_javascript and extra_css
• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

Each time I navigate to a new page in my documentation and then click Search, "Initializing search" appears below the search field. The string displays for a longer time as the number of words in the documentation increases.

Expected behaviour

Each time I navigate to a new page in my documentation and then click Search, "Type to start searching" should appear immediately. I can accept initialization on the first search in my content set, but not on subsequent searches.

Actual behaviour

Each time I navigate to a new page in my documentation and then click Search, "Initializing search" appears below the search field. The string displays for a longer time as the number of words in the documentation increases.

Steps to reproduce

1. Create a content set.
2. Build the content.
3. View a topic in a server (for example, "Open with Live Server" in Visual Studio Code).
4. Click in the Search field.
5. Time the persistence of the "Initializing Search" message.
6. View another topic and repeat steps 4 and 5.
7. Add more topics and repeat steps 1 through 6.

Note that I created several very long topics in order to get the problem to appear. My actual topics would never be as long as these samples, but I have over 7,000 topics in my content set, so I think my sample topics are a fair test.

initializing-search.zip

Package versions

• Python: Python 3.10.2
• MkDocs: version 1.2.3
• Material: 8.1.11

Configuration

site_name: My site

theme:
  name: material
  features:
    - navigation.instant

plugins:
  - search
  
nav:
    - My Product Documentation:
      - Topic One: topic_1.md
      - Topic Two: topic_2.md
      - Topic Three: topic_3.md
      - Topic Four: topic_4.md
      - Topic Five: topic_5.md

System information

Windows 10
Firefox 100.0.2 (64-bit), Chrome 102.0.5005.62 (Official Build) (32-bit)

[Bernardo C.D.A. Vasconcelos]

Hi, just chiming in to say that I had this problem using the git installation. That is, git method outlined in the documentation for the material-theme (downloading the repository and adding the overrides to the config yml file). When I changed to using the pip version, it started working properly (all this in the local build).

[ms2art]

Thanks, @bcdavasconcelos . I think I used pip, so I can't try that, but it's good to know that the problem can be solved.

[Martin Donath]

Thanks for reporting. Two problems are at play here:

1. You have several very, very large documents. This leads to a search index of 4MB, which your browser must first download. This is why the initializing search phase might take a little longer. Furthermore, the MkDocs search plugin generates an index that is significantly larger than it needs to be. Insiders contains a rewrite of the search plugin that can cut down the index size to ~50% of the previous size, which will also significantly reduce indexing time. Currently, my measurements show that indexing takes 1.77s, so expect indexing to take <1s with the new implementation.

2. You stumbled upon an edge case in instant loading. When the sitemap.xml is fetched (which Material for MkDocs uses as a reference to know which page is internal and which is external), the prefix of each URL is stripped, so instant loading also works on other domains than the one you configured, e.g. in Netlify previews. Now, since you didn't define an index.md, which is always the root page that is used as a prefix to be stripped from all other pages and always comes first in sitemap.xml, the prefix elimination algorithm doesn't work: http://localhost:8000/topic_1/ is used for prefix elimination, passing the following URLs to instant loading, all of which will 404:

   `["1/","2/","3/","4/","5/"]`
   

   If you define an index.md (which I tested), instant loading will work correctly, as the URLs are now correct:

   ["","topic_1/","topic_2/","topic_3/","topic_4/","topic_5/"]
   

   Note that the presence of an index.md is essential for MkDocs to work, which is why it always generates an index.md upon mkdocs new. It just doesn't work without an index.md.

You might consider switching to Insiders, if your documentation is this large.

Closing, as on our side, no change is required.

[ms2art]

One follow-up: Is it the size of the documents that affects the index size, or is it the number of words? My example used just a few very large test documents, so I could get a large number of words into the content more easily. In reality, I have a large number of much smaller documents.

[Martin Donath]

The number of words across all documents. For example, the index for the entire Material for MkDocs site weighs 400kb.

[ms2art]

The "initializing" delay cropped up again recently. Could the folder relationship between sitemap.xml and the HTML files be a problem? I'm looking at entries in sitemap.xml like:

<url>
     <loc>http://127.0.0.1:5500/docs/documentation/index.html</loc>
     <lastmod>2022-07-27</lastmod>
     <changefreq>daily</changefreq>
</url>

Where sitemap.xml and index.html are located like this:

sitemap.xml
  docs
    documentation
      index.html

[Martin Donath]

Not sure, nothing changed on that front. I'm going to need a reproducible example to look into this. If you manage to create one, you can create a new issue with a zip file containing the reproduction attached.

[ms2art]

Ok. I should have mentioned that my folder structure had to change to what I showed above, some time after I got search working, which is why I wondered about the folder structure. I'll see if I can put together an example. Thanks for your reply.