lord-re / tinysearch

πŸ” Tiny, full-text search engine for static websites built with Rust and Wasm

Home Page:https://endler.dev/2019/tinysearch

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

tinysearch

CI

TinySearch is a lightweight, fast, full-text search engine. It is designed for static websites.

TinySearch is written in Rust, and then compiled to WebAssembly to run in the browser.
It can be used together with static site generators such as Jekyll, Hugo, zola, Cobalt, or Pelican.

Demo

How it works

tinysearch is a Rust/WASM port of the Python code from the article "Writing a full-text search engine using Bloom filters". It can be seen as an alternative to lunr.js and elasticlunr, which are quite heavy for smaller websites and require a lot of JavaScript.

The idea of tinysearch is to generate a small, self-contained WASM module from a list of articles on your website and run it directly on the frontend inside browsers.

Users

Limitations

  • Only searches for entire words. There are no search suggestions (yet).
  • Since we bundle all search indices for all articles into one static binary, we recommend to only use it for small- to medium-size websites. Expect around 4kB (non-compressed) per article.

Installation

wasm-pack is required to build the WASM module. Install it with

cargo install wasm-pack

To optimize the JavaScript output, you'll also need terser:

npm install terser -g

If you want to make the WebAssembly as small as possible, we recommend to install binaryen as well. On macOS you can install it with homebrew:

brew install binaryen

Alternatively, you can download the binary from the release page or use your OS package manager.

After that, you can install tinysearch itself:

cargo install tinysearch

Usage

As an input, we require a JSON file, which contains a the content you like to index. Check out this example file).

tinysearch fixtures/index.json

(You can take a look at the code examples for different static site generators here.)

This will create a WASM module and the JavaScript glue code to integrate it into your homepage. You can open the demo.html from any webserver to see the result.

For example, Python has a built-in webserver for testing:

python3 -m http.server 

then browse to http://0.0.0.0:8000/demo.html to see the result.

For advanced usage options, try

tinysearch --help

Please check what's required to host WebAssembly in production -- you will need to explicitly set mime gzip types.

Docker

If a full Rust setup, you can also use our nightly-built Docker images.

Build

Available buid args:

  • WASM_REPO
  • WASM_BRANCH
  • TINY_REPO
  • TINY_BRANCH
  • TINY_MAGIC (for a magic number see tinysearch#111)

Demo

wget https://raw.githubusercontent.com/tinysearch/tinysearch/master/fixtures/index.json
docker run $PWD:/tmp tinysearch/cli index.json

Custom repo/branch build

docker build --build-arg WASM_BRANCH=master --build-arg TINY_MAGIC=64 -t tinysearch/cli .

By default most recent stable alpine rust image is used. To get nightly just run

docker build --build-arg RUST_IMAGE=rustlang/rust:nightly-alpine -t tinysearch/cli:nightly .

Maintainers

  • Matthias Endler (@mre)
  • Jorge-Luis Betancourt (@jorgelbg)
  • Mad Mike (@fluential)

License

tinysearch is licensed under either of

at your option.

About

πŸ” Tiny, full-text search engine for static websites built with Rust and Wasm

https://endler.dev/2019/tinysearch

License:Apache License 2.0


Languages

Language:Rust 85.8%Language:Makefile 4.3%Language:HTML 4.1%Language:Dockerfile 3.9%Language:C 1.9%