bennetthardwick / docs.rs

crates.io documentation generator

Docs.rs

Docs.rs (formerly cratesfyi) is an open source project to host documentation of crates for the Rust Programming Language.

Docs.rs automatically builds crates' documentation released on crates.io using the nightly release of the Rust compiler.

The README of a crate is taken from the readme field defined in Cargo.toml. If a crate doesn't have this field, no README will be displayed.

Redirections

Docs.rs is using semver to parse URLs. You can use this feature to access crates' documentation easily. Example of URL redirections for clap crate:

URL	Redirects to documentation of
https://docs.rs/clap	Latest version of clap
https://docs.rs/clap/~2	2.* version
https://docs.rs/clap/~2.9	2.9.* version
https://docs.rs/clap/2.9.3	2.9.3 version (you don't need = unlike semver)
https://docs.rs/clap/*/clap/struct.App.html	Latest version of this page (if it still exists).

Badges

You can use badges to show state of your documentation to your users. The default badge will be pointed at the latest version of a crate. You can use version parameter to show status of documentation for any version you want.

Badge will display in blue if docs.rs is successfully hosting your crate documentation, and red if building documentation failing.

Example badges for mio crate:

URL	Badge
Latest version: https://docs.rs/mio/badge.svg
Version 0.4.4: https://docs.rs/mio/badge.svg?version=0.4.4
Version 0.1.0: https://docs.rs/mio/badge.svg?version=0.1.0

Development

We strongly recommend using docker-compose, which will make it easier to get started without adding new users and packages to your host machine.

Getting started

Make sure you have docker-compose and are able to download ~10GB data on the first run.

git clone https://github.com/rust-lang/docs.rs.git docs.rs
cd docs.rs
cp .env.sample .env

docker-compose build  # This builds the docs.rs binary

# Build a sample crate to make sure it works
# This sets up the docs.rs build environment, including installing the nightly
# Rust toolchain. This will take a while the first time but will be cached afterwards.
docker-compose run web build crate regex 1.3.1

# This starts the web server but does not build any crates.
# If you want to build crates, see below under `build` subcommand.
# It should print a link to the website once it finishes initializing.
docker-compose up

If you need to store big files in the repository's directory it's recommended to put them in the ignored/ subdirectory, which is ignored both by git and Docker.

Running tests

Tests are run outside of the docker-compose environment, and can be run with:

cargo test

Some tests require access to the database. To run them, set the CRATESFYI_DATABASE_URL in .env to the url of a PostgreSQL database, and set the AWS_ACCESS_KEY_ID, S3_ENDPOINT, and AWS_SECRET_ACCESS_KEY variables. We have some reasonable default parameters in .env.sample.

For example, if you are using the docker-compose environment to run tests against, you can launch only the database and s3 server like so:

docker-compose up -d db s3

If you don't want to use docker-compose, see the wiki page on developing outside docker-compose for more information on how to setup this environment. Note that either way, you will need docker installed for sandboxing with Rustwide.

Docker-Compose

The services started by Docker-Compose are defined in docker-compose.yml. Three services are defined:

name	access	credentials	description
web	http://localhost:3000	N/A	A container running the docs.rs binary
db	postgresql://cratesfyi:password@localhost:5432/	-	Postgres database used by web
s3	http://localhost:9000	cratesfyi - secret_key	Minio (simulates AWS S3) used by web

Rebuilding Containers

To rebuild the site, run docker-compose build. Note that docker-compose caches the build even if you change the source code, so this will be necessary anytime you make changes.

FAQ

I keep getting the error standard_init_linux.go:211: exec user process caused "no such file or directory" when I use docker-compose.

You probably have CRLF line endings. This causes the hashbang in the docker-entrypoint to be /bin/sh\r instead of /bin/sh. This is probably because you have git.autocrlf set to true, set it to input instead.

CLI

Starting web server

# This command will start web interface of docs.rs and you can access it from
# http://localhost:3000/`
docker-compose up

build subcommand

# Builds <CRATE_NAME> <CRATE_VERSION> and adds it into database
# This is the main command to build and add a documentation into docs.rs.
# For example, `docker-compose run web build crate regex 1.1.6`
docker-compose run web build crate <CRATE_NAME> <CRATE_VERSION>

# Builds every crate and adds them into database
# (beware: this may take months to finish)
docker-compose run web build world

# Builds a local package you have at <SOURCE> and adds it to the database.
# The package does not have to be on crates.io.
# The package must be on the local filesystem, git urls are not allowed.
docker-compose run -v "$(realpath <SOURCE>)":/build web build crate --local /build

database subcommand

# Adds a directory into database to serve with `staticfile` crate.
docker-compose run web database add-directory <DIRECTORY> [PREFIX]

# Updates github stats for crates.
# You need to set CRATESFYI_GITHUB_USERNAME, CRATESFYI_GITHUB_ACCESSTOKEN
# environment variables in order to run this command.
# You can set this environment variables in ~/.cratesfyi.env file.
docker-compose run web database update-github-fields

If you want to explore or edit database manually, you can connect to the database with the psql command.

# NOTE: this creates a new container that will keep running after you exit
# To remove the new container, run `docker-compose down`.
docker exec -it "$(docker-compose run -d db && sleep 1)" psql -U cratesfyi
# You only need to start the container once, in the future you can attach to it like so:
docker exec -it "$(docker container ps | grep postgres | cut -d ' ' -f 1)" psql -U cratesfyi

The database contains a blacklist of crates that should not be built.

# List the crates on the blacklist
docker-compose run web database blacklist list

# Adds <CRATE_NAME> to the blacklist
docker-compose run web database blacklist add <CRATE_NAME>

# Removes <CRATE_NAME> from the blacklist
docker-compose run web database blacklist remove <CRATE_NAME>

daemon subcommand

# Run a persistent daemon which queues builds and starts a web server.
# Warning: This will try to queue hundreds of packages on crates.io, only start it
# if you have enough resources!
docker-compose run -p 3000:3000 web daemon

Changing the build environment

To make a change to the build environment and test that it works on docs.rs, see the wiki.

Contact

Docs.rs is run and maintained by the docs.rs team. You can find us in #docs-rs on Discord.