MirrorZ

Your next MirrorS is not MirrorS, nor MirrorSes, it's MirrorZ.

Demo

https://mirrorz.org. Currently mirrorz.json for each MirrorS is real-time generated by their maintainer.

Also legacy webpages (for w3m and noscript users) are provided in https://mirrorz.org/_/

A speed test script oh-my-mirrorz.py is also provided. See README to more info.

Intro

MirrorS are heterogeneous. It is hard for a single mirror to provide all mirrors, so differences occur.

For end users, this is not a good experience as they need to search for available mirrors.

To make things easy, MirrorZ is intended to include all mirrorS, so a unified interface is needed.

Data Format v1.4 (draft)

Each MirrorS participating in MirrorZ should provide a mirrorz.json with the following fields.

{
  "version": 1.4,
  "site": {
    "url": "https://example.org",
    "logo": "https://example.org/img/logo.svg",
    "logo_darkmode": "https://example.org/img/logo-white.svg",
    "abbr": "EXAMPLE",
    "name": "样例镜像站",
    "homepage": "https://blog.example.org",
    "issue": "https://github.com/example/issues",
    "request": "https://github.com/example/mirror-request",
    "email": "admin@example.com",
    "group": "QQ: 10086 and/or Telegram @something",
    "disk": "may be any string showing usage of disk, e.g. usage",
    "note": "may be any string; like speed limit or connection limit",
    "big": "/speedtest/1000mb.bin"
  },
  "info": [
    {
      "distro": "Debian",
      "category": "os",
            "urls": [
                {
                    "name": "10.7.0 (amd64, CD installer with xfce)",
                    "url": "/debian-cd/current/amd64/iso-cd/debian-10.7.0-amd64-xfce-CD-1.iso"
                }
            ]
    }
  ],
  "mirrors": [
    {
      "cname": "AOSP",
      "desc": "Android 操作系统源代码",
      "url": "/AOSP",
      "status": "S",
      "help": "/help/AOSP/",
      "upstream": "https://android.googlesource.com/mirror/manifest",
      "size": "596G"
    },
    {
      "cname": "AUR",
      "desc": "Arch Linux 用户软件库",
      "url": "https://aur.tuna.tsinghua.edu.cn",
      "status": "S1612195849X1612196849N",
      "help": "/help/AUR/",
      "upstream": "https://aur.archlinux.org/"
    }
  ]
}

Notes

version is optional for version 1.x
previous versions of this protocol could be found in git history; protocols in v1.x are back-ward compatible
site provides the global info about one MirrorS
only site.url and site.abbr are mandatory
site.logo should not be of format ico. Also, at least 64x64 resolution is required
site.logo_darkmode is used when browser uses dark mode. Note that it should be set only after site.logo is set
site.url should not end with slash /
site.big should be a valid url to a big file, used by oh-my-mirrorz.py for speed testing
info is used for category view
the name of info.distro should be agreed and have a mapping, maintained in cname.json
mirrors is used for list view
the name of mirrors.cname should be agreed and have a mapping, maintained in cname.json
mirrors.desc may differ for each MirrorS since there are excludes for some MirrorS
mirrors.desc may be empty
if mirrors.url begins with a slash /, it should be appended to site.url to form a full url
mirrors.status is a concat of strings of pattern [A-Z](\d+)?. Only one main status is allowed; the number of auxiliary status is not limited.
- S1600000000: successful. (optional) last successful ended unix timestamp
- Y1600000000: syncing. (optional) start to sync unix timestamp
- F1600000000: failed. (optional) last attempt to sync unix timestamp
- P1600000000: paused. (optional) the unix timetamp sync stopped
- U: unknown
- X1600000000: (auxiliary) next scheduled sync unix timestamp
- N1600000000: (auxiliary) new mirror. (optional) unix timestamp the repo added
- O1600000000: (auxiliary) old successful timestamp, used only when it is syncing or failed
mirrors.help may be empty, or the same rule as mirrors.url
mirrors.upstream, mirrors.size may be empty

Backend and Frontend

For each MirrorS, it should provide a json file of the above format (recommended name mirrorz.json) and allow CORS of mirrorz.org on that file (CORS on all domains is recommended as some other sites may also use this; and useful for debuging).

MirrorS may provide mirrorz.json using their mirror servers, or any other valid url that reflects the real-time status of their mirror. For example, for TUNA-series MirrorS, a CloudFlare worker is deployed (currently another server status.tuna.wiki is used instead of CF worker as it is slow on generating json file).

MirrorS should provide mirrorz.json with the name inside processed by the mapping cname.json. In the generating scripts of MirrorS, cname.json should be real-time fetched from mirrorz.org/static/json/cname.json

The list of participating MirrorS should be maintained in src/config/mirrors.js.

For the front end, previously a naive one is implemented using JQuery and Jekyll, currently a modern frontend is implemented by @CircuitCoder using React, one MirrorS may provide their FrontEnd for rendering, for example mirrorz.org/{tuna,ustc,sjtug,hit}/.

mirrorz.org should only be statically generated.

With user's consent, mirrorz.org may use Cookie or other methods to archive personalized rendering such as turning off some MirrorS when the json file is large and/or slow to download.

Contributing and Developing

MirrorS

For participating MirrorS, it should add and maintain their url to mirrorz json file in src/config/mirrors.js, in the url https is needed, and the url should be widely accessible (not limited to campus) as all users of mirrorz.org would request that url.

MirrorS may also contribute their mirrorz.json generating scripts in the directory scripts. The standard script is scripts/tunasync/mirrorz.py, all the details are specified there and unclear points of the data format is explaned there.

Frontend

One may use

yarn --frozen-lockfile
yarn start

to start a local server.

To use static files, 404 redirection (emulate Github pages) and legacy pages, one may use

yarn --frozen-lockfile
yarn full_start

to start a local server. Python3 is required in this case.

Deployment

For dynamic web page part, only

yarn build

is required.

However if one also wants to deploy the static web page, one may use

yarn build
yarn legacy_build

Note that legacy_build has dependencies on files yarn build has made.

Misc

Currently *.json is ignored in .gitignore. If one wants to commit a json file, they should use git add -f.

uestclug / mirrorz