ooni / sysadmin

System administration tools

https://ooni.org

ooni/sysadmin Issues

Slack bridge is not transmitting messages to IRC channel
Closed 2 years ago2
RSS by country feed is not available
Updated 3 years ago
Metadb Replica Access - postgres user
Closed 4 years ago1
no:assignee
Closed 4 years ago
MetaDB - Time to replication
Closed 4 years ago
psql: FATAL: database "metadb" does not exist
Closed 4 years ago4
Incident: very slow rsync between ams-ps1 and datacollector
Closed 5 years ago4
Bouncer giving out down test-helpers for ~16 hours
Closed 4 years ago5
Monitoring of hosts was down for several weeks
Closed 4 years ago5
Slacktopus is not transmitting messages to IRC endpoint
Closed 4 years ago3
API down for ~10h
Closed 4 years ago1
registry.proteus.test.ooni.io was expired for some hours
Closed 4 years ago4
Measurements were not uploaded properly in OONI Probe android 2.0.0
Closed 4 years ago1
Half of 2019-01-05 bucket stalled for 10d
Closed 4 years ago2
DiskStuck for ~10m at AMS leading to measurements loss
Closed 4 years ago7
Almost-OOM at b.web-connectivity.th.ooni.io
Closed 4 years ago2
OpenDNS resolver unreachability triggers alert storm
Closed 4 years ago2
a.collector was not running properly
Closed 4 years ago5
Open recursive resolver at ooni-zoo.infra.ooni.io used for amplification attack
Closed 4 years ago3
b.echo.th.ooni.io possibly down for 8 hours
Updated 4 years ago2
[FIRING] Lots of `scrape_samples_scraped` lost Now ~ 2.83k, 24h ago ~ 27.96k.
Closed 4 years ago1
Write documentation for docker-trampoline
Closed 4 years ago2
Drop brie.darkk.net.ru from monitoring
Closed 4 years ago2
Prometheus / Grafana: store data for longer times
Closed 4 years ago10
Make the cans public and move the to the open data account
Closed 4 years ago
Incident: blocked pipeline on 2019-12-10
Closed 4 years ago1
y3zq5fwelrzkkv3s.onion ams-wcth2 unreachable
Closed 4 years ago2
Implement a deb based deployment process
Closed 5 years ago2
Monitoring: fix frequent alerts around .onion services being unreachable
Closed 5 years ago
Migrate collectors to the oonified host
Closed 5 years ago1
LetsEncrypt: make sure we have correct hooks
Closed 5 years ago4
c.collector non usable with Debian because of TLS < 1.2
Closed 5 years ago4
Update all vms on jessie to buster
Closed 5 years ago2
Drop third collector VM
Closed 5 years ago1
Update ooni run playbook to use docker
Closed 5 years ago3
Ensure all our infrastructure is redeployable
Closed 5 years ago5
Improve availability of our infrastructure aka "oonification of probe services"
Closed 5 years ago7
Create VM to host testing collector
Closed 5 years ago1
Test master issue
Closed 5 years ago
Child issue
Closed 5 years ago
Upgrade TLS used by web-connectivity test-helper
Closed 5 years ago1
amsmetadb running out of disk space
Closed 5 years ago2
Delete or add to the inventory currently untracked machines
Closed 5 years ago
Change the uid scheme
Closed 5 years ago1
Create a periodic database dump of the metadb
Closed 5 years ago1
2019-06-21 pipeline stall
Closed 5 years ago5
Devops before leaving
Closed 5 years ago1
Setup a system for sharing the metadb with third parties
Closed 5 years ago22
GH hosts missing from inventory
Closed 5 years ago3
labs.ooni.io down
Closed 6 years ago1