SQL Scripts
Useful SQL scripts, split from DevOps Bash tools, for which this is now a submodule.
Hari Sekhon
Cloud & Big Data Contractor, United Kingdom
(you're welcome to connect with me on LinkedIn)
Inventory
aws_athena_cloudtrail_ddl.sql
- AWS Athena DDL to setup up integration to query CloudTrail logs from Athenabigquery_billing_*.sql
- Google BigQuery billing queries for GCP services eg. highest cost services, most used GCP products, recent charges etc.mysql_*.sql
- MySQL / MariaDB queries for DBA investigating + performance tuning- tested on MySQL 5.5, 5.6, 5.7, 8.0 and MariaDB 5.5, 10.x
mysql_info.sql
- summary overview, useful to debug yourmysql.user
table auth effects (intendedUSER()
vs actualCURRENT_USER()
)
postgres_*.sql
- PostgreSQL queries for DBA investigating + performance tuning- tested on PostgreSQL 8.4, 9.x, 10.x, 11.x, 12.x
- postgres_info.sql - big summary overview, recommend you start here
You can quickly test the PostgreSQL / MySQL scripts using postgres.sh
/ mysqld.sh
/ mariadb.sh
in the DevOps Bash tools repo, which boots a docker container and drops in to mysql
/ psql
shell with this directory mounted at /sql
and used as $PWD
for fast easy sourcing eg.
postgres:
\i /sql/postgres_query_times.sql
mysql:
source /sql/mysql_sessions.sql
or just
\i postgres_query_times.sql
\. mysql_sessions.sql
since the $PWD
is set to /sql
for convenience.
See Also:
- .psqlrc - advanced PostgreSQL psql client config
- psql.sh - quickly connect to PostgreSQL with command line switches inferred from environment variables
- mysql.sh - quickly connect to MySQL / MariaDB with command line switches inferred from environment variables
- postgres.sh - one-touch PostgreSQL, boots docker container and drops you in to
psql
shell. Version can be given as an argument - mysqld.sh / mariadb.sh - one-touch MySQL / MariaDB, boots docker container and drops you in to
mysql
shell. Version can be given as an argument - sqlcase.pl - autocases your SQL code
- I use this a lot and call it via hotkey configured in my .vimrc
- there are
*case.pl
specializations for most of the major RDBMS and distributed SQL systems, even several NoSQL systems, using each one's language specific keywords
-
DevOps Bash Tools - 300+ DevOps Bash Scripts, Advanced
.bashrc
,.vimrc
,.screenrc
,.tmux.conf
, CI configs & Utility Code Library - AWS, GCP, Kubernetes, Kafka, Docker, Hadoop, SQL, Hive, Impala, PostgreSQL, MySQL, LDAP, Spotify API & MP3 tools, Git, Code & build linting, package management for Linux / Mac / Python / Perl / Ruby / NodeJS / Golang, and lots more random goodies -
DevOps Python Tools - 80+ DevOps CLI tools for AWS, Hadoop, HBase, Spark, Log Anonymizer, Ambari Blueprints, AWS CloudFormation, Linux, Docker, Spark Data Converters & Validators (Avro / Parquet / JSON / CSV / INI / XML / YAML), Elasticsearch, Solr, Travis CI, Pig, IPython
-
The Advanced Nagios Plugins Collection - 450+ programs for Nagios monitoring your Hadoop & NoSQL clusters. Covers every Hadoop vendor's management API and every major NoSQL technology (HBase, Cassandra, MongoDB, Elasticsearch, Solr, Riak, Redis etc.) as well as message queues (Kafka, RabbitMQ), continuous integration (Jenkins, Travis CI) and traditional infrastructure (SSL, Whois, DNS, Linux)
-
DevOps Perl Tools - 25+ DevOps CLI tools for Hadoop, HDFS, Hive, Solr/SolrCloud CLI, Log Anonymizer, Nginx stats & HTTP(S) URL watchers for load balanced web farms, Dockerfiles & SQL ReCaser (MySQL, PostgreSQL, AWS Redshift, Snowflake, Apache Drill, Hive, Impala, Cassandra CQL, Microsoft SQL Server, Oracle, Couchbase N1QL, Dockerfiles, Pig Latin, Neo4j, InfluxDB), Ambari FreeIPA Kerberos, Datameer, Linux...
-
HAProxy Configs - 80+ HAProxy Configs for Hadoop, Big Data, NoSQL, Docker, Elasticsearch, SolrCloud, HBase, Cloudera, Hortonworks, MapR, MySQL, PostgreSQL, Apache Drill, Hive, Presto, Impala, ZooKeeper, OpenTSDB, InfluxDB, Prometheus, Kibana, Graphite, SSH, RabbitMQ, Redis, Riak, Rancher etc.
-
Dockerfiles - 50+ DockerHub public images for Docker & Kubernetes - Hadoop, Kafka, ZooKeeper, HBase, Cassandra, Solr, SolrCloud, Presto, Apache Drill, Nifi, Spark, Mesos, Consul, Riak, OpenTSDB, Jython, Advanced Nagios Plugins & DevOps Tools repos on Alpine, CentOS, Debian, Fedora, Ubuntu, Superset, H2O, Serf, Alluxio / Tachyon, FakeS3