DevOps-Bash-tools/README.md

Hari Sekhon - DevOps Bash Tools

70+ Shell Scripts, Advanced Bashrc & Utility Code Library used by all my other GitHub repos.

For more advanced Systems Administration scripts in other languages, see the repos listed at the bottom of the page.

• Scripts - Linux systems administration scripts

  • installation scripts for various OS packages (RPM, Deb, Apk) for various Linux distros (Redhat RHEL / CentOS / Fedora, Debian / Ubuntu, Alpine)
  • install if absent scripts for Python and Perl modules - good for combining with first attempt to install via system packages, and then these will pull from PyPI or CPAN only those modules which aren't installed. This speeds up builds and uses the standard packaged modules where possible. This is also more reliable than getting random compile errors from CPAN as libraries update and introduce bugs or needing to install too many dev libraries
  • install scripts for Jython and build tools like Gradle and SBT for when Linux distros don't provide packaged versions or where the packaged versions are too old
  • Git branch management
  • utility scripts used from other scripts

• .bashrc - shell tuning and sourcing of .bash.d/*.sh

• .bash.d/*.sh - thousands of lines of advanced bashrc code, aliases, functions and environment variables for:

  • Linux & Mac
  • SCM - Git, Mercurial, Svn
  • Docker
  • Kubernetes
  • Kafka
  • Vagrant
  • automatic GPG and SSH agent handling
  • and lots more - see .bash.d/README for a more detailed list

• lib/*.sh - Bash utility libraries full of functions for Docker, environment, CI detection (Travis CI, Jenkins), port and HTTP url availability content checks etc.

• kafka_wrappers/*.sh - scripts to make Kafka cli usage easier including auto-setting Kerberos to source TGT from environment and auto-populating broker and zookeeper addresses. These are auto-added to the $PATH when .bashrc is sourced. For something similar for Solr, see solr_cli.pl in DevOps Perl Tools

• Programming language linting:

  • Python (syntax, pep8, pre-byte-compiling)
  • Perl
  • Java
  • Scala
  • Ruby
  • Bash / Shell
  • Misc (whitespace, custom enforced checks like not calling quit() in python etc)

• Build System and CI linting:

  • Make
  • Maven
  • SBT
  • Gradle
  • Travis CI

• Data format validation using programs from my DevOps Python Tools repo:

  • CSV
  • JSON
  • Avro
  • Parquet
  • INI / Properties files (Java)
  • LDAP LDIF
  • XML
  • YAML

Currently utilized in the following GitHub repos:

• Advanced Nagios Plugins Collection - 450+ programs covering every major Hadoop & NoSQL technology and Linux/Unix based infrastructure technologies
• DevOps Python Tools - 75+ command line tools
• DevOps Perl Tools - 25+ command line tools
• Perl Lib - Perl utility library
• PyLib - Python utility library
• Lib-Java - Java utility library
• Nagios Plugin Kafka - Kafka Nagios Plugin written in Scala with Kerberos support

Pre-built Docker images are available for those repos (which include this one as a submodule) and the "docker available" icon above links to an uber image which contains all my github repos pre-built. There are Centos, Alpine, Debian and Ubuntu versions of this uber Docker image containing all repos.