Bash script for displaying relevant system information and performing prerequisite checks for Cloudera Manager, CDH and CDP Data Center (CDP-DC) installations.
Motivation: Ensuring that the long list of required and recommended prerequisites are correctly applied during a Hadoop Cluster Deployment (or similar) engagement is manual, time-consuming, and error-prone (not to mention mind-numbing).
Non-Goals: This is not intended to replace or compete with the Cloudera Support Interface (CSI), which includes a detailed cluster validation report.
For details on the checks performed, refer to the following:
The following screenshot shows a run on an configured (or misconfigured)
system:
And here's the output on the same server after addressing all the issues:
Currently there are two ways to run this script: The first method is to obtain the inspection result from a single host targeting the host that executed the script. The second method is to obtain the inspection results from multiple hosts by using Ansible to automate distribution and execution of the script file(s) and gathering the inspection results from the multiple target hosts.
- For using the distributed mode, run
yum install ansible - For using the AD domain controller checks, run
yum install perl-Convert-ASN1 bind-utils - For using the AD delegated user privilege checks, run
yum install openldap-clients
Running the script is easy as it is intentionally written in Bash to avoid any dependencies. This is to avoid dependency hell in restrictive customer environments. It does not run on Mac OS. Tested on RHEL/CentOS 6.7 and 7.3 - see the vagrant/ subfolder for details. Requires root/superuser permissions to run.
To run:
./prereq-check-dev.sh
This requires the libraries in lib/, which includes both Bash and Perl
libraries. See Usage for details.
To build/update the single file version of the script, run:
./build.sh
This produces the file prereq-check.sh. See Usage for details.
Usage for build.sh:
% ./build.sh -h
NAME:
build.sh - prereq-check.sh build script
SYNOPSIS:
build.sh [options]
OPTIONS:
-h, --help
Show this message
-a, --auto-build
Watch dependencies for changes and automatically rebuild on changes
With the -a option, the build script can watch dependencies and automatically
rebuild the combined prereq-check.sh. It also runs shellcheck (Bash Lint) if
it's installed. For example:
% ./build.sh -a
Sun Aug 20 11:21:19 SGT 2017 [INFO] Found 'shellcheck', will run post-build Bash lint
Sun Aug 20 11:21:19 SGT 2017 [INFO] Watching dependencies for changes...
Sun Aug 20 11:21:32 SGT 2017 [INFO] Wrote to prereq-check.sh and updated Vagrant hard-links
Sun Aug 20 11:21:32 SGT 2017 [INFO] Running 'shellcheck prereq-check.sh'...
Sun Aug 20 11:21:33 SGT 2017 [INFO] shellcheck: All good
Sun Aug 20 11:21:41 SGT 2017 [INFO] Wrote to prereq-check.sh and updated Vagrant hard-links
Sun Aug 20 11:21:41 SGT 2017 [INFO] Running 'shellcheck prereq-check.sh'...
In prereq-check.sh line 1434:
unused_var=
^-- SC2034: unused_var appears unused. Verify it or export it.
Sun Aug 20 11:21:42 SGT 2017 [INFO] shellcheck: Found above warnings/errors
$ ./prereq-check.sh -h
NAME:
prereq-check.sh - Cloudera Manager & CDH Prerequisites Checks v1.4.1
SYNOPSIS:
prereq-check.sh [options]
OPTIONS:
-h, --help
Show this message
-a, --addc domain
Run tests against Active Directory Domain Controller
-p, --privilegetest LdapURI KDC_Manager ClouderaOU_DN KDC_Manager_Password
Run tests against Active Directory delegated user for Direct to AD integration
http://blog.cloudera.com/blog/2014/07/new-in-cloudera-manager-5-1-direct-active-directory-integration-for-kerberos-authentication/
-c, --cdsw CDSW_FQDN CDSW_Master_IP
Run CDSW pre-requisite checks
Please report any bugs or feature requests using the Github Issues here. Better yet, send pull requests!