CentOS 7 Simple Kickstart

Background: I have supported Redhat since its inception and every *nix/VMS/Dynix prior to that. I have assisted countless companies in migrating from proprietary unix to Redhat Linux. Sorry to say, this has reached its end. Redhat have embraced systemd, a framework that has a strong potential to compromise corporations and individual privacy. Going forward I will be assisting companies in moving away from Redhat Linux and CentOS.

I will keep this repo around for a while. Feel free to fork it if you find it at all useful. My focus will be on CoreOS [0] for containers and Alpine Linux [1] for everything else.

[0] https://coreos.com/ [1] https://alpinelinux.org/

Goals:

Create an extremely simple and trimmed down CentOS 7 kickstart environment that produces small CentOS 7 images.
Use nearly the same method that a person might use to create this on bare metal servers in a data-center.
One could even build servers in a data-center off their laptop if they add a bridged or NAT interface to the Kickstart VM.
Use only what Apple provides us on the Mac, plus VirtualBox if it is not already installed.
Avoid requiring sudo or root on the laptop.
Avoid vagrant as it is not repeatable in the data-center.
Learn how kickstart works and what is required to build a simple CentOS 7 machine.

___

Notational Conventions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119.

The key words "MUST (BUT WE KNOW YOU WON'T)", "SHOULD CONSIDER", and "REALLY SHOULD NOT" are to be interpreted as described in RFC-6919.

Requirements:

Mac Laptop with VirtualBox installed.
Access to the internet on TCP ports 80 (http), 443 (https) and 873 (rsync)
About 100 GB of free space on your hard drive. Actual usage may be much less depending on number of VM you create.
Some type of command line interface. Most folks use Terminal on Mac.

___

Steps

In these steps, we will be syncing the public CentOS and Fedora EPEL repos to a staging location on your laptop, then starting a local low privileged instance of rsync and apache on your private vboxnet interface.

From there, the kickstart process will read your kickstart configuration files and build the first VM, using the files from your laptop. After the first VM is built, all future VM's will then be built directly from your "kickstart server" VM from step2.

Install VirtualBox if you have not done so already. See virtualbox.org
Clone this repo. See https://github.com/ for instructions on how to clone repos.


mkdir -p ~/build/centos7_simple_kickstart/scripts


git clone https://github.com/ohdns/centos7_simple_kickstart.git


rsync -av centos7_simple_kickstart/. ~/build/centos7_simple_kickstart/scripts/.

This is so that you have a working copy you can edit. Please do feel free to fork this.

OPTIONAL: Edit the .cfg files in ~/build/centos7_simple_kickstart/scripts and replace the SHA512 hashes with your own.
I would show how to generate a SHA512 shadow hash, but each OS and each version of Python behaves dramatically different in this area.

The default password for ohadmin and root will be centos7
The ohadmin account will be used for automation; and temporarily, the way you ssh to your VM for manual changes or container deployments.

It is assumed that you will change all passwords and ssh keys before doing something risky, such as bridging your kickstart VM onto a non private network
Step 1: Execute ~/build/centos7_simple_kickstart/scripts/oh repo_sync

This step will rsync the CentOS and Fedora EPEL repos to your laptop in a staging location.

This will run peridoically from the kickstart VM, pulling and mirroring the updated repos from your laptop, so run this step once per week.
Step 2: Execute ~/build/centos7_simple_kickstart/scripts/oh ssh_config

This step will create a ssh key pair for each new VM that this script supports. You should only have to run this step once.
Step 3: Execute ~/build/centos7_simple_kickstart/scripts/oh build_ks

This step will perform the following:
Start up a local apache instance on 192.168.120.1 on a new private network of 192.168.120.0/24
Start up a local rsync daemon on a high port on 192.168.120.1
Kickstart your new kickstart VM server. You should only have to run this step once.

When the first VM starts, you should see a CentOS ISO install screen.
At this point, hit the TAB key, backspace over quiet and type:

cmdline ip=192.168.120.10 netmask=255.255.255.0 ks=http://192.168.120.1:8888/c7_server.cfg nicdelay=20

This will build the first VM (the kickstart server role) and will rsync the Yum repos to itself, pulling from your laptop. Get a cup of coffee or tea while this runs.

Once this step completes, you should be able (from your laptop terminal) to type: ssh ohkickstart

Now let us deploy some server roles using our shiny new Kickstart VM

When the Kickstart VM completes building:

Step 4: (Optional) Execute ~/build/centos7_simple_kickstart/scripts/oh build_docker to spin up a docker VM.

Unless you change the oh script functions, the VM's will use up to 1.5 GB of ram each. Most laptops do not have more than 16 GB of ram. By all means, fiddle with the memory allocation to see what you can create. CentOS 7 can run in a small amount of memory and with a tiny disk, but it wants some disk space to stage temporary files.

When this step completes, you should be able to type: ssh ohdocker from your laptop terminal.
Build a DevOps Workstation / Laptop Image
Step 5: (Optional) Execute ~/build/centos7_simple_kickstart/scripts/oh build_devops

This step will create a DevOps workstation with many of the tools and services one might need such as mcollective, ruby, perl, python, various compilers and libraries, etc...

___

Some Challenges For You!

Keep SELinux enabled, no matter what you plan to install.
Instead of disabling SELinux, read up on setting booleans (setsebool -P boolean or getsebool -a), creating rules, setting file and directory contexts (semanage fcontext)
Confine as many applications as you can, to the least amount of privilages required to run the application.
Use audit2allow, audit2why or grep through /var/log/audit/audit.log to see why something was denied.
As a last resort, set a user or process to unconfined or permissive instead of disabling SELinux.
Prefer a chroot restricted SFTP over unrestricted SSH trusts when feasible.
You can accomplish the same behavior of rsync using SFTP Chroot + LFTP and it's mirror subsystem, without having to expose your entire system or provide shells to people. It is faster than rsync and more secure.
For custom applications, clearly define:
where the application binaries should be installed. i.e. /opt/application_name/{etc,lib,include,bin,sbin}
where the application logs should reside. i.e. /data/application_name/{var,logs}
where transitory data should reside. i.e. /data/application_name/{data,db}
or better, use containers! to keep the OS pristine and easy to patch.
Configure your applications, users, and group posix permissions to allow the right folks to view logs, restart services, or otherwise manage their job role without having to elevate privs to root. If non sysadmins are needing root or sudo to get their job done, then the sysadmin has more work to do, sorry. Get a sysadmin, a developer, a devops person into a room and sort it out. You will save thousands of hours or work for yourselves.
Aside from the Automation account, avoid SSH key trusts when you can. Proper automation is derrived from codified instructions that execute on orchestration servers from within data-center, not from a laptop.
Avoid enabling root login via ssh. In fact, avoid logging into any VM's if you can.
The base configuration belongs in kickstart and customization needs to be done in a configuration management and orchestration system.
All services need to be initialized by a proper startup script when the server OS starts up.
Avoid using sudo. If you find yourself using sudo su - or sudo -s then you really just need the root password. Sudo is not a security tool. It is designed to allow a sysadmin to delegate commands as a user, to a set of users. If you can get a root shell through sudo, then it is misconfigured. Sudo is potentially a critical security risk if your users have un-restricted sudo and they get phished. This is equally a risk in both a production customer facing environment and a development or continuous integration environment, as both ultimately touch customer data.

Create yourself a counter. Every time you type sudo, subtract 10 engineering points.

___

Known Issues and Limitations / TO-DO's:

LIMITATION and OPTIONAL: To use NTP from the kickstart server to our laptop, we have to break our own rule one time and use either root or sudo to modify /private/etc/ntp-restrict.conf on our laptop to allow a query.
sudo echo "restrict 192.168.120.0/24" >> /private/etc/ntp-restrict.conf ; sudo pkill -HUP ntpd
SUB-OPTIMAL: There is one manual step to create the DHCP/PXE/Yum/Kickstart server. This is probably ok, since we should not be doing this often.
SUB-OPTIMAL: This method currently lacks end-to-end validation of the RPM GPG signatures. (Work In Progress, Contacting CentOS Team.)
TO-DO: ~~You will need to manually add your SSH public key in the ~/build/centos7_simple_kickstart/scripts/c7*cfg~~ Completed and Rearchitected.
TO-DO: ~~Merge all the steps into 1 script as functions~~. Completed.
TO-DO: ~~Add cron entry to the _kickstart_ VM to periodically sync its mirror of CentOS and EPEL from the Laptop.~~ Completed.
WORKS AS DESIGNED: There is currently a bit of customization in the kickstart files. This is on purpose. We start with functional customizations in kickstart, so that anyone can easily figure out what needs to be customized in Ansible, Puppet, Chef, cfengine or whatever their configuration management flavor preference may be.
WORKS AS DESIGNED: I renice this script to avoid blocking any work you are doing. This means if your laptop is under a heavy load, this kickstart build process may go very slow by design.
WORKS AS DESIGNED: Since we are not using Vagrant, you would have to script startup/shutdown yourself.

Examples:

PATH=${PATH}:/Applications/VirtualBox.app/Contents/MacOS;export PATH

Clean / Graceful Power Off:
VBoxManage controlvm c7_docker acpipowerbutton

Power On:
VBoxManage startvm c7_docker

Power On Headless (No GUI / console):
VBoxManage startvm --type headless c7_docker

If you use VirtualBox a bit from the command line, then you may with to update the PATH in your ~/.bash_profile

License: WTFPL see http://www.wtfpl.net/txt/copying/

Disclaimer: This software repository contains scripts that are for educational purposes only. This repo contains default passwords that must not be used anywhere beyond VirtualBox on your laptop for educational purposes only. DO NOT use this to deploy a production environment unless you have properly changed all defaults, changed settings to reflect that which is approved for your environment and have properly tested this in a lab and staging area that matches your live environments. The author of these scripts assumes no responsibility for damages to persons or property. Do not bridge any VM's to a network outside of localnet without first understanding the potential consequences and having the appropriate personnel in your organization validate and accept the risks.

TL;DR - Get someone else to sign off and accept the risk for using someone elses scripts in your environment.

20180608

higeon / centos7_simple_kickstart