scripts-misc
============
This project contains a collection of scripts which I have developed
over the years to simplify common and recurring tasks and to make my
life easier.
Environment
-----------
Some scripts require environment variables to be set up to provide
personal data and adjust to the local setup. Common settings will
be listed in this section.
Script specific settings will be listed in the respective sections.
Scripts will source $HOME/.myconfig to set the personal preferences.
To be able to share this file between different machines, it should
source a machine specific config.
The following environment variables are recognized from $HOME/.myconfig:
TMPDIR= # /var/tmp
# company email address
MY_COMPANY_EMAIL= # "me@suse.com"
# email address preferred for software projects
MY_EMAIL= # "me@myfavoriteproject.org"
test -e ${HOME}/.mylocalconfig && . ${HOME}/.mylocalconfig
test -e ${HOME}/.mymachine-${HOSTNAME} && . ${HOME}/.mymachine-${HOSTNAME}
$HOME/.mymachine-* should contain:
BUILDENVHOME=/space/bev
BUILDEXTRADIRS="/space /space1"
SUSEKERNEL=/space/suse_kernel/kernel-source
CHECKPATCH=/space/kernel/linux-2.6/scripts/checkpatch.pl
SPACEDIR=/space
Help
----
Each script comes with a help. It can be invoked with '-h'.
installkernel/removekernel
--------------------------
This script is based on a script I once got from Thomas Renninger.
Its origins may well be older. It is capable of installing a kernel
or individual modules on the local or a remote machine. It will also
update the boot loader accordingly.
Optionally it is able to strip the kernel and its modules. This is
useful to speed up the transfer and to reduce its foot print on the
remote machine.
The script needs to be executed from the kernel source directory (ie.
where 'make' was executed).
Usage installkernel [-o <builddir> ] [-a <arch>] [-l] [-f] [-i] [-n] [-s] [-j <n>] [hostname]
installkernel [-i] [-a <arch>] [-s] -m "module_list" [hostname]
removekernel [-l] [-n] [-f] [-i] [<version>] [hostname]
Options
-n: nomodule
-a <arch>: architecture(x86_64/i386)
-o <build_dir>: location of build files (O=... kernel option)
-b: edit bootloader configuration manually
-f: don't ask (doesn't update bootloader)
-i: don't update bootloader/initrd
-d: do update kdump initrd
-m: modules to install
-j <n>: parallelize actions that can be parallelized
-l: use lilo
-s: strip kernel/modules
-O: set a one time boot entry to the newly installed kernel
(like grub(2-)once).
Description:
'installkernel' installs the entire kernel (optionally without modules)
or individual modules. If no hostname is specified, it will install the
kernel on the local machine.
'-n' will install only the kernel image, not the modules.
'-m' will install the list of modules specified after the option. When
specifying more than one module, the list needs to appear in quotes.
'-o <build_dir>' will look for the kernel binaries on the directory
specified. This is useful when building with the O=<build_dir> parameter.
'-i' will prevent the boot loader from being update. This will also prevent
the initrd from being built.
'-d' will invoke 'mkdumprd -f' for the new kernel so it can be used for
kexec, too.
'-s' will cause the kernel image and the modules to be stripped.
'-a <arch>' if the kernel was built for a different architecture than the
build host (for example i586 on x86_64) this option must be specified.
Otherwise, 'make install' will attempt to recreate the kernel configuration.
'-j <n>' will parallelize the 'make install' process.
'-b' allows the user to edit the boot loader configuration manually.
'-f' runs the command in non-interactive mode, ie all user queries are
suppressed.
'-l' uses 'lilo' instead of 'grub' on x86_64 and i386 platforms.
'-O' creates a one time boot entry for grub(2) for the newly created kernel.
This option is only allowed when installkernel is run when the initrd is
rebuild as well.
'removekernel' does the opposite ie delete a specific kernel. If no
version is specified it will remove the default kernel - ie the one
which 'vmlinux' or 'vmlinuz' links to.
git_show_suse.sh / git_show_xorg.sh
-----------------------------------
git_show_suse.sh is based on a script I once got from Thomas Renninger.
Create a list of patches in SUSE kernel or X11:XOrg project style.
When executed in Linus' tree the upstream commit ID will be added.
When executed from another cloned kernel repository the upstream URL
and commit ID will be added. When the repository is local or the
upsream URL contains 'suse' no repor or commit ID will be specified.
For X11:XOrg the patch state will be prepended to the name.
Usage: git_show_suse.sh [-m N|n|u|U[<release_number>]|T] [-M "<mainline message>"] [-r "<reference string>"] [-d <directory>] [<range>]
git_show_xorg.sh [-m N|n|u|U[<release_number>]|T] [-M "<mainline message>"] [-r "<reference string>"] [-d <directory>] [<range>]
Options:
-m The following values are recognized:
N: "Not applicable"
n: "never"
u: "To be upstreamed"
U "Upstream", an optional revision string may be specified
Default: u
-M <message> will replace the default message generated by the -m option with a
different string. This string will be printed in the Patch-mainline: preamble
line of the patch.
-r <sting> an optional reference string. If none is specified the Reference:
line in the patch preamble will be left empty.
-d <directory>: output to directory <directory> instead of the current one.
-v verbose: print list of files.
<range> a range of patches. This can either be a single commit or a range of
commits like HEAD^^..HEAD. Default is HEAD^..HEAD.
Description:
'-m' Allows to override the default state message 'u'. In the kernel case, the
script will attempt to determine if the patch is from a public repo and will
mark it accordingly.
'-M' will replace the default (according to the state set with '-m') state
string.
'-r' allows to add a 'Reference:' string. It should contain a bug and/or
FATE ID.
'-d' allows to specify a directory where the patches should be placed.
'-v' displays the list of patch files generated.
<range> allows to specify the list of patches. This can be either a list of
<id>, or <id1>..<id2>, ..<id2>, <id1>.. or a mixture thereof.
TODO:
An 'upstream' should be added for X11:XOrg flavored patches as well. The '-m'
option needs to be reconsidered reflecting recent changes in requirements for
the kernel.
Requires: git, rpmbuild, sudo, coreutils, util-linux
rpm-git
-------
This scripts when executed from a directory containing an unpacked source
RPM and executes the %prep stage to unpack the sources in a subdirectory
'src/' and apply patches relevant patches. If the environment variable
BUILDENV is set, it take the specified path as the path to the buildroot
and tries to determine the architecture from BUILDENV. This way the
architecture specific macros of the target project are used to correctly
follow conditional statements. If no BUILDENV is set, the script will
execute %prep stage for the current system. If BUILDENV is set, BUILDENVHOME
needs to be set as well. This is the directory where the changeroot
environment exists.
Once all patches have been applied, it creates a git repo in 'src/'
and set up a .gitignore.
rpm-git [-f <fuzz>] [-F <flavor>] [-t] [specfile]
Options:
-f <fuzz>: specify maximum fuzz level. default: 0.
-F <flavor>: specify flavor for multibuild spec file (default: standard)
-t: test mode, will not create a src/ directory.
-a <arch>: specify architecture if not the same as host.
'-f': Set a allowed fuzz level. This is useful for older versions of some
packages where a certain amount of fuzz was still allowed.
'-F': In a multibuild environment, the spec file contains the pattern
'@FLAVOR@' which is to be replaced by the build flavor to be used.
The default is 'standard' which is the fallback in most cases.
'-t': Test mode: don't create a 'src/' directory and git repo. Useful
to test if all patches apply.
'-a': specify an architecture in case it cannot be derived from BUILDENV.
A specfile needs to be specified only if the current directory contains
more than one.
TODO: Also catch rpm macros set in the projconf in the current directory
is an OBS package directory.
setup-bev.sh
------------
Set up a build changeroot. Given a list of 'BuildRequires: ' in a file
<PRODUCT>.desc, the script set up a fully installed build chroot
with all the BuildRequires and their dependencies installed.
This chroot is meant to be used with the localbuild.sh and my-rpmbuild
scripts below.
The 'BuildRequires: ' lines have the same format as in a spec-file
and can be taken straight from a spec file.
The changeroot can be used to build software for the system installed
in the chroot, instead of the host system. With binfmt and kvm this
even works for other architectures.
This script requires the BUILDENVHOME environment variable to be set.
It specifies where the chroot directory is to be created.
setup_bev.sh [-a <arch>] [-r <repository>] [-d <directory>] [-n] <PRODUCT> [POST]
Options:
-a <arch>: select architecture: i586, x86_64
-r <repository>: set repository, default: standard
-d <directory>: use alternate directory for build env
-n: do not strip trailing _* from target
<PRODUCT>: product name: openSUSE-13.2, SLE-12-SP1, etc.
POST: For SLE - if not to be included in the product name
- GA Update Update:Test (default: GA)
'-a <arch>': specify the architecture for which to create the environment.
'-r <repository>': set name of the binary repository
'-d <directory>': overrides BUILDENVHOME
<PRODUCT>: the product name in OBS.
POST: Used for SLE - specifies the version (ie. GA, Update, Update:Test)
if this isn't to be sepcified with the product.
This script will also set up a home and an entry to the /etc/passwd file
for the current user (the one set in $USER) so that a login into the
chroot is possible.
If the environment variable $SPACEDIR is set it will place the package
cache for the packages to install in $SPACEDIR/osbuild-packagecache.
Furthermore it will create $SPACEDIR in the chroot.
localbuild.sh
-------------
Mount the current directory ${PWD} into chroot environment specified
by $BUILDENV or -s <SYSTEM> and execute <command> there. It can also
be used to start an interactive shell in the BUILDROOT.
This supplements setup-bev.sh which is used to set up the chroot
environment. Alongside the current directory, all directories listed
in BUILDEXTRADIRS are bind-mounted into the chroot at the same location
relative to the root of the chroot as they are on the host system.
localbuild.sh [-s <SYSTEM>] [-l|-i|<command>]
Options:
-s <SYSTEM> := openSUSE-<version> | SLE..
or set the BUILDENV environment variable
-l: start interactive shell in local home directory.
-i: info only: print buildroot directory.
-e <ENV>: add environment setting ENV to command. <ENV> must be of the form FOO=BAR.
<command>: run <command> in current directory but in build environment.
If run without command, start interactive shell in current directory.
use BUILDEXTRADIRS to specify extra directories to include in build environment.
use BUILDENVHOME to specify location of build environment.
'-s': Set the target chroot system. Can be used in place of the BUILDENV
environment variable.
'-l': start an interactive shell in the bind mounted home directory
in the chroot.
'-i': Just liste the name of the chroot.
'<command>': execute <command> in the chroot. This can be used to build
in a build environment that is different from the host system.
'-e': specify an environment that is to be set in the chrooted environment
before the command gets executed. Use the form FOO=BAR, if this contains
spaces, quote the entire string.
Requires: coreutils, sudo, grep, util-linux
my-rpmbuild
-----------
rpmbuild package in chroot environment. Executed from a rpm source
directory, this script checks for the presence of a src directory.
If none exists, it will run rpm-git to create one. It creates a linked
source tree in the subdirectory .build calls localbuild.sh to mount the
package into a a buildroot environment (change root) and to perform
the rpm build stage specified with the 'b' option (the --short-circuit
option is set to not overwrite any of the previous build stages).
The mount point in the chrooted environment will be the same as on the
host.
This can be used to build modified sources for a target system that's
different from the host system in exactly the same way as it would be
done in the build system. Furthermore, when mounting the build directory
on a test system in the same location as on the host, all paths to the
source files are set up so that gdb will find them.
my-rpmbuild [-n][-s][-b buildopt][-e buildenv][--with[out]-<option>][-v] [specfile]
Options:
-n: do not create a git repo
-s: don't build in a separated `lndir'ed` directory
-y: say 'yes' to every question
-a <arch>: specify architecture if not the same as host.
-b <buildopt>: <buildopt>: -bc -bb -bi -bs
-e <buildenv>: the build environment overrides BUILDENV if set
-with[out]-<option>: add this --with[out]-* option to rpmbuild
-v: run rpmbuild verbose
<specfile>: the specfile (if applicable with path)
'-n': do not create a git repo of the package if none exists. It
requires the directory .build to exist and contain the code to
build.
'-s': do not create a speparate lndir'ed build directory. Some packages
don't build well in such an environment (qmake of Qt5 is an example
for a tool which fails in an lndir'ed environment in a non-trivial
way).
'-y': When being asked questions in interactive mode, always say 'yes'.
Useful when run in batch mode.
'-a <arch>': specifies the architecture of the target system if it
is different from the host and cannot be determined from the
change root build environment.
'-b <buildopt>': the rpmbuild build stage to execute: supported values
are: '-bc, -bb, -bi, -bs'.
'-e <buildenv>':
'specfile': specify the spec file if it cannot be guessed.
Requires: requires for localbuild.sh and rpm-git, lndir, rpmbuild, sed
rpm-bisect
----------
Bisect a project and create a tarball for inclusion in source rpm.
This script can be run even in the middle of a running bisect. It
saves/restores the current state, memorizes its state (by saving it
to a file with a unique identifier) so the bisect can be continued
at any time. The tarball will be copied to the directory containing
the source rpm. No actual checkout of the bisected version will be
done.
rpm-bisect command [options]
Run a bisect, output a tar-archive of the results and store the
state in a config file.
Command:
init <name> <target_dir> [-p <package> ] [-c <compression> ] [-v] (bad good ... | -r bad ... good)
Options:
<name>: name to identify the bisect (ie. bugzilla ID)
<target_dir>: directory to copy the tarball to
-r: reverse good/bad
-p <package>: specify package name (default: git dir name)
-c <compression>: specify compression for tar ball
(default: bz2)
-v: verbose, print version string used in tarball
good [-v] <name>
bad [-v] <name>
Requires: core-utils, sed, git
setup-kernel
------------
Create a source RPM from a kernel version from a git repo.
Run from the top directory of a SUSE kernel repo will create a kernel-source
directory with a kernel version specified on the command line from a
specified git repo instead of using the checked out version of the SUSE kernel.
The script will use the provided config files and convert them to work
with the specified kernel version.
setup-kernel [-V <upstream_tag>|<upstream_version>|<upstream_branch>][-a <arch> -f <flavor>]|--vanilla] <upstream_src>
Options:
-V <upstream_tag>|<upstream_version>|<upstream_branch>: default: master
Upstream version (either sha-1 or tag or HEAD of branch)
-a <arch>: use <arch> as architecture instead of current arch.
-f <flavor>: explicitely use flavor <flavor> for sequence-patch.sh
--vanilla: prepare for vanilla flavor
<upstream_src>: patch to upstream directory
Requires: core-utils, sed, git, awk
get-debug
---------
Download package, debugsource and debuginfo for this package (for a given
or the latest version for the specified product), unpack the rpms and copy
debuginfo files into the binary/library subdirectories so they can be found
easily by objdump and gdb.
get-debug -h | [ -a arch ] product package [ version ]
-h help: display this text
product: 13.2, 42.1, tumbleweed ...
package: package name
version: full package version (with arch
NOTE: you need to specify either <arch> or <version>
Requires: wget, rpm, zypper, cpio, grep, sed
tarup-version
-------------
Create a compressed tar archive of a given or current version of a git
repo and place the result in the dirctory specified.
tarup-version [-v][-c <compression>][-r <sha1>][-p package] targetdir
Options:
-v: verbose - print package & version
-c <compression> set archive compression. Default: bz2
-r <sha1> set version id. Default: HEAD
-p package: name of package. Default: name of git directory
-V version: package version to put in tarball and directory names
targetdir: directory to put archive
Requires: git, sed
suspend-test
------------
Cycle test suspend resume: on each cycle wait a configurable amount of time,
set wakeup time with a configurable delay, suspend (or hibernate).
To terminate, remove specified file in /dev/shm/. Optionally slecify maximum
number of cycles to run on the command line.
suspend-test [-h] [-s <sleep_delay>] [-w <wakeup_delay>] [-c <cycles>]
Options:
-h: hibernate (default: suspend)
-w <wakeup_delay>: set delay between suspend and wakeup (default 15 sec)
-s <suspend_delay>: set delay before suspend (defaut 8 sec)
-c <cycles>: number of cycles to run (default: indefinite)
-h: usage
Requires: coreutils, systemd or pm-utils
create-gitignore
----------------
Supplement script to rpm-git, creates a gitignore file containing generated
autotool files like Makefile.in or configure.in.
It searches for the top of the git tree and puts the ignore list into
.git/info/exclude.
No command line options, no help available.
Requires: coreutils, findutils, sed
osc-wrapper
-----------
Wrapper for osc to automatically replace ':' with '_' in 'osc checkout's.
When called as iosc_wrapper, it adds the innerweb API URL.
Transparent wrapper, no extra help available.
Requires: coreutils, osc ;-)
create-tarball-from-git
-----------------------
Create a tarball from a git repo with correct version number in its name
and top directory.
create-tarball-from-git [-v <version>] [-d <outdir>] [-c <compression>] | [-h]
Creaqte a tarball from HEAD or a given version of a git repo.
Options:
-e <pattern>: exclude glob(7) pattern when searching for tag
-m <pattern>: match glob(7) pattern when searching for tag
-v <version>: git version to use. default: HEAD
-d <outdir>: directory to place file. default:
-c <compression>: comression to use. Vaild are:
gz, Z, bz2, xz, lzma, zip. default: bz2.
Requires: coreutils, git
restore-rpms
------------
Roll back packages to a previously stored state. The state is stored
simply by dumping the output of `rpm -qa` into a file. Default file
name is `list.pre`.
restore-rpms [ <pre-file> ] || -h
-h : help
<pre-file>: file with rpm-list to get back to (generate with
rpm -qa)
default: pre.list
Requires: diffutils, grep, sed, zypper
create_mr
---------
Piece together a maintenance request in a staging project from a package in
IBS or OBS. This allows the package to be modified before being submmitted
as a maintenance request.
It supports to create a package under a different name than the source
package to start a new code stream.
create_mr [-o|-d <dest_repo>] <source_repo> <package_name> <product_repo>
[<release_package_name>]
-h: help
-o: Use latest scratch repositry instead of creating a new one
-d <dest_repo>: Use <dest_repo> as scratch repository
<source_repo>: [OBS:|IBS:]<reponame>
OBS/IBS are set appropriately.
Requires: osc
install-all
-----------
Install all RPM-files in a directory in a subdirectory install/.
install-all
Requires:
rpm, cpio
mount_export_install
--------------------
This script will look for ISOs in DISTROOT and check if they are
loop-mounted to a matching directories in MOUNTPOINT, if needed
it will create them.
It will export this directory for NFS for the network EXPORT and
register it as nfs mount point and ftp directory with SLP.
DISTROOT, MOUNTPOINT and EXPORT are environment variables set in
the script itself.
mount_export_install
Requires: coreutils, grep, util-linux, nfs-kernel-server,
coreutils, openslp
list_binary_packages
--------------------
Lists all binary packages (without version and extension) for
source packages listed in PACKAGE_LIST for all architectures
listed in ARCH_LIST for the repositories in REPO_LIST in
project PROJECT. The variables are set in a file whose name
is passed on the command line.
list_binary_packages <input_file>
Requires: osc
ls-bin
------
In an OBS package directory, list all binary packages built for
a given repository, arch and optionally flavor.
ls-bin -M flavor repository arch
Requires: grep, sed
uppatch.sh
----------
Utility to apply a set of patches to a git-ed source tree.
When needing to apply a set of 'git show' style patches, git am may
often not do the job as it is not tolerant to whitespace changes.
This script retries by using 'patch' in case 'git am' fails.
uppatch.sh [-h|-?|--help|filelist...]
Options:
-h|-?|--help: Usage
Either specify filelist or set environment variable "PATCHSET"
In an OBS package directory, after running rpm-git, apply a set of
patches (git show style).
Requires: git, patch, coreutils