aconfmgr

by CyberShadow

CyberShadow / aconfmgr

A configuration manager for Arch Linux

517 Stars 15 Forks Last release: Not found 692 Commits 0 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

aconfmgr: A configuration manager for Arch Linux Build Status

aconfmgr
is a package to track, manage, and restore the configuration of an Arch Linux system. Its goals are:
  • Quickly configure a new system, or restore an existing system according to a saved configuration
  • Track temporary/undesired changes to the system's configuration
  • Identify obsolete packages and maintain a lean system

aconfmgr
tracks the list of installed packages (both native and external), as well as changes to configuration files (
/etc/
). Since the system configuration is described as shell scripts, it is trivially extensible.

screenshot

Table of Contents

Usage

Installation

Simply clone (or download+unzip) the GitHub repository.

aconfmgr
will install dependencies as needed during execution. An AUR package is also available.

First run

Run

aconfmgr save
to transcribe the system's configuration to the configuration directory. This will create the file
99-unsorted.sh
in the configuration directory, as well as other files describing the system configuration. (The configuration directory will usually be
~/.config/aconfmgr
, or
./config
if running directly from git, or it can be overridden with
-c
.)

On the first run,

aconfmgr
will likely include some files which you may not want to include in your system configuration. These can be temporary or auto-generated files which are not directly owned by a package. To prevent
aconfmgr
from including these files in the configuration, create e.g.
10-ignores.sh
in the configuration directory, with the lines e.g.
IgnorePath '/path/to/file.ext'
or
IgnorePath '/path/to/dir/*'
. (See ignoring files for details.) Delete everything from the configuration directory except that file and re-run
aconfmgr save
to regenerate a configuration minding these ignore rules.

Once

aconfmgr save
finishes, you should review the contents of
99-unsorted.sh
, and sort it into one or more new files (e.g.:
10-base.sh
,
20-drivers.sh
,
30-gui.sh
,
50-misc.sh
...). The files should have a
.sh
extension, and use
bash
syntax. I suggest adding a comment for each package describing why installing the package was needed, so it is clear when the package is no longer needed and can be removed.

During this process, you may identify packages or system changes which are no longer needed. Do not sort them into your configuration files - instead, delete the file

99-unsorted.sh
, and run
aconfmgr apply
. This will synchronize the system state against your configuration, thus removing the omitted packages. (You will be given a chance to confirm all changes before they are applied.)

Note: you don't need to run

aconfmgr
via
sudo
. It will elevate as necessary by invoking
sudo
itself.

Maintenance

The configuration directory should be versioned using a version control system (e.g. Git). Ideally, the file

99-unsorted.sh
should not be versioned - it will only be created when the current configuration does not reflect the current system state, therefore indicating that there are system changes that have not been accounted for.

Periodic maintenance consists of running

aconfmgr save
; if this results in uncommitted changes to the configuration directory, then there are unaccounted system changes. The changes should be reviewed, sorted, documented, committed and pushed.

Restoring

To restore a system to its earlier state, or to set up a new system, simply make sure the correct configuration is in the configuration directory and run

aconfmgr apply
. You will be able to preview and confirm any actual system changes.

Modus operandi

The

aconfmgr
script has two subcommands:
  • aconfmgr save
    calculates the difference between the current system's configuration and the configuration described by the configuration directory, and writes it back to the configuration directory.
  • aconfmgr apply
    applies the difference between the configuration described by the configuration directory and the current system's configuration, installing/removing packages and creating/editing configuration files.

The configuration directory contains shell scripts, initially generated by the

save
subcommand, and then usually edited by the user. Evaluating these scripts will compile a system configuration description in the
output
directory. The difference between that directory's contents, and the actual current system configuration, dictates the actions ultimately taken by
aconfmgr
.

aconfmgr save
will write the difference to the file
99-unsorted.sh
(under the configuration directory) as a series of shell commands which attempt to bring the configuration up to date with the current system. When starting with an empty configuration, this difference will consist of the entire system description. Since the script only appends to that file, it may end up undoing configuration changes done earlier in the scripts (e.g. removing packages from the package list). It is up to the user to refactor the configuration to remove redundancies, document changes, and improve maintainability.

aconfmgr apply
will apply the differences to the actual system.

The contracts of both commands are that they are mutually idempotent: after a successful invocation of either, invoking either command immediately after will be a no-op.

Packages

Background: On Arch Linux, every installed package is installed either explicitly, or as a dependency for another package. Packages can also have mandatory (hard) or optional dependencies. You can view this information using

pacman -Qi 
("Install Reason", "Depends On", "Optional Deps").

aconfmgr
only tracks explicitly-installed packages, ignoring their hard dependencies. Therefore:
  • aconfmgr save
    will only save installed packages that are marked as explicitly installed.
  • Installed packages that are neither explicitly installed, nor are hard dependencies of other installed packages, are considered prunable orphans and will be removed.
  • Packages that are only optional dependencies of other packages must be listed explicitly, otherwise they will be pruned.
  • aconfmgr apply
    removes unlisted packages by unpinning them (setting their install reason as "installed as a dependency"), after which it prunes all orphan packages. If the package is still required by another package, it will remain on the system (until it is no longer required); otherwise, it is removed.
  • Packages that are installed and explicitly listed in the configuration will have their install reason set to "explicitly installed".

Advanced Usage

Configuration syntax

The configuration files use

bash
syntax. The easiest way to learn the syntax is to run
aconfmgr save
and examine its output (
99-unsorted.sh
).

Some simple helper functions are defined in

src/helpers.bash
(sourced automatically). You are encouraged to examine their implementation - their main goal is not so much to provide an API as simply to make the generated configuration terser and more readable. As such, their use is in no way required, and they can be substituted with their underlying implementations.

The list of provided helper functions:

  • AddPackage [--foreign] PACKAGE...
    - Adds a package to the list of packages to be installed.
  • RemovePackage [--foreign] PACKAGE...
    - Removes an earlier-added package to the list of packages to be installed.
  • IgnorePackage [--foreign] PACKAGE...
    - Adds a package to the list of packages to be ignored.
  • AddPackageGroup GROUP
    - Adds all packages currently in the given group to the list of packages to be installed.
  • CopyFile PATH [MODE [OWNER [GROUP]]]
    - Copies a file from the
    files
    subdirectory to the output.
  • CopyFileTo SRC-PATH DST-PATH [MODE [OWNER [GROUP]]]
    - As above, but allows source and output paths to vary.
  • CreateFile PATH [MODE [OWNER [GROUP]]]
    - Creates an empty file, to be included in the output. Prints its absolute path to standard output.
  • GetPackageOriginalFile PACKAGE PATH
    - Extracts the original file from a package's archive for inclusion in the output. Prints its absolute path to standard output.
  • CreateLink PATH TARGET [OWNER [GROUP]]
    - Creates a symbolic link with the specified target.
  • RemoveFile PATH
    - Removes an earlier-added file.
  • SetFileProperty PATH TYPE VALUE
    - Sets a file property.
  • IgnorePath PATTERN
    - Adds the specified pattern to the list of ignored paths.
  • AddFileContentFilter PATTERN FUNCTION
    - Adds the specified rule to the list of file content filters.

You are free to define your own custom helper functions as part of your configuration, e.g. by placing them in a

00-helpers.sh
file.

Some examples can be found on the project wiki.

Ignoring some changes

Ignoring files

Some files will inevitably neither belong to or match any installed packages, nor can be considered part of the system configuration. This can include:

  • Temporary / cache / auto-generated / lock / pipe / pid / timestamp / database / backup / log files
  • Files managed by third-party package managers, esp. programming languages' package managers (pip, gem, npm)
  • Virtual machine disk images

Other files may not be desirable to include in the managed system configuration because they are security-sensitive (e.g. sshd private keys).

To declare a group of files to be ignored by

aconfmgr
, you can use the provided
IgnorePath
function, e.g.:
IgnorePath '/var/lib/pacman/local/*' # package metadata
IgnorePath '/var/lib/pacman/sync/*.db' # repos
IgnorePath '/var/lib/pacman/sync/*.db.sig' # repo sigs

Make sure to quote the pattern argument to prevent globbing at configuration parse time.

Ignoring parts of files

If only some parts of a file are relevant to the system configuration, and the rest can be safely omitted or normalized to some default value, a filter rule can be configured to normalize how the contents of a file appears to aconfmgr. For example, this is applicable if a file contains a timestamp or some randomly-generated or frequently-changing value, which can be safely discarded.

To configure a file content rule, first define a function which specifies how to filter the file. The function is called with the file name as the only parameter, the file contents on its stdin, and is expected to provide the filtered contents on its stdout:

function NetworkManagerConnectionFilter() {
    grep -v '^timestamp='
}

Then, register the rule by calling the

AddFileContentFilter
helper function:
AddFileContentFilter '/etc/NetworkManager/system-connections/*.nmconnection' NetworkManagerConnectionFilter

Only one function may be configured per unique pattern. The most recently added rule takes precedence over any rules with matching patterns preceding it.

Ignoring packages

To ignore the presence of some packages on the system, you can use the

IgnorePackage
function:
IgnorePackage linux-git

aconfmgr save
will not update the configuration based on ignored packages' presence or absence, and
aconfmgr apply
will not install or uninstall them. The packages should also not be present in the configuration's package list, of course. To ignore a foreign package (e.g. a non-AUR foreign package), use the
--foreign
switch (e.g.
IgnorePackage --foreign my-flobulator
).

Inlining files

In the output generated by

aconfmgr save
, non-empty new or modified files are copied in their entirety to the configuration directory, under the
/files/
subdirectory, accompanied by a
Copyfile
invocation. However, this can sometimes be wasteful or unwieldy if the file consists of a single line, or has one line changed out of a thousand. This section describes some techniques to move the edits into the configuration files entirely.

Inlining file content entirely

Instead of copying, the file's can be inlined using a

bash
heredoc, or even an
echo
:
# Enable Magic SysRq
echo "kernel.sysrq = 1" > "$(CreateFile /etc/sysctl.d/99-sysrq.conf)"

https://wiki.archlinux.org/index.php/Getty#Have_boot_messages_stay_on_tty1

cat > "$(CreateFile /etc/systemd/system/[email protected]/noclear.conf)" <

Inlining file edits

It is also possible to generate an output file that is a modification of the original. For this purpose, the helper function

GetPackageOriginalFile
is provided. The function will extract the indicated file from the archive in pacman's package cache, downloading it first if necessary.
# Append some options to systemd's system.conf
cat >> "$(GetPackageOriginalFile systemd /etc/systemd/system.conf)" <

The Augeas package can assist in editing configuration files:

AconfNeedProgram augtool augeas n # Install augeas now, if needed
aug() { augtool --root="$output_dir/files" "[email protected]" ; }

DNS

GetPackageOriginalFile filesystem /etc/resolv.conf > /dev/null aug set '/files/etc/resolv.conf/nameserver[101]' 127.0.0.1 # dnsmasq aug set '/files/etc/resolv.conf/nameserver[102]' 8.8.8.8 # Google aug set '/files/etc/resolv.conf/nameserver[103]' 8.8.4.4 # Google

Managing multiple systems

You can use the same configuration repository to manage multiple sufficiently-similar systems. One way of doing so is e.g. Git branches (having one main branch plus one branch per machine, and periodically merge in changes from the main branch into the machine-specific branches); however, it is simpler to use shell scripting:

AddPackage coreutils
# ... more common packages ...

if [[ "$HOSTNAME" == "home.example.com" ]] then AddPackage nvidia AddPackage nvidia-utils # ... more packages only for the home system ... fi

To manage different versions of a file for different systems, you can inline the file, or copy a different version depending on the hostname.

An example for inlining

/etc/hosts
:
f="$(GetPackageOriginalFile filesystem /etc/hosts)"
echo '1.2.3.4 home.example.net' >> "$f"
if [[ "$HOSTNAME" == laptop.example.net ]]
then
    echo '127.0.1.1 laptop.example.net laptop' >> "$f"
fi

If the file is binary, or too large to make inlining practical,

CopyFileTo
can be used:
CopyFileTo "/etc/hosts-$HOSTNAME" "/etc/hosts"

This will use

files/etc/hosts-home.example.net
for the
home.example.net
machine, and so on.

Comparisons

aconfmgr vs. Puppet/Ansible

Although

aconfmgr
calls itself a configuration manager, it has a number of core distinctions from the more well-known ones. One big distinction is that
aconfmgr
manages only one system - the one it is running on. It does not depend on any background services or network agents to work.

Another big distinction is the scope, and the direction of the flow of information. To clarify:

  • Puppet and Ansible have limited discovery abilities, whereas

    aconfmgr
    attempts to discover and save the system configuration in its entirety. There is no Puppet/Ansible command that can be run to save a running system's configuration to a file which, when executed, will produce a similarly-configured system; however, this is the goal of
    aconfmgr save
    .
  • Puppet and Ansible manage the system's configuration insofar as it is defined by the configuration file, whereas

    aconfmgr
    manages the entire system; as such, the absence of an item in the configuration file indicates its absence on the system. This is not true for Puppet or Ansible, where, to remove a package, one must first push a configuration file that explicitly indicates that the package is to be removed, only after which can all mentions of the package be removed from the configuration file.

aconfmgr vs. NixOS

There are some similarities between the NixOS configuration file and

aconfmgr
: they both attempt to describe the entire system configuration from a text file, with any changes in the configuration reflecting on the system. However, while
NixOS
forbids directly editing files under its control,
aconfmgr
doesn't. As with Puppet/Ansible,
aconfmgr
differs in that it provides a mechanism to transcribe changes in system state back to the configuration, making it idempotent.

Another difference is that

NixOS
provides a specialized syntax for many common configuration settings of managed software (e.g. allowing syntax such as
boot.loader.grub.device = "/dev/sda";
).
aconfmgr
doesn't provide this directly, though this can be achieved to some extent using tools such as Augeas, without also sacrificing the extent of possible configuration to only the predefined set of available options.

aconfmgr vs. lostfiles

lostfiles is a "simple script that identifies files not owned and not created by any Arch Linux package".

aconfmgr
provides a superset of its functionality, not least the ability to save exclusions to a configuration file.

aconfmgr vs. etckeeper

etckeeper allows storing

/etc
in a version control system.
aconfmgr
allows this as well, although it does not directly provide a way to automatically merge configuration files with upstream package versions. This can be done manually, by inlining file changes.

Limitations

  • Dependencies where more than one package provides something (e.g.
    fcron
    and
    cronie
    provide
    cron
    ) are not tracked, and the desired dependency must be pinned or added to the configuration manually.
  • Installing AUR packages that depend on virtual packages (such as
    java-environment
    ) is not currently implemented. The desired dependency can be manually specified in the configuration, or a supported AUR helper can be used instead.

License

Copyright (c) 2016-2018 aconfmgr authors.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.