Homemaker

Homemaker is a lightweight tool for straightforward and efficient management of *nix configuration files found in the user's home directory, commonly known as dot-files. It can also be readily used for general purpose system bootstrapping, including installing packages, cloning repositories, etc. This tool is written in Go, requires no installation, has no dependencies and makes use of simple configuration file structure inspired by make to generate symlinks and execute system commands to aid in configuring a new system for use.

Table of Contents

• Motivation
• Installation
• Configuration
  • Environment Variables
  • Command Macros
  • Task and Macro Variants
  • Conditional Execution
• Usage
• Sample

Motivation

Ever since switching to using Linux as my daily driver operating system, I have been searching for a way to effectively manage settings between different computers (and system reinstalls on the same machine) while avoiding the accumulation of cruft that plagues our home directories.

Specifically, I required a solution that had the following characteristics:

• Do not require software installation

  The user may not have root privileges on all the computers that she accesses, making it difficult to make use software that has additional dependencies. Even if the user has root on the machine in question, it is sub-optimal to require her to install software packages just to bootstrap the system.

• Support configuration variants

  The user should be able to use cherry-pick which settings they wish to use. Different system installs have different configuration requirements; it must be possible to share common settings while keeping the unique bits unique to each machine. Furthermore, it should be possible to store all configuration settings in one location.

• Make no assumptions about synchronization

  While Git often works well for managing dot-files, it is not ideal for every situation. I've seen some applications add and remove seemingly randomly-named files within their configuration directories, while others store their settings in large, opaque binary blobs. Managing configuration settings for such poorly-behaved applications may be easier through, Dropbox, rsync, or some other utility outside of version control.

• Be easy to read and modify the source

  As a programmer, I always consider the underlying technology of a tool when deciding whether or not to use it. The user should feel empowered to change and test the application without having to deal with the archaic incantations otherwise known as shell scripts. Scripting languages can work well but are closely tied to the environment in which they are executed.

It soon became apparent to me that utility which met all of my requirements for simply did not exist. After making do with a hastily hacked-together Python script for a couple of months, I decided that this problem deserved a clean, formal solution. I settled on building this new utility in Go because in addition to the language syntax being clear and easy to understand, executables built by the Go compiler are statically linked, making them highly portable. Just drop the binary on your system and you are ready! The result of my work is Homemaker; I hope that you find it suitable for your needs.

Installation

If you already have the Go environment and toolchain set up, you can get the latest version by running:

$ go get github.com/FooSoft/homemaker

Otherwise, you can use the pre-built binaries for the platforms below:

• homemaker_darwin_386.tar.gz
• homemaker_darwin_amd64.tar.gz
• homemaker_linux_386.tar.gz
• homemaker_linux_amd64.tar.gz
• homemaker_linux_arm.tar.gz
• homemaker_windows_386.tar.gz
• homemaker_windows_amd64.tar.gz

Configuration

Configuration files for Homemaker can be authored in your choice of TOML, JSON, or YAML markup languages. Being the easiest to read out of the three, TOML will be used for the example configuration files. Worry not if you are unfamiliar with this format; everything you need to know about it will be shown below.

Let's start by looking at a basic example configuration file,

example.toml

.toml/.tml

.yaml/.yml

.json

[tasks.default]
    links = [
        [".config/fish"],
        [".config/keepassx"],
        [".config/terminal"],
        [".config/vlc"],
        [".gitconfig"],
        [".xinputrc"],
    ]

We could have just as easily written this configuration in JSON (or YAML for that matter), but it's subjectively uglier:

{
    "tasks": {
        "default": {
            "links": [
                [".config/fish"],
                [".config/keepassx"],
                [".config/terminal"],
                [".config/vlc"],
                [".gitconfig"],
                [".xinputrc"]
            ]
        }
    }
}

To create symlinks based on the contents of the TOML file from before, we invoke the

homemaker

$ homemaker example.toml /mnt/data/config

To get a better idea of what

/mnt/data/config

Usage: homemaker [options] conf src
https://foosoft.net/projects/homemaker/

Parameters:
  -clobber
        delete files and directories at target
  -dest string
        target directory for tasks (default "/home/alex")
  -force
        create parent directories to target (default true)
  -nocmds
        don't execute commands
  -nolinks
        don't create links
  -task string
        name of task to execute (default "default")
  -unlink
        remove existing links instead of creating them
  -variant string
        execution variant for tasks and macros
  -verbose
        verbose output

For the purpose of our illustration,

src

/mnt/data/config

-dest

Another useful parameter is

task

default

default

-task=taskname

[tasks.flatline]
    links = [
        [".config/syncthing", ".config/syncthing_flatline"],
        [".s3cfg"],
        [".sabnzbd"],
        [".ssh", ".ssh_flatline"],
    ]

[tasks.wintermute]
    links = [
        [".config/syncthing", ".config/syncthing_wintermute"],
        [".ssh", ".ssh_wintermute"],
    ]

Here we see two tasks, named after the computers that will be using them,

flatline

wintermute

flatline

wintermute

.ssh

.ssh

.ssh_flatline

.ssh_wintermute

You may have noticed that each entry in the

links

Now that we have machine specific tasks defined in our configuration file, it would be nice to still be able to share configuration settings that are common to the two computers. We can do this by adding a

dep

[tasks.common]
    links = [
        [".config/fish"],
        [".config/keepassx"],
        [".config/terminal"],
        [".config/vlc"],
        [".gitconfig"],
        [".xinputrc"],
    ]

[tasks.flatline]
    deps = ["common"]
    links = [
        [".config/syncthing", ".config/syncthing_flatline"],
        [".s3cfg"],
        [".sabnzbd"],
        [".ssh", ".ssh_flatline"],
    ]
[tasks.wintermute]
    deps = ["common"]
    links = [
        [".config/syncthing", ".config/syncthing_wintermute"],
        [".ssh", ".ssh_wintermute"],
    ]

Homemaker will process the dependency tasks before processing the task itself.

Sometimes, just linking a config file is not enough, because the content of the configuration file needs to be adapted to the target and we do not want to maintain several different versions of the same file. For such use cases, Homemaker supports templates. The configuration syntax for templates is the same as for links.

[tasks.template]
    templates = [
        [".gitconfig"]
    ]

In the template file, the go templating syntax is used for the customization of the configuration file. With the

.Env

[user]
name = "John Doe"
{{if eq .Env.USER "john"}}
    email = "[email protected]"
{{else}}
    email = "[email protected]"
{{end}}

In addition to creating links and processing templates, Homemaker is capable of executing commands on a per-task basis. Commands should be defined in an array called

cmds

dest

cmdspre

cmdspost

The example task below will clone and install configuration files for Vim into the

~/.config

links

[tasks.vim]
    cmds = [
        ["rm", "-rf", ".config/vim"],
        ["git", "clone", "https://github.com/FooSoft/dotvim", ".config/vim"],
    ]
    links = [
        [".vimrc", "${HM_DEST}/.config/vim/.vimrc"],
        [".vim", "${HM_DEST}/.config/vim/.vim"],
    ]

Environment Variables

Homemaker supports the expansion of environment variables for both command and link blocks as well as for dependencies. This is a good way of avoiding having to hard code absolute paths into your configuration file. To reference an environment variable simply use

${ENVVAR}

$ENVVAR

ENVVAR

• HM_CONFIG

  Path to the homemaker configuration file.

• HM_TASK

  Task name invoked from the command line.

• HM_SRC

  Source directory for link creation.

• HM_DEST

  Destination directory for link creation.

• HM_VARIANT

  Variant used for task and macro execution.

Environment variables can also be set within tasks block by assigning them to the

envs

[tasks.default]
    envs = [
        ["MYENV1", "foo"],        # set MYENV1 to foo
        ["MYENV2", "foo", "bar"], # set MYENV2 to foo,bar
        ["MYENV3"],               # clear MYENV3
    ]

It should be pointed out that it is possible to reference other environment variables using the syntax shown in the first part of this section. This makes it possible to expand variables like

PATH

Command Macros

It is often convenient to execute certain commands repeatedly within task blocks to install packages, clone git repositories, etc. Homemaker provides macro blocks for this purpose; you can specify a command prefix and suffix that is used to wrap the parameters you provide. For example, you can declare a macro for

apt-get install

[macros.install]
    prefix = ["sudo", "apt-get", "install", "-y"]

Macros can be referenced from commands by prefixing the macro name with the

@

[tasks.python]
    cmds = [
        ["@install", "python-dev", "python-pip", "python3-pip"]
    ]

Macros can have dependencies just like tasks. The

git clone

[macros.clone]
    deps = ["git"]
    prefix = ["git", "clone"]

[tasks.git]
    cmds = [
        ["@install", "git"]
    ]

Macros help reduce the clutter that comes from the repeated commands which must be executed to bootstrap a new system. When executed with the

verbose

Task and Macro Variants

If you wish to use this tool in a truly cross-platform and cross-distribution manner without authoring multiple configuration files, you will have to provide information to Homemaker about the environment it is running in. Different operating systems and distributions use different package managers and package names; we solve this problem with task and macro variants.

For example, if you want to write a generic macro for installing packages that works on both Ubuntu and Arch Linux, you can define the following variants (Ubuntu uses the apt package manager and Arch Linux uses pacman).

[macros.install__ubuntu]
    prefix = ["sudo", "apt-get", "install"]

[macros.install__arch]
    prefix = ["sudo", "pacman", "-S"]

The double underscore characters signify that the following identifier is a variant decorator. In most cases, you only have to think about variants when you are writing task and macro definitions, not when using them. For example, to see how to use the

install

[tasks.tmux]
    cmds = [["@install", "tmux"]]

Notice that the package manager is conveniently abstracted by the

install

install

$ homemaker --variant=ubuntu example.toml /mnt/data/config

Tasks can be be decorated much like commands:

[tasks.vim__server]
    cmds = [["@install", "vim-nox"]]

[tasks.vim]
    cmds = [["@install", "gvim"]]

In the above example, we avoid installing

gvim

The command below will execute the

vim__server

$ homemaker --variant=server example.toml /mnt/data/config

Both of the commands below will execute the

vim

$ homemaker --variant=foobar example.toml /mnt/data/config
$ homemaker example.toml /mnt/data/config

If for some reason you wish to explicitly reference the base task from the decorated task, you can add a dependency that contains a variant override as shown in the somewhat contrived examples below:

[tasks.foo]
[tasks.foo__specific]
    deps = ["foo__"]         # executes foo and foo_specific

[tasks.bar__specific]
[tasks.bar]
    deps = ["bar__specific"] # executes bar_specific and bar

Although variants are somewhat of an advanced topic as far as Homemaker features are concerned, they can be used to provide some basic conditional functionality to your configuration file without significantly increasing complexity for the user.

Conditional Execution

Homemaker provides a facility for determining whether or not a given task should execute at runtime; this is accomplished with the

accepts

rejects

cmds

• accepts

  Execute commands non-interactively; any non-zero return code will cause the task to be skipped.

• rejects

  Execute commands non-interactively; any return code of zero code will cause the task to be skipped.

The intent of this feature is to allow tasks to "early out" when the work they carry out has already been completed. In the example below, we use the

which

which

[tasks.fish]
    rejects = [["which", "fish"]]
    cmds = [["@install", "fish"], ["chsh", "-s", "/usr/bin/fish"]]
    links = [[".config/fish/config.fish"]]

The

accepts

rejects

Usage

Executing Homemaker with the

-help

• clobber

  By default, Homemaker will only remove identically-named symlinks at the destination directory. Using this parameter will cause Homemaker to be more aggressive and delete clashing files and entire directories as well. This can be useful for getting rid of the default configuration settings some applications write when you run them for the first time, but should obviously be used with caution.

• dest

  This parameter specifies destination where Homemaker is to create symlinks. This will default to the home directory for the current user, and as long as you are just using this application to manage dot-files, will probably never need to be changed.

• force

  Sometimes dot-files for an application are nested within parent directories that must exist in order to allow the symlink to be successfully created (for example the

  .config

  directory in

  .config/vlc

  ). As this is the expected behavior, this parameter defaults to

  true

  ; however you can explicitly disable it if required. You can specify the access permissions of directories created by force by providing a third array item in the link descriptor. For example, if you wanted the

  .ssh

  directory to be created with mode

  700

  , you could write the following:

  [".ssh/id_rsa.pub", ".ssh_flatline/id_rsa.pub", "0700"]

  . Notice that you can specify permissions in octal notation by adding a leading zero value (the

  0x

  prefix signifies hexadecimal).

• nocmds

  Do not execute commands for the

  cmds

  blocks inside of tasks.

• nolinks

  Do not create links for the

  links

  blocks inside of tasks.

• task

  This parameter is used to specify which task Homemaker will process when executed. It defaults to the

  default

  task, which should be used when creating a configuration file that does not have system-specific tasks specified.

• unlink

  Sometimes it's useful to "uninstall" links previously created by Homemaker. When running with the

  unlink

  flag, the tool will delete the links created by the tasks provided. This flag automatically sets the

  nocmds

  flag as well, because it makes no sense to execute commands when performing an uninstall operation.

• variant

  When using homemaker across different operating systems or distributions it can be useful to be able to perform conditional command and task execution, allowing for variation in things like package names and package management tools. This parameter is used for specifying the name of the variant that should be used.

• verbose

  When something isn't going the way you expect, you can use this parameter to make Homemaker to log everything it is doing to console.

Sample

Below is a sample configuration file which should help to illustrate how Homemaker can be used in practice.

#
# macros
#

[macros.clone]
    deps = ["git"]
    prefix = ["git", "clone"]
[macros.install]
    prefix = ["sudo", "dnf", "install", "-y"]

development

[tasks.dev]
    deps = ["git", "vim", "node", "python", "golang"]
    cmds = [[
        "@install",
        "make",
        "automake",
        "gcc",
        "gcc-c++",
        "cmake",
        "the_silver_searcher",
        "meld",
        "ncurses-compat-libs",
    ]]
[tasks.git]
    cmds = [["@install", "git"]]
    links = [[".gitconfig"]]
[tasks.golang]
    envs = [["GOPATH", "${HM_DEST}/projects/go"]]
    cmds = [["mkdir", "-p", "$GOPATH"], ["@install", "golang"]]
[tasks.node]
    cmds = [["@install", "nodejs", "npm"]]
[tasks.python]
    cmds = [["@install", "python-devel", "python-pip"]]
[tasks.vim]
    deps = ["vimrc"]
    cmds = [["@install", "vim-X11", "vim-enhanced"]]
[tasks.vimrc]
    rejects = [["test", "-d", ".config/vim"]]
    cmds = [["@clone", "https://github.com/FooSoft/dotvim.git", ".config/vim"]]
    links = [
        [".vim", "$HM_DEST/.config/vim/.vim"],
        [".vimrc", "$HM_DEST/.config/vim/.vimrc"],
        [".eslintrc.json"],
    ]

general

[tasks.fusion]
    cmds = [["/home/alex/projects/dotfiles/bin/fusion.sh"]]
[tasks.virtualbox]
    cmds = [["@install", "VirtualBox"]]
[tasks.nvidia]
    deps = ["fusion"]
    cmds = [["@install", "akmod-nvidia"]]
[tasks.vlc]
    deps = ["fusion"]
    cmds = [["@install", "vlc"]]
[tasks.dropbox]
    deps = ["fusion"]
    cmds = [["@install", "dropbox"]]
[tasks.fish]
    rejects = [["which", "fish"]]
    cmds = [["@install", "fish"], ["chsh", "-s", "/usr/bin/fish"]]
    links = [[".config/fish/config.fish"]]
[tasks.common_term]
    cmds = [["@install", "openssh-server", "fzf", "htop", "p7zip", "unrar", "tmux", "whois", "rsync"]]
    links = [["bin"]]
[tasks.ibus]
    cmds = [["@install", "ibus", "ibus-anthy", "ibus-qt"]]
[tasks.default]
    deps = ["fusion", "common_term", "ibus", "vlc", "dropbox", "dev"]
    cmds = [["@install", "gimp", "keepassxc", "speedcrunch"]]
    links = [[".profile"]]