Over the last couple of days, I published a survey targeted at people who manage dotfiles (ie. their configuration files for various software), asking for how they manage their dotfiles, and their thoughts about it.
I ended up receiving about 230 responses, which was pretty unexpected! Thanks to everyone who took the survey! :)
The full survey results can be found here on Google Forms.
For PRT455 (Software Engineering Practice), a 4th year uni subject, I needed to pick a software project to build that involves software engineering best practices. You know: regression testing, iterative development, TDD, CI, and requirements engineering. I chose dotfiles management software as my topic. The reason for this is that it has a nice scope - simple to implement a basic one, but plenty of room for fancy features, and every program I've tried so far has been lacking in some area. Bash scripts are too finicky, Ansible too complex, etc. etc.
I personally love configuring dotfiles, and have tried several management programs, including custom bash scripts, Dotbot, Ansible, SaltStack, and currently rcm.
This survey was part of requirements engineering. I wanted to obtain a general idea from the community on the current state of dotfiles management in the real world.
The ultimate aim of this project is to build a dotfiles management program based on initial requirements, requirements elicited from this survey, and further requirements that come up during the iterative development process.
I'll just go through each question in the order they were presented in the survey. Mostly presenting results, with analysis and comments as well. Probably doesn't follow good practice, but at least it's been helpful to me with requirements analysis.
Which dotfiles management software do you currently use?
And... voilà a pie chart.
Sorry, google charts wouldn't let me sort pie pieces.
Some interesting things to note:
- A huge number entered None as the software. I think for most cases this means they manage dotfiles, but do management (like symlinking) by hand.
- Self developed scripts and Stow made up a majority of the software used (54.5% together). This interested me, because I'd hardly heard of Stow being used until now.
- There is a large range of unique programs used - about 28 in this survey.
The full list of unique different software from responses is below:
- Borg Backup
- custom python script with gnu stow
- git with an alias : config='/usr/bin/git --git-dir=/home/x/.myconf --work-tree=/home/x'
- GNU Stow
- Manually made symlinks
- plain mercurial repo
- Rsync ml (not sure if typo - can't find it)
- self developed scripts
- Super User Spark
If you selected None to the previous question, what would convince you to start using such software?
I added this question towards the end of the survey upon seeing so many 'None' answers, so didn't end up with many responses (about 8).
Out of these answers, all of them seemed pretty happy with their current setup. Responses included:
- "Not much, really. I have no need for it. Setting up a new machine takes me less than a minute already"
- "... nothing, really"
Some comments showed an interest in using management software, if they had some time or the entry point was easy:
- "A basic walk through"
- "having some time to try them out"
It would seem that any new management software would have to have a low learning curve and good documentation!
Which method do you use for tracking changes or syncing your dotfiles?
The responses to this showed that Git was by far the most popular method, comprising 75% of respondents. After that was 14.7% not using any form of syncing or change tracking. The remaining 10% used a range of methods, including cloud syncing (eg. Dropbox), rsync, or Mercurial.
Some dotfile managers work directly with git repositories, but due to about 25% not using git, it probably should support a simple directory of files as well.
On how many computers (virtual or physical) do you currently have your dotfiles?
This shows quite clearly that a majority of users who manage configuration files manage them on multiple computers. In fact, over 61% of respondents manage dotfiles over 3 or more computers.
Now this possibly doesn't mean that these are the same dotfile setups on each computer each respondent has - but we can draw the following:
- an overwhelming majority of people in this demographic need to work with dotfiles on several computers
- thus it would suggest that syncing between computers and host-specific configuration features would be important in a dotfiles management workflow.
What OS's do you have your dotfiles installed on?
Unsurprisingly for this demographic, 99% of respondents use a GNU/Linux distro. However, many manage dotfiles on multiple OS's. 19% OSX, 11% Windows, 8.7% BSD, and the rest are either outliers or very opinionated. :P
Thus a requirement for software would be that it be highly cross platform - Linux, OSX, Windows, and BSD at least.
What is your favourite feature of your chosen dotfiles management software?
This and further questions allowed optional text input responses. Note that these are comments about the specific software that the respondent is using - I haven't grouped them by software in analysis, since I'm wanting to get a general idea about what requirements are important or not in general.
For this question, the most common response was a one word answer: "simplicity". See word cloud below:
word cloud created with WordItOut
Other things respondents liked:
- ease of use
- should 'just work'
- minimal dependencies
- version control
- isolate files per host
- declarative configuration
What annoys you most about your selected dotfiles management software?
Continuing the trend that most users are generally happy with their setup, the most common response here was "nothing". About 16 out of the 93 responses (17%) to this question were not annoyed with their software at all.
Of the others that did have problems, annoyances included:
- needing to manually sync changes
- differences between software use on different OS's (eg. requiring sudo on linux)
- having to maintain scripts themselves
- difficult to remove symlinks set up by a script
- no packages in distro repos for software in use
- lack of features
- can't force overwriting symlinks
- lot of work required
- complexity (the opposite of the simplicity that users liked!)
- requiring extra dependencies such as python
- difficult to merge changes
- bootstrapping process
- not well tested
Note that some of these (this and previous question), such as syncing and merging changes, probably relate more to the repository in which the dotfiles are stored, rather than the management software itself.
What feature doesn't your software have, but you most want it to have?
Once again, many were content or couldn't think of anything. Some suggestions were:
- per-machine branches (relating to host-specific config)
- compression of archives
- parallel processes for installations - eg. run as fast as possible
- templating (eg. for host-specific config, api keys, passwords)
- easy bootstrapping
- setting file permissions
- revision info per-file (note: if using git or another vcs, this is probably built in - could be handy though to include something in the management software to make this more accessible)
- alert to commit/push/sync changes
- self sustained - everything in the repo for bootstrapping (relates to request for easier bootstrapping)
- ability to install system software as well
- encryption support
- "Built in config directory presents for common apps. Once I install Thunderbird I'd like the app to prompt me for adding it's account config to the list of tracked locations" - really interesting idea! I'm sure this could be something to consider looking into.
- auto sync changes
How do you manage configuration differences between machines?
Here, the most popular response was git branches. Other methods included:
- loading config from local (unsynced) config files in managed files. (eg.
- templates with conditionals
- check for the hostname in the configuration files (note: this would only work in config files that allowed for this - eg. json configuration could not)
- host specific support in the management software (eg. placing files in a
host-HOSTNAMEdirectory in the repo)
- taking care to not require different configuration between machines
Do you use any helper programs (such as mir.qualia for host-specific config)?
A majority do not. (53/64 responses)
The few that did, used programs including mir.qualia (host-specific config), pystache (templating), and own scripts.
Are there any features you find useful and would particularly like to mention?
This is sort of a follow on to the "favourite feature" question to see if there might be other features useful but not necessarily favourite. It didn't produce many responses, but out of the few it's worth mentioning:
- enabling specific items from the dotfiles repo (ie. installing a subset of dotfiles)
- ncurses-like user interface
- software being widely available so it's always easy to find/install
- repo file structure doesn't have to reflect structure of deployed files
If you store your dotfiles in a public repo, would you like to share the link below?
Several respondents were happy to share, and are presented below along with the dotfiles management software used (at the time of writing).
Thanks for sharing! Hopefully this will provide some inspiration to readers! :)
Now having analysed the results, it looks like I have some work to do! Namely, begin choosing a programming language (probably rust at this stage), planning an API or UI, and deciding on how it should work (eg. what liked features from the survey should be included and disliked features should be improved on).
If you have read this far, thanks for reading, and I hope it was interesting!
Stay tuned, maybe sometime there will be yet another dotfiles management program to try out. :)