credit: xkcd http://xkcd.com/303/
Compiling takes a long time
- Maybe you are using
make
. - Maybe make is polling the filesystem.
- Maybe you have too many files and too many subdirectories.
I was recently reviewing some articles about build wars and trying to figure out what makes a good and efficient build system. The claim was that using gulp
you have streams
and you pipe them all together and the build becomes very efficient. That’s one aspect. Both grunt
and gulp
support a watch
method. Instant feedback is great to have to become more productivity and most sought after. In order to get instant feedback you need to be able to detect changes to your files and be able to perform the minimum amount of work that is required. Why clean objects that are still perfectly valid? Why run make clean
at all?
Nice and fancy, but are the tools that I use daily harnessing that power?
Django has runserver
At work, I use django
in some of the projects. Here is my favorite command:
./manage.py runserver
I have become fond of the autoreload
feature. If you look at the django.utils.autoreload
code, you will see that it tries to use inotify
if it is available.
inotify
from Wikipedia:
Inotify (inode notify) is a Linux kernel subsystem that acts to extend filesystems to notice changes to the filesystem, and report those changes to applications. It replaces an earlier facility, dnotify, which had similar goals.
In other words, rather than polling the filesystem to check what new changes came in, the kernel subsystem notifies you. If you have a daemon running for your compile job, it translates in some time spent walking a tree up-front; but a net effect that it does not need to keep doing that each time; later updates are handled more efficiently.
gulp.watch
With gulp
(a build streaming process - written in javascript) allows you to use gulp.watch
which monitors file changes and runs the callback functions you provide.
However, gulp.watch
is provided by vinyl-fs
which itself re-exports watch
from glob-watcher
, which in turns uses gaze
.
grunt-contrib-watch
With grunt
you can leverage the grunt-contrib-watch
plugin which allows you to monitor the file changes.
Just as gulp.watch
this also leverages gaze
.
gaze
https://github.com/shama/gaze:
A globbing fs.watch wrapper built from the best parts of other fine watch libs. Compatible with Node.js 0.10/0.8, Windows, OSX and Linux.
Here is a sample from the documentation:
Back to make
I wanted to search if make
was able to run as a daemon and use inotify
. This is what I found after a bit of googling and searching stackoverflow.
supplementing make
Possibilities are to use tools / libraries and teach make
about them. These are the contenders…
quote from the inotify-tools
github page:
[inotify-tools] is a package of some commandline utilities relating to inotify.
The general purpose of this package is to allow inotify’s features to be used from within shell scripts.
fswatch
quote from the fswatch
github page:
A cross-platform file change monitor with multiple backends: Apple OS X File System Events API, *BSD kqueue, Linux inotify and a stat-based backend.
make
alternatives using inotify
If make
does not support inotify
there are some known alternatives that do. It may be tedious to port the Makefiles though; which is why I would look into making make
better rather than rip-and-replace with a different solution.
tup
What is tup?
Tup is a file-based build system for Linux, OSX, and Windows. It inputs a list of file changes and a directed acyclic graph (DAG), then processes the DAG to execute the appropriate commands required to update dependent files. Updates are performed with very little overhead since tup implements powerful build algorithms to avoid doing unnecessary work. This means you can stay focused on your project rather than on your build system.
After previous discoveries, I tried to look up build systems that leverage inotify
; I found tup
. The website publishes a comparison between make
and tup
.