This is a somewhat limited approximation of dirdeps for gmake. It started out as a thought exercise, since I'd previously asserted that gmake could not do it. Well it can - not all of it certainly, but enough to be quite useful.
We cannot handle all the bells and whistles that bmake can, but we can still manage the basics to handle orchestrating a large build correctly.
Apart from efficiency, a key advantage of using DIRDEPS is that all the makefiles involved in orchestrating the build are easy to read and understand. This is rarely true of top-level makefiles. (Ok it is also not true of the full blown dirdeps but that can be considered a black box, and it is not just for top-level builds).
Along the way, I also had to implement sys.gmk (first makefile read by each instance of gmake) and a number of other useful analogs for the centralized makefiles commonly used with bmake.
The concept is quite simple. We locate a Makefile.depend* (I use GNUmakefile.depend* for gmake) in each directory to be made.
Each such makefile sets DIRDEPS to a list of paths relative to SRCTOP that need to be built before that directory. A DIRDEP might be qualified with a suffix to describe the TARGET_SPEC for which it should be built. An unqualified DIRDEP is built with the TARGET_SPEC specified when reading the makefile or the current one. The makefile then includes dirdeps.gmk to do the work.
This happens recursively and in the process we build a tree dependency graph which we hand over to gmake to build in the optimal order, in parallel with no race conditions - provided DIRDEPS are accurate. The model works just as well regardless of where you start your build from - so top-level and leaf builds all work the same way, with all the same benefits.
In the bmake version, a Makefile.depend file looks like:
# Autogenerated - do NOT edit!
DIRDEPS = \
tools/thing.host \
lib/sjg \
.include <dirdeps.mk>
and we can have a kernel module track the files read and executed etc to build each target - kept in a .meta file for each target which alone greatly adds to the reliability of update builds, and the syscall traces can be leveraged to auto-update the above and keep it accurate.
For gmake it will be a bit more crude. For a start; we'll assume that all GNUmakefile.depend* are manually maintained, and of course we have nothing like meta mode.
There is a vpath PATTERN DIRLIST directive, which lets us do:
vpath %.gmk ${GMAKESYSPATH}
so that:
include dirdeps.gmk
should just work, and at least with recent (4.4) gmake, it does. For older versions we might need to spell it out:
include ${GMKSYSDIR}/dirdeps.gmk
where GMKSYSDIR is either set in the environment or by sys.gmk (see below) which we can cause to be included before each makefile by setting something like MAKEFILES=${SRCTOP}/gmk/sys.gmk in the environment. The Gmake Manual frowns on this usage - but this is a good use for it.
Since I tend to use a wrapper around make for conditioning the environment, that's simple.
We could also run gmake with -I ${GMKSYSDIR} which is easy enough with the mk wrapper, but that's a bit ugly for anyone not using it.
Below I will assume a recent version of gmake and will use mk to invoke it.
With bmake we take advantage of a number of generic makefiles (*.mk) traditionally found in /usr/share/mk/ on BSD systems, but in a src tree they might be in share/mk/ or even mk/.
I've done equivalents for gmake as *.gmk.
The first makefile read by bmake is sys.mk and it (and anything it includes) can pre-condition the build environment as needed. We want the same functionality here.
As noted above this is handled by putting MAKEFILES=${SRCTOP}/gmk/sys.gmk into the environment before invoking gmake.
Since I've now also implemented a useful options.gmk we could make sys.gmk more generic, but as is, it is geared towards the DIRDEPS build setup.
I dislike dribbling generated files all over the src tree, so keeping a clean separation of SRCTOP and OBJTOP is a given:
# to be safe, we will be explicit here
-include ${GMKSYSDIR}/local.sys.gmk
# we need this in a variable
MAKEFILE_PREFERENCE ?= GNUmakefile makefile Makefile
ifndef sys_gmk0
# these do not change
SRCTOP ?= ${GMKSYSDIR}/..
OBJROOT ?= ${SRCTOP}/../obj
SRCTOP := $(abspath ${SRCTOP})
OBJROOT := $(abspath ${OBJROOT})
export SRCTOP OBJROOT
endif
The default TARGET_SPEC_VARS is just MACHINE.
If more is needed then, local.sys.gmk can set TARGET_SPEC_VARS, TARGET_OBJ_SPEC_VARS and TARGET_SPEC_VARS_x as needed.
For each instance of gmake, sys.gmk will need to decompose TARGET_SPEC to set the individual TARGET_SPEC_VARS correctly. The below does the trick:
define set-target-spec-var
$(eval k=$(word $1,${TARGET_SPEC_VARS}))
$(eval v=$(word $1,$2))
$(if $k, $(if $v,$(eval $k := $v)))
endef
...
ifneq "${TARGET_SPEC}" ""
# decompose ${TARGET_SPEC} to set the individual values
_tspec := $(subst ${comma}, ,${TARGET_SPEC})
TARGET_SPEC_VARS_x ?= 1 2 3 4
$(foreach i,${TARGET_SPEC_VARS_x},$(call set-target-spec-var,$i,${_tspec}))
endif
Apart from not being able to convert the number of words in TARGET_SPEC_VARS into a range of indices (we could resort to $(shell)) it's not much different than what we do with bmake.
If local.sys.gmk does not set TARGET_OBJ_SPEC_VARS, we likely have to resort to a $(shell) invocation to reverse the order of TARGET_SPEC_VARS but we only do that if TARGET_OBJ_SPEC_SEP is /:
TARGET_SPEC_VARS ?= MACHINE
ifneq "$(words ${TARGET_SPEC_VARS})" "1"
...
TARGET_OBJ_SPEC_SEP ?= .
ifeq "${TARGET_OBJ_SPEC_VARS}" ""
ifeq "${TARGET_OBJ_SPEC_SEP}" "/"
# for this case reversing order of TARGET_SPEC_VARS is better
TARGET_OBJ_SPEC_VARS = $(shell x= ; for w in ${TARGET_SPEC_VARS}; do x="$$w $$x"; done; echo $$x)
export TARGET_OBJ_SPEC_VARS
endif
endif
TARGET_OBJ_SPEC_VARS ?= ${TARGET_SPEC_VARS}
TARGET_OBJ_SPEC ?= $(subst ${space},${TARGET_OBJ_SPEC_SEP},$(foreach v,${TARGET_OBJ_SPEC_VARS},${$v}))
Since gmake does not automatically handle this, we have to do it manually by noting the value of CURDIR. If it is under ${OBJTOP} then we are good otherwise we need to adjust things (obj.gmk will do that if needed):
ifdef HOST_TARGET
ifeq "${MACHINE}" "host"
OBJTOP ?= ${OBJROOT}/${HOST_TARGET}
endif
ifeq "${MACHINE}" "host32"
OBJTOP ?= ${OBJROOT}/${HOST_TARGET32}
endif
endif
TARGET_OBJ_SPEC ?= ${MACHINE}
OBJTOP ?= ${OBJROOT}/${TARGET_OBJ_SPEC}
# deduce _CURDIR _OBJDIR and RELDIR from ${CURDIR}
ifeq "${CURDIR}" "${SRCTOP}"
RELDIR = .
_CURDIR := ${CURDIR}
_OBJDIR := ${OBJTOP}
else
ifeq "${OBJTOP}" "${CURDIR}"
RELDIR = .
_CURDIR := ${SRCTOP}
_OBJDIR := ${CURDIR}
else
# are we under ${OBJTOP}/ ?
RELDIR := $(subst ${OBJTOP}/,,${CURDIR})
ifneq "${RELDIR}" "${CURDIR}"
_CURDIR := ${SRCTOP}/${RELDIR}
_OBJDIR := ${CURDIR}
else
# are we under ${SRCTOP}/ ?
RELDIR := $(subst ${SRCTOP}/,,${CURDIR})
ifneq "${RELDIR}" "${CURDIR}"
_CURDIR := ${CURDIR}
_OBJDIR = ${OBJTOP}/${RELDIR}
else
# we have no clue where we are!
$(warning ${CURDIR} is not in/under ${SRCTOP}/ or ${OBJTOP}/)
undefine RELDIR
undefine _CURDIR
undefine _OBJDIR
endif
endif
endif
endif
This is the one that makes the build work. It is included by each GNUmakefile.depend*
Having constructed the fully qualified list of dirdeps for the current directory eg.:
${SRCTOP}/${RELDIR}.${TARGET_SPEC}
We can turn that into a list of qualified depend files to try and include:
${SRCTOP}/${RELDIR}/${DEPENDFILE_PREFIX}.${TARGET_SPEC}
We use the following to do the actual including of the next depend file while setting DEP_RELDIR and DEP_TARGET_SPEC appropriately. We want a qualified depend file if it exists, otherwise an unqualified one should do:
define include-depend
$(eval DEP_TARGET_SPEC=$(subst .,,$(suffix $1)))
$(eval DEP_RELDIR=$(patsubst %/,%,$(subst ${SRCTOP}/,,$(dir $1))))
$(if ${DEBUG_DIRDEPS},$(info Looking for $1 DEP_TARGET_SPEC=${DEP_TARGET_SPEC} DEP_RELDIR=${DEP_RELDIR}),)
$(eval -include $(shell test -s $1 && echo $1 || echo $(basename $1)))
endef
invoked via:
$(eval $(foreach dep,${_more_depends},$(call include-depend,${dep})))
That's actually not too shabby.
Gmake does not appear to provide a means of knowing if a target has been defined, so we need to do that ourselves, otherwise we get lots of warnings about duplicate targets. The following does the trick:
# we have to do target tracking ourselves to avoid complaints
define if-new-target
$(eval t=_${1}_target)
$(if ${$t},,$1)
$(eval $t=1)
endef
invoked thus:
# add dependencies to the graph
${_this_dirdep}: ${_dirdeps}
# now which ones have not been made targets yet?
_newdeps := $(sort $(foreach dep,${_dirdeps},$(call if-new-target,${dep})))
ifneq "${_newdeps}" ""
${_newdeps}:
$(call build-dirdep)
endif
We want to be able to:
mk -j8 -f dirdeps.gmk some/dir.host other/dir.i386
which should behave exactly as if we had a GNUmakefile.depend file containing:
DIRDEPS= \
some/dir.host \
other/dir.i386 \
include dirdeps.gmk
Unfortunately gmake's $(filter) does not seem able to match a pattern like %/% so we have to resort to a shell script:
# (shell is needed because filter cannot do %/%)
_dirdeps := $(sort $(shell for f in ${MAKECMDGOALS}; do echo "$$f"; done | grep /))
and we had to use grep since case "$$f" in */*) echo $$f;; esac caused a parsing error - the ) closed the $(shell.
Inefficient, but it's a one-off and it works.
After computing the tree dependencies as described above, dirdeps.gmk handles visiting them in the correct order. A dirdep in which no makefile from ${MAKEFILE_PREFERENCE} can be found, will be ignored. This makes handling optional directories, sparse trees, and just refactoring fallout all simple.
If OBJTOP is defined, dirdeps.gmk will use build-dirdep-obj which will auto-create an obj dir and run gmake in that directory with VPATH set to the correct src dir and with the makefile we found. Otherwise we just visit each the src dir using build-dirdep.
In either case TARGET_SPEC will be set to the suffix of the dirdep and MACHINE will be set to the first word in TARGET_SPEC. As noted above sys.gmk will deal with TARGET_SPEC.
This makefile handles the logic normally found in a top-level makefile to decide what to build.
For any given target dirdeps-targets.gmk looks for a directory of that name under ${SRCTOP}/targets/ and anything else in DIRDEPS_TARGETS_DIRS, that contains a GNUmakefile.depend* file.
Given the tree from our dirdeps example below and:
$ cat targets/tests/GNUmakefile.depend DIRDEPS= tests/prog include dirdeps.gmk
and a top level makefile:
$ cat GNUmakefile
all:
@echo Finished ${DIRDEPS_TARGETS}
include dirdeps-targets.gmk
is all we need to turn tests into a top-level target:
$ mk tests Checking /homes/sjg/work/gmk/tests/lib/b for host ... gmake[1]: Entering directory '/homes/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b' Building /homes/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b/libb.a Finished making tests/lib/b for host .. .. gmake[1]: Entering directory '/homes/sjg/work/gmk/obj/amd64/tests/prog' Building /homes/sjg/work/gmk/obj/amd64/tests/prog/prog Finished making tests/prog for amd64 gmake[1]: Leaving directory '/homes/sjg/work/gmk/obj/amd64/tests/prog' Checking /homes/sjg/work/gmk for amd64 ... gmake[1]: Entering directory '/homes/sjg/work/gmk/obj/amd64' Finished targets/tests gmake[1]: Leaving directory '/homes/sjg/work/gmk/obj/amd64' $
Another cool feature missing in gmake is the ability to automate control of the directory where files will be generated. Granted; this can be a source of confusion to those unfamiliar with the concept.
As noted above; dirdeps.gmk compensates by doing the auto-obj task itself when building each dirdep.
This makefile will do the same thing - for when dirdeps.mk is not involved, for example when we bypass MAKELEVEL=0:
mk --machine 1,amd64 -C tests/prog
will start gmake in tests/prog with MAKELEVEL=1 and MACHINE=amd64. We want it to use the same _OBJDIR that dirdeps.gmk would have.
We rely on sys.gmk having set _CURDIR and _OBJDIR. If CURDIR is the same as _CURDIR then we are not in our desired _OBJDIR. We do a subset of what dirdeps.gmk does - setting VPATH=${_CURDIR}, and running MAKE in ${_OBJDIR} with the correct makefile.
At MAKELEVEL > 0, init.gmk will include obj.gmk if ${MK_AUTO_OBJ} is "yes". It gets set by options.gmk.
We set _SKIP_BUILD=obj to let {init,lib,prog}.gmk know they should do nothing else, since the real work is already being done via a sub-make.
This makefile is included by both lib.gmk and prog.gmk to deal with common logic. It will include local.init.gmk, ${_CURDIR}/../GNUmakefile.inc and options.gmk.
As noted at MAKELEVEL > 0 it will include obj.gmk if needed.
If _SKIP_BUILD is not defined it also sets up the basic targets:
ifndef _SKIP_BUILD all: afterbuild afterbuild: realbuild realbuild: beforebuild beforebuild: endif
and defaults for CC, CXX etc. and rules to use them:
CC ?= cc
CXX ?= c++
AR ?= ar
LD ?= ld
COMPILE.c = ${CC} -c -o $@ ${CPPFLAGS} ${CFLAGS}
COMPILE.cxx = ${CXX} -c -o $@ ${CPPFLAGS} ${CXXFLAGS}
LINK.c = ${CC} -o $@ ${CLDFLAGS} ${LDADD} ${LDADD_LAST}
.c.o:
${COMPILE.c} $<
.cc.o .cxx.o:
${COMPILE.cxx} $<
srcs2objs = $(foreach e,.c .cc .cxx .cpp,$(subst $e,.o,$(filter %$e,${SRCS})))
Rules for building libraries.
Rules for building non-libraries.
This makefile is a subset of the bmake options.mk. For each option (op) listed in OPTIONS_DEFAULT_NO we set MK_${op} if not already set. It uses:
# $(call set-option,op,default)
define set-option
$(call debug-options,op=$1 default=$2)
$(if ${${OPTION_PREFIX}$1}, , \
$(if $(or ${NO_$1},${NO$1},${WITHOUT_$1}), \
$(eval ${OPTION_PREFIX}${1}=no), \
$(if ${WITH_$1}, $(eval ${OPTION_PREFIX}${1}=yes), \
$(eval ${OPTION_PREFIX}${1}=${2}))))
$(call debug-options,${OPTION_PREFIX}$1=${${OPTION_PREFIX}$1})
endef
Ie. if any of NO_${op} NO${op} or WITHOUT_${op} are defined, the value will be no, if WITH_${op} is defined the value will be yes, otherwise the value will be the default (no).
The same is done for OPTIONS_DEFAULT_YES except the default is yes.
Finally any OPTIONS_DEFAULT_DEPENDENT are processed. Each entry is of the form op/default_op. We set MK_${op} as above but the default comes from ${MK_${default_op}}.
The example below (available in http://www.crufty.net/ftp/pub/sjg/gmk-tests.tar.gz ) consists of:
tests/GNUmakefile.inc tests/lib/GNUmakefile.inc tests/lib/a/GNUmakefile tests/lib/a/GNUmakefile.depend tests/lib/a/GNUmakefile.depend.host tests/lib/b/GNUmakefile tests/lib/b/GNUmakefile.depend tests/lib/b/GNUmakefile.depend.host tests/lib/d/GNUmakefile tests/lib/d/GNUmakefile.depend tests/lib/e/GNUmakefile tests/lib/e/GNUmakefile.depend tests/prog/GNUmakefile tests/prog/GNUmakefile.depend tests/tools/GNUmakefile.inc tests/tools/tool/GNUmakefile tests/tools/tool/GNUmakefile.depend.host
of which the following all list tests/tools/tool.host as a dependency:
tests/lib/a/GNUmakefile.depend tests/lib/e/GNUmakefile.depend tests/prog/GNUmakefile.depend
and tests/lib/a depends on tests/lib/b and tests/lib/e, while tests/lib/b depends on tests/lib/d and tests/lib/e.
Since tests/tools/tool/GNUmakefile.depend.host is:
DIRDEPS = \
tests/lib/a \
include dirdeps.gmk
it needs tests/lib/a built for host. note that tests/lib/a/GNUmakefile.depend.host is just:
DIRDEPS = \
tests/lib/b \
include dirdeps.gmk
(so tests/lib/b also needs to build for host) while tests/lib/a/GNUmakefile.depend is:
DIRDEPS = \
tests/tools/tool.host \
tests/lib/b \
tests/lib/e
include dirdeps.gmk
Also note that there is no GNUmakefile.depend.host in tests/lib/b so we will re-use its GNUmakefile.depend.
Initially I had all of the GNUmakefile end up including tests/GNUmakefile.inc which has just:
all:
# sys.gmk already set this
_CURDIR ?= ${CURDIR}
PROG ?= $(notdir ${_CURDIR})
THING ?= ${PROG}
ifneq "${MAKELEVEL}" "0"
all: realbuild
realbuild:
@echo Building ${CURDIR}/${THING}
@echo Finished making ${RELDIR} for ${TARGET_SPEC}
endif
so we didn't actually build anything, we just claim to.
Handy for testing dirdeps.gmk but not so much for obj.gmk.
As we can see below, everything get's built in the optimal order and nothing is visited more than once:
$ mk-i386 -C tests/prog gmake: Entering directory '/h/sjg/work/gmk/tests/prog' Checking /h/sjg/work/gmk/tests/lib/b for host ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b' Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b/libb.a Finished making tests/lib/b for host gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b' Checking /h/sjg/work/gmk/tests/lib/a for host ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a' Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a/liba.a Finished making tests/lib/a for host gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a' Checking /h/sjg/work/gmk/tests/tools/tool for host ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool' Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool/tool Finished making tests/tools/tool for host gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool' Checking /h/sjg/work/gmk/tests/lib/d for i386 ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/lib/d' Building /h/sjg/work/gmk/obj/i386/tests/lib/d/libd.a Finished making tests/lib/d for i386 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/lib/d' Checking /h/sjg/work/gmk/tests/lib/e for i386 ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/lib/e' Building /h/sjg/work/gmk/obj/i386/tests/lib/e/libe.a Finished making tests/lib/e for i386 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/lib/e' Checking /h/sjg/work/gmk/tests/lib/b for i386 ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/lib/b' Building /h/sjg/work/gmk/obj/i386/tests/lib/b/libb.a Finished making tests/lib/b for i386 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/lib/b' Checking /h/sjg/work/gmk/tests/lib/a for i386 ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/lib/a' Building /h/sjg/work/gmk/obj/i386/tests/lib/a/liba.a Finished making tests/lib/a for i386 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/lib/a' Checking /h/sjg/work/gmk/tests/prog for i386 ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/prog' Building /h/sjg/work/gmk/obj/i386/tests/prog/prog Finished making tests/prog for i386 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/prog' gmake: Leaving directory '/h/sjg/work/gmk/tests/prog' $
and it all works just as well in parallel (see below).
Also if we change the origin, the graph changes accordingly - only that which is needed is built:
$ mk-amd64 -j8 -C tests/lib/b gmake: Entering directory '/h/sjg/work/gmk/tests/lib/b' Checking /h/sjg/work/gmk/tests/lib/d for amd64 ... Checking /h/sjg/work/gmk/tests/lib/b for host ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/amd64/tests/lib/d' gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b' Building /h/sjg/work/gmk/obj/amd64/tests/lib/d/libd.a Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b/libb.a Finished making tests/lib/d for amd64 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/amd64/tests/lib/d' Finished making tests/lib/b for host gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b' Checking /h/sjg/work/gmk/tests/lib/a for host ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a' Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a/liba.a Finished making tests/lib/a for host gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a' Checking /h/sjg/work/gmk/tests/tools/tool for host ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool' Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool/tool Finished making tests/tools/tool for host gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool' Checking /h/sjg/work/gmk/tests/lib/e for amd64 ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/amd64/tests/lib/e' Building /h/sjg/work/gmk/obj/amd64/tests/lib/e/libe.a Finished making tests/lib/e for amd64 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/amd64/tests/lib/e' Checking /h/sjg/work/gmk/tests/lib/b for amd64 ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/amd64/tests/lib/b' Building /h/sjg/work/gmk/obj/amd64/tests/lib/b/libb.a Finished making tests/lib/b for amd64 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/amd64/tests/lib/b' gmake: Leaving directory '/h/sjg/work/gmk/tests/lib/b' $
Note too that when building for the pseudo machine host we use an objdir named for HOST_TARGET (set by mk) which helps ensure that a tree shared via NFS with lots of different machines will just work.
The mk tool is reading .sandbox-env to condition the environment and identify the top of the tree (sandbox):
export SB_MAKE=gmake export SRCTOP=$SB export GMKSYSDIR=$SB/gmk export MAKEFILES=$GMKSYSDIR/sys.gmk SB_PATH=$PATH
The MAKEFILES=$GMKSYSDIR/sys.gmk is key.
You can run the tests as:
$ mk -j8 -f gmk/dirdeps.gmk tests/prog.i386 tests/prog.amd64
and it will build tests/prog for both i386 and amd64 but all the host bits will still be built only once.
If we add DEBUG_DIRDEPS=1 we get to see all the plumbing in action:
$ mk DEBUG_DIRDEPS=1 -j8 -f gmk/dirdeps.gmk tests/prog.i386 tests/prog.amd64 INCLUDED_FROM=/h/sjg/work/gmk/gmk/local.sys.gmk dirdeps: /h/sjg/work/gmk/tests/prog.amd64 /h/sjg/work/gmk/tests/prog.i386 _more_depends= /h/sjg/work/gmk/tests/prog/GNUmakefile.depend.amd64 /h/sjg/work/gmk/tests/prog/GNUmakefile.depend.i386 Looking for /h/sjg/work/gmk/tests/prog/GNUmakefile.depend.amd64 DEP_TARGET_SPEC=amd64 DEP_RELDIR=tests/prog -including /h/sjg/work/gmk/tests/prog/GNUmakefile.depend.inc ... dirdeps: DEP_RELDIR=tests/prog DEP_TARGET_SPEC=amd64 DIRDEPS=tests/tools/tool.host tests/lib/a tests/lib/e /h/sjg/work/gmk/tests/prog.amd64: /h/sjg/work/gmk/tests/tools/tool.host /h/sjg/work/gmk/tests/lib/a.amd64 /h/sjg/work/gmk/tests/lib/e.amd64 _more_depends= /h/sjg/work/gmk/tests/tools/tool/GNUmakefile.depend.host /h/sjg/work/gmk/tests/lib/a/GNUmakefile.depend.amd64 /h/sjg/work/gmk/tests/lib/e/GNUmakefile.depend.amd64 Looking for /h/sjg/work/gmk/tests/tools/tool/GNUmakefile.depend.host DEP_TARGET_SPEC=host DEP_RELDIR=tests/tools/tool -including /h/sjg/work/gmk/tests/tools/tool/GNUmakefile.depend.inc ... dirdeps: DEP_RELDIR=tests/tools/tool DEP_TARGET_SPEC=host DIRDEPS=tests/lib/a /h/sjg/work/gmk/tests/tools/tool.host: /h/sjg/work/gmk/tests/lib/a.host ...
and so on.
It is easier to read without that noise though:
$ mk -j8 -f gmk/dirdeps.gmk tests/prog.i386 tests/prog.amd64 Checking /h/sjg/work/gmk/tests/lib/b for host ... Checking /h/sjg/work/gmk/tests/lib/d for amd64 ... Checking /h/sjg/work/gmk/tests/lib/d for i386 ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/lib/d' gmake[1]: Entering directory '/h/sjg/work/gmk/obj/amd64/tests/lib/d' gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b'
Note that since tests/lib/d and tests/lib/b (for host) do not have any dependencies (empty DIRDEPS) they get built first.
Building /h/sjg/work/gmk/obj/i386/tests/lib/d/libd.a Building /h/sjg/work/gmk/obj/amd64/tests/lib/d/libd.a Finished making tests/lib/d for i386 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/lib/d' Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b/libb.a Finished making tests/lib/d for amd64 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/amd64/tests/lib/d' Finished making tests/lib/b for host gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b' Checking /h/sjg/work/gmk/tests/lib/a for host ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a' Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a/liba.a Finished making tests/lib/a for host gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a' Checking /h/sjg/work/gmk/tests/tools/tool for host ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool' Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool/tool Finished making tests/tools/tool for host gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool' Checking /h/sjg/work/gmk/tests/lib/e for amd64 ... Checking /h/sjg/work/gmk/tests/lib/e for i386 ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/amd64/tests/lib/e' gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/lib/e' Building /h/sjg/work/gmk/obj/amd64/tests/lib/e/libe.a Building /h/sjg/work/gmk/obj/i386/tests/lib/e/libe.a Finished making tests/lib/e for amd64 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/amd64/tests/lib/e' Finished making tests/lib/e for i386 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/lib/e' Checking /h/sjg/work/gmk/tests/lib/b for amd64 ... Checking /h/sjg/work/gmk/tests/lib/b for i386 ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/amd64/tests/lib/b' gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/lib/b' Building /h/sjg/work/gmk/obj/amd64/tests/lib/b/libb.a Building /h/sjg/work/gmk/obj/i386/tests/lib/b/libb.a Finished making tests/lib/b for amd64 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/amd64/tests/lib/b' Finished making tests/lib/b for i386 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/lib/b' Checking /h/sjg/work/gmk/tests/lib/a for amd64 ... Checking /h/sjg/work/gmk/tests/lib/a for i386 ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/amd64/tests/lib/a' gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/lib/a' Building /h/sjg/work/gmk/obj/amd64/tests/lib/a/liba.a Building /h/sjg/work/gmk/obj/i386/tests/lib/a/liba.a Finished making tests/lib/a for amd64 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/amd64/tests/lib/a' Finished making tests/lib/a for i386 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/lib/a' Checking /h/sjg/work/gmk/tests/prog for i386 ... Checking /h/sjg/work/gmk/tests/prog for amd64 ... gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/prog' gmake[1]: Entering directory '/h/sjg/work/gmk/obj/amd64/tests/prog' Building /h/sjg/work/gmk/obj/i386/tests/prog/prog Building /h/sjg/work/gmk/obj/amd64/tests/prog/prog Finished making tests/prog for i386 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/prog' Finished making tests/prog for amd64 gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/amd64/tests/prog' gmake: Nothing to be done for 'tests/prog.amd64'. $
This is really just a proof of concept.
I haven't used gmake seriously since the mid '90s, so it is quite possible others could improve on my implementation.
The real dirdeps (with bmake) works incredibly well. This implementation provides only the basic functionality but is far more capable that I expected would be possible.
A key missing feature is the ability to filter DIRDEPS in weird and wonderful ways, and this implementation cannot automatically adapt to different requirements by simply tweaking TARGET_SPEC_VARS.
Filtering I address by including local.dirdeps-filter.gmk at the appropriate point. The implementation is left as an exercise for the suitably motivated reader ;-)
If interested; you can download this version from https://www.crufty.net/ftp/pub/sjg/gmk.tar.gz
While dirdeps*.gmk and sys.gmk are the key bits, the rest of init.gmk, lib.gmk, obj.gmk, options.gmk and prog.mk are quite functional.
Feedback and improvements are always welcome.
| Author: | sjg@crufty.net |
|---|---|
| Revision: | $Id: gmake-dirdeps.txt,v 6a4ee0e3d493 2026-02-16 20:19:47Z sjg $ |