DIRDEPS for gmake
*****************

This is a somewhat limited approximation of dirdeps_ for ``gmake``.
It started out as a thought exercise, since I'd previously asserted
that ``gmake`` could not do it.  Well it can - not all of it certainly,
but enough to be quite useful.

We cannot handle all the bells and whistles that bmake_ can, but we
can still manage the basics to handle orchestrating a large build
correctly.

Apart from efficiency, a key advantage of using ``DIRDEPS`` is that
all the makefiles involved in orchestrating the build are easy 
to read and understand.  This is rarely true of top-level makefiles.
(Ok it is also not true of the full blown dirdeps_ but that can be
considered a *black box*, and it is not *just* for top-level builds).

Along the way, I also had to implement ``sys.gmk`` (first makefile read
by each instance of ``gmake``) and a number of other useful analogs
for the centralized makefiles commonly used with bmake_.

DIRDEPS in brief
================

The concept is quite simple.
We locate a ``Makefile.depend*`` (I use ``GNUmakefile.depend*`` for
``gmake``) in each directory to be made.

Each such makefile sets ``DIRDEPS`` to a list of paths relative to
``SRCTOP`` that need to be built before that directory.
A ``DIRDEP`` might be qualified with a suffix to describe the
``TARGET_SPEC`` for which it should be built.
An unqualified ``DIRDEP`` is built with the ``TARGET_SPEC`` specified
when reading the makefile or the current one.
The makefile then includes ``dirdeps.gmk`` to do the work.

This happens recursively and in the process we build a tree dependency
graph which we hand over to ``gmake`` to build in the optimal order,
in parallel with no race conditions - provided ``DIRDEPS`` are
accurate.  The model works just as well regardless of where you start
your build from - so top-level and leaf builds all work the same way,
with all the same benefits.

In the bmake_ version, a ``Makefile.depend`` file looks like::

	# Autogenerated - do NOT edit!

	DIRDEPS = \
		tools/thing.host \
		lib/sjg \


	.include <dirdeps.mk>

and we can have a kernel module track the files read and executed etc
to build each target - kept in a `.meta file`_ for each target which
alone greatly adds to the reliability of update builds, and the
syscall traces can be leveraged to auto-update the above and keep it
accurate. 

For ``gmake`` it will be a bit more crude.
For a start; we'll assume that all ``GNUmakefile.depend*`` are manually
maintained, and of course we have nothing like `meta mode`_.

There is a ``vpath PATTERN DIRLIST`` directive, which lets us do::

	vpath %.gmk ${GMAKESYSPATH}

so that::

	include dirdeps.gmk

should *just work*, and at least with recent (4.4) ``gmake``, it does.
For older versions we might need to spell it out::

	include ${GMKSYSDIR}/dirdeps.gmk

where ``GMKSYSDIR`` is either set in the environment or by ``sys.gmk``
(see below) which we can cause to be included before each makefile by
setting something like ``MAKEFILES=${SRCTOP}/gmk/sys.gmk`` in the
environment.  The Gmake Manual frowns on this usage - but this is a
good use for it.

Since I tend to use a `wrapper around make`_ for conditioning the
environment, that's simple.

We could also run ``gmake`` with ``-I ${GMKSYSDIR}``
which is easy enough with the mk_ wrapper, but that's a bit ugly for
anyone not using it.

Below I will assume a recent version of ``gmake`` and will use mk_ to
invoke it.

Generic makefiles: ``*.gmk``
============================

With bmake_ we take advantage of a number of generic makefiles (``*.mk``)
traditionally found in ``/usr/share/mk/`` on BSD systems, but in a src
tree they might be in ``share/mk/`` or even ``mk/``.

I've done equivalents for ``gmake`` as ``*.gmk``.

``sys.gmk``
-----------

The first makefile read by bmake_ is ``sys.mk`` and it (and anything
it includes) can pre-condition the build environment as needed.
We want the same functionality here.

As noted above this is handled by putting
``MAKEFILES=${SRCTOP}/gmk/sys.gmk`` into the environment before
invoking ``gmake``.

Since I've now also implemented a useful ``options.gmk`` we could make
``sys.gmk`` more generic, but as is, it is geared towards the
``DIRDEPS`` build setup.

Establishing ``SRCTOP`` and ``OBJTOP``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I dislike dribbling generated files all over the src tree, so keeping
a clean separation of ``SRCTOP`` and ``OBJTOP`` is a given::

	# to be safe, we will be explicit here
	-include ${GMKSYSDIR}/local.sys.gmk
	
	# we need this in a variable
	MAKEFILE_PREFERENCE ?= GNUmakefile makefile Makefile
	
	ifndef sys_gmk0
	# these do not change
	SRCTOP ?= ${GMKSYSDIR}/..
	OBJROOT ?= ${SRCTOP}/../obj
	SRCTOP := $(abspath ${SRCTOP})
	OBJROOT := $(abspath ${OBJROOT})
	export SRCTOP OBJROOT
	endif


Handling ``TARGET_SPEC`` and ``TARGET_SPEC_VARS``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The default ``TARGET_SPEC_VARS`` is just ``MACHINE``.

If more is needed then, ``local.sys.gmk`` can set
``TARGET_SPEC_VARS``, ``TARGET_OBJ_SPEC_VARS`` and 
``TARGET_SPEC_VARS_x`` as needed.

For each instance of ``gmake``, ``sys.gmk`` will need to decompose
``TARGET_SPEC`` to set the individual ``TARGET_SPEC_VARS`` correctly.
The below does the trick::

	define set-target-spec-var
	       $(eval k=$(word $1,${TARGET_SPEC_VARS}))
	       $(eval v=$(word $1,$2))
	       $(if $k, $(if $v,$(eval $k := $v)))
	endef

	...
	
	ifneq "${TARGET_SPEC}" ""
	# decompose ${TARGET_SPEC} to set the individual values
	_tspec := $(subst ${comma}, ,${TARGET_SPEC})
	TARGET_SPEC_VARS_x ?= 1 2 3 4
	$(foreach i,${TARGET_SPEC_VARS_x},$(call set-target-spec-var,$i,${_tspec}))
	endif

Apart from not being able to convert the number of words in
``TARGET_SPEC_VARS`` into a range of indices (we could resort to
``$(shell)``) it's not much different than what we do with bmake_.

If ``local.sys.gmk`` does not set ``TARGET_OBJ_SPEC_VARS``,
we likely have to resort to a ``$(shell)``
invocation to reverse the order of ``TARGET_SPEC_VARS``
but we only do that if ``TARGET_OBJ_SPEC_SEP`` is ``/``::

	TARGET_SPEC_VARS ?= MACHINE 
	ifneq "$(words ${TARGET_SPEC_VARS})" "1"
	...
	TARGET_OBJ_SPEC_SEP ?= .
	ifeq "${TARGET_OBJ_SPEC_VARS}" ""
	ifeq "${TARGET_OBJ_SPEC_SEP}" "/"
	# for this case reversing order of TARGET_SPEC_VARS is better
	TARGET_OBJ_SPEC_VARS = $(shell x= ; for w in ${TARGET_SPEC_VARS}; do x="$$w $$x"; done; echo $$x)
	export TARGET_OBJ_SPEC_VARS
	endif
	endif
	TARGET_OBJ_SPEC_VARS ?= ${TARGET_SPEC_VARS}
	TARGET_OBJ_SPEC ?= $(subst ${space},${TARGET_OBJ_SPEC_SEP},$(foreach v,${TARGET_OBJ_SPEC_VARS},${$v}))

Setting ``_CURDIR`` and ``_OBJDIR``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Since ``gmake`` does not automatically handle this, we have to do it
manually by noting the value of ``CURDIR``.  If it is under
``${OBJTOP}`` then we are good otherwise we need to adjust things
(``obj.gmk`` will do that if needed)::

	ifdef HOST_TARGET
	ifeq "${MACHINE}" "host"
	OBJTOP ?= ${OBJROOT}/${HOST_TARGET}
	endif
	ifeq "${MACHINE}" "host32"
	OBJTOP ?= ${OBJROOT}/${HOST_TARGET32}
	endif
	endif
	TARGET_OBJ_SPEC ?= ${MACHINE}
	OBJTOP ?= ${OBJROOT}/${TARGET_OBJ_SPEC}
	
	# deduce _CURDIR _OBJDIR and RELDIR from ${CURDIR}
	ifeq "${CURDIR}" "${SRCTOP}"
	RELDIR = .
	_CURDIR := ${CURDIR}
	_OBJDIR := ${OBJTOP}
	else
	ifeq "${OBJTOP}" "${CURDIR}"
	RELDIR = .
	_CURDIR := ${SRCTOP}
	_OBJDIR := ${CURDIR}
	else
	# are we under ${OBJTOP}/ ?
	RELDIR := $(subst ${OBJTOP}/,,${CURDIR})
	ifneq "${RELDIR}" "${CURDIR}"
	_CURDIR := ${SRCTOP}/${RELDIR}
	_OBJDIR := ${CURDIR}
	else
	# are we under ${SRCTOP}/ ?
	RELDIR := $(subst ${SRCTOP}/,,${CURDIR})
	ifneq "${RELDIR}" "${CURDIR}"
	_CURDIR := ${CURDIR}
	_OBJDIR = ${OBJTOP}/${RELDIR}
	else
	# we have no clue where we are!
	$(warning ${CURDIR} is not in/under ${SRCTOP}/ or ${OBJTOP}/)
	undefine RELDIR
	undefine _CURDIR
	undefine _OBJDIR
	endif
	endif
	endif
	endif
	

``dirdeps.gmk``
---------------

This is the one that makes the build work.  It is included by each
``GNUmakefile.depend*``

Having constructed the fully qualified list of ``dirdeps`` for the current
directory eg.::

	${SRCTOP}/${RELDIR}.${TARGET_SPEC}

We can turn that into a list of qualified depend files to try and include::

	${SRCTOP}/${RELDIR}/${DEPENDFILE_PREFIX}.${TARGET_SPEC}

We use the following to do the actual including of the next depend file
while setting ``DEP_RELDIR`` and ``DEP_TARGET_SPEC`` appropriately.
We want a qualified depend file if it exists, otherwise an unqualified
one should do::

	define include-depend
		$(eval DEP_TARGET_SPEC=$(subst .,,$(suffix $1)))
		$(eval DEP_RELDIR=$(patsubst %/,%,$(subst ${SRCTOP}/,,$(dir $1))))
		$(if ${DEBUG_DIRDEPS},$(info Looking for $1 DEP_TARGET_SPEC=${DEP_TARGET_SPEC} DEP_RELDIR=${DEP_RELDIR}),)
		$(eval -include $(shell test -s $1 && echo $1 || echo $(basename $1)))
	endef

invoked via::

	$(eval $(foreach dep,${_more_depends},$(call include-depend,${dep})))

That's actually not too shabby.

Target tracking
~~~~~~~~~~~~~~~

Gmake does not appear to provide a means of knowing if a target has
been defined, so we need to do that ourselves, otherwise we get
lots of warnings about duplicate targets.
The following does the trick::

	# we have to do target tracking ourselves to avoid complaints
	define if-new-target
		$(eval t=_${1}_target)
		$(if ${$t},,$1)
		$(eval $t=1)
	endef

invoked thus::

	# add dependencies to the graph
	${_this_dirdep}: ${_dirdeps}
	# now which ones have not been made targets yet?
	_newdeps := $(sort $(foreach dep,${_dirdeps},$(call if-new-target,${dep})))
	ifneq "${_newdeps}" ""
	${_newdeps}:
		$(call build-dirdep)
	endif

Limited pattern matching
~~~~~~~~~~~~~~~~~~~~~~~~

We want to be able to::

	mk -j8 -f dirdeps.gmk some/dir.host other/dir.i386

which should behave exactly as if we had a ``GNUmakefile.depend`` file
containing::

	DIRDEPS= \
		some/dir.host \
		other/dir.i386 \

	include dirdeps.gmk

Unfortunately gmake's ``$(filter)`` does not seem able to match a pattern
like ``%/%`` so we have to resort to a ``shell`` script::

	# (shell is needed because filter cannot do %/%)
	_dirdeps := $(sort $(shell for f in ${MAKECMDGOALS}; do echo "$$f"; done | grep /))

and we had to use ``grep`` since ``case "$$f" in */*) echo $$f;;
esac`` caused a parsing error - the ``)`` closed the ``$(shell``.

Inefficient, but it's a one-off and it works.

building dirdeps
~~~~~~~~~~~~~~~~

After computing the tree dependencies as described above,
``dirdeps.gmk`` handles visiting them in the correct order.
A ``dirdep`` in which no makefile from ``${MAKEFILE_PREFERENCE}`` can
be found, will be ignored.
This makes handling optional directories, sparse trees, and just
refactoring fallout all simple.

If ``OBJTOP`` is defined, ``dirdeps.gmk`` will use
``build-dirdep-obj`` which will auto-create an obj dir
and run ``gmake`` in that directory with ``VPATH`` set to the correct
src dir and with the makefile we found.
Otherwise we just visit each the src dir using ``build-dirdep``.

In either case ``TARGET_SPEC`` will be set to the suffix of the
``dirdep`` and ``MACHINE`` will be set to the first word in
``TARGET_SPEC``.  As noted above ``sys.gmk`` will deal with
``TARGET_SPEC``.

dirdeps-targets.gmk
-------------------

This makefile handles the logic normally found in a top-level
makefile to decide what to build.

For any given target ``dirdeps-targets.gmk`` looks for a directory of
that name under ``${SRCTOP}/targets/`` and anything else in
``DIRDEPS_TARGETS_DIRS``, that contains a ``GNUmakefile.depend*``
file.

Given the tree from our `dirdeps example`_ below and::

	$ cat targets/tests/GNUmakefile.depend
	DIRDEPS= tests/prog

	include dirdeps.gmk

and a top level makefile::

	$ cat GNUmakefile
	all:
        	@echo Finished ${DIRDEPS_TARGETS}

	include dirdeps-targets.gmk

is all we need to turn ``tests`` into a top-level target::

	$ mk tests
	Checking /homes/sjg/work/gmk/tests/lib/b for host ...
	gmake[1]: Entering directory '/homes/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b'
	Building /homes/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b/libb.a
	Finished making tests/lib/b for host
	..
	..
	gmake[1]: Entering directory '/homes/sjg/work/gmk/obj/amd64/tests/prog'
	Building /homes/sjg/work/gmk/obj/amd64/tests/prog/prog
	Finished making tests/prog for amd64
	gmake[1]: Leaving directory '/homes/sjg/work/gmk/obj/amd64/tests/prog'
	Checking /homes/sjg/work/gmk for amd64 ...
	gmake[1]: Entering directory '/homes/sjg/work/gmk/obj/amd64'
	Finished targets/tests
	gmake[1]: Leaving directory '/homes/sjg/work/gmk/obj/amd64'
	$


``obj.gmk``
-----------

Another cool feature missing in ``gmake`` is the ability to
automate control of the directory where files will be generated.
Granted; this can be a source of confusion to those unfamiliar with
the concept.

As noted above; ``dirdeps.gmk`` compensates by doing the auto-obj task
itself when building each ``dirdep``.

This makefile will do the same thing - for when ``dirdeps.mk`` is not
involved, for example when we bypass ``MAKELEVEL=0``::

	mk --machine 1,amd64 -C tests/prog

will start ``gmake`` in tests/prog with ``MAKELEVEL=1`` and
``MACHINE=amd64``.
We want it to use the same ``_OBJDIR`` that ``dirdeps.gmk`` would
have.

We rely on ``sys.gmk`` having set ``_CURDIR`` and ``_OBJDIR``.
If ``CURDIR`` is the same as ``_CURDIR`` then we are not in our
desired ``_OBJDIR``.
We do a subset of what ``dirdeps.gmk`` does - setting
``VPATH=${_CURDIR}``, and running ``MAKE`` in ``${_OBJDIR}`` with the
correct makefile.

At ``MAKELEVEL > 0``, ``init.gmk`` will include ``obj.gmk`` if
``${MK_AUTO_OBJ}`` is ``"yes"``.  It gets set by ``options.gmk``.

We set ``_SKIP_BUILD=obj`` to let ``{init,lib,prog}.gmk`` know they
should do nothing else, since the real work is already being done via a
sub-make.


``init.gmk``
------------

This makefile is included by both ``lib.gmk`` and ``prog.gmk`` to deal
with common logic.
It will include ``local.init.gmk``, ``${_CURDIR}/../GNUmakefile.inc``
and ``options.gmk``.

As noted at ``MAKELEVEL > 0`` it will include ``obj.gmk`` if needed.

If ``_SKIP_BUILD`` is not defined it also sets up the basic targets::

	ifndef _SKIP_BUILD
	all: afterbuild
	afterbuild: realbuild
	realbuild: beforebuild
	beforebuild:
	endif

and defaults for ``CC``, ``CXX`` etc. and rules to use them::

	CC ?= cc
	CXX ?= c++
	AR ?= ar
	LD ?= ld
	
	COMPILE.c = ${CC} -c -o $@ ${CPPFLAGS} ${CFLAGS}
	COMPILE.cxx = ${CXX} -c -o $@ ${CPPFLAGS} ${CXXFLAGS}
	LINK.c = ${CC} -o $@ ${CLDFLAGS} ${LDADD} ${LDADD_LAST}
	
	.c.o:
		${COMPILE.c} $<
	
	.cc.o .cxx.o:
		${COMPILE.cxx} $<
	
	srcs2objs = $(foreach e,.c .cc .cxx .cpp,$(subst $e,.o,$(filter %$e,${SRCS})))

``lib.gmk``
-----------

Rules for building libraries.

``prog.gmk``
------------

Rules for building non-libraries.

``options.gmk``
---------------

This makefile is a subset of the bmake_ ``options.mk``.
For each option (``op``) listed in ``OPTIONS_DEFAULT_NO`` we set
``MK_${op}`` if not already set.
It uses::

	# $(call set-option,op,default)
	define set-option
		$(call debug-options,op=$1 default=$2)
		$(if ${${OPTION_PREFIX}$1}, , \
		  $(if $(or ${NO_$1},${NO$1},${WITHOUT_$1}), \
		    $(eval ${OPTION_PREFIX}${1}=no), \
	            $(if ${WITH_$1}, $(eval ${OPTION_PREFIX}${1}=yes), \
		      $(eval ${OPTION_PREFIX}${1}=${2}))))
		$(call debug-options,${OPTION_PREFIX}$1=${${OPTION_PREFIX}$1})
	endef
	
Ie. if any of ``NO_${op}`` ``NO${op}`` or ``WITHOUT_${op}`` are
defined, the value will be ``no``, 
if ``WITH_${op}`` is defined the value will be ``yes``, otherwise
the value will be the default (``no``).

The same is done for ``OPTIONS_DEFAULT_YES`` except the default is
``yes``.

Finally any ``OPTIONS_DEFAULT_DEPENDENT`` are processed.
Each entry is of the form ``op/default_op``.
We set ``MK_${op}`` as above but the default comes from
``${MK_${default_op}}``.

DIRDEPS Example
===============

The example below (available in
http://www.crufty.net/ftp/pub/sjg/gmk-tests.tar.gz )
consists of::

	tests/GNUmakefile.inc
	tests/lib/GNUmakefile.inc
	tests/lib/a/GNUmakefile
	tests/lib/a/GNUmakefile.depend
	tests/lib/a/GNUmakefile.depend.host
	tests/lib/b/GNUmakefile
	tests/lib/b/GNUmakefile.depend
	tests/lib/b/GNUmakefile.depend.host
	tests/lib/d/GNUmakefile
	tests/lib/d/GNUmakefile.depend
	tests/lib/e/GNUmakefile
	tests/lib/e/GNUmakefile.depend
	tests/prog/GNUmakefile
	tests/prog/GNUmakefile.depend
	tests/tools/GNUmakefile.inc
	tests/tools/tool/GNUmakefile
	tests/tools/tool/GNUmakefile.depend.host

of which the following all list ``tests/tools/tool.host``
as a dependency::

	tests/lib/a/GNUmakefile.depend
	tests/lib/e/GNUmakefile.depend
	tests/prog/GNUmakefile.depend

and ``tests/lib/a`` depends on ``tests/lib/b`` and ``tests/lib/e``, 
while ``tests/lib/b`` depends on ``tests/lib/d`` and ``tests/lib/e``.

Since ``tests/tools/tool/GNUmakefile.depend.host`` is::

	DIRDEPS = \
		tests/lib/a \

	include dirdeps.gmk

it needs ``tests/lib/a`` built for ``host``.
note that ``tests/lib/a/GNUmakefile.depend.host`` is just::

	DIRDEPS = \
		tests/lib/b \

	include dirdeps.gmk

(so ``tests/lib/b`` also needs to build for ``host``)
while ``tests/lib/a/GNUmakefile.depend`` is::

	DIRDEPS = \
		tests/tools/tool.host \
		tests/lib/b \
		tests/lib/e

	include dirdeps.gmk

Also note that there is no ``GNUmakefile.depend.host`` in ``tests/lib/b``
so we will re-use its ``GNUmakefile.depend``.

Initially I had  all of the ``GNUmakefile`` end up including
``tests/GNUmakefile.inc`` which has just::

	all:
	
	# sys.gmk already set this
	_CURDIR ?= ${CURDIR}
	
	PROG ?= $(notdir ${_CURDIR})
	THING ?= ${PROG}
	
	ifneq "${MAKELEVEL}" "0"
	all: realbuild
	
	realbuild:
		@echo Building ${CURDIR}/${THING}
		@echo Finished making ${RELDIR} for ${TARGET_SPEC}
	endif

so we didn't actually build anything, we just claim to.

Handy for testing ``dirdeps.gmk`` but not so much for ``obj.gmk``.

As we can see below, everything get's built in the optimal order
and nothing is visited more than once::

	$ mk-i386 -C tests/prog
	gmake: Entering directory '/h/sjg/work/gmk/tests/prog'
	Checking /h/sjg/work/gmk/tests/lib/b for host ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b'
	Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b/libb.a
	Finished making tests/lib/b for host
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b'
	Checking /h/sjg/work/gmk/tests/lib/a for host ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a'
	Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a/liba.a
	Finished making tests/lib/a for host
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a'
	Checking /h/sjg/work/gmk/tests/tools/tool for host ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool'
	Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool/tool
	Finished making tests/tools/tool for host
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool'
	Checking /h/sjg/work/gmk/tests/lib/d for i386 ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/lib/d'
	Building /h/sjg/work/gmk/obj/i386/tests/lib/d/libd.a
	Finished making tests/lib/d for i386
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/lib/d'
	Checking /h/sjg/work/gmk/tests/lib/e for i386 ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/lib/e'
	Building /h/sjg/work/gmk/obj/i386/tests/lib/e/libe.a
	Finished making tests/lib/e for i386
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/lib/e'
	Checking /h/sjg/work/gmk/tests/lib/b for i386 ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/lib/b'
	Building /h/sjg/work/gmk/obj/i386/tests/lib/b/libb.a
	Finished making tests/lib/b for i386
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/lib/b'
	Checking /h/sjg/work/gmk/tests/lib/a for i386 ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/lib/a'
	Building /h/sjg/work/gmk/obj/i386/tests/lib/a/liba.a
	Finished making tests/lib/a for i386
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/lib/a'
	Checking /h/sjg/work/gmk/tests/prog for i386 ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/prog'
	Building /h/sjg/work/gmk/obj/i386/tests/prog/prog
	Finished making tests/prog for i386
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/prog'
	gmake: Leaving directory '/h/sjg/work/gmk/tests/prog'
	$

and it all works just as well in parallel (see below).

Also if we change the origin, the graph changes accordingly - only
that which is needed is built::

	$ mk-amd64 -j8 -C tests/lib/b
	gmake: Entering directory '/h/sjg/work/gmk/tests/lib/b'
	Checking /h/sjg/work/gmk/tests/lib/d for amd64 ...
	Checking /h/sjg/work/gmk/tests/lib/b for host ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/amd64/tests/lib/d'
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b'
	Building /h/sjg/work/gmk/obj/amd64/tests/lib/d/libd.a
	Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b/libb.a
	Finished making tests/lib/d for amd64
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/amd64/tests/lib/d'
	Finished making tests/lib/b for host
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b'
	Checking /h/sjg/work/gmk/tests/lib/a for host ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a'
	Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a/liba.a
	Finished making tests/lib/a for host
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a'
	Checking /h/sjg/work/gmk/tests/tools/tool for host ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool'
	Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool/tool
	Finished making tests/tools/tool for host
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool'
	Checking /h/sjg/work/gmk/tests/lib/e for amd64 ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/amd64/tests/lib/e'
	Building /h/sjg/work/gmk/obj/amd64/tests/lib/e/libe.a
	Finished making tests/lib/e for amd64
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/amd64/tests/lib/e'
	Checking /h/sjg/work/gmk/tests/lib/b for amd64 ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/amd64/tests/lib/b'
	Building /h/sjg/work/gmk/obj/amd64/tests/lib/b/libb.a
	Finished making tests/lib/b for amd64
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/amd64/tests/lib/b'
	gmake: Leaving directory '/h/sjg/work/gmk/tests/lib/b'
	$

Note too that when building for the pseudo machine ``host``
we use an objdir named for ``HOST_TARGET`` (set by mk_)
which helps ensure that a
tree shared via NFS with lots of different machines will *just work*.

The mk_ tool is reading ``.sandbox-env`` to condition the environment
and identify the top of the tree (*sandbox*)::

	export SB_MAKE=gmake
	export SRCTOP=$SB
	export GMKSYSDIR=$SB/gmk
	export MAKEFILES=$GMKSYSDIR/sys.gmk
	SB_PATH=$PATH

The ``MAKEFILES=$GMKSYSDIR/sys.gmk`` is key.

You can run the tests as::

	$ mk -j8 -f gmk/dirdeps.gmk tests/prog.i386 tests/prog.amd64

and it will *build* ``tests/prog`` for both ``i386`` and ``amd64``
but all the ``host`` bits will still be built only once.

If we add ``DEBUG_DIRDEPS=1`` we get to see all the plumbing in
action::

	$ mk DEBUG_DIRDEPS=1 -j8 -f gmk/dirdeps.gmk tests/prog.i386 tests/prog.amd64
	INCLUDED_FROM=/h/sjg/work/gmk/gmk/local.sys.gmk
	dirdeps:  /h/sjg/work/gmk/tests/prog.amd64  /h/sjg/work/gmk/tests/prog.i386
	_more_depends= /h/sjg/work/gmk/tests/prog/GNUmakefile.depend.amd64  /h/sjg/work/gmk/tests/prog/GNUmakefile.depend.i386
	Looking for /h/sjg/work/gmk/tests/prog/GNUmakefile.depend.amd64 DEP_TARGET_SPEC=amd64 DEP_RELDIR=tests/prog
	-including /h/sjg/work/gmk/tests/prog/GNUmakefile.depend.inc ...
	dirdeps: DEP_RELDIR=tests/prog DEP_TARGET_SPEC=amd64 DIRDEPS=tests/tools/tool.host tests/lib/a tests/lib/e
	/h/sjg/work/gmk/tests/prog.amd64:  /h/sjg/work/gmk/tests/tools/tool.host  /h/sjg/work/gmk/tests/lib/a.amd64  /h/sjg/work/gmk/tests/lib/e.amd64
	_more_depends= /h/sjg/work/gmk/tests/tools/tool/GNUmakefile.depend.host  /h/sjg/work/gmk/tests/lib/a/GNUmakefile.depend.amd64  /h/sjg/work/gmk/tests/lib/e/GNUmakefile.depend.amd64
	Looking for /h/sjg/work/gmk/tests/tools/tool/GNUmakefile.depend.host DEP_TARGET_SPEC=host DEP_RELDIR=tests/tools/tool
	-including /h/sjg/work/gmk/tests/tools/tool/GNUmakefile.depend.inc ...
	dirdeps: DEP_RELDIR=tests/tools/tool DEP_TARGET_SPEC=host DIRDEPS=tests/lib/a 
	/h/sjg/work/gmk/tests/tools/tool.host:  /h/sjg/work/gmk/tests/lib/a.host
	...

and so on.

It is easier to read without that noise though::

	$ mk -j8 -f gmk/dirdeps.gmk tests/prog.i386 tests/prog.amd64
	Checking /h/sjg/work/gmk/tests/lib/b for host ...
	Checking /h/sjg/work/gmk/tests/lib/d for amd64 ...
	Checking /h/sjg/work/gmk/tests/lib/d for i386 ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/lib/d'
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/amd64/tests/lib/d'
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b'

Note that since ``tests/lib/d`` and ``tests/lib/b`` (for ``host``) do
not have any dependencies (empty ``DIRDEPS``) they get built first.
::

	Building /h/sjg/work/gmk/obj/i386/tests/lib/d/libd.a
	Building /h/sjg/work/gmk/obj/amd64/tests/lib/d/libd.a
	Finished making tests/lib/d for i386
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/lib/d'
	Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b/libb.a
	Finished making tests/lib/d for amd64
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/amd64/tests/lib/d'
	Finished making tests/lib/b for host
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/b'
	Checking /h/sjg/work/gmk/tests/lib/a for host ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a'
	Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a/liba.a
	Finished making tests/lib/a for host
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/lib/a'
	Checking /h/sjg/work/gmk/tests/tools/tool for host ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool'
	Building /h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool/tool
	Finished making tests/tools/tool for host
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/freebsd11-amd64/tests/tools/tool'
	Checking /h/sjg/work/gmk/tests/lib/e for amd64 ...
	Checking /h/sjg/work/gmk/tests/lib/e for i386 ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/amd64/tests/lib/e'
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/lib/e'
	Building /h/sjg/work/gmk/obj/amd64/tests/lib/e/libe.a
	Building /h/sjg/work/gmk/obj/i386/tests/lib/e/libe.a
	Finished making tests/lib/e for amd64
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/amd64/tests/lib/e'
	Finished making tests/lib/e for i386
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/lib/e'
	Checking /h/sjg/work/gmk/tests/lib/b for amd64 ...
	Checking /h/sjg/work/gmk/tests/lib/b for i386 ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/amd64/tests/lib/b'
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/lib/b'
	Building /h/sjg/work/gmk/obj/amd64/tests/lib/b/libb.a
	Building /h/sjg/work/gmk/obj/i386/tests/lib/b/libb.a
	Finished making tests/lib/b for amd64
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/amd64/tests/lib/b'
	Finished making tests/lib/b for i386
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/lib/b'
	Checking /h/sjg/work/gmk/tests/lib/a for amd64 ...
	Checking /h/sjg/work/gmk/tests/lib/a for i386 ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/amd64/tests/lib/a'
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/lib/a'
	Building /h/sjg/work/gmk/obj/amd64/tests/lib/a/liba.a
	Building /h/sjg/work/gmk/obj/i386/tests/lib/a/liba.a
	Finished making tests/lib/a for amd64
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/amd64/tests/lib/a'
	Finished making tests/lib/a for i386
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/lib/a'
	Checking /h/sjg/work/gmk/tests/prog for i386 ...
	Checking /h/sjg/work/gmk/tests/prog for amd64 ...
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/i386/tests/prog'
	gmake[1]: Entering directory '/h/sjg/work/gmk/obj/amd64/tests/prog'
	Building /h/sjg/work/gmk/obj/i386/tests/prog/prog
	Building /h/sjg/work/gmk/obj/amd64/tests/prog/prog
	Finished making tests/prog for i386
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/i386/tests/prog'
	Finished making tests/prog for amd64
	gmake[1]: Leaving directory '/h/sjg/work/gmk/obj/amd64/tests/prog'
	gmake: Nothing to be done for 'tests/prog.amd64'.
	$
	


Conclusion
==========

This is really just a proof of concept.

I haven't used ``gmake`` seriously since the mid '90s, so it is quite
possible others could improve on my implementation.

The real dirdeps_ (with bmake_) works incredibly well.
This implementation provides only the basic functionality
but is far more capable that I expected would be possible.

A key missing feature is the ability to filter ``DIRDEPS``
in weird and wonderful ways, and this implementation cannot
automatically adapt to different requirements by simply tweaking
``TARGET_SPEC_VARS``.

Filtering I address by including 
``local.dirdeps-filter.gmk`` at the appropriate point.
The implementation is left as an exercise for the suitably motivated
reader ;-)

If interested; you can download this version from
https://www.crufty.net/ftp/pub/sjg/gmk.tar.gz

While ``dirdeps*.gmk`` and ``sys.gmk`` are the key bits,
the rest of ``init.gmk``, ``lib.gmk``, ``obj.gmk``, ``options.gmk``
and ``prog.mk`` are quite functional.

Feedback and improvements are always welcome.

--------

.. _bmake: http://www.crufty.net/sjg/help/bmake.htm
.. _dirdeps: http://www.crufty.net/sjg/docs/dirdeps.htm
.. _`wrapper around make`: http://www.crufty.net/sjg/docs/sb-tools.htm
.. _mk: http://www.crufty.net/sjg/docs/sb-tools.htm#mk
.. _`meta mode`: http://www.crufty.net/sjg/docs/bmake-meta-mode.htm
.. _`.meta file`: http://www.crufty.net/sjg/docs/bmake-meta-mode.htm

:Author: sjg@crufty.net
:Revision: $Id: gmake-dirdeps.txt,v 6a4ee0e3d493 2026-02-16 20:19:47Z sjg $
