Discussion:
[1003.1(2008)/Issue 7 0000763]: If rule has no prerequisites or commands, and target is non-existent, imagine it's been updated
(too old to reply)
Austin Group Bug Tracker
2013-10-10 02:18:06 UTC
Permalink
The following issue has been SUBMITTED.
======================================================================
http://austingroupbugs.net/view.php?id=763
======================================================================
Reported By: dwheeler
Assigned To: ajosey
======================================================================
Project: 1003.1(2008)/Issue 7
Issue ID: 763
Category: Shell and Utilities
Type: Clarification Requested
Severity: Editorial
Priority: normal
Status: Under Review
Name: David A. Wheeler
Organization:
User Reference:
Section: make
Page Number: 2911
Line Number: 95628-95636
Interp Status: ---
Final Accepted Text:
======================================================================
Date Submitted: 2013-10-10 02:18 UTC
Last Modified: 2013-10-10 02:18 UTC
======================================================================
Summary: If rule has no prerequisites or commands, and target
is non-existent, imagine it's been updated
Description:
Many people want to have "make" dependencies automatically generated.
However, many tools and approaches (such as automake) that do this depend
on a poorly-documented interpretation of make's specification. I'd like to
see this interpretation clarified and clearly stated.

As explained in "Advanced Auto-Dependency Generation"
(http://mad-scientist.net/make/autodep.html), it's common to create a
"dependency" file for each source file that contains the dependency
statement for the target. This normally works, but it creates a serious
problem when done the obvious way: "if you remove or rename a prerequisite
file (say a C .h file), make will stop with a fatal error, complaining that
the target doesn't exist".

Tom Tromey found a solution, which appears to be the approach used by
automake. IF you mention the file as a target in the makefile with no
commands and no prerequisites, then if the file doesn't exist, make is
supposed to silently pretend it was recreated. The same approach is
recommended here: http://scottmcpeak.com/autodepend/autodepend.html (and
probably many other places).

This solution works in GNU make, and I suspect it works in many makes.
However, this solution assumes a particular interpretation of the rules in
make's "EXTENDED DESCRIPTION" that is not immediately obvious from the
POSIX text (at best).

I'd like to see this stated clearly, so that it can be relied on by systems
that automatically generate dependencies.

Desired Action:
After "If there are no commands listed for the target, the target shall be
treated as up-to-date." append the following:

"If a target has no prerequisites or commands, and the target of the rule
is a nonexistent file, then `make' imagines this target to have been
updated whenever its rule is run. This implies that all targets depending
on this one will always have their commands run."

(Note that this text is similar to the GNU make section 4.7 text:
http://www.gnu.org/software/make/manual/html_node/Force-Targets.html).


======================================================================

Issue History
Date Modified Username Field Change
======================================================================
2013-10-10 02:18 dwheeler New Issue
2013-10-10 02:18 dwheeler Status New => Under Review
2013-10-10 02:18 dwheeler Assigned To => ajosey
2013-10-10 02:18 dwheeler Name => David A. Wheeler
2013-10-10 02:18 dwheeler Section => make
2013-10-10 02:18 dwheeler Page Number => 2911
2013-10-10 02:18 dwheeler Line Number => 95628-95636
======================================================================
Schwarz, Konrad
2013-10-10 10:56:41 UTC
Permalink
-----Original Message-----
======================================================================
http://austingroupbugs.net/view.php?id=763
======================================================================
Summary: If rule has no prerequisites or commands, and target
is non-existent, imagine it's been updated
After "If there are no commands listed for the target, the target shall be
"If a target has no prerequisites or commands, and the target of the rule
is a nonexistent file, then `make' imagines this target to have been
updated whenever its rule is run. This implies that all targets depending
on this one will always have their commands run."
The proposed addition is just special case of the (quoted) existing text --
make imagines this target to have been updated regardless of whether
it has prerequisites or whether it exists or not. And targets are
updated when one of their prerequisites has been (or is treated as) updated.

I think it is better for a specification to focus on general principles
rather than on special cases. The make specification is already so long
that the general principles are easily overlooked.

Note that GCC's -MP option caters to this mechanism.
David A. Wheeler
2013-10-10 17:08:40 UTC
Permalink
Post by Schwarz, Konrad
The proposed addition is just special case of the (quoted) existing text --
If your position is that the current specification already covers this case,
with the proposed semantics, great! If there's general consensus that the spec
already requires this, then by all means close out the proposal with no change to the spec.
Is there such a consensus?

I read the text several times and could not convince myself
that the spec really required this. That could be a problem with me,
though, and not with the spec :-).
Post by Schwarz, Konrad
I think it is better for a specification to focus on general principles
rather than on special cases.
Sure.
Post by Schwarz, Konrad
Note that GCC's -MP option caters to this mechanism.
Yes, that's why I want to make sure it's actually standard.
Post by Schwarz, Konrad
The make specification is already so long
that the general principles are easily overlooked.
Here I completely disagree. The current "make" spec has so little
functionality that many (most?) people ignore the spec.

A vast number, for example, assume make == GNU make.
When I Google for "portable makefile", my #1 result is
"http://oreilly.com/catalog/make3/book/ch07.pdf", which says:
"What do we mean by a portable makefile? As an extreme example, we want a
makefile that runs without change on any system that GNU make runs on."
Notice the assumption that OF COURSE only GNU make matters. Same thing here:
http://skramm.blogspot.com/2013/04/writing-portable-makefiles.html

GNU make's a great tool! But I presume everyone on this list believes
it's important to have standards too. For standards to be useful they must
include the necessary functionality most people want, and since people's
wants change, the standards need to be extended over time.
Which is what I'm trying to do.

The "make" specification NEEDS to get bigger, because it fails to
include functionality that most of its users require. I don't think
much has to be added; a few general-purpose additions,
almost always borrowed from systems already in use, should fix the problem.

--- David A. Wheeler
Joerg Schilling
2013-10-10 22:47:33 UTC
Permalink
"David A. Wheeler" <dwheeler-OyCmK+MLziRWk0Htik3J/***@public.gmane.org> wrote:

I did not yet read the previous mail, but I like to give some comments on the
rest frombelow.
Post by David A. Wheeler
Yes, that's why I want to make sure it's actually standard.
Post by Schwarz, Konrad
The make specification is already so long
that the general principles are easily overlooked.
Here I completely disagree. The current "make" spec has so little
functionality that many (most?) people ignore the spec.
Correct, the current POSIX make spec still does not include enough to allow
portable makefiles that can be used on a wide variety of platforms.
Post by David A. Wheeler
A vast number, for example, assume make == GNU make.
This seems to be a problem, in special as the number of usefully supported
target platforms is limited with gmake.

Although my "smake" is much older than gmake, I was planning to withdraw it in
favor of gmake in the mid 1990s. Then I realized that there a a lot of
platforms where gmake is not usable execpt for very simple makefiles and in
1997, I decided do continue with smake.

At that time, I found that the best solution is to have a portable make program
and make smake _very_ portable.
Post by David A. Wheeler
When I Google for "portable makefile", my #1 result is
"What do we mean by a portable makefile? As an extreme example, we want a
makefile that runs without change on any system that GNU make runs on."
http://skramm.blogspot.com/2013/04/writing-portable-makefiles.html
These seem to say the same: make the make program portable... but gmake has
problems on CRLF systems and on systems that do not implement fork()/exec() the
same way as on UNIX and the makefile parser in gmake does not honor space
characters and backslashes the way it is documented in the POSIX standard.
Post by David A. Wheeler
GNU make's a great tool! But I presume everyone on this list believes
it's important to have standards too. For standards to be useful they must
include the necessary functionality most people want, and since people's
wants change, the standards need to be extended over time.
Which is what I'm trying to do.
The "make" specification NEEDS to get bigger, because it fails to
include functionality that most of its users require. I don't think
much has to be added; a few general-purpose additions,
almost always borrowed from systems already in use, should fix the problem.
If there would be more make programs with the minimal set of features I
identified as the minimum set that is needed for portability across platforms
(i.e. make programs that allow to auto configure makefiles for the target
platforms of interest), it would be easier for me to make people believe that
my software is very portable.

I found three programs that implement a minimal set of features that is needed
to implement portability at all and that have these features overlap to allow
to use them in portable makefiles: My smake, gmake and SunPro make.

SunPro make is only available for Solaris and Linux, gmake us usable on the
BSDs in addition and smake runs on aprox. 30 different platforms. IIRC, I had
to remove aprox. 5 platforms from the list of supported platforms if I did only
have gmake available.

BTW: BSD make is not implementing a sufficient set of overlapping features
to make it compatible enough with the other implementations to allow uniform
make files. Internal macros are expanded in a different way if VPATH is in use
and there are a few other deviations. BSD make would need to either become
incompatible to it's versions from between 1995 to now or to implement a switch.

In general, I see problems with defining more features in POSIX as some make
implementations may get into problems if the standard defines a behavior that
is not compatible ti their implementation and if we like to enhance the
features, we definitely need do make such decisions that will cause one or the
other make implementaion incompatible to the new standard.

My current solution is to distribute a bootstrap using a shell script to
compile smake and then let people use the resultant smake.

Jörg
--
EMail:joerg-3Qm2Liu6aU2sY6utFDHCwYAplN+***@public.gmane.org (home) Jörg Schilling D-13353 Berlin
js-CFLBMwTPW48UNGrzBIF7/***@public.gmane.org (uni)
joerg.schilling-8LS2qeF34IpklNlQbfROjRvVK+***@public.gmane.org (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
David A. Wheeler
2013-10-11 00:38:17 UTC
Permalink
Post by Joerg Schilling
Correct, the current POSIX make spec still does not include enough to allow
portable makefiles that can be used on a wide variety of platforms.
... I found that the best solution is to have a portable make program
and make smake _very_ portable.
Currently that's the most practical solution. But it should not STAY that way.
Let's improve the spec!
Post by Joerg Schilling
BTW: BSD make is not implementing a sufficient set of overlapping features
to make it compatible enough with the other implementations to allow uniform
make files. Internal macros are expanded in a different way if VPATH is in use
and there are a few other deviations. BSD make would need to either become
incompatible to it's versions from between 1995 to now or to implement a switch.
In general, I see problems with defining more features in POSIX as some make
implementations may get into problems if the standard defines a behavior that
is not compatible ti their implementation and if we like to enhance the
features, we definitely need do make such decisions that will cause one or the
other make implementaion incompatible to the new standard.
I think that's already solved. Any make (including BSD make)
only needs to implement POSIX features if includes the special target ".POSIX".
So BSD could retain its unusual features by default, and switch to
POSIX requirements on VPATH (or whatever) only when .POSIX is set.

We want to avoid gratuitous differences, but if changes MUST be made,
there is already a mechanism for doing so.

--- David A. Wheeler
Steffen "Daode" Nurpmeso
2013-10-11 12:26:17 UTC
Permalink
"David A. Wheeler" <dwheeler-OyCmK+MLziRWk0Htik3J/***@public.gmane.org> wrote:
|On Fri, 11 Oct 2013 00:47:33 +0200, Joerg Schilling <Joerg\
|.Schilling-8LS2qeF34IpklNlQbfROjRvVK+***@public.gmane.org> wrote:
|> In general, I see problems with defining more features in \
|> POSIX as some make
|> implementations may get into problems if the standard defines \
|> a behavior that
|> is not compatible ti their implementation and if we like to enhance the
|> features, we definitely need do make such decisions that \
|> will cause one or the
|> other make implementaion incompatible to the new standard.
|
|I think that's already solved. Any make (including BSD make)
|only needs to implement POSIX features if includes the special \
|target ".POSIX".
[.]
|We want to avoid gratuitous differences, but if changes MUST be made,
|there is already a mechanism for doing so.

Dealing with the standard itself should be made unique first, in
that behaviour needs to be non-implementation-defined.

GMAKE=make
SMAKE=smake
BSDMAKE=bsdmake

T1=./t1.mk
T2=./t2.mk

runner() {
echo gmake && ${GMAKE} CFLAGS=XCFLAGS MKFILE=${1} -f ${1}
echo smake && ${SMAKE} CFLAGS=XCFLAGS MKFILE=${1} -f ${1}
echo bsdmake && ${BSDMAKE} CFLAGS=XCFLAGS MKFILE=${1} -f ${1}
}
cat << \! > ${T1}
# dummy line
CFLAGS = -O1 -g
all: ;@echo "TOP: CFLAGS=$(CFLAGS)"; $(MAKE) -f $(MKFILE) all-recur
all-recur: ;@echo "RECUR: CFLAGS=$(CFLAGS)"
!
runner ${T1}
sed -e '1c\
.POSIX:
' < ${T1} > ${T2}
runner ${T2}
rm -f ${T1} ${T2}

|> My current solution is to distribute a bootstrap using a shell script to
|> compile smake and then let people use the resultant smake.

If all packages would be so user-friendly and be bootstrappable
with whatever make(1)!

|--- David A. Wheeler

--steffen
Schwarz, Konrad
2013-10-11 07:52:51 UTC
Permalink
-----Original Message-----
On Thu, 10 Oct 2013 10:56:41 +0000, "Schwarz, Konrad"
Post by Schwarz, Konrad
The proposed addition is just special case of the (quoted) existing
text --
I read the text several times and could not convince myself
that the spec really required this. That could be a problem with me,
though, and not with the spec :-).
Unfortunately, you snipped the part of my letter which shows that
it does.
Post by Schwarz, Konrad
The make specification is already so long
that the general principles are easily overlooked.
Here I completely disagree. The current "make" spec has so little
functionality that many (most?) people ignore the spec.
Well, I can only speak for myself, but in practice I
refer to the POSIX web pages much more often than to the GNU info pages.
This is not to put down the GNU documentation, which is excellent,
but that for my needs, reading a concise specification
is usually more efficient than wading through reams of text.

I disagree that POSIX make in its current form is lacking significantly
in functionality. Make is a "little language" designed
to complement the general-purpose shell; nearly
all use made of the various Make extensions can be handled
easily in the shell (be it setting environment variables,
invoking Make in a special environment, or generating Makefiles).
Recipes in Make are specified as shell commands, so use of Make
already requires knowledge of a shell language. Why burden the
user with two languages, when one suffices?
Joerg Schilling
2013-10-11 09:26:25 UTC
Permalink
Post by Schwarz, Konrad
I disagree that POSIX make in its current form is lacking significantly
in functionality. Make is a "little language" designed
to complement the general-purpose shell; nearly
all use made of the various Make extensions can be handled
easily in the shell (be it setting environment variables,
invoking Make in a special environment, or generating Makefiles).
Recipes in Make are specified as shell commands, so use of Make
already requires knowledge of a shell language. Why burden the
user with two languages, when one suffices?
I disagree, make is intended to automate things and it does not seem to be wise
to require make to call the shell and the shell to call make again just because
functionality is missing in make.

There are several things that are important and that are missing in POSIX make:

- conditionals. If all make implementations did support recursive macro
expansion and pattern macro expansion, something similar to if-then-else
could be implemented using constructs like this to include target
specific make files:

# Global definitions:
#
# Define magic unique cookie
#
###########################################################################
_UNIQ= .XxZzy-

DEF_ROOT= $(SRCROOT)
DEF_DIR= DEFAULTS

_DEFAULTSROOT= $(_UNIQ)$(DEFAULTSROOT)
__DEFAULTSROOT= $(_DEFAULTSROOT:$(_UNIQ)=$(DEF_ROOT))
DEFLTSROOT= $(__DEFAULTSROOT:$(_UNIQ)%=%)

include $(DEFLTSROOT)/xxx


- A way to automatically set up the automake relevant macros.
Smake (being automake aware) supports this as built in feature
and sets up:

MAKE = smake # This is the name under which make was called
MAKEFLAGS = -pr
MAKE_ARCH = i386
MAKE_BRAND =
MAKE_DOMAIN =
MAKE_FLAGS = -pr
MAKE_HOST = opt
MAKE_HWSERIAL = 448596686
MAKE_ISALIST = amd64 pentium_pro+mmx pentium_pro pentium+mmx pentium i486 i386 i86
MAKE_LEVEL = 1
MAKE_MACH = i86pc
MAKE_MODEL = i86pc
MAKE_M_ARCH = i86pc
MAKE_NAME = smake # This is the official name if the "make"
# implementation to allow make dependent
# makefiles
MAKE_OS = sunos
MAKE_OSDEFS = -D__SVR4
MAKE_OSREL = 5.11
MAKE_OSVERSION = snv_147
MAKE_SHELL_FLAG = -ce
MAKE_SHELL_IFLAG = -c
MAKE_VERSION = 1.2.3

gmake and SunPro make support (different ways) to call shell commands
to fill up these macros and this is slow - but it works.

In general, macros derived from platform specific strings are used
with macro substitution code similar to the code from above are used
to create target/platform oriented file names and the related files
are included.


- A method to implement auto-dependencies.

Method 1) This could be done by writing rules that create
xxx.d files that are first made and then "include"ed

gmake has a bug in this area as it does not use t
known rules to make files to "include" while it on
the other side evaluates rules (e.g. SCCS rules) to
fetch "makefile" before trying to read it.

SunPRO make does not implement "include list of files"
that would be needed for projects with more than one
source.

smake does things right here and could be used as master

Method 2) Standardize the method introduced in 1986 by SunPro make
and the Sun C-compiler and that was enhanced in the
early 1990s. This method is suported by gcc since
aprox. 1997, but it is not supported by gld. To enable
it in SunPro make, you need to include: .KEEP_STATE:

- Compiler dependencies are auto created by the
compiler while doing compilation in case the
environment:

SUNPRO_DEPENDENCIES="some/unique/path <tgt>"

with <tgt> set to $@ for the current command.
This is supported by gcc and Sun Studio.

- Linker dependencies are created by setting the
environment:

SGS_SUPPORT=libmakestate.so.1

where libmakestate.so.1 is a shared library that
will be dlopen-ed by "ld" and whose interface
functions are called by "ld". libmakestate.so.1
then writes a linker dependency file.

This is a shortened description, a standard would need
more information.

The following low level make features have been identified as the minimum
requirements:

include list of files This is already in the standard.

appending to a macro macro += val

suffix macro replacement out= $(macro:string1=string2)

pattern macro replacement out= $(macro:op%os=np%ns)

Of course, it would help to standardize on something like VPATH (smake supports
much more) for convenience.

Note that if we did standardize the automated dependency creation for
source-compilation and linking, it would be possible to write 100% safe
hierarchical make file systems.


Jörg
--
EMail:joerg-3Qm2Liu6aU2sY6utFDHCwYAplN+***@public.gmane.org (home) Jörg Schilling D-13353 Berlin
js-CFLBMwTPW48UNGrzBIF7/***@public.gmane.org (uni)
joerg.schilling-8LS2qeF34IpklNlQbfROjRvVK+***@public.gmane.org (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
David A. Wheeler
2013-10-11 12:54:00 UTC
Permalink
... make is intended to automate things and it does not seem to be wise
to require make to call the shell and the shell to call make again just because
functionality is missing in make.
I agree. The standard for "make" should include the functionality
necessary for typical cases, without having to do a lot of work-arounds.
- conditionals.
Agreed. All major "make" implementations, to my knowledge, implement
if-then-else style conditionals. Tools like automake have to re-implement
conditionals, because the POSIX standard fails to define a STANDARD way
to use conditionals.
- A way to automatically set up the automake relevant macros.
I don't know if this is *strictly* necessary, but I really like the idea
of a large set of standard make macros with preset values that
people can depend on. It'd greatly reduce the need to run a "configure"
and implement work-arounds just to find out basic, trivial information.

Since a makefile or user would be free to override them, this seems
unlikely to cause harm.
- A method to implement auto-dependencies.
Agreed. Everyone works out a way to do auto-dependencies, but
they are totally platform-dependent. I use automake to work around
these limitations, but that should not be necessary, and that just
moves the portability problem; automake then has to grovel about
and embed lots of platform-dependent stuff.


--- David A. Wheeler
Joerg Schilling
2013-10-11 13:31:03 UTC
Permalink
Post by David A. Wheeler
... make is intended to automate things and it does not seem to be wise
to require make to call the shell and the shell to call make again just because
functionality is missing in make.
I agree. The standard for "make" should include the functionality
necessary for typical cases, without having to do a lot of work-arounds.
- conditionals.
Agreed. All major "make" implementations, to my knowledge, implement
if-then-else style conditionals. Tools like automake have to re-implement
conditionals, because the POSIX standard fails to define a STANDARD way
to use conditionals.
But please note that I am not proposing to implement if-then-else in make.
I am just proposing to implement nestable suffix and pattern macro expansions.
This together with include allows conditionals.

BTW: simple on/off conditionals can be implemented by putting a '#' into some
macros, but there is no official way to have this sign in a makefile.
smake for this reason predefines: NUMBER_SIGN = #
Post by David A. Wheeler
- A method to implement auto-dependencies.
Agreed. Everyone works out a way to do auto-dependencies, but
they are totally platform-dependent. I use automake to work around
these limitations, but that should not be necessary, and that just
moves the portability problem; automake then has to grovel about
and embed lots of platform-dependent stuff.
The schily makefile system supports auto dependencies for most platforms since a
long time. This even works for the brain damaged cl.exe but I need a longer
shell script to do that.

Jörg
--
EMail:joerg-3Qm2Liu6aU2sY6utFDHCwYAplN+***@public.gmane.org (home) Jörg Schilling D-13353 Berlin
js-CFLBMwTPW48UNGrzBIF7/***@public.gmane.org (uni)
joerg.schilling-8LS2qeF34IpklNlQbfROjRvVK+***@public.gmane.org (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
David A. Wheeler
2013-10-11 18:00:02 UTC
Permalink
Post by Joerg Schilling
- conditionals....
But please note that I am not proposing to implement if-then-else in make.
I am just proposing to implement nestable suffix and pattern macro expansions.
This together with include allows conditionals.
I've already proposed adding nestable macro expansions to the specification,
which to my knowledge all "make" implementations *already* do. Amusingly,
I recently posted a comment mentioning that these can sometimes implement conditionals:
http://austingroupbugs.net/view.php?id=336

However, I think few make users will be happy with just that.
If-then-else logic is very familiar to everyone, and practically all "makes" implement it;
they just don't agree on the syntax of it.

--- David A. Wheeler
Joerg Schilling
2013-10-11 19:45:55 UTC
Permalink
Post by David A. Wheeler
Post by Joerg Schilling
- conditionals....
But please note that I am not proposing to implement if-then-else in make.
I am just proposing to implement nestable suffix and pattern macro expansions.
This together with include allows conditionals.
I've already proposed adding nestable macro expansions to the specification,
which to my knowledge all "make" implementations *already* do. Amusingly,
http://austingroupbugs.net/view.php?id=336
computed conditionaly (the way I recently posted it) allow much more and this
feature has been invented by me and Simon Ney in February 1993 (when Solaris
x86 came out). They need patern macro substitutions, but this is already
supported by all major make implementations - it just needs to be written down
for the standard.
Post by David A. Wheeler
However, I think few make users will be happy with just that.
If-then-else logic is very familiar to everyone, and practically all "makes" implement it;
they just don't agree on the syntax of it.
With computed conditionals, you may include 1, 2 or more different files that
carry different make rules depending on how you write the pattern matching. This
feature is well tested since more than 20 years now.

We should document it in the make rationale or examples section.

Jörg
--
EMail:joerg-3Qm2Liu6aU2sY6utFDHCwYAplN+***@public.gmane.org (home) Jörg Schilling D-13353 Berlin
js-CFLBMwTPW48UNGrzBIF7/***@public.gmane.org (uni)
joerg.schilling-8LS2qeF34IpklNlQbfROjRvVK+***@public.gmane.org (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
David A. Wheeler
2013-10-11 18:04:40 UTC
Permalink
Post by Joerg Schilling
- conditionals.
I'll try to generate a proposal over this weekend and submit it to this committee.

--- David A. Wheeler
Joerg Schilling
2013-10-11 19:31:00 UTC
Permalink
Post by David A. Wheeler
Post by Joerg Schilling
- conditionals.
I'll try to generate a proposal over this weekend and submit it to this committee.
As mentioned, we would just need pattern macro substitutions.

This first appeared in SunPro make in 1986, then it was copied into gmake and
smake. BSDmake had a halfbaken implementation, but a friend and I fixed this
aprox. 10 years ago, so it should be available now.

Note that it is important not to try to standardize things that are completely
missing in a mayor implementation.


Jörg
--
EMail:joerg-3Qm2Liu6aU2sY6utFDHCwYAplN+***@public.gmane.org (home) Jörg Schilling D-13353 Berlin
js-CFLBMwTPW48UNGrzBIF7/***@public.gmane.org (uni)
joerg.schilling-8LS2qeF34IpklNlQbfROjRvVK+***@public.gmane.org (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
David A. Wheeler
2013-10-11 22:41:38 UTC
Permalink
Post by Joerg Schilling
As mentioned, we would just need pattern macro substitutions.
I think most people would be very unhappy with that.
Almost all make implementations include if-then-else make conditionals, including GNU make, makepp,
FreeBSD/OpenBSD/NetBSD make, Microsoft nmake, AT&T/Alcatel-Lucent nmake, and fastmake.
Makefile generators, like automake and Cmake, also support make conditionals.

I'm currently gathering information and creating a draft to be submitted here:
https://docs.google.com/document/d/1oUR7iMnaNzkeT3TTOS-Gwul6_V3TE8caIDAd1FwPyNc/edit?usp=sharing

I think it comes down to either (1) supporting GNU make style "ifeq"/"ifdef" syntax, or
(2) supporting *BSD make style ".if" syntax. GNU make is more popular, but
slightly clunky and doesn't begin with the reserved ".".

Comments?

--- David A. Wheeler
Dan Kegel
2013-10-11 23:19:55 UTC
Permalink
Post by David A. Wheeler
I think it comes down to either (1) supporting GNU make style "ifeq"/"ifdef" syntax, or
(2) supporting *BSD make style ".if" syntax. GNU make is more popular, but
slightly clunky and doesn't begin with the reserved ".".
Comments?
Popular counts for a lot.
- Dan
David Holland
2013-10-13 08:36:43 UTC
Permalink
Post by Dan Kegel
Post by David A. Wheeler
I think it comes down to either (1) supporting GNU make style "ifeq"/"ifdef" syntax, or
(2) supporting *BSD make style ".if" syntax. GNU make is more popular, but
slightly clunky and doesn't begin with the reserved ".".
Comments?
Popular counts for a lot.
So does "principled". I am not the least bit interested in adding more
gmake warts to BSD make.

(In point of fact though, because there are two entirely different
competing implementations, this is probably something POSIX should
avoid for the time being.)
--
David A. Holland
dholland-S783fYmB3Ccdnm+***@public.gmane.org
David A. Wheeler
2013-10-13 12:07:29 UTC
Permalink
Post by David Holland
(In point of fact though, because there are two entirely different
competing implementations, this is probably something POSIX should
avoid for the time being.)
If there were no competing implementations, there'd be little reason
to have a standard. Just use the One True Implementation (and
its documentation).

Nearly all make implementations have if...then...else.
A very large number of makefiles *use* if...then...else.
But there's no standard syntax for if...then...else, so
in practice POSIX just isn't enough for portable makefiles.

It may be that no agreement can be found, but that
shouldn't be assumed going in. Let's try.

--- David A. Wheeler
Joerg Schilling
2013-10-13 13:39:08 UTC
Permalink
Post by David A. Wheeler
Post by David Holland
(In point of fact though, because there are two entirely different
competing implementations, this is probably something POSIX should
avoid for the time being.)
If there were no competing implementations, there'd be little reason
to have a standard. Just use the One True Implementation (and
its documentation).
The concept of POSIX is not to standardize things if there is no agrement.
Post by David A. Wheeler
Nearly all make implementations have if...then...else.
If you like to call gmake "all"....
Post by David A. Wheeler
A very large number of makefiles *use* if...then...else.
But there's no standard syntax for if...then...else, so
in practice POSIX just isn't enough for portable makefiles.
Not many makfiles use if then else and the makefiles that use them are
non-portable and offer usability only to the limited number of platforms where
gmake works as dumumented.

It is a fact that POSIX does not offer enough for portable makefiles, but there
is no need to standardize something that is not yet agreed on. Pattern macro
expansion is agreed on (*) and if it is standardized together with recursive
expansion, it allows to use a common method for conditionals. This method is
sufficient for portability to all known OS platforms (as demonstrated by the
schily makefile system since 20 years).

*) Aprox. 10 years ago, BSD make did not implement _recursive_ pattern macro
expansion but at that time Hartmut Brandt and I fixed this for FreeBSD.
If the other BSDs did take the code, it should be available on all BSDs.

Jörg
--
EMail:joerg-3Qm2Liu6aU2sY6utFDHCwYAplN+***@public.gmane.org (home) Jörg Schilling D-13353 Berlin
js-CFLBMwTPW48UNGrzBIF7/***@public.gmane.org (uni)
joerg.schilling-8LS2qeF34IpklNlQbfROjRvVK+***@public.gmane.org (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
,
David Holland
2013-10-13 19:45:39 UTC
Permalink
Post by Joerg Schilling
It is a fact that POSIX does not offer enough for portable makefiles, but there
is no need to standardize something that is not yet agreed on. Pattern macro
expansion is agreed on (*) and if it is standardized together with recursive
expansion, it allows to use a common method for conditionals. This method is
sufficient for portability to all known OS platforms (as demonstrated by the
schily makefile system since 20 years).
*) Aprox. 10 years ago, BSD make did not implement _recursive_ pattern macro
expansion but at that time Hartmut Brandt and I fixed this for FreeBSD.
If the other BSDs did take the code, it should be available on all BSDs.
By recursive pattern macro expansion you mean? (sorry if the answer is
somewhere in the 4000 unread mails I have in this mailbox...) Can you
post a quick example of the syntax you expect FreeBSD's make to
accept?
--
David A. Holland
dholland-S783fYmB3Ccdnm+***@public.gmane.org
Joerg Schilling
2013-10-14 11:12:40 UTC
Permalink
Post by David Holland
Post by Joerg Schilling
It is a fact that POSIX does not offer enough for portable makefiles, but there
is no need to standardize something that is not yet agreed on. Pattern macro
expansion is agreed on (*) and if it is standardized together with recursive
expansion, it allows to use a common method for conditionals. This method is
sufficient for portability to all known OS platforms (as demonstrated by the
schily makefile system since 20 years).
*) Aprox. 10 years ago, BSD make did not implement _recursive_ pattern macro
expansion but at that time Hartmut Brandt and I fixed this for FreeBSD.
If the other BSDs did take the code, it should be available on all BSDs.
By recursive pattern macro expansion you mean? (sorry if the answer is
somewhere in the 4000 unread mails I have in this mailbox...) Can you
post a quick example of the syntax you expect FreeBSD's make to
accept?
This happened aprox. 10 ago, AFIK, the problem was that from some construct
like:

$(name: op%os= np%ns)

only "name" was subject to macro expansions (probaly not even allowing more
than a single string literal macro name at this place). "op", "os, "np", "ns"
have been taken as string literals.

Recursive in this context means that any text pattern may be subject to macro
expansion.

Jörg
--
EMail:joerg-3Qm2Liu6aU2sY6utFDHCwYAplN+***@public.gmane.org (home) Jörg Schilling D-13353 Berlin
js-CFLBMwTPW48UNGrzBIF7/***@public.gmane.org (uni)
joerg.schilling-8LS2qeF34IpklNlQbfROjRvVK+***@public.gmane.org (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
David A. Wheeler
2013-10-13 21:35:48 UTC
Permalink
Post by Joerg Schilling
The concept of POSIX is not to standardize things if there is no agrement.
Of course! That is true for *all* standards efforts.

But agreements have be ironed out somewhere.
Agreements are often created inside standards groups like this one,
because sometimes independent implementations implement an
idea with different syntax (which is true in this case).

It may not be possible to gain agreement in this case...
but it is unlikely to happen without trying.
Post by Joerg Schilling
Post by David A. Wheeler
Nearly all make implementations have if...then...else.
If you like to call gmake "all"....
No. Here's a list of make implementations that I *know*
have if...then..else capabilities:
- GNU make (gmake)
- makepp
- FreeBSD make
- NetBSD make (same origin as FreeBSD make)
- OpenBSD make (same origin as FreeBSD make)
- AT&T nmake
- Alcatel-Lucent nmake (related to AT&T make)
- Microsoft nmake (part of Visual Studio, unrelated to AT&T nmake)
- Opus software make

That's 9 makes, depending on how you count.

In addition, the following systems generate makefiles and have to
create their own if-then-else system because portable "make" lacks it:
- automake (part of autotools suite)
- cmake

There may be others, too. But I think it's clear that that this is
NOT a gmake-specific thing. Makes *without* if-then-else are unusual.
Post by Joerg Schilling
Not many [makefiles] use if then else and the makefiles that use them are
non-portable and offer usability only to the limited number of platforms where
gmake works as dumumented.
*LOTS* of makefiles use if-then-else (including automake Makefile.am files),
though I don't know what the percentages are.

It's true that including conditionals in makefiles makes them non-portable.
But that is the very problem I'm trying to fix. The problem isn't that people
are using if-then-else, the problem is that POSIX does not provide even the
basic functionality they want. POSIX make could be useful as-is to far
more people if just a few additions were made.
Post by Joerg Schilling
Pattern macro
expansion is agreed on (*) and if it is standardized together with recursive
expansion, it allows to use a common method for conditionals.
I presume you mean the same thing as "recursive variable indirection in makefiles"
such as $(var1$(var2)), as I separately proposed in:
http://austingroupbugs.net/view.php?id=336

I certainly agree that those should *also* be in the standard; they are widely implemented
and very useful. But they're awkward to use as the *sole*
conditional mechanism; a simple if-then-else is better when you want if-then-else.
I think you really want both; they work very nicely together.

--- David A. Wheeler
Joerg Schilling
2013-10-14 09:37:26 UTC
Permalink
Post by David A. Wheeler
No. Here's a list of make implementations that I *know*
- GNU make (gmake)
- makepp
- FreeBSD make
- NetBSD make (same origin as FreeBSD make)
- OpenBSD make (same origin as FreeBSD make)
- AT&T nmake
- Alcatel-Lucent nmake (related to AT&T make)
- Microsoft nmake (part of Visual Studio, unrelated to AT&T nmake)
- Opus software make
That's 9 makes, depending on how you count.
This is an ideosyncratic definiton of what a "make" program is. How many of
these programs would be able to pass a POSIX compliance test?

If you list only halfway POSIX compliant implementations, you would find that
most make implementations do not implement if-then-else.
Post by David A. Wheeler
In addition, the following systems generate makefiles and have to
- automake (part of autotools suite)
- cmake
These programs are no make implementations and it is really counter productive
to mention them at all. Note that the intention of POSIX is to create native
usability across different platforms. These two programs are just an automated
way of editing makefiles in hope to make them fit to the current target.

If you look at what POSIX likes to establish, there are only two aproaches to
create portability without editing files in order to make them fit to no more
than one single target:

- David Korn and Glenn Fowlers nmake, but this is done without using
the POSIX make syntax and AFAIK this needs a complex bootstrap proces
to create the portable nmake.

- My makefilesystem that has been created since 1993, that _is_ based on
POSIX make syntax + a few extensions.

BTW: if you look into nmake files and compare them to leaf make files from the
schily makefile system, you will find that they are nearly identical although
both have been implemented wihout knowning about the respective other.

The intention of POSIX is to standardize things that are already available and
that allows to work without editing files to make them fit to a target, just to
be able to do a single run of the software.

You are trying to push a method that is supported by a single make program and
you did not yet verify that the method you are proposing will get us to the
expected goal. The method I am proposing is already implementing compatibility
to several popular make implementations and it has been verified during the
past 20 years that this method allows to give portability to all known
platforms (even non-POSIX ones) without the need of editing files. Note that
the method I am psoposing allows to compile the same source tree simultaneously
inside the original source directories if this tree is network mountned to all
target platforms.
Post by David A. Wheeler
*LOTS* of makefiles use if-then-else (including automake Makefile.am files),
though I don't know what the percentages are.
Looking at those makefiles, it turns out that they are mostly non-portable.
Did you ever try to use these systems on a certified POSIX platform like
Solaris?
Post by David A. Wheeler
It's true that including conditionals in makefiles makes them non-portable.
But that is the very problem I'm trying to fix. The problem isn't that people
are using if-then-else, the problem is that POSIX does not provide even the
basic functionality they want. POSIX make could be useful as-is to far
more people if just a few additions were made.
If-then-else is not needed, conditional code is. You need to understand that
there are programming lanuages that are not based on if-then-else. "Make" is
such a language.
Post by David A. Wheeler
Post by Joerg Schilling
Pattern macro
expansion is agreed on (*) and if it is standardized together with recursive
expansion, it allows to use a common method for conditionals.
I presume you mean the same thing as "recursive variable indirection in makefiles"
http://austingroupbugs.net/view.php?id=336
I see no relation to pattern macro expansion.

Here is a typical UNIX make man page that explains it:

http://schillix.berlios.de/man/man1s/make.1s.html


Jörg
--
EMail:joerg-3Qm2Liu6aU2sY6utFDHCwYAplN+***@public.gmane.org (home) Jörg Schilling D-13353 Berlin
js-CFLBMwTPW48UNGrzBIF7/***@public.gmane.org (uni)
joerg.schilling-8LS2qeF34IpklNlQbfROjRvVK+***@public.gmane.org (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Steffen "Daode" Nurpmeso
2013-10-14 13:09:14 UTC
Permalink
Joerg Schilling <Joerg.Schilling-8LS2qeF34IpklNlQbfROjRvVK+***@public.gmane.org> wrote:
|If you look at what POSIX likes to establish, there are only \
|two aproaches to
|create portability without editing files in order to make \
|them fit to no more
|than one single target:
|
|- David Korn and Glenn Fowlers nmake, but this is done without using
| the POSIX make syntax and AFAIK this needs a complex bootstrap proces
| to create the portable nmake.
|
|- My makefilesystem that has been created since 1993, that _is_ based on
| POSIX make syntax + a few extensions.

While off-topic i think this thread wouldn't be complete without
a reference to the research POSIX mk(1) [1,2]. Especially the
possibility to use true regular expressions, as in

(.+)\.$O:R: \1.c
$CC $CFLAGS $stem1.c

and shell evaluation in include directives, e.g.,

MK=x.x
<`{echo '# Extremely powerful mechanisms,' > $MK;\
echo '# including regular expression patters and meta rules' >> $MK;\
echo 'all:' >> $MK;\
echo ' echo We are working the rule' >> $MK;\
echo $MK\
}

seem to be really wicked, and they fit in unbelievable ~215 K of
source code ([3]).

[1] "Maintaining Files on Plan 9 with Mk",
<http://plan9.bell-labs.com/sys/doc/mk.html>
[2] <http://swtch.com/plan9port/unix/man/mk1.html>
[1] <http://swtch.com/plan9port/unix/mk-with-libs.tgz>

|The intention of POSIX is to standardize things that are already \
|available and
|that allows to work without editing files to make them fit \
|to a target, just to
|be able to do a single run of the software.

Given that this would apply to a later version of POSIX and needs
according adjustments in the make(1) incarnations which strive to
support that standard version, i wonder why such powerful POSIX
standardized tools like the sh(1) and regular expressions should
be used solely by standard tools like sed(1) or grep(1). A marker
like '.POSIX-EXTENDED:' or similar could be the required knob to
enable regular expressions and extended patterns in targets and
prerequisits and shell evaluation in include lines.

--steffen
David A. Wheeler
2013-10-14 13:59:20 UTC
Permalink
Post by Joerg Schilling
.... Here's a list of make implementations that I *know*
.,,
That's 9 makes, depending on how you count.
This is an ideosyncratic definiton of what a "make" program is. How many of
these programs would be able to pass a POSIX compliance test?
Probably ALL of the ones I listed; they'd at least get close.
The POSIX "make" specification has very little functionality,
so it's easy to meet. That's the problem; the spec for make doesn't include
enough functionality, so real makefiles are often not portable, or people
have to create big workaround tools to simulate missing "make" functionality.
Post by Joerg Schilling
If you list only halfway POSIX compliant implementations, you would find that
most make implementations do not implement if-then-else.
I don't think that's true, but that's not relevant anyway.

Something doesn't need to be implemented by a majority of
implementations to be added to POSIX. The usual preference is that it's
been implemented at least once, and that threshold has been amply met.
Post by Joerg Schilling
In addition, the following systems generate makefiles and have to
- automake (part of autotools suite)
- cmake
These programs are no make implementations and it is really counter productive
to mention them at all.
They're important because the purpose of both tools is, in part, to work around
the limited facilities of standard make. By identifying the extra mechanisms they
provide make, we can find evidence of what is missing in the make spec.
For example, most automake inputs I've seen use conditionals;
this suggests that conditionals should be in make itself.
Post by Joerg Schilling
If-then-else is not needed, conditional code is.
All we need is a Turing machine, but
most people do not *want* to code on a Turing machine.

A vast number of makes, and make-like systems, add if-then-else
because it's a familiar system for dealing with conditionals.
There's nothing wrong with adding a facility that is
simple, clear, and familiar :-).
Post by Joerg Schilling
Pattern macro expansion is agreed on (*)...
http://schillix.berlios.de/man/man1s/make.1s.html
This is the macro variable pattern substitution proposal,
which I previously proposed and we both discussed here, correct?:
http://austingroupbugs.net/view.php?id=519

If so, then great, that has already been accepted.

--- David A. Wheeler
Joerg Schilling
2013-10-14 14:36:33 UTC
Permalink
Post by David A. Wheeler
Post by Joerg Schilling
Pattern macro expansion is agreed on (*)...
http://schillix.berlios.de/man/man1s/make.1s.html
This is the macro variable pattern substitution proposal,
http://austingroupbugs.net/view.php?id=519
If so, then great, that has already been accepted.
So we are done for conditionals already ;-)


Jörg
--
EMail:joerg-3Qm2Liu6aU2sY6utFDHCwYAplN+***@public.gmane.org (home) Jörg Schilling D-13353 Berlin
js-CFLBMwTPW48UNGrzBIF7/***@public.gmane.org (uni)
joerg.schilling-8LS2qeF34IpklNlQbfROjRvVK+***@public.gmane.org (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Rob Landley
2013-10-26 16:18:21 UTC
Permalink
On Fri, Oct 11, 2013 at 3:41 PM, David A. Wheeler
Post by David A. Wheeler
I think it comes down to either (1) supporting GNU make style
"ifeq"/"ifdef" syntax, or
Post by David A. Wheeler
(2) supporting *BSD make style ".if" syntax. GNU make is more
popular, but
Post by David A. Wheeler
slightly clunky and doesn't begin with the reserved ".".
Comments?
Popular counts for a lot.
Not just popular: currently usable with the installed base of package
builds.

A bit behind on the list (first week at new day job), but this thread
interests me because I need to write a new "make" for my android
self-hosting project. I gave a talk about this at ELC in February and a
follow-up talk at Ohio LinuxFest last month. The former is archived at
and the latter was recorded but not yet
posted by the conference organizers.

I already have Linux down to the smallest self-hosting build
environment as part of my Aboriginal Linux projects. The seven packages
are linux, uClibc, busybox, gcc 4.2, binutils, make, and bash. (I can
probably ditch bash now, but busybox ash was buggy/incomplete when I
did this.) That's why I became busybox maintainer for a while:
upgrading busybox until it could build Linux From Scratch.

But Android has a "no GPL in userspace" policy, so if android is to
become self-hosting it needs different components. uClibc can be
replaced by the musl libc (BSD licensed), I'm writing toybox to replace
busybox and bash (public domain), and http://ellcc.org and
http://lld.llvm.org are making good progress replacing the toolchain
with Apple's LLVM.

That leaves make. I need to write a new one to replace gmake. I was
originally planning to do so in the context of combining my fork of
Fabrice Bellard's old "tinycc" project with qemu's Tiny Code Generator
backend to produce a "qcc" (part of which involves turning tcc/qcc into
a multicall binary that can be called as ld, strip, readelf...), but
llvm will clearly get there first (I don't have time to _start_ on qcc
until after toybox's 1.0 release), so I'll probably add it to toybox
instead.

And my _only_ criteria for the new make will be "does it build
packages". Starting with the above minimal self-hosting set, then
building http://linuxfromscratch.org, then building book II of that
(Beyond Linux From Scratch), then trying to natively bootstrap Linux
distros on top of the resulting system, and at about that time maybe
poking at BSD a bit if I can ever get pcbsd to install _without_ zfs.

I was planning to start by implementing the posix subset of make, but
that isn't enough to actually build anything released this century. If
you like, I can try to document what I add on top of that (and what
packages need it), and possibly I can make a config entry in toybox to
do "posix only" mode so others can also watch package builds break?
(More likely a config entry to switch off the stuff that only breaks a
_few_ packages, rather than half of linux from scratch. Or maybe output
a debug line when a non-posix codepath is triggered? Hmmm...)

The musl-libc.org guys have been pushing patches upstream to remove
some of the obvious glibc-isms they don't feel like implementing.
(Unlike uClibc, musl does _not_ #define
NO_REALLY_THIS_IS_GLIBC_ALL_HAIL_STALLMANS_LEFT_NOSTRIL and rather a
lot breaks if you don't do that. There is a #define __linux__ but
people check for __GLIBC__ or even the glibc version macros instead.
Sigh.) I'm not sure I'll have the energy to do that sort of thing, but
I can document it for others who want to...

Rob
David A. Wheeler
2013-10-26 20:23:07 UTC
Permalink
Not just popular: currently usable with the installed base of package builds.
A bit behind on the list (first week at new day job), but this thread
interests me because I need to write a new "make" for my android
self-hosting project.
...
I was planning to start by implementing the posix subset of make, but
that isn't enough to actually build anything released this century.
Hi Rob! Yes, that's similar to my position. Clearly there are some programs
that stick with just with the POSIX make spec, but a lot of people just
give up and use nonstandard extensions. But this is a *fixable* problem.

To me, adding conditionals to make is an obvious need.
IIRC, "make" is the only utility in POSIX with a long rationale about
why it doesn't include "newer and more leading-edge features"
such as "ifdef" (a functionality almost everyone supports, though with
incompatible semantics).
If you like, I can try to document what I add on top of that (and what
packages need it), and possibly I can make a config entry in toybox to
do "posix only" mode so others can also watch package builds break?
I'd be interested! However, different people depend
on things, and it'd be easy to argue that the list just applies to you.

Could you do a more general survey of "what nonstandard
make extensions are most popularly depended on",
including software with a variety of licenses?
I understand if you don't have the time (I don't), but if someone would
be willing to do that, the results would be especially compelling.

--- David A. Wheeler
Rich Felker
2013-11-14 17:00:44 UTC
Permalink
Post by David A. Wheeler
Not just popular: currently usable with the installed base of package builds.
A bit behind on the list (first week at new day job), but this thread
interests me because I need to write a new "make" for my android
self-hosting project.
....
I was planning to start by implementing the posix subset of make, but
that isn't enough to actually build anything released this century.
Hi Rob! Yes, that's similar to my position. Clearly there are some programs
that stick with just with the POSIX make spec, but a lot of people just
give up and use nonstandard extensions. But this is a *fixable* problem.
I think the easiest "fix" to this problem is not making possibly
controversial changes to POSIX make that might conflict with different
make variants' extensions and that would still leave us with a make
utility that's seriously deficient from a standpoint of many
real-world application needs, but a permissive-licensed (or public
domain) reimplementation of GNU make, written as a strictly conforming
POSIX application with its build system written in POSIX make. While
this would be a bit of work, it doesn't look prohibitive. GNU make is
among the smallest GNU programs, and its functionality is fairly
well-documented in the manual (although I admit on multiple occasions
I've had to test behavior to make sense of ambiguities in the manual).
Post by David A. Wheeler
Could you do a more general survey of "what nonstandard
make extensions are most popularly depended on",
including software with a variety of licenses?
I understand if you don't have the time (I don't), but if someone would
be willing to do that, the results would be especially compelling.
I can't speak for everyone, but my usage pattern with make is to write
the Makefile in such a way to minimize its project-specific knowledge
and have it derive the dependency tree from the source tree at
invocation time. For this purpose, patterned implicit rules (using %),
glob ("wildcard") expansions, and complex textual substitutions are
the most important features.

The features I'm NOT a fan of (but others seem to use) are those that
shift make further away from being a pure-declarative language.

Rich

Ranjit Singh
2013-10-27 18:11:00 UTC
Permalink
Post by Rob Landley
On Fri, Oct 11, 2013 at 3:41 PM, David A. Wheeler
Post by David A. Wheeler
I think it comes down to either (1) supporting GNU make style
"ifeq"/"ifdef" syntax, or
Post by David A. Wheeler
(2) supporting *BSD make style ".if" syntax. GNU make is more
popular, but
Post by David A. Wheeler
slightly clunky and doesn't begin with the reserved ".".
Comments?
Popular counts for a lot.
Not just popular: currently usable with the installed base of package
builds.
I'd prefer the GNU form, out of the two, but above that Knuth-
style \ifdef etc; so long as we have some form of generic \if
which can be used similarly to shell, with builtin [ .. ] or
any other (simple) command such as grep -q, using make macros
as normal.

That does rather obviate the need for if{eq,neq} I know, and if
one extends sh expansion to make macros in such a context, also
if{def,ndef}. Personally I'd keep \ifdef (both) along with a
\defined macro similar to CPP, and lose "ifeq" since it causes
user-confusion for learners of gmake, in the bracketed-form, and
specifying it in the same manner with two variants, is just ugly,
as is the requirement for string quoting (in the other form.)
We can do better.

If we specified: \if [ .. ] we could restrict it not to word-split,
and indeed to use fnmatch for RHS of = if that is deemed useful
(it is ime, since it condenses a "case", but requires quoting of
the RHS when it is a macro one wishes to treat as a literal), ie:
make it equivalent to ksh/bash [[ ]] but with less typing. (It
does not /have/ to be "\if" ofc it just stands out better here.)
\if ( .. ) could simply be numeric, and the other form would ofc
be: \if command. [1]

So I guess I'm pointing out that POSIX could sidestep and simply
specify cleaner forms, in a manner similar to ::= although that
may not appeal to you. However, existing applications would
continue to work with current implementation extensions, exactly
as they currently do with ":=".
Given that we are adding ::= and the other changes to make,
implementors and application authors might as well deal with a
clean POSIX "make 2.0", imo.

This opens the door to things like:
\switch $(macro) \n \case "$(dir)" \n .. \n \case * \n .. \n \end
..which uses exactly the same implementation as \if \elif \else
\fi, and loops like \while and \for.
Granted these could be done with vanilla tokens, but then the
distinction between make, and shell embedded in make, is much
less clear when scanning, tabs notwithstanding.

Functional macros are then \define foo(a, b, c..) \n .. \n \endef
and used via: \foo(argA, argB, argC, \1, \2) and one may leave
\def foo bar for: @foo@ m1-style substitution, which in
conjunction with include/-include begins to obviate the need for
automake.

Much of this can be done externally, as can the whole thing, but
there clearly is demand for the flexibility consequent upon it
being done by make itself (or we would not have divergent extant
implementations to consider.)

We need to specify also that a make should reload itself, as
gmake does, when an include'd file is changed (istr -include is
in the offing, so if this is done already, apologies for noise.)
Post by Rob Landley
But Android has a "no GPL in userspace" policy, so if android
is to become self-hosting it needs different components.
That is curious: one can only surmise that is a political/
commercial decision, given the usage of GPL components on Darwin
and *BSD, such as gmake in MirBSD ports.

Even more remarkable given that Android is an application of
Linux, a GPL component if ever I saw one, and one which requires
gmake to build, last I heard. I guess tivoization is alive and
kicking.

Regards,
Ranjit.

[1] Yes, I am precluding a standard subshell, but not:
var=$(capture) && [ $var = *.ext ] nor: { ( subshell ) && ..; }
although I would not personally allow braced compounds myself,
nor embedded: if ..; then .. fi and so on.
I'd prefer something relatively simple, that can be run via exec*
where external execution is required, and not leave scope for
something which may require system() after parsing.
I realise popen() is a hole in this, but I still want a simpler
format than sh entire, and would prefer not to implement via
popen() at all, since it leaves scope for exactly that.

This is all speculative discussion, ofc, hopefully in tune with
what we are aiming for, and much more discussion is needed if
we are to follow Geoff's exhortation to step back and think
of the bigger picture, of make as a format entire: whence
one is forced to think of make as a language, even if it has
never to date had a specified grammar.

In this regard, I favour a restricted grammar, at the sh
statement level, for conditional expressions which are not
simple boolean tests/numeric expressions. That is ofc new
functionality; although it can be mimicked in gmake with
ifeq "yes" "$(shell cmd >/dev/null && echo yes)"
or wrapped with:
$(if $(shell cmd >/dev/null && echo yes),non-empty,empty)
..it is not as flexible ime, as well as being a pain to maintain,
by comparison with: \if command.

Further if one is already implementing $(shell ..) or has !=,
it is not a great leap to implement:
\if command \n exit0 \n else.. even if one does not wish to
parse anything at all, and/or prefers the full flexibility of
system()/popen().

Either the \if is followed by a bracket, or it is not, in which
case one simply passes the line to sh and acts on the exit
status from wait{,pid}().
Line continuation is already a basic requirement.
--
"One can be a gentleman, without being a push-over."
Rob Landley
2013-11-14 07:50:22 UTC
Permalink
Post by Ranjit Singh
Post by Rob Landley
But Android has a "no GPL in userspace" policy, so if android
is to become self-hosting it needs different components.
That is curious: one can only surmise that is a political/
commercial decision, given the usage of GPL components on Darwin
and *BSD, such as gmake in MirBSD ports.
It's partially pragmatic: there's no such thing as "The GPL" anymore.
The linux kernel and Samba can't share code even though they implement
two ends of the same protocol and both are GPL; the kernel GPLv2 only,
Samba GPLv3 or later. Poor QEMU is caught between wanting kernel
drivers to invert into device emulations and binutils/gdb code to
invert into processor emulations, and it can't do both. (It sided with
the kernel, although its TCG subdirectory is BSD licensed.) People who
don't understand the situation say "just license your project GPLv2 or
later", but then you can't accept code from _either_ source.

The GPL was a terminal node in a giant directed graph of license
convertability allowing developers to simplify all licensing decisions
to "is it GPL compatible Y/N" and thus not be lawyers. And then the FSF
split it into incompatible factions (something Sun tried with CDDL and
got laughed off the stage) and leveraged their copyright assignments to
move a critical mass of projects to a new license incompatible with the
existing GPL, and it turns out that copyleft only works when there's a
single category killer license everyone agrees on. Once it's got
factions, they start attacking each other and people start trying to
escape the conflict by coming up with independent "GPL-next" projects
and applying creative commons to source code, and you get
http://xkcd.com/927/

Now in the absence of a universal receiver everybody under the age of
30 seems to be switching to universal donor. I'm trying to convince
people that the Napster style civil disobedience approach is NOT
HELPING. "No license specified" being the #1 license on github isn't
laziness, it's because "kids these days" (and Dan Berstein) lump
together copyright and patent as too dumb to live when applied to
anything digital so you just ignore it until it dies, thanks so much
RIAA/MPAA/FSF...

Alas, "should I do 2 clause BSD, or 3 clause, or MIT, or Apache, or..."
leads us right back to no license specified territory for the people
who don't want to be a lawyer. (Do these slight wording differences
matter? Do they INTERACT?) So my message to them is "Explicit public
domain is a viable option". People who want to view a simple permission
statement like "Permission to use, copy, modify, and/or distribute this
software for any purpose with or without fee is hereby granted." as a
license are free to do so, and the rest of us can remember Jim
Butterfield's disk bonus pack of public domain software bundled with
the commodore 64 in 1982 and sold retail at Sears and how he _wasn't_
eaten by wolves. Nor were the authors of libtomcrypt and lzma. You
don't have to be clever about it, you can just get the code out there.

Anyway: yeah, open source licensing is in flux right now. The FSF did
one of those petard self-hoisting things and the rest of us have to
clean up the mess. But there's a reason BSD-licensed projects like
http://musl-libc.org and http://ellcc.org took off in the past few
years.
Post by Ranjit Singh
Even more remarkable given that Android is an application of
Linux, a GPL component if ever I saw one,
Hence "in userspace". And there are some components grandfathered in,
but if you add any more you violate the trademark licensing guidelines
and can't describe the result as "Android" in your marketing materials.
Post by Ranjit Singh
and one which requires
gmake to build, last I heard. I guess tivoization is alive and
kicking.
I've seen people try hard to stick with the last GPLv2 release until
the replacement's ready. MacOS X was on gcc 4.2.1 with binutils 2.17
for a long time until they sponsored enough LLVM work to use the
result. (Qualcomm also sponsored some Open64 work, which is a
GPLv2-only C compiler, but I think their motivation was "it's
implemented in C, not C++". They seem to have fallen in line behind
llvm now...)

(Of course the FSF responded to people sticking with the last v2
releases of stuff by deleting the binutils 2.17 tarball off its'
servers and replacing it with a tarball containing GPLv3 source code,
because that's how they define freedom. Some of us had it cached and
noticed when the sha1sum changed, but the real fix is for
http://lld.llvm.org/ to mature to the point http://ellcc.org can use
it.)

Rob
Joerg Schilling
2013-10-13 11:35:01 UTC
Permalink
Post by David A. Wheeler
Post by Joerg Schilling
As mentioned, we would just need pattern macro substitutions.
I think most people would be very unhappy with that.
Almost all make implementations include if-then-else make conditionals, including GNU make, makepp,
FreeBSD/OpenBSD/NetBSD make, Microsoft nmake, AT&T/Alcatel-Lucent nmake, and fastmake.
Makefile generators, like automake and Cmake, also support make conditionals.
Why do you like to introduce something that is obviously not agreed on while
something that is agreed on would just need to be put into the standard?
Post by David A. Wheeler
https://docs.google.com/document/d/1oUR7iMnaNzkeT3TTOS-Gwul6_V3TE8caIDAd1FwPyNc/edit?usp=sharing
I think it comes down to either (1) supporting GNU make style "ifeq"/"ifdef" syntax, or
(2) supporting *BSD make style ".if" syntax. GNU make is more popular, but
slightly clunky and doesn't begin with the reserved ".".
I think that all make maintainers would be very unhappy if you like to
standardize something that people do not agree on. This is a gmake specific
thing and I would not say that gmake is popular. It seems that rather people
are forced to use gmake because a Linux dominated development creates
nonportable makefles by using vendor specific extensions without looking at
portability.

Make is a language that was designed without if-then-else but tht has other
constructs that allow conditionals.

Let us standardize what's available already and document how to use this for
conditionals.

Jörg
--
EMail:joerg-3Qm2Liu6aU2sY6utFDHCwYAplN+***@public.gmane.org (home) Jörg Schilling D-13353 Berlin
js-CFLBMwTPW48UNGrzBIF7/***@public.gmane.org (uni)
joerg.schilling-8LS2qeF34IpklNlQbfROjRvVK+***@public.gmane.org (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Bruce Korb
2013-10-14 16:40:17 UTC
Permalink
Post by Joerg Schilling
I think that all make maintainers would be very unhappy if you like to
standardize something that people do not agree on. This is a gmake specific
thing and I would not say that gmake is popular. It seems that rather people
are forced to use gmake because a Linux dominated development creates
nonportable makefles by using vendor specific extensions without looking at
portability.
I believe David allowed that the BSD syntax would be okay,
but that the GNU syntax was a bit closer to common.
My two cents worth is that:

a) it is highly useful to have conditionals in make, and
b) the BSD syntax is cleaner.

but I don't care as much about syntax as about the facility.
Even if there are convolutions that will boil down to the
desired effect, clean syntax is important. (So I like that
BSD syntax is readily distinguished from other make file tokens.)
David A. Wheeler
2013-10-11 17:44:47 UTC
Permalink
Post by Schwarz, Konrad
Post by David A. Wheeler
I read the text several times and could not convince myself
that the spec really required this. That could be a problem with me,
though, and not with the spec :-).
Post by Schwarz, Konrad
The proposed addition is just special case of the (quoted) existing
text --
...
Post by Schwarz, Konrad
Unfortunately, you snipped the part of my letter which shows that
it does.
Sorry, I didn't mean to. I quoted:
"If there are no commands listed for the target, the target shall be
treated as up-to-date."

Konrad Schwarz then said:
"The proposed addition is just special case of the (quoted) existing text --
make imagines this target to have been updated regardless of whether
it has prerequisites or whether it exists or not. And targets are
updated when one of their prerequisites has been (or is treated as) updated."

So once target 1 with no prereqs or command is treated as up-to-date,
the rest of the make spec implies that all other targets that depend on target 1
will execute their commands (in turn) because target 1 has been "updated".
My original concern is that a reader might think that they don't need to update anything
that depends on target 1. But the text does say that just THAT target shall
be treated as up-to-date, not anything else, so that may be an absurd concern.

Does anyone object to this interpretation?
If that's the general understanding, then I guess there's no need to
change the spec, and I could withdraw the proposal.
I appreciate the comments.

--- David A. Wheeler
Loading...