HOWTO Autoconfiscate a Project into the GNU Autotools
Written by Bob Proulx <bob@proulx.com>
License
Copyright (C) 2003, 2005, 2006, 2009 Bob Proulx
This document is free; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You can get a copy of the GNU GPL at at <http://www.gnu.org/copyleft/gpl.html>.
The purpose of this document
When I started to learn the GNU autoconf and automake tools I searched the documentation and read all that I could find about them. The Goat book[1] is a good source of information. Also the online texinfo documentation is quite good. However, I found that in all of the existing documentation, HOWTOs, books, and examples that they all dealt with pieces of larger and more complicated projects. Since the projects were larger the examples from them were not complete examples. It was hard to get going. There were not simple examples that someone could start out small with and learn the process. And they all left out some of what I felt were important details which were needed to really understand the tools. That was my incentive to write this mini-HOWTO.
The intended audience are those just getting started with autoconf and automake as a developer. This is intended for people who have been building projects that others have created. They are familiar with the basic operation of running configure and make to build and install packages but have not been creating those packages themselves. Because of that I am making several simplifying assumptions. I am sure the experienced developer will find details which I have either ignored or treated in only the barest way. That is intended. This is a first introduction to using autoconf and automake as a developer and not a second book on the subject.
Versions of the autotools
I wrote and tested this document with autoconf-2.59 and automake-1.8.4. Certainly other versions will work. Newer versions are preferable. But older versions will be less desireable due to behavior differences.
Getting started
Let's jump right in with a small program. Let's assume a top level directory with a 'src' subdirectory below it. This is a good structure for most sizes of projects. Even if you only have a very small project I recommend this directory structure. Therefore I am going to push you into learning how to develop with this type of directory structure right from the beginning to encourage forming good habits. It can grow and this style is a good one which grows easily. Later we will add 'lib', 'doc', and 'test' subdirectories but for now let's start small. A single directory with source code in it.
Following tradition, here is the program we are going to package. Unless you have one of your own place this in a file main.c.
#include <stdio.h>
int main()
{
printf("hello\n");
return 0;
}
Your directory structure should now look like this.
hello hello/src hello/src/main.c
Creating the automake files
We are now ready to start the process of "autoconfiscating" this project. Let's start in the source directory and get the Makefile set up there. Create the Makefile.am file for the autotools to use to build the Makefile file. Those are created from Makefile.am files by automake. Well, not precisely. The automake program produces Makefile.in files using the Makefile.am files and the autoconf program will process Makefile.in files into Makefiles. Each program has a part in the process.
In the Makefile.am file place the name of the binary exectuable to be produced in the 'bin_PROGRAMS' whitespace delimited list variable. Place the sources in the 'progname_SOURCES' list. (The file processing is done with string manipulation. The name of the program becomes the name of the sources list. This allows you to have multiple build targets in the same directory because they will be named differently.)
bin_PROGRAMS = hello hello_SOURCES = main.c
For our hello world program that is all we need. That is the entire contents of the file. If there were more sources they would all be listed there in the file. Simple enough.
Your directory structure should now look like this.
hello hello/src hello/src/Makefile.am hello/src/main.c
Every directory from the top down in a hierarchy of build files will need a Makefile[2]. Therefore we need to create another Makefile.am file in the top level of our project directory tree to be converted to a Makefile. Our Makefiles will recursively call the Makefiles of subdirectories of this project. We do that by using the SUBDIRS variable to list out all of the directories which are part of this project. Ours will start with simply the 'src' directory.
SUBDIRS = src
Hey, that was not so bad. We have the top level Makefile.am done. We can see that it sets one option. It lists the subdirectories of the project.
Your directory structure should now look like this.
hello hello/Makefile.am hello/src hello/src/Makefile.am hello/src/main.c
We are completely through with automake. Were you expecting more effort? Half of the process is complete.
Creating the autoconf files
Create a file in the top level of your project named 'configure.ac'. Place the following text in the file.
dnl Process this file with autoconf to produce a configure script. AC_INIT([hello],[1.1]) AM_INIT_AUTOMAKE([-Wall]) AC_PROG_CC AC_PROG_RANLIB AC_CONFIG_FILES([ src/Makefile Makefile ]) AC_OUTPUT
Autoconf is driven by the m4 text processor. These files are really just m4 files. The 'dnl' string is an m4 comment. It tells m4 to delete through newline. That line and the newline are deleted. This is slightly differenent than a '#' comment. The '#' comments are passed through to the later files but the 'dnl' comments are not seen in the downstream files. This file is used to create the configure script.
The AC_INIT macro initializes the autoconf system. It takes as arguments the name of the package or program plus the version number of it. I have fully quoted all arguments. The quote characters are the square brackets which you may think of as "" in shells scripts.
The AM_INIT_AUTOMAKE macro initializes the automake system. As it happens automake is designed to work with autoconf and therefore reads the same files.
The purpose of autoconf is to automatically determine many aspects of the compilation machine. The macros listed invoke the minimum needed to deduce the commands that automake uses internally. It is possible and likely that you will use other commands too. In which case the list shown here would grow to include your other commands which might vary from system to system. But for now we will just go with the minimum. This covers the list of tools used by automake in the Makefile.in file.
The last is to actually use an "output" function. The AC_OUTPUT macro processes the listed files. It looks for a .in version of the files and processes the .in file into a file named without the .in. Example AC_OUTPUT([Makefile]) processes Makefile.in into Makefile. The .in file may have many symbolic names which will be expanded. This is a big 'sed' like edit operation which changes @VARIABLE@ style strings into the autoconf determined values. That is, @CFLAGS@ will be converted into "-g -O" or other appropriate text.
Your directory structure should now look like this.
hello hello/Makefile.am hello/configure.ac hello/src hello/src/Makefile.am hello/src/main.c
We are now done with the majority of the autoconfiscation process. We have covered automake files and autoconf files.
Conforming to the GNU Standards
The GNU standards require a certain amount of structure for all of their projects. GNU programs are required to have specific documentation files in place. This is a Good Thing. It provides a consistent framework for disparate projects. Let's take care of this right now. These are AUTHORS, COPYING, ChangeLog, INSTALL, NEWS, and README. The use and contents of those files should be obvious.
You have two choices. The first and best choice is to follow the GNU standards and to create these files placing the appropriate documentation in each file. Any documentation is better than no documentation. Even a few lines of text pointing to other locations is better than no information at all.
The second choice is to set the 'foreign' option to automake. You can do this by putting "AUTOMAKE_OPTIONS = foreign" in the top level Makefile.am file. The 'foreign' option relaxes the rules. With the 'foreign' option enabled the docs say, "Automake will check for only those things which are absolutely required for proper operations." But that is not the normal mode of operation. I won't recommend it. But it is available to use it to simplify the process to the bare minimum possible.
Your directory structure should now look like this.
hello hello/AUTHORS hello/COPYING hello/ChangeLog hello/INSTALL hello/Makefile.am hello/configure.ac hello/NEWS hello/README hello/src hello/src/Makefile.am hello/src/main.c
Generating the configure script and Makefiles
How do we get the process going? There are Makefile.am files and the configure.ac file but still no Makefiles. Normally we use a configure script to create the Makefiles but we don't have that script yet. How do we bootstrap the process?
The 'autoreconf' tool is specificially designed to run the other autotools in the right sequence to create and bring up to date all of the generated files. Use 'autoreconf' to auto-magically determine what needs to be reconfigured. Have it install the rest of the scripts which the autotools system uses. This will create the configure script which will be used to configure the project.
autoreconf --install
This will create many other files in your directory. Those are all part of the autotools infrastructure. One file which will always be created is the "configure" script. That script will drive the rest of the process. Run the configure script now to create your Makefiles.
./configure
You should now have a complete set of Makefiles for the entire project. Each Makefile has all of the standard targets for building programs, testing, installing and making distributions. You should build your program.
make
Perhaps you will want to turn off optimization to make debugging of the program more clear? In that case specify the flag on the configure line. Specifying this on the command line will record the options used in the config.status file and will invoke the same options again if the config.status script is invoked.
./configure CFLAGS=-g make clean make
Making a distribution
Speaking of distributions, how do you make one? By building the system fully and then using the 'distcheck' target. This creates a versioned tar file of all of the files. The distcheck target runs 'make dist' first to create the distribution and then checks it to make sure it is all really there. You should always run distcheck to ensure that everything is consistent and ready for distribution.
make distcheck
Time to stop and take another breath. If all of the above went well for you then you are done. You have a working autoconf/automake based configure and build process. It is a bare minimum structure. But a good place to start. Take some time right now to play with your system. Browse the files and get familiar with them.
Going further
Hopefully you are now comfortable with a bare bones autoconfiscation of a program. But we are not done yet. We can do better than just a minimal system.
Building from clean source
A common and recommended methodology is to keep all of your source files in revision control but not to check in any generated files[3]. Every so often you will want to make sure can build your program using only the files from clean source. Otherwise if there is a problem with your configuration the only way you will know is when downstream builders of your project run into the problem and gripe to you with bug reports. You want to be able to find those problems yourself first before sending your package out.
A clean check out of sources yields a flow similar to this.
...check out clean sources... autoreconf --install make
This is exactly what the maintainer-clean target is designed to do for you too. If it is properly configured it will remove all of the generated files and allow you to quickly test your system without the trouble it takes to start from just checked out sources. However, it is always possible that things will not be properly configured. Crosschecking with a fresh checkout is a more stringent test.
Automatically running the autotools commands
Starting with a clean copy of the source files and building should be a routine part of all projects. But being routine it also is easy to forget to run 'autoreconf --install' first to create the configure script. In that case the make program will complain that it can't find a Makefile. But of course you need to run autoreconf and configure to generate it. Do'h!
make make: *** No targets specified and no makefile found. Stop. ...Do'h!... autoreconf --install make make: *** No targets specified and no makefile found. Stop. ...Do'h**2!... ./configure make
Normally after unpacking a tar.gz distribution archive you will have a configure script but you will not have any Makefiles. They have not been built on your system yet. If you type 'make' you will get an error message saying no makefile found. But you will have a configure script. The configure script will make the Makefiles.
tar xzvf hello-1.1.tar.gz && cd hello-1.1 make make: *** No targets specified and no makefile found. Stop. ...Do'h!... ./configure make
It is possible to detect these cases and Do The Right Thing automagically. This is not strictly necessary. But this avoids the minor frustration experienced when it is forgotten. This also only works with GNU make.
If you are using GNU make then the GNUmakefile will be highest priority makefile. In it we can place commands which will automatically run autoreconf and configure. This is not a requirement of the GNU system. But it is a nice feature. It automatically does the right thing in the case that the user unpacks a distribution and types 'make'.
Here is one possible implementation of a GNUmakefile which implements this feature.
# This makefile is used only if you run GNU make.
# This provides a convenience of automatically running configure
# if it has not previously been run.
SHELL = /bin/sh
have-Makefile := $(shell test -f Makefile && echo yes)
have-configure := $(shell test -f configure && echo yes)
ifeq ($(have-Makefile),yes)
include Makefile
else
ifeq ($(have-configure),yes)
all:
@echo "There seems to be no Makefile in this directory."
@echo "Running ./configure before running 'make'."
./configure
@$(MAKE)
else
all:
@echo "There seems to be no Makefile in this directory."
@echo "There seems to be no configure script in this directory."
@echo "Running 'autoreconf' to generate the configure script."
autoreconf --install
@$(MAKE)
endif
endif
# Tell make to not build goals in this directory in parallel. This
# is necessary in case someone tries to build multiple targets on
# one command line.
.NOTPARALLEL:
Your directory structure should now look like this.
hello hello/AUTHORS hello/COPYING hello/ChangeLog hello/INSTALL hello/GNUmakefile hello/Makefile.am hello/configure.ac hello/NEWS hello/README hello/src hello/src/Makefile.am hello/src/main.c
Note that there will be additional files which you have generated. But this is the list of files that you have created yourself for this project.
The GNUmakefile is an extra file to be distributed with your project. It needs to be added to the list of files in the EXTRA_DIST variable so that it will be included in distributions of your package. Edit your top level Makefile.am and add it there. The variable will look like this.
EXTRA_DIST = GNUmakefile
Now with these additions in place a user can unpack a tar.gz distribution, or a maintainer can check out fresh sources, and in both cases only a single 'make' command is needed to completely configure and build the system. Here is an example from the perspective of a user unpacking your project.
tar xzvf hello-1.1.tar.gz cd hello-1.1 make
Ah. Isn't that nice?
Configuring a single source tree for different options
Some people desire to configure a single source tree in multiple different ways. One build tree may have a debug option while another will have maximum optimization while a third may have profiling enabled. In order to have this all come from the same source files the object files must be built in their own directory tree.
The configure script supports that capability. Instead of running configure in the top level source directory run it in your top level build directory. This will create all of the Makefiles in a separate build directory. It can only be done if the source directory has not been frozen into a previous configuration. The 'distclean' target will reset things to a distribution clean state.
make distclean mkdir ../hello-build-debug cd ../hello-build-debug ../hello-1.1/configure CFLAGS=-g make
Wow. That was a lot. But you now have a fully automated configure and build system.
Adding documentation
Our program to produce a friendly greeting is functional. But where is the documentation? Every program should have some documentation. The GNU standards do not require a man page but instead institute texinfo documentation. But other standards such as the Debian standards do require man pages. I personally like man pages for syntax reference and info pages for expanded user manuals.
Let's start with a man page. I won't provide an example man page. There are many available. Create a man page for your program and for this example call it 'hello.1'.
We also need to include the rules for the man page in the Makefile.am file. Edit your Makefile.am file in your source directory. Put the man page in a man_MANS variable.
man_MANS = hello.1
Your directory structure should now look like this.
hello hello/AUTHORS hello/COPYING hello/ChangeLog hello/INSTALL hello/GNUmakefile hello/Makefile.am hello/configure.ac hello/NEWS hello/README hello/src hello/src/Makefile.am hello/src/main.c hello/src/hello.1
This will include the man page in distributions and produce rules to install it along with your executables. This is the simple case.
Actually I recommend using 'help2man' to generate your man page from your program's online usage documentation. But that is another tutorial. Read the documention provided with 'help2man' for details of how to do use it with your program.
Using a library
Our program has started out small. But over time we find that we are always greeting users with "hello" and wish to place this code in a library so that our other programs can share this functionality. We need a common library of greeting functions which can by used by our expanded system. Make a new directory for the library.
Here is the new hello.h file.
#ifndef hello_c_INCLUDED #define hello_c_INCLUDED void hello(void); #endif
Here is the new hello.c file.
#include <stdio.h>
#include "hello.h"
void hello(void)
{
printf("hello\n");
}
Here is the new main.c file.
#include <stdio.h>
#include "hello.h"
int main()
{
hello();
printf("working\n");
return 0;
}
This new library directory needs a Makefile therefore we create a Makefile.am for automake to process. This library is not going to be installed in the system /usr/lib directory. We just need it for our project. Therefore it is a "noinstall" library.
Here is the new library Makefile.am file.
noinst_LIBRARIES = libhello.a libhello_a_SOURCES = \ hello.c hello.h
Your directory structure should now look like this.
hello hello/AUTHORS hello/COPYING hello/ChangeLog hello/INSTALL hello/GNUmakefile hello/configure.ac hello/Makefile.am hello/NEWS hello/README hello/lib/Makefile.am hello/lib/hello.1 hello/lib/hello.c hello/lib/hello.h hello/src hello/src/Makefile.am hello/src/main.c
The files are in place, the library Makefile.am is done. But the top level Makefile.am needs to know to build the library. And the configure.ac file needs to know to process the library Makefile.in file. You can probably write these two without help. Add the new lib directory to the top level Makefile.am SUBDIRS variable. Add the library Makefile to the configure.ac.
Your top level Makefile.am file would look like this.
SUBDIRS = lib src EXTRA_DIST = GNUmakefile
Your configure.ac file would look like this.
dnl Process this file with autoconf to produce a configure script. AC_INIT([hello],[1.1]) AM_INIT_AUTOMAKE([-Wall]) AC_PROG_CC AC_PROG_RANLIB AC_CONFIG_FILES([ lib/Makefile src/Makefile Makefile ]) AC_OUTPUT
And we need to tell the program directory Makefile.am to include extra compile and link options to the program so that it can use the library.
Your source directory Makefile.am file should look like this.
bin_PROGRAMS = hello hello_SOURCES = main.c AM_CPPFLAGS = -I$(top_srcdir)/lib -I$(top_srcdir) LDADD = ../lib/libhello.a
A few important things to note about this example. One is that we always reference the source directory through $(top_srcdir) and not a relative path. We will be building in a build directory and the source may be a long ways away from us. But the targets we build will be in our build area and so for any target we build we will reference it through a relative path. The library is a target we built and needs to be referenced using a relative path.
You will need to reconfigure to bring all of the generated Makefiles and configure script up to date. Run 'autoreconf'.
autoreconf ./configure make
Everything has been done. You now have a multiple directory build system that uses a library.
Adding defines to your compilation
It is possible that the program you are working with requires defines. Probably not your program that you wrote. But sometimes other programs will need to have special flags added to the compilation. Let me contrive an example. A program needs the define 'foo' set to a value. You want to pass the option -Dfoo=bar to the program when it compiles. This is a CPPFLAGS option and should go there. Set it like this in the Makefile.am file.
AM_CPPFLAGS = -I$(top_srcdir)/lib -I$(top_srcdir) -Dfoo=bar
AM_CPPFLAGS? What is that? Why do we use AM_something? You may have been expecting to set CPPFLAGS in the Makefile.am file but you can't do that. Why not? The answer is because those are user variables.
Run './configure --help' on any of the configure based programs you have available. You will see this in the documentation.
Some influential environment variables:
CC C compiler command
CFLAGS C compiler flags
LDFLAGS linker flags, e.g. -L<lib dir> if you have libraries in a
nonstandard directory <lib dir>
CPPFLAGS C/C++ preprocessor flags, e.g. -I<include dir> if you have
headers in a nonstandard directory <include dir>
CPP C preprocessor
Which means those variables are user variables. You are not allowed to overwrite them with your own settings. Respect the user override capability. That means that every variable has its own automake name which gets appended. In this case AM_CPPFLAGS instead of CPPFLAGS is what you set in the Makefile.am and it will be handle properly.
You could set all include and link options with AM_CPPFLAGS and AM_LDFLAGS. But those are more of a last resort for programs that are special and need them. Normally you would use the AM_CPPFLAGS and LDADD variables which are designed for that purpose. But this need is common enough among programs that are not natively developed with the autotools in mind that I am mentioning it expicitly.
Footnotes
[1] Homepage for GNU Autoconf, Automake and Libtool, also known as "The Goat Book" by Gary V. Vaughan, Ben Elliston, Tom Tromey and Ian Lance Taylor, first published in October 2000 by New Riders publishing but also freely available on the web at https://web.archive.org/web/20091218163351/http://sources.redhat.com/autobook/ .
[2] I am not going to talk about "Recursive make considered harmful".
[3] Alexandre Duret-Lutz <duret_g@lrde.epita.fr> has written an excellent FAQ entry on this subject. At this time it is not published and my best pointer to it is here, http://mail.gnu.org/archive/html/automake/2003-02/msg00036.html.
For more information
Here is the online gnu documentation for autoconf. http://www.gnu.org/software/autoconf/manual/