diff --git a/docs/Contributing.md b/docs/Contributing.md index 38615fd..011de6e 100644 --- a/docs/Contributing.md +++ b/docs/Contributing.md @@ -17,7 +17,7 @@ have a copy of redo. Nowadays, redo is good enough for real production use, and some people are using it for real work. That said, it has -not reached version 1.0 and there are surely still bugs. +not reached version 1.0 and there are always going to be bugs. If you run into a problem, it's really helpful if you report it to the mailing list below (with or without subscribing first). We really want to @@ -29,21 +29,21 @@ Small feature additions are also welcome, but you might want to ask on the mailing list before you start working on it. The code is still evolving and might not be the same by the time you submit your pull request. -The best things you can do for redo are: +Contributions to [redoconf](/cookbook/redoconf-simple/) are very welcome. +For example, you might want to contribute detectors for new libraries or +compilers. + +Some of the best things you can do for redo are: - Convert your projects to using it. Without users, no project is successful. -- Build new infrastructure around redo, especially things to make it easier - for people to get started. For example, an automake-like tool that filled - in default redo build rules for common program types would probably be - very popular. - - Convince your favourite OS distro to build and include (up to date!) redo - packages. + packages. We especially need someone to help with getting redo included + in debian. - Help proofreading this documentation, and send patches or pull requests - with improvements. + with improvements and additions. - Join the mailing list (below) to discuss design changes or bugs. @@ -52,7 +52,7 @@ The best things you can do for redo are: # Mailing list -You should join the `redo-list@googlegroups.com` mailing list. +You should join the `redo-list@googlegroups.com` mailing list! You can find the mailing list archives here: diff --git a/docs/FAQBasics.md b/docs/FAQBasics.md index 8fd4b7e..471af23 100644 --- a/docs/FAQBasics.md +++ b/docs/FAQBasics.md @@ -35,16 +35,11 @@ specific to ruby programs, or python programs, or Java or .Net programs. redo isn't like those systems; it's more like make. It doesn't know anything about your system or the language your program is written in. -The good news is: redo will work with *any* programming language with about -equal difficulty. The bad news is: you might have to fill in more details -than you would if you just use ANT to compile a Java program. - -So the short version is: cross-platform builds are about equally easy in -make and redo. It's not any easier, but it's not any harder. - -It would be possible to make an automake-like or cmake-like tool that -generates .do files for your project, just like automake generates -Makefiles. But that's beyond the scope of redo itself. +However, there is a new project called [redoconf](/cookbook/redoconf-simple/) +which is now part of the redo distribution. It works kind of like +`autoconf` does with make; drop it into your project and it will help with +auto-detection, cross-compiling, and portability, so you can concentrate on +actually writing your program. # Can I set my dircolors to highlight .do files in ls output? diff --git a/docs/FAQInterop.md b/docs/FAQInterop.md index 8070e24..72bd1ed 100644 --- a/docs/FAQInterop.md +++ b/docs/FAQInterop.md @@ -1,13 +1,17 @@ # Is redo compatible with autoconf? -Yes. You don't have to do anything special, other than the above note about -declaring dependencies on config.h, which is no worse than what you would -have to do with make. +Yes. You don't have to do anything special, other than making sure you +`redo-ifchange config.h` (where `config.h` is generated by autoconf) in +whatever redo script you use to compile your object files. This +is about the same as what you would have to do with make. # Is redo compatible with automake? -Hells no. You can thank me later. But see next question. +Not exactly; there is no point in generating Makefiles if you aren't going +to use make. However, we now include a similar project, +[redoconf](/cookbook/redoconf-simple/), which provides a lot of the same +features in a redo project that automake provides for a make project. # Is redo compatible with make? @@ -22,7 +26,7 @@ In a file called myproject.stamp.do: make -C myproject all So, to amend our answer to the previous question, you *can* use -automake-generated Makefiles as part of your redo-based project. +automake-generated Makefiles as part of your hybrid redo/make-based project. # Is redo -j compatible with make -j? diff --git a/docs/Roadmap.md b/docs/Roadmap.md index 9c7666a..de8c602 100644 --- a/docs/Roadmap.md +++ b/docs/Roadmap.md @@ -309,21 +309,3 @@ consider bundling redo with a particular (optional) sh implementation, and maybe also unix-like tools, that it will use by default. An obvious candidate would be busybox, which has a win32 version called [busybox-w32](https://frippery.org/busybox/). - - -### redoconf - -redo is fundamentally a low-level tool that doesn't know as much about -compiling specific programming languages as do higher-level tools like -[cmake](https://cmake.org/). - -Similarly, `make` doesn't know much about specific programming languages -(and what it does know is hopelessly out of date, but cannot be deleted or -updated because it would break backward compatibility with old Makefiles). -This is why `autoconf` and `automake` were created: to automatically fill in -the language- and platform-specific blanks, while letting `make` still -handle executing the low level instructions. - -It might be useful to have a redo-native autoconf/automake-like system, -although you can already use autoconf with redo, so this might not be -essential. diff --git a/docs/cookbook/redoconf-simple/.gitignore b/docs/cookbook/redoconf-simple/.gitignore new file mode 100644 index 0000000..fd66d9f --- /dev/null +++ b/docs/cookbook/redoconf-simple/.gitignore @@ -0,0 +1,2 @@ +/out +/out.* diff --git a/docs/cookbook/redoconf-simple/all.do b/docs/cookbook/redoconf-simple/all.do new file mode 100644 index 0000000..ddc8e33 --- /dev/null +++ b/docs/cookbook/redoconf-simple/all.do @@ -0,0 +1,15 @@ +# Run from the source dir. +# +# We'll make sure the out/ dir exists and that +# a C++ compiler is available, then redo out/all, +# which is implemented in the file all.od. +# +# Note that a "normal" project might not have an all.do +# at all; the end user would be expected to make an +# output dir, run ../configure, and then redo from there. +# But we want this file to build as part of the redo +# examples, so there needs to be a toplevel all.do in +# each example. +# +. ./skip-if-no-cxx.sh +redo-ifchange out/all diff --git a/docs/cookbook/redoconf-simple/all.hpp b/docs/cookbook/redoconf-simple/all.hpp new file mode 100644 index 0000000..7c78212 --- /dev/null +++ b/docs/cookbook/redoconf-simple/all.hpp @@ -0,0 +1,11 @@ +// Precompiled header. +// +// Include your most commonly used, but least +// frequently changing, header files here, to +// speed up compilation. +#ifndef __ALL_HPP + +#include +#include + +#endif // __ALL_HPP diff --git a/docs/cookbook/redoconf-simple/all.od b/docs/cookbook/redoconf-simple/all.od new file mode 100644 index 0000000..8fb0446 --- /dev/null +++ b/docs/cookbook/redoconf-simple/all.od @@ -0,0 +1,3 @@ +# This script gets called from inside the out/ +# directory when someone runs "redo" or "redo all". +redo-ifchange hello diff --git a/docs/cookbook/redoconf-simple/all.rc.od b/docs/cookbook/redoconf-simple/all.rc.od new file mode 100644 index 0000000..eba15fd --- /dev/null +++ b/docs/cookbook/redoconf-simple/all.rc.od @@ -0,0 +1,34 @@ +# Specify which redoconf modules to use when building +# our program. +# +# The rc_include function is available in every *.od +# script in a redoconf project. +rc_include \ + rc/CXX.required.rc \ + rc/Wextra.rc \ + rc/all.hpp.precompiled.rc \ + rc/openssl__ssl.h.rc \ + rc/openssl__opensslv.h.rc \ + rc/libssl.rc \ + rc/libm.rc + +# We'll link *all* our binaries with the detected +# $LIBSSL by default. We rc_included rc/libssl.rc +# instead of rc/libssl.required.rc, so if it's +# not available, this line won't add any +# libraries. It'll be up to the program to check +# for HAVE_LIBSSL before trying to call it. +# +# If we wanted to only link $LIBSSL into certain +# binaries, we would include $LIBSSL explicitly in +# the .list.od file for those binaries. Since we're +# putting it here, it doesn't have to be in the .list +# file. +rc_appendln LIBS "$LIBSSL" + +# Combine all the redoconf settings we've collected +# above, and write them to $3 (all.rc). all.rc is +# a special .rc file that is used when generating +# the automatic ./compile and ./link scripts, among +# others. +rc_save diff --git a/docs/cookbook/redoconf-simple/clean.do b/docs/cookbook/redoconf-simple/clean.do new file mode 100644 index 0000000..996a074 --- /dev/null +++ b/docs/cookbook/redoconf-simple/clean.do @@ -0,0 +1,2 @@ +rm -f *~ .*~ +rm -rf out out.* diff --git a/docs/cookbook/redoconf-simple/configure b/docs/cookbook/redoconf-simple/configure new file mode 100755 index 0000000..736916f --- /dev/null +++ b/docs/cookbook/redoconf-simple/configure @@ -0,0 +1,3 @@ +#!/bin/sh +S="$(dirname "$0")" +. "$S/redoconf/configure.sh" diff --git a/docs/cookbook/redoconf-simple/configure.help b/docs/cookbook/redoconf-simple/configure.help new file mode 100644 index 0000000..f1f54f1 --- /dev/null +++ b/docs/cookbook/redoconf-simple/configure.help @@ -0,0 +1,15 @@ +# Automatically generated by redoconf/_all.rc.od - do not edit +ARCH Architecture prefix for output (eg. i686-w64-mingw32-) +CC C compiler name (cc) +CPPFLAGS Extra C preprocessor flags (eg. -I... -D...) +CFLAGS Extra C compiler flags (eg. -O2 -g) +OPTFLAGS C/C++ compiler flag overrides (eg. -g0) +LINK Linker name (cc) +LDFLAGS Extra linker options (eg. -s -static) +LIBS Extra libraries to always link against (eg. -lsocket) +STATIC Link libraries and binaries statically +LIBSSL Extra linker options for 'libssl libcrypto' +LIBM Extra linker options for 'libm' +CXX C++ compiler name (c++) +CXXFLAGS Extra C++ compiler flags (eg. -O2 -g) +PREFIX Change installation prefix (usually /usr/local) diff --git a/docs/cookbook/redoconf-simple/configured.do b/docs/cookbook/redoconf-simple/configured.do new file mode 100644 index 0000000..acb5b64 --- /dev/null +++ b/docs/cookbook/redoconf-simple/configured.do @@ -0,0 +1,9 @@ +# Ensure that an out/ directory exists and that +# it has been configured (ie. ../configure has been run). +[ -d out ] || (mkdir out && cd out && ../configure) + +# By declaring a dependency on this file *after* running +# configure, we can tell redo that reconfiguration is +# needed if this file ever disappears (for example, if +# the whole out/ directory disappears). +redo-ifchange out/default.do diff --git a/docs/cookbook/redoconf-simple/hello.cc b/docs/cookbook/redoconf-simple/hello.cc new file mode 100644 index 0000000..b4fe9ce --- /dev/null +++ b/docs/cookbook/redoconf-simple/hello.cc @@ -0,0 +1,13 @@ +#include "all.hpp" +#include "ssltest.h" + +using namespace std; + +int main() { + cout << "Hello, world!" + << endl + << "libssl version " + << hex << libssl_version() + << endl; + return 0; +} diff --git a/docs/cookbook/redoconf-simple/hello.list.od b/docs/cookbook/redoconf-simple/hello.list.od new file mode 100644 index 0000000..201d4a1 --- /dev/null +++ b/docs/cookbook/redoconf-simple/hello.list.od @@ -0,0 +1,14 @@ +rc_include all.rc + +# We include $LIBM here just as example of how to +# specify an auto-detected library as part of the +# linker settings for a binary. We don't actually +# use it for anything. Note that $LIBM was detected +# up above in the rc_include, because all.rc depends +# on rc/libm.rc. If the library is not available, +# $LIBM will be blank. +cat <<-EOF + hello.cc + ssltest.c + $LIBM +EOF diff --git a/docs/cookbook/redoconf-simple/index.md b/docs/cookbook/redoconf-simple/index.md new file mode 100644 index 0000000..aafaee4 --- /dev/null +++ b/docs/cookbook/redoconf-simple/index.md @@ -0,0 +1,779 @@ +### redoconf: Platform detection and C/C++ programs + +In the redo source tree you can now find a companion toolkit, redoconf, +which provides a handy way to build C and C++ programs that work on +different Unix variants and Windows 10's WSL (Windows Subsystem for Linux), +including the ability to use cross compilers and to compile the same source +tree for multiple platforms in different output directories. + +(redo and redoconf probably also work on Cygwin or MSYS on Windows, but +these are less well tested.) + +In this tutorial, we'll construct a new cross-platform-capable C++ project +from scratch, step by step, using redo and redoconf. + +### Intended for copy-and-paste + +redoconf is intended to be used by copying it directly into your source +repository, not by adding it as an external dependency. Otherwise you have a +chicken-and-egg problem: redoconf is the dependency checking and platform +independence tool, so how will you check for redoconf on your user's +platform? + +redoconf also does not make backward-compatibility guarantees in the same +way that redo does. With redo, we try very hard to ensure that when a new +version comes out, all programs that used to build with redo will continue +to build just fine; all new features must not interfere with old behaviour. +With redoconf, we don't make this promise. Upgrading redoconf might break +your project in small ways, so you have to change a few things (hopefully +for the better). We still try not to break things, but it's not a promise +like it is with redo. + +In these ways (copy-and-paste design; no compatibility guarantees), redoconf +is very much like autoconf and automake, and that's on purpose. Those very +intentional design decisions have made those programs successful for decades. + +That said, we do depart slightly from how autoconf and automake work. Those +packages provide command-line tools which *generate* customized files into +your source tree. This was important because they depend on the M4 macro +processor, which not everyone has available, and moreover, they wanted to +use autoconf to configure the source code for GNU M4, so they had an even +deeper chicken-and-egg problem. + +But they didn't have redo. Because redo's interface is so simple - +implemented in +[minimal/do](/FAQBasics/#do-end-users-have-to-have-redo-installed-in-order-to-build-my-project) +in a few hundred lines of shell script - and redo is powerful enough to run +the entire build system, we can use redo instead of M4 macros. And we can +use minimal/do when building redo, which means we can use redoconf to build +redo, and there are no circular dependencies. + +Anyway, the net result is that instead of *generating* customized files into +your source project like autoconf does, you can just *copy* the redoconf +files verbatim into a redoconf subdirectory of your project, and later +update them to a new version later in the same way. + +Let's do that as the first step to building our C++ test project. If you +want to play with the completed results, get the [redo +source code](https://github.com/apenwarr/redo) and look in the +`docs/cookbook/redoconf-simple/` directory. (Note: in that directory, +redoconf is a symlink rather than a copy, because the redo source repo +already contains a copy of redoconf.) + + +### Paste redoconf into your project + +Let's get started! (Replace `/path/to/redo` in this example with the path +to your copy of the redo source code, or the directory containing the +redoconf you want to copy.) +```shell +$ mkdir redoconf-simple +$ cd redoconf-simple +$ cp -a /path/to/redo/redoconf redoconf +``` + +Now we have to provide a `configure` script. It doesn't do much other than +call into `configure.sh` inside the redoconf directory. +
+ +(One important job the configure script does is to provide a value for `$S`, +the path to the top of the source tree, relative to the output directory. +The convention with autoconf, which we copy here, is to always *run* +configure from the directory where you want to produce the output. Then the +path to the configure program - which is in `$0` - will also be the path to +the source directory. To make this work, you need a configure script to be +in the top level of your source directory.) + +As you can see, unlike autoconf, which generates a new configure script for +your specific project, our configure script is just a few lines of +boilerplate. + + +### Run ./configure + +redoconf is ready to go! Let's configure it and see what it can detect. +```shell +$ ./configure +./configure: run this script from an empty output directory. + For example: + (mkdir out && cd out && ../configure && redo -j10) +``` + +Oops. Unlike autoconf, redoconf strongly encourages you to create a +*separate* output directory from the rest of your source code. Autoconf +lets you get away with building your source and output in the same place. +As a result, users often don't even know it supports separate source and +output directories. But even worse, developers often don't *test* its +behaviour with separate source and output directories, which sometimes +causes them to break the feature by accident. To cut down on variation, +redoconf just always pushes you to use a separate output directory, and +gives you a hint for how to use it. Let's run the command it suggested, +except we'll skip the `redo` part since there is nothing to redo yet. +```shell +$ (mkdir out && cd out && ../configure) +``` + +No output. In the unix tradition, that means it worked. + +We've departed again from autoconf here. When you run an autoconf +configure script, it immediately tries to detect everything it needs. This +is necessary because autoconf is independent of make; it wants to detect +things *before* you run make, and is usually responsible for generating your +Makefile. + +On the other hand, redoconf fundamentally depends on redo - or at least +[minimal/do](/FAQBasics/#do-end-users-have-to-have-redo-installed-in-order-to-build-my-project). +That's mostly good news because, for example, it can run its detection +scripts in parallel using redo's parallelism features, thus making it much +faster than autoconf on modern machines. But on the other hand, it doesn't +want to force you to use any particular version of redo. You might use +minimal/do, or the "official" redo whose documentation you are reading +(sometimes known as apenwarr/redo), or some other compatible version. In +order to avoid that choice, the redoconf configure script just sets up the +output directory and then quits. + +So okay, if the detection is going to be done by redo anyway, why do we even +need a configure script? + +Good question! The reason is that redo scripts (.do files) don't take +command-line parameters. Redo tries very hard to make the build process for +each target completely repeatable, so that you can `redo path/to/whatever` +and have it build the `whatever` output file in exactly the same way as if +you did `cd path/to && redo whatever` or if `redo all` calls +`path/to/whatever` as one of its dependencies. + +When you're starting to build a project, though, sometimes you need to +provide command line options. One of them is the implicit one we mentioned +above: the current directory, at the time you run the configure script, is +recorded to be the output directory. And from there, we remember `$S`, +which is the relative path to the source tree. But there are other options, +too: +```shell +$ (cd out && ../configure --help) +Usage: ../configure [options...] [KEY=value] [--with-key=value] + + --prefix= Change installation prefix (usually /usr/local) + --host= Architecture prefix for output (eg. i686-w64-mingw32-) + --enable-static Link binaries statically + --disable-shared Do not build shared libraries + --{dis,en}able-optimization Disable/enable optimization for C/C++ + --{dis,en}able-debug Disable/enable debugging flags for C/C++ + -h, --help This help message + +No extra help yet: configure.help is missing. +``` + +This project doesn't have any files yet, so the option list is pretty short. +But a very important one is `--host`, which lets you specify which +cross-compiler to use. (The name `--host` is also copied from autoconf, and +has a very confusing history that dates back to the early gcc project. It +means the "host architecture" of the programs that we will produce. Despite +the confusion, we decided to stick with the autoconf naming convention here. +A name like `--target` might be more obvious, but in autoconf that means +something else entirely.) + +Anyway, to get started, we didn't provide any options, so we'll get the +defaults. The configure script then configured our output directory for us +by creating a few files: +```shell +$ ls out +Makefile _flags default.do@ redoconf.rc src + +$ cat out/_flags +# Auto-generated by ../configure + +$ cat out/src +.. + +$ cat out/redoconf.rc +# Automatically generated by ../configure +read -r S &2; exit 1) + CFLAGS= + CXXFLAGS='' + PCH1='' + PCH2='' + PCH3='' + ;; +esac +case $PCH in + 1) FLAGS_PCH=$PCH1 ;; + 2) FLAGS_PCH=$PCH2 ;; + 3) FLAGS_PCH=$PCH3 ;; +esac +. ./_compile + +$ cat _compile +#!/bin/sh -e +# Run the C/C++ compiler. +# Assumes config variables (CFLAGS, etc) are already set. +t="$1" d="$2" i="$3" +IFS=" +" +set -f +$CC -o "$t" \ + -MMD -MF "$d" \ + $CPPFLAGS $CFLAGS $CXXFLAGS $FLAGS_PCH $xCFLAGS $OPTFLAGS \ + -c "$i" +``` + +Let's not worry too much about these files - which are auto-generated anyway +- other than to note that they embed the actual values of things like `CC` +and `CPPFLAGS` that were detected in `rc/CC.rc`. The reason we do that is +because having the shell load our .rc files (sometimes recursively) can be a +little slow, and in a large project, we might end up doing it hundreds or +thousands of times. It's better to do the substitution once, and produce a +`./compile` script, than to repeat the substitutions over and over. It's +also convenient to know you can just run +`./compile hello.o.tmp hello.deps hello.cc` at any time and be +sure that it is always compiled the same way, with no impact from +environment variable settings. + +### Choosing the feature detectors for our project + +We cheated and skipped a step above. We can build .rc files individually, +but that doesn't automatically feed them into the `./compile` script, which +happens to only include `rc/CC.rc` by default (since it's absolutely +essential for compiling C programs). If we want to include other compiler +options, header files, and libraries, we'll need to tell redoconf the +complete set of detectors that are relevant to our project. We do that with +the special file `all.rc`, which is generated from `all.rc.od`. Let's +assemble one for our example project: +
+ +And here's the result: +```shell +$ redo all.rc +redo all.rc +redo rc/CXX.required.rc +redo rc/CXX.rc +Trying C++ compiler: 'c++' + CXX = 'c++' + LINK = 'c++' + HAVE_CXX = '1' +redo rc/Wextra.rc +redo rc/Wall.rc + CPPFLAGS += '-Wall' +redo rc/Wextra.rc (resumed) + CPPFLAGS += '-Wextra' +redo rc/all.hpp.precompiled.rc + CXXFLAGS_PCH_LANG += '-x +c++-header' + CXXFLAGS += '-Winvalid-pch' + CXXFLAGS_PCH += '-include +all.hpp' + CXXFLAGS_PCH_FPIC += '-include +all.hpp.fpic' + PRE_CXX_TARGETS += 'all.hpp.gch' + PRE_CXX_TARGETS_FPIC += 'all.hpp.fpic.gch' +redo rc/openssl__ssl.h.rc + HAVE_OPENSSL__SSL_H = '1' +redo rc/openssl__opensslv.h.rc + HAVE_OPENSSL__OPENSSLV_H = '1' +redo rc/libssl.rc +redo rc/pkg-config.rc + PKG_CONFIG = 'pkg-config' + HAVE_PKG_CONFIG = '1' +redo rc/libssl.rc (resumed) + CPPFLAGS += '' + LIBSSL += '-lssl +-lcrypto' + HAVE_LIBSSL = '1' +redo rc/libm.rc + HAVE_LIBM = '1' + LIBM += '-lm' +redo all.rc (resumed) + LIBS += '-lssl +-lcrypto' +``` + +Notice the slightly odd formatting when we append multiple words to the same +variable: there is a newline separating `-lssl` and `-lcrypto`, for example. + +Now that we have an `all.rc.od`, the `compile` script will have more stuff +in it. + +Let's look at some examples of the built-in detector scripts we're using. +You can find them all in the `redoconf/rc/` directory. This project won't +define any extra ones, but if you wanted to, you could put them in your own +`rc/` directory of your source tree. If there's an overlap between your +`rc/` directory and the `redoconf/rc/` directory, yours will override the +built-in one. So feel free to copy detectors from `redoconf/rc/` into your +own `rc/` and customize them as needed. + +Here's a script that just detects whether the compiler supports gcc's +`-Wextra` (extra warnings) option. If you use it, it also pulls in the +`-Wall` detector by default: +
+ +And here's the detector that checks if a given header file is available. +This is how we handle the request for `rc/openssl__ssl.h.rc` in `all.rc` for +example. It will define `HAVE_OPENSSL__SSL_H` (the two underscores mean a +`/` character in the path) if `openssl/ssh.h` exists.
+ +Here's one that detects libssl (both headers, for compile time, and +libraries, for link time) using pkg-config, if available: +
+ +...and so on. + + +### ../configure help messages + +Now that we've taught redoconf which detectors we want for our project, +and run through the detectors at least once, another file gets produced as a +side effect. This is the list of project-specific configuration options +available to `../configure`: +
+ +Which now show up in the `--help` output: +```shell +$ ../configure --help +Usage: ../configure [options...] [KEY=value] [--with-key=value] + + --prefix= Change installation prefix (usually /usr/local) + --host= Architecture prefix for output (eg. i686-w64-mingw32-) + --enable-static Link binaries statically + --disable-shared Do not build shared libraries + --{dis,en}able-optimization Disable/enable optimization for C/C++ + --{dis,en}able-debug Disable/enable debugging flags for C/C++ + -h, --help This help message + +Project-specific flags: + CC=... C compiler name (cc) + CFLAGS=... Extra C compiler flags (eg. -O2 -g) + CPPFLAGS=... Extra C preprocessor flags (eg. -I... -D...) + CXX=... C++ compiler name (c++) + CXXFLAGS=... Extra C++ compiler flags (eg. -O2 -g) + LDFLAGS=... Extra linker options (eg. -s -static) + LIBM=... Extra linker options for 'libm' + LIBS=... Extra libraries to always link against (eg. -lsocket) + LIBSSL=... Extra linker options for 'libssl libcrypto' + LINK=... Linker name (cc) + OPTFLAGS=... C/C++ compiler flag overrides (eg. -g0) +``` + +You should include the auto-generated `configure.help` file in your source +repository, so that end users will get the right help messages the first +time they try to build your project. + + +### Precompiled headers + +Redoconf supports the concept of precompiled headers, which can make +compilation of large C++ projects go a lot faster. You can have one +precompiled header for C++ source files and one for C source files. + +To make it work, we have to actually precompile the header, plus add some +compiler options to every other compilation to make it recognized the +precompiled headers. We activate all that for our header, `all.hpp`, +using the `rc/all.hpp.precompiled.rc` detector, which does this: +```shell +$ redo rc/all.hpp.precompiled.rc +redo rc/all.hpp.precompiled.rc + CXXFLAGS_PCH_LANG += '-x +c++-header' + CXXFLAGS += '-Winvalid-pch' + CXXFLAGS_PCH += '-include +all.hpp' + CXXFLAGS_PCH_FPIC += '-include +all.hpp.fpic' + PRE_CXX_TARGETS += 'all.hpp.gch' + PRE_CXX_TARGETS_FPIC += 'all.hpp.fpic.gch' +``` + +Notice that this works even though we haven't created our actual precompiled +header, `all.hpp`, yet! That's because the detector just makes sure the +compiler can handle precompiled headers; it doesn't precompile the header +itself. That's handled by redoconf's `default.o.od`, which does something +like `redo-ifchange $PRE_CXX_TARGETS`, where `$PRE_CXX_TARGETS` is created +by the precompilation detector above. + +Here's the actual header we want to use: +
+ + +### Compiling source files + +With all that done, we're ready to actually compile a program! Here's our +first source file: +
+ +and a header it needs: +
+ +Since we've already told redoconf what feature detectors our project needs, +redoconf can do the rest. There is a `redoconf/default.o.od` that can handle +compilation and autodependencies. We'll just ask it to produce the output. +(If you want, you can provide your own `default.o.od` which will take +precedence over the default one. But that's rarely needed.) +```shell +$ redo hello.o +redo hello.o +redo cxx.precompile +redo redoconf.h +redo rc_vars +redo all.hpp.gch +``` + +The `default.o.od` script depends on `cxx.precompile`, which does all the +things needed before compiling a C++ program (in particular, it builds all +the `$PRE_CXX_TARGETS`). As you can see, it built `all.hpp.gch` (the +precompiled `all.hpp`) and `redoconf.h` (redoconf's equivalent of autoconf's +`config.h`). + +Speaking of `redoconf.h`, here's the rest of our toy program. This one +provides a different version of a function depending on whether libssl is +available. It also uses redoconf to detect which header files are available +to be included: +
+ + +### Linking + +Final programs in unix are a little annoying, because they don't have file +extensions. That means redo (and redoconf) can't easily have a generalized +rule for how to build them. + +In redoconf, we handle that by looking for `filename.list` whenever you ask +for `filename`. If it exists, we assume `filename` is a binary that you +want to produce by linking together all the object files and libraries in +`filename.list`. Furthermore, the .list file can be automatically +generated, by calling `filename.list.od`. Here's the one for our test +program: +
+ +And here's what happens: +```shell +$ redo hello +redo hello +redo link +redo hello.list +redo ssltest.o +redo cc.precompile + +$ ./hello +Hello, world! +libssl version 100020cf +``` + +(Notice that we had to run `cc.precompile` as a prerequisite for `ssltest.o` +here. That's because `ssltest.c` is a C program, not C++, so it has +different pre-compilation dependencies than C++. We didn't declare any +plain-C precompiled headers, but if we did, they would have happened at +this stage.) + + +### Cross compiling + +One nice thing about redoconf is it handles cross compiling without you +having to do any extra work. Let's try cross compiling our test program for +Windows, using the gcc-mingw32 compiler (assuming you have it installed). +If you have `wine` installed, redoconf will also try to use that for running +your produced binaries. +```shell +$ redo -j5 test +redo test +redo all +redo hello +redo link +redo rc/_init.rc +redo _all.rc +redo rc/CC.rc +Trying C compiler: 'x86_64-w64-mingw32-cc' +Trying C compiler: 'x86_64-w64-mingw32-gcc' + CPPFLAGS += '-I.' + CC = 'x86_64-w64-mingw32-gcc' + LINK = 'x86_64-w64-mingw32-gcc' + LDFLAGS += '-static-libgcc +-static-libstdc++' + ARCH = 'x86_64-w64-mingw32-' + HAVE_CC = '1' +redo rc/zdefs.rc +'-Wl,-z,defs' doesn't work on this platform; skipped. +redo all.rc +redo rc/CXX.required.rc +redo rc/CXX.rc +Trying C++ compiler: 'x86_64-w64-mingw32-c++' + CXX = 'x86_64-w64-mingw32-c++' + LINK = 'x86_64-w64-mingw32-c++' + HAVE_CXX = '1' +redo rc/Wextra.rc +redo rc/Wall.rc + CPPFLAGS += '-Wall' +redo rc/Wextra.rc (resumed) + CPPFLAGS += '-Wextra' +redo rc/all.hpp.precompiled.rc + CXXFLAGS_PCH_LANG += '-x +c++-header' + CXXFLAGS += '-Winvalid-pch' + CXXFLAGS_PCH += '-include +all.hpp' + CXXFLAGS_PCH_FPIC += '-include +all.hpp.fpic' + PRE_CXX_TARGETS += 'all.hpp.gch' + PRE_CXX_TARGETS_FPIC += 'all.hpp.fpic.gch' +redo rc/openssl__ssl.h.rc + HAVE_OPENSSL__SSL_H = '' +redo rc/openssl__opensslv.h.rc + HAVE_OPENSSL__OPENSSLV_H = '' +redo rc/libssl.rc +redo rc/pkg-config.rc + PKG_CONFIG = 'pkg-config' + HAVE_PKG_CONFIG = '1' +redo rc/libssl.rc (resumed) + HAVE_LIBSSL = '' + LIBSSL = '' +redo rc/libm.rc + HAVE_LIBM = '1' +redo all.rc (resumed) + LIBS += '' +redo hello.list +redo hello.o +redo compile +redo _compile +redo cxx.precompile +redo redoconf.h +redo rc_vars +redo all.hpp.gch +redo ssltest.o +redo cc.precompile +redo run +redo rc/run.rc +redo rc/windows.h.rc + HAVE_WINDOWS_H = '1' +redo rc/run.rc (resumed) +Considering RUN='' +Considering RUN='wine64' + RUN = 'wine64' + CAN_RUN = '1' +redo test (resumed) +Hello, world! +libssl version 0 +``` + +I didn't have a copy of libssl available on Windows, so `$HAVE_LIBSSL` and +`$LIBSSL` were not set. As a result, `ssltest.o` reverted to the fallback +version of its function, and reported `libssl version 0`. But it worked! + + +### Housekeeping + +We want `cd out && ./configure && redo` to work, so we'll need an `all` +target. Note that, unlike a plain redo project, redoconf projects should +use an `all.od`, because you want to `redo all` in each output directory, not +the source directory: +
+ +We should also make sure that our little program works, so let's add a `test` +target too: +
+ +Finally, let's provide a `clean.do` (not `clean.od` in this case) which +will delete any `out` and `out.*` subdirectories of the source tree. We +could also provide a `clean.od` file for deleting files from a given +output directory, but it's usually easier to simply delete the output +directory entirely and start over. +
diff --git a/docs/cookbook/redoconf-simple/redoconf b/docs/cookbook/redoconf-simple/redoconf new file mode 120000 index 0000000..f0a8b6b --- /dev/null +++ b/docs/cookbook/redoconf-simple/redoconf @@ -0,0 +1 @@ +../../../redoconf \ No newline at end of file diff --git a/docs/cookbook/redoconf-simple/skip-if-no-cxx.sh b/docs/cookbook/redoconf-simple/skip-if-no-cxx.sh new file mode 100644 index 0000000..dbf55c6 --- /dev/null +++ b/docs/cookbook/redoconf-simple/skip-if-no-cxx.sh @@ -0,0 +1,14 @@ +# Make sure there is an out/ directory and +# the configure script was run. +redo-ifchange configured + +# Abort if we can't find a C++ compiler for +# this platform. +if ! (cd out && + . ./redoconf.rc && + rc_include rc/CXX.rc && + [ -n "$HAVE_CXX" ]); then + echo "$1: No C++ compiler: skipping." >&2 + exit 0 +fi + diff --git a/docs/cookbook/redoconf-simple/ssltest.c b/docs/cookbook/redoconf-simple/ssltest.c new file mode 100644 index 0000000..37a7c6b --- /dev/null +++ b/docs/cookbook/redoconf-simple/ssltest.c @@ -0,0 +1,28 @@ +// An example of how to change code behaviour based on +// redoconf autodetection. +#include "redoconf.h" + +#if HAVE_LIBSSL + +#if HAVE_OPENSSL__SSL_H +#include +#endif + +#if HAVE_OPENSSL__OPENSSLV_H +#include +#endif + +unsigned long libssl_version() { + SSL_library_init(); + return OPENSSL_VERSION_NUMBER; +} + +#else // HAVE_LIBSSL + +unsigned long libssl_version() { + // Library not present + return 0; +} + +#endif // HAVE_LIBSSL + diff --git a/docs/cookbook/redoconf-simple/ssltest.h b/docs/cookbook/redoconf-simple/ssltest.h new file mode 100644 index 0000000..4c0cc61 --- /dev/null +++ b/docs/cookbook/redoconf-simple/ssltest.h @@ -0,0 +1,14 @@ +#ifndef __SSLTEST_H +#define __SSLTEST_H + +#ifdef __cplusplus +extern "C" { +#endif + +unsigned long libssl_version(); + +#ifdef __cplusplus +} +#endif + +#endif // __SSLTEST_H diff --git a/docs/cookbook/redoconf-simple/test.do b/docs/cookbook/redoconf-simple/test.do new file mode 100644 index 0000000..58e19b8 --- /dev/null +++ b/docs/cookbook/redoconf-simple/test.do @@ -0,0 +1,4 @@ +# This is run as part of building the examples inside +# the redo source tree. See all.do for details. +. ./skip-if-no-cxx.sh +redo out/test diff --git a/docs/cookbook/redoconf-simple/test.od b/docs/cookbook/redoconf-simple/test.od new file mode 100644 index 0000000..e1034db --- /dev/null +++ b/docs/cookbook/redoconf-simple/test.od @@ -0,0 +1,33 @@ +# Run a simple test of our hello binary. +# +# The 'run' script is automatically generated by +# redoconf when requested (by redo-ifchange) and if +# it's possible (ie. we are not cross compiling). A +# typical project might choose to fail if it is not +# available, but because this script runs as part of +# the redo source tree, we'll skip the test politely +# if it's not possible to run. + +# 'test' is a target you always want to run when +# requested, even if it passes. +redo-always + +# Make sure the program (all) is built, and that the +# redoconf program runner (run) is built, if possible. +redo-ifchange all run + +# Test the program only if redoconf knows how to run +# produced binaries. It will try to run programs directly, +# or try wine if you are cross-compiling windows programs. +# If the run script is nonexistent (run.od didn't produce +# output), we'll skip it, because we are presumably cross +# compiling for an incompatible platform. +if [ -x ./run ]; then + ./run ./hello | tee "$3" >&2 + if ! grep "^Hello" "$3" >/dev/null; then + echo "Hello program did not say hello!" >&2 + exit 10 + fi +else + echo "Non-native platform: test skipped." >&2 +fi diff --git a/mkdocs.yml b/mkdocs.yml index e8f86c9..04af578 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -28,7 +28,8 @@ plugins: - cookbook/container/*.initrd - cookbook/container/*.layer - cookbook/container/*.list - - cookbook/c/out.* + - cookbook/*/out/* + - cookbook/*/out.* - "t/*" - "*.eps" - "*.gz" @@ -51,6 +52,7 @@ nav: - Text substitution (default.do, redo-always, redo-stamp): cookbook/defaults/index.md - R plots and LaTeX to pdf (side effects, multiple outputs): cookbook/latex/index.md - Docker and kvm containers (from scratch): cookbook/container/index.md + - Portable C++ programs (using redoconf): cookbook/redoconf-simple/index.md - FAQ: - Basics: FAQBasics.md - Semantics: FAQSemantics.md diff --git a/redoconf/rc/Wextra.rc.od b/redoconf/rc/Wextra.rc.od index 1d843fd..7016e94 100644 --- a/redoconf/rc/Wextra.rc.od +++ b/redoconf/rc/Wextra.rc.od @@ -1,4 +1,4 @@ -rc_include rc/CC.rc +rc_include rc/Wall.rc rc_appendln CPPFLAGS "-Wextra" if rc_compile cc nolink; then diff --git a/redoconf/rc/libssl.rc.od b/redoconf/rc/libssl.rc.od index 1f608d1..e614c7f 100644 --- a/redoconf/rc/libssl.rc.od +++ b/redoconf/rc/libssl.rc.od @@ -2,6 +2,8 @@ rc_include rc/CC.rc rc/pkg-config.rc prog=" #include + +void f() { SSL_library_init(); } " rc_pkg_detect LIBSSL "libssl libcrypto" \ diff --git a/redoconf/rc_vars.od b/redoconf/rc_vars.od index c6cf4b3..e207796 100644 --- a/redoconf/rc_vars.od +++ b/redoconf/rc_vars.od @@ -15,7 +15,7 @@ cquote() { } ( - echo "# Automatically generated by autovars.od" + echo "# Automatically generated by rc_vars.od" for k in $(echo "$RC_KEYS" | sort); do [ "$k" != "RC_INCLUDES" ] || continue eval v=\$$k