Add an extra compile step to a custom compiler/language in CMake

Question

This is a bit of follow-up to an earlier question I posted. My basic problem was to build a application with Gambit Scheme.

While the solution suggested in the question mentioned above works, it is kinda cumbersome so I decided to try and add Gambit Scheme as custom compiler/language to CMake. Following the suggestions in this question, I created the following files:

cmake/CMakeDetermineGambitCompiler.cmake:

# Find the compiler
find_program(
    CMAKE_Gambit_COMPILER 
        NAMES "gambitc"
        HINTS "${CMAKE_SOURCE_DIR}"
        DOC "Gambit Scheme compiler" 
)

mark_as_advanced( CMAKE_Gambit_COMPILER )

set( CMAKE_Gambit_SOURCE_FILE_EXTENSIONS scm;six )
# Remember this as a potential error
set( CMAKE_Gambit_OUTPUT_EXTENSION .c )
set( CMAKE_Gambit_COMPILER_ENV_VAR "" )

# Configure variables set in this file for fast reload later on
configure_file( ${CMAKE_CURRENT_LIST_DIR}/CMakeGambitCompiler.cmake.in
                ${CMAKE_PLATFORM_INFO_DIR}/CMakeGambitCompiler.cmake )

cmake/CMakeGambitInformation.cmake:

# This file sets the basic flags for the GAMBIT compiler

# Generate the C files
set( CMAKE_Gambit_COMPILE_OBJECT
    "<CMAKE_Gambit_COMPILER> -o <OBJECT> -c <SOURCE>"
)

# Build a executable 
set( CMAKE_Gambit_LINK_EXECUTABLE 
    "<CMAKE_Gambit_COMPILER> -o <TARGET> -exe <OBJECTS>"
)

set( CMAKE_Gambit_INFORMATION_LOADED 1 )

cmake/CMakeGambitCompiler.cmake.in:

set( CMAKE_Gambit_COMPILER "@CMAKE_Gambit_COMPILER@" )
set( CMAKE_Gambit_COMPILER_LOADED 1 )
set( CMAKE_Gambit_SOURCE_FILE_EXTENSIONS @CMAKE_Gambit_SOURCE_FILE_EXTENSIONS@ )
set( CMAKE_Gambit_OUTPUT_EXTENSION @CMAKE_Gambit_OUTPUT_EXTENSION@ )
set( CMAKE_Gambit_COMPILER_ENV_VAR "@CMAKE_Gambit_COMPILER_ENV_VAR@" )

cmake/CMakeTestGambitCompiler.cmake:

# For now do nothing
set( CMAKE_Gambit_COMPILER_WORKS 1 CACHE INTERNAL "" )

Then, in my project root I have two more files:

CMakeTexts.txt:

cmake_minimum_required( VERSION 3.10...3.18 )

if( ${CMAKE_VERSION} VERSION_LESS 3.12 )
    cmake_policy( VERSION ${CMAKE_MAJOR_VERSION}.${CMAKE_MINOR_VERSION} )
endif()

# Give the project a name
project( cmake-scheme-template NONE )

# Build simple Gambit Scheme program
list( APPEND CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake")
enable_language( Gambit )
add_executable( ${PROJECT_NAME} main.scm )

The actual code to build, main.scm:

;;; Simple Scheme example
(begin (write "Hello, Schemer!")
       (newline))

Giving the following structure:

project_root/
    cmake/
        CMakeDetermineGambitCompiler.cmake
        CMakeGambitCompiler.cmake.in
        CMakeGambitInformation.cmake
        CMakeTestGambitCompiler.cmake
    CMakeLists.txt
    main.scm

While this works for a single file, as soon as I added another source file, I need Gambit to first create a link file for all C files that are generated from Scheme sources. Here's a simple example:

Say I add a second file, factmodule.scm:

;;; This is a simple Scheme module that provides a functions that will 
;;; calculate the factorial of a number n
(define fact
  (lambda (n)
    (if (zero? n)
        1
        (* n (fact (- n 1))))))

And update main.scm:

(begin (write "Hello, Schemer!")
       (newline)
       (write "10! = ")
       (write (number->string (fact 10)))
       (newline))

To build this "by hand", I do the following:

$ gambitc -c factmodule.scm main.scm                      # generate C files from Scheme
$ gambitc -o link_file.c -link factmodule.c main.c        # generate a link file
$ gambitc -obj factmodule.c main.c link_file.c            # compile the C files in object files
$ gcc -o myexec -factmodule.o main.o link_file.o -lgambit # link the final executable

My problem is the second step, creating the link file. Ideally, I'd like to add to cmake/CMakeGambitInformation.cmake something like:

# Generate the C, link, and object files
set( CMAKE_Gambit_COMPILE_OBJECT
    "<CMAKE_Gambit_COMPILER> -o <OBJECT> -c <SOURCE>"        # generate C files
    "<CMAKE_Gambit_COMPILER> -o link_file.c -link <OBJECTS>" # generate link file
    "<CMAKE_Gambit_COMPILER> -obj <OBJECTS>"                 # compile C and link files
)

But there are two problems to this. One, <OBJECTS> holds the generated C files; it is my understand that the commands given in CMAKE_Gambit_COMPILE_OBJECT are executed on a per source file base. Two, I obviously want to run the last two commands only once, but before the command given to CMAKE_Gambit_LINK_EXECUTABLE are invoked.

Is there a way to execute custom commands after the objects are created but before they're linked?

I've looked into some other compilers in CMake/Modules but couldn't really find a language that does that. And this entry seems to be the most complete documentation for adding a new language, the official doc seems not to really mention it.

Answer 1

The following hacks should achieve what you are after. The solution releaves that gsc and cmake don't always play nice with each other, as they both have their own way of handling file extensions implicitly. Anyway, let's get started.

The series of commands I intend to reproduce from within cmake (with slightly different filenames) is

gsc -c linkstub.scm
gsc -c factmodule.scm
gsc -c main.scm
gsc -obj linkstub.scm
gsc -obj factmodule.scm
gsc -obj main.scm
gsc -o linkstub.c -link factmodule.c main.c
gsc -obj linkstub.c
gsc -o test -exe factmodule.o main.o linkstub.o

Here, linkstub.scm is an empty (generated) dummy file. We need this for the final linking, because we can't modify the <OBJECTS> list that is passed to CMAKE_Gambit_LINK_EXECUTABLE. The corresponding linkstub.c file will be the actual link file generated in by gsc -o linkstub.c -link .... And that is achieved with add_custom_command and its PRE_LINK option (execute a command every time a target is supposed to be linked). The same custom command also compiles the linkstub.c into an object file; the previous linkstub.o is overwritten by the newly created one, before the final linking starts. This way we can trick CMake to swallow a linkstub.o that is updated whenever all other objects files have been compiled.

Let get's started. We need to change the compiler command and the object extension in their respective files:

set(CMAKE_Gambit_COMPILE_OBJECT
    "<CMAKE_Gambit_COMPILER> -o <SOURCE>.c -c <SOURCE>"
    "<CMAKE_Gambit_COMPILER> -o <OBJECT> -obj <SOURCE>.c")

set(CMAKE_Gambit_OUTPUT_EXTENSION .o)

This is somewhat dirty, as it will leave build artifacts in your source tree (maybe you could just prepend a path to the build tree - here, we'll simply delete them automatically later). Note that compiling .scm into .o on a per-source basis also scales better, as the linking step in your original attempt can get quite large (all .c files are compiled down into an executable). Next, the placeholder for the link file:

set(linkstub ${CMAKE_CURRENT_BINARY_DIR}/linkstub.scm)
file(WRITE ${linkstub} "")

Again, it this is only a mean to get the corresponding object file into the object dependencies of the target. Now all the rest:

set(sources
    factmodule.scm
    main.scm)

list(TRANSFORM sources
    APPEND ".c"
    OUTPUT_VARIABLE cSources)

list(TRANSFORM cSources
    PREPEND "${CMAKE_CURRENT_SOURCE_DIR}/"
    OUTPUT_VARIABLE cSources)

list(REMOVE_ITEM cSources
    linkstub.scm.c)

add_executable(test ${sources} ${linkstub})

add_custom_command(TARGET test
    PRE_LINK
    COMMAND ${CMAKE_Gambit_COMPILER}
    -o
    ${CMAKE_CURRENT_BINARY_DIR}/linkstub.scm.c
    -link
    ${cSources}
    COMMAND ${CMAKE_Gambit_COMPILER}
    -o 
    ${CMAKE_CURRENT_BINARY_DIR}/CMakeFiles/test.dir/linkstub.scm.o
    -obj
    ${CMAKE_CURRENT_BINARY_DIR}/linkstub.scm.c
    COMMAND ${CMAKE_COMMAND} -E remove ${cSources})

The last command does all the magic. It re-writes linkstub.scm.c, compiles it into an object file, and finally removes all generated .scm.c files in the source tree. Note that there is an ugly, hard-coded CMakeFiles/test.dir/ in one of the paths, there should be a way to query this path from the target to get around this.

Note again that the file extensions are somewhat crucial here, and the solution is brittle. CMake seems to absolutely need object files with .o appended, i.e. .scm.o. gsc however, will - if not told otherwise - generate a .c instead of the .scm.c, which causes the -link step to produce symbols that don't match the ones compiled into .scm.o.

As a side note, this approach obviously doesn't handle any dynamic/implicit dependencies between scheme sources - if you change factmodule.scm in a way that main.scm needs to be re-trans- and re-compiled, this won't be resolved for you. As far as I know, there is currently no way to teach CMake that you would like to register a custom dependency scanner that parses .scm files.

It seems that a plain old makefile might do a better job.