How do you track the build count of your library when there are multiple authors using version control?

Question

I don't know if this is something common for people to do or not, but I personally always keep track of the number of times I built my code. That is, both the number of times I called make and the number of times the build was successful.

My current solution

I have a simple code that takes a file as parameter, opens it, increments the number inside and overwrites it. This code is compiled, first thing when make is called.

Immediately after, ./increase_build build.txt is called which increments the number of times I called make to build the library.

Then, the code is compiled and the lib file is made (with ar cq ...). After that, ./increase_build libbuild.txt is called that increments the number of successful builds. Finally the tests are built.

This is an example of one of my Makefiles.

Why I got concerned

This always worked fine, until I started using version control. There seemed like there is no problem: I am the sole author of my own libraries and I add features one by one.

One day though, I was testing branching and merging (I use git for myself and svn at work), so I added one feature in a branch and changed something in master and I merged the two. Now the build count files have different values.

The problem

The problem is, let's say at the time of branch, the build count is 100. Then I write something in branch and the build count gets to 110. I write something in master branch and the build count gets to 120. When I merge the two, I see one is 110 and one is 120 (which is a conflict by the way). The correct solution would be to set the build to 130.

However, I can't (read I don't want to) go back to the commit where the branch took off and find that it was 100 so I compute 100+(110-100)+(120-100) = 130! I want this to be automatic

The question

Well the question is obvious: How do I do this? How do I keep track of my build count (not commit count!) when I'm working with version control? I don't want an answer that is based on a feature in the version control, because the problem arises anew if I change version control system.

What I though could work was to add one line in the build count file for every build, something say with data and time. Then the build number would be the number of lines in the build count files. Also, unless I get two builds on two branches that were done the EXACT same time then merging the files would be just the union of the two.

I wonder though, are there any better solutions to this? Is what I want (build counts) even worth the effort?

P.S. If you are wondering why I do it with both the number of builds and the number of successful builds, that's just something personal. I like to see how much rebuild I get for small typos and errors I make when I code.

Edit: I program in C and C++, so a solution in either works for me.

Answer 1

I'm going to post and accept the implementation of my own idea as the answer, as it seems to be the most practical.

So here's the solution:

• With every build, add one line to the build file containing the following data:
  • Date
  • Time
  • A random number
• With every merge, keep lines from both build files
• The number of builds is the total number of lines in the build file.

Each line of the build file needs to be unique. The date and time make that very nearly unique. It is very unlikely that two people issue a build on their own branch at the same time. However, it may happen. Therefore, a random number is generated and added to decrease that chance.

There is one problem though. If you seed srand with time(NULL), then since both builds are supposedly at the same time, the generated number could also happen to be the same. Therefore, the random number generator could be seeded with a different number such as clock() or the milliseconds part of gettimeofday(). Even if not randomly generated, those numbers themselves could be put instead of a random number.

In the case where still two lines ended up the same, I'll apply the ostrich algorithm.

Update:

I implemented it and everything works fine. Finally, I used clock_gettime(CLOCK_MONOTONIC, ...) and printed the nanoseconds obtained by this function as the random number. The reason I didn't use clock() was that since the program was quite short, it ran less than the resolution of clock() and therefore I kept getting 0s.

Update:

Here is the final code I wrote (with some parts of it stolen from somewhere else!). You may need -lrt on some platforms.

/*
 * This program is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 */

#include <time.h>
#include <stdio.h>
#include <stdlib.h>

#ifdef _WIN32
#include <windows.h>

struct timespec
{
    long tv_sec;
    long tv_nsec;
};

/* Note: I copy-pasted this from internet (https://stackoverflow.com/questions/5404277/porting-clock-gettime-to-windows/5404467#5404467)
 * I tweaked it to return nanoseconds instead of microseconds
 * It is much more complete than just finding tv_nsec, but I'm keeping it for possible future use. */
LARGE_INTEGER getFILETIMEoffset(void)
{
    SYSTEMTIME s;
    FILETIME f;
    LARGE_INTEGER t;

    s.wYear = 1970;
    s.wMonth = 1;
    s.wDay = 1;
    s.wHour = 0;
    s.wMinute = 0;
    s.wSecond = 0;
    s.wMilliseconds = 0;
    SystemTimeToFileTime(&s, &f);
    t.QuadPart = f.dwHighDateTime;
    t.QuadPart <<= 32;
    t.QuadPart |= f.dwLowDateTime;
    return t;
}

int clock_gettime(int X, struct timespec *tv)
{
    LARGE_INTEGER t;
    FILETIME f;
    double microseconds;
    static LARGE_INTEGER offset;
    static double frequencyToNanoseconds;
    static int initialized = 0;
    static BOOL usePerformanceCounter = 0;

    if (!initialized)
    {
        LARGE_INTEGER performanceFrequency;
        initialized = 1;
        usePerformanceCounter = QueryPerformanceFrequency(&performanceFrequency);
        if (usePerformanceCounter)
        {
            QueryPerformanceCounter(&offset);
            frequencyToNanoseconds = (double)performanceFrequency.QuadPart/1000000000.0;
        }
        else
        {
            offset = getFILETIMEoffset();
            frequencyToNanoseconds = 0.010;
        }
    }
    if (usePerformanceCounter)
        QueryPerformanceCounter(&t);
    else
    {
        GetSystemTimeAsFileTime(&f);
        t.QuadPart = f.dwHighDateTime;
        t.QuadPart <<= 32;
        t.QuadPart |= f.dwLowDateTime;
    }

    t.QuadPart -= offset.QuadPart;
    microseconds = (double)t.QuadPart/frequencyToNanoseconds;
    t.QuadPart = microseconds;
    tv->tv_sec = t.QuadPart/1000000000;
    tv->tv_nsec = t.QuadPart%1000000000;
    return 0;
}

#ifndef CLOCK_MONOTONIC
#define CLOCK_MONOTONIC 0       /* not used anyway */
#endif
#endif

int main(int argc, char **argv)
{
    time_t now_sec;
    struct tm *now;
    FILE *bout;
    struct timespec now_clk;
    if (argc < 2)
    {
        printf("Usage: %s build_file_name\n\n", argv[0]);;
        return EXIT_FAILURE;
    }
    bout = fopen(argv[1], "a");
    if (!bout)
    {
        printf("Could not open file: %s\n\n", argv[1]);
        return EXIT_FAILURE;
    }
    time(&now_sec);
    now = gmtime(&now_sec);
    fprintf(bout, "%02d/%02d/%04d %02d:%02d:%02d", now->tm_mday, now->tm_mon+1, now->tm_year+1900, now->tm_hour, now->tm_min, now->tm_sec);
    clock_gettime(CLOCK_MONOTONIC, &now_clk);
    fprintf(bout, " %ld\n", now_clk.tv_nsec);
    return EXIT_SUCCESS;
}

Hopefully this will be of help to someone.

Update

After ~9 months of using this, I can say this has been quite useful. Some observations are:

• On Windows, the last element given by the implementation of clock_gettime is quite small, half of the times with the same value. Nevertheless, it still makes it a bit more random.
• On Linux, the last element is indeed quite random.
• Every now and then I had to do a "Build" commit just to get the lines in the build files committed so I can merge. However, this can be avoided with git stash.
• Using this almost always results in a conflict when merging, but is very trivial to resolve it (just remove the diff markers as lines from both files are wanted).
• wc -l is your friend.

Answer 2

Your solution with a build log (one line per build) seems pretty clever. You could add the IP (or Mac address) of the machine doing the build do you timstample so you would remove the risk of duplicated line. However depending of your VCS you probably have to merge manually your build log file. With git you can configure it so the merge will always keep both versions (and eventually sort lines by date, etc.)

Answer 3

I can't (read I don't want to) go back to the commit where the branch took off and find that it was 100 so I compute ... I want this to be automatic

Dirty idea: each build add string to versioned file (or to one from pair PASS/FAIL depending on result), slightly different for each branch. Merge branches will require manual merging of this signal file(s), where differences in strings decoration make this task easier. wc -l later will count numbers

Answer 4

Because the build number is not a feature of the branch you are in, it should be tracked differently. I don't use git, but for SVN, we have a system at work which builds a particular branch by copying it to a particular tag, adding some artefacts specific to that tag (your build number would be a prime example of the sort of thing to add), and committing only if the build succeeded.

In other words, there is a designated place (a tag name in SVN, or it could be a separate repo) where you only do builds, and this is the only place where you build, and that is where the build number information is stored and updated. Your build script would look something like

# I don't know git -- this is all very much pseudocode

# Where did you commit the code you want to build?
source=git://server/path/to/my/branch

# Replace builddir tree with yours
git replace git://server/special/place/build/thisproject with code from $source

cd /tmp
git checkout git://sever/special/place/build/thisproject into new builddir
cd builddir

update local version-controlled file buildnumber+=1

if make
    # Build was successful
    git commit buildnumber
    copy build artefacts to where-ever
endif

cd /tmp
rm -rf /tmp/builddir      

There is a race condition; if somebody checks in a build request after yours, but somehow ends up reaching the server first, you will end up building their check-in.

This can probably be made a lot simpler by using a designated build host like with Hudson/Jenkins.