Git submodules and rebar

Question

My application uses Mochiweb. As I understand, rebar fetches the latest version from Github when I run make, because there is a line in rebar.config:

{deps, [
  {mochiweb, ".*",
   {git, "git://github.com/mochi/mochiweb.git", "master"}}

My application has a VCS and it is git. So, essentially I have one git repository inside another:

myapp
 .git
 deps
  mochiweb
   .git
 src
 etc

I know that adding a git repository inside another one is not a good idea (git add .). Git submodules functionality should be used instead.

So, I added the deps/mochiweb directory as a submodule to the main git repository.

The problem is that when another developer clones the main repository he has to init and update the submodules first in order to get deps/mochiweb (otherwise it would be empty).

If the developer just runs make right after he clones the main repository, the Makefile says the following:

ERROR: Dependency dir deps/mochiweb failed application validation with reason:

{missing_app_file,"deps/mochiweb"}

make: *** [all] Error 1

My question is: What is the proper way of adding another app to the deps of an Erlang app to allow easy updates by the other developers without using git submodules?

Answer 1

What is the proper way of adding another app to the deps of an Erlang app to allow easy updates by the other developers without using git submodules?

Add the app to the rebar.config and use:

./rebar update-deps

For updating. The first time, you need to use:

./rebar get-deps

See: https://github.com/basho/rebar/wiki/Rebar-commands

Now, back to your error.

My feeling is that you have an (almost) empty directory for mochiweb in your deps, probably as a result of playing with Git submodules. When you run the get-deps command, rebar silently discards mochiweb, since the directory is already there. But it expects an OTP application and looks for the mochiweb.app file, which is not there (the directory is empty). Therefore, the error. If my interpretation is correct, you can simply do:

rm -rf deps/mochiweb
./rebar get-deps

Having a look to your rebar.config would help, though.

Answer 2

I think you're looking for the rebar get-deps command. Take a look at the Makefile and rebar.config used by Riak for a good example.

Answer 3

In our internal Erlang projects we use Git subtree merge approach to reference dependencies. In my view even though rebar get-deps is handly way to get dependencies for a github hosted project it is not that good in a corporate environment:

1. github needs to be accessible from every build-machine (and the sources for the dependencies are hardcoded in rebar.config)
2. You rely on dependent project visioning which is not that precise (can you point to the specific commit?). Not only it isn't possible to reproduce the state of your project in the past (with all the dependencies at that time) but even worse, your project can also be easily broken once some dependency is updated on github (without updating its version).
3. What if you want to customise dependent project? Like a small change in a dependent project's 'rebar.config' file?

So instead of get-deps we use a separate branch (+ remote pointing to specific github project) for every dependent project and git subtree merge it into 'dep' directory on our project branch. Effectively we store all the dependencies in the main branch but that way we solve all above problems:

1. You get entire code base by pulling the project main branch (or dev etc.) from our internal git repositories. It is immediately buildable.
2. The precise versions of all dependencies are subtree merged into our main branch and we update to a newer version of a dependency only when we want/need to: just switch to dependency branch, do git pull and them subtree merge some specific version of the dependency into main branch. At any time you can get any past version of the entire project.
3. Dependency can be customised in its own branch (or another customised branch) once it is time to update the dependency: you will have to git merge the latest code with your changes but it is usually not a problem. You can even point your remote to your own repository (forked on github or internal).

Of course we would not publish our project in this form (with all dependencies included) on 'github' but you can strip all dependencies (by removing the 'dep' folder) on a separate branch and publish the result. The drawback of this approach is a little bit of hassle with dependency updates: instead of rebar get-deps you have to do git push + git merge -s subtree for every dependency you want to update but it is just a consequence of the strict dependency management and easiness of the build.