building a tensorflow based android app with tensorflow as a repository

Question

This is like a follow up question from Build an android app using Tensorflow. I'd like to separate the android example project from tensorflow git repo and be able to build it separately with tensorflow as a dependency. This is my folder structure

my_project
|-- WORKSPACE
|-- android
|   |-- BUILD
|   `-- ...
|-- tensorflow
|   |-- tensorflow
|   |   |   |-- workspace.bzl
|   |   |   |-- tensorflow.bzl
|   |   |   `-- ...
|   |-- WORKSPACE
|   |-- BUILD
.    `-- ...

where android app is just a copy of Tensorflow Android example. The root WORKSPACE file has following contents:

workspace(name = "my_android_app")

local_repository(
     name = "org_tensorflow",
     path = "tensorflow", # Relative path to the tensorflow workspace
 )

load('//android:workspace.bzl', 'android_workspace')
android_workspace()

 # Specify the minimum required bazel version.
load("@org_tensorflow//tensorflow:tensorflow.bzl", "check_version")
check_version("0.3.1")

android/workspace.bzl looks like this

load('@org_tensorflow//tensorflow:workspace.bzl', 'tf_workspace')

def android_workspace():
  tf_workspace()

and android/BUILD has the same contents as Tensorflow Android example BUILD except that I've prefixed @org_tensorflow to //tensorflow everywhere, for example

"@org_tensorflow//tensorflow:tensorflow.bzl"
  "@org_tensorflow//tensorflow/contrib/android:android_tensorflow_inference_jni",
"@org_tensorflow//tensorflow/core:android_tensorflow_lib",

when I try to build the main target tensorflow_demo, it gives this error

no such package 'tensorflow': Package crosses into repository @org_tensorflow and referenced by '//android:libtensorflow_demo.so'.
ERROR: Analysis of target '//android:tensorflow_demo' failed; build aborted.

EDIT:

I was able to decouple Tensorflow Android demo from Tensorflow source, thanks to Kristina. You can use the following git as a template for your Tensorflow Android projects. https://github.com/devinsaini/tensorflow_android

Answer 1

Ah, okay, I see what you're doing. Thanks for elaborating.

The problem here is a bit subtle: copts = tf_copts() calls a function in @org_tensorflow that looks like this:

def tf_copts():
  return (["-DEIGEN_AVOID_STL_ARRAY",
           "-Iexternal/gemmlowp",
           "-Wno-sign-compare",
           "-fno-exceptions"] +
          if_cuda(["-DGOOGLE_CUDA=1"]) +
          if_android_arm(["-mfpu=neon"]) +
          if_x86(["-msse4.1"]) +
          select({
              "//tensorflow:android": [
                  "-std=c++11",
                  "-DTF_LEAN_BINARY",
                  "-O2",
              ],
              "//tensorflow:darwin": [],
              "//tensorflow:windows": [
                "/DLANG_CXX11",
                "/D__VERSION__=\\\"MSVC\\\"",
                "/DPLATFORM_WINDOWS",
                "/DEIGEN_HAS_C99_MATH",
                "/DTENSORFLOW_USE_EIGEN_THREADPOOL",
              ],
              "//tensorflow:ios": ["-std=c++11"],
              "//conditions:default": ["-pthread"]}))

Basically a switch statement to use different flags on different platforms. However, because load()s are evaluated before select()s, your BUILD file suddenly "contains" references to //tensorflow.

To fix, the easiest options are to either hard-code the copts you need or redefine tf_copts in your local repository.

How to track down incorrect references in general

I was actually pretty baffled by the error you were getting, so here's a brain dump of how one usually goes about debugging this kind of thing.

The usual way to track down this kind of cross reference is 1) to examine your BUILD file (maybe the deps are just wrong) or 2) print out how Bazel actually "sees" the BUILD file. For example, if you wanted to print the evaluated build rule in this case, you could do:

$ bazel query --output=build //:libtensorflow_demo.so
# /home/kchodorow/gitroot/so41153199/BUILD:18:1
cc_binary(
  name = "libtensorflow_demo.so",
  tags = ["manual", "notap"],
  deps = ["//:demo_proto_lib_cc", "@org_tensorflow//tensorflow/contrib/android:android_tensorflow_inference_jni", "@org_tensorflow//tensorflow/core:android_tensorflow_lib", "@org_tensorflow//tensorflow/contrib/android:jni/version_script.lds"],
  srcs = ["//:jni/box_coder_jni.cc", "//:jni/imageutils_jni.cc", "//:jni/object_tracking/config.h", "//:jni/object_tracking/flow_cache.h", "//:jni/object_tracking/frame_pair.cc", "//:jni/object_tracking/frame_pair.h", "//:jni/object_tracking/geom.h", "//:jni/object_tracking/gl_utils.h", "//:jni/object_tracking/image-inl.h", "//:jni/object_tracking/image.h", "//:jni/object_tracking/image_data.h", "//:jni/object_tracking/image_neon.cc", "//:jni/object_tracking/image_utils.h", "//:jni/object_tracking/integral_image.h", "//:jni/object_tracking/jni_utils.h", "//:jni/object_tracking/keypoint.h", "//:jni/object_tracking/keypoint_detector.cc", "//:jni/object_tracking/keypoint_detector.h", "//:jni/object_tracking/log_streaming.h", "//:jni/object_tracking/object_detector.cc", "//:jni/object_tracking/object_detector.h", "//:jni/object_tracking/object_model.h", "//:jni/object_tracking/object_tracker.cc", "//:jni/object_tracking/object_tracker.h", "//:jni/object_tracking/object_tracker_jni.cc", "//:jni/object_tracking/optical_flow.cc", "//:jni/object_tracking/optical_flow.h", "//:jni/object_tracking/sprite.h", "//:jni/object_tracking/time_log.cc", "//:jni/object_tracking/time_log.h", "//:jni/object_tracking/tracked_object.cc", "//:jni/object_tracking/tracked_object.h", "//:jni/object_tracking/utils.h", "//:jni/object_tracking/utils_neon.cc", "//:jni/rgb2yuv.cc", "//:jni/rgb2yuv.h", "//:jni/yuv2rgb.cc", "//:jni/yuv2rgb.h"],
  linkopts = ["-landroid", "-ljnigraphics", "-llog", "-lm", "-z defs", "-s", "-Wl,--version-script", "@org_tensorflow//tensorflow/contrib/android:jni/version_script.lds"],
  linkstatic = True,
  linkshared = True,
)

However, this doesn't actually help you in this case, because it doesn't show the copts!

So, after running the query, I ended up "binary searching" the BUILD rule: I commented out everything but the srcs, tried building, then gradually uncommented parts until I got the error.