How to build and run tests against Chromium based WebView on KitKat

Question

Kitkat uses chromium based webview, and directory "external/chromium_org" contains code of chromium.

There are integration tests for android webview in the directory:

"external/chromium_org/android_webview/javatests".

I have read the wiki here(https://code.google.com/p/chromium/wiki/AndroidTestInstructions), and I went through it smoothly in the chromium source directory(Not in AOSP, just download the chromium standalone).

But, in AOSP/external/chromium_org, I can not build the test package out. The first assert error is lacking NDK, I made a link to the standalone chromium's NDK. Still, it can not build out with an error:

"ninja: Entering directory `out/Debug'
 ninja: error: loading 'build.ninja': No such file or directory
"

---

My questions are listed below:

Q1: Would you please point out how to build the test package and run the tests for KitKat WebView in AOSP?

Q2: Or, we can only build and run these test in Chromium source directory? If so, how can I guarantee the the tested webview is same with the one used in the KitKat? Because the target of AndroidWebViewTest.apk is AndroidWebView.apk, not the webview in KitKat system image which is build out of code in "AOSP/external/chromium_org".

Q3: If I want to make sure KitKat Chromium webview works well on a specific device, which test cases are needed? Currently, I want to include android.webkit in CTS and the integration test AndroidWebViewTest.apk, will these enough? Do I need to leverage more test cases?

Answer 1

The code you get from AOSP/external/chromium_org is a stripped version of the Chromium source and can't be used to build and run tests (other than the Android CTS tests). What you need to do is check out Chromium code and build it.

The specific build and invocation commands I use are:

# Set up your environment
. build/android/envsetup.sh

# Generate the ninja files
GYP_GENERATORS="ninja" build/gyp_chromium

# Use -C out/Release for release build
ninja -C out/Debug android_webview_apk android_webview_test_apk

# Install the webview test shell
adb install out/Debug/apks/AndroidWebView.apk

# Connect a device and run tests
build/android/test_runner.py instrumentation --test-apk AndroidWebViewTest --test_data webview:android_webview/test/data/device_files

The error you're seeing is probably because you didn't run gyp to generate the ninja files or didn't pass the right folder to ninja with the -C option.

To answer your qestions:

Q1: Per above - you can't run the Chromium instrumentation tests from the AOSP checkout.

Q2: See above for instructions on running tests. The upstream Chromium code is obviously newer than the AOSP/external/chromium_org code. The way to contribute to the WebView is to submit your patch to upstream Chromium code and wait for that to be included in a future Android release. Patches against code in the AOSP/external/chromium_org will not be accepted.

Unfortunately if you were to replace the AOSP/external/chromium_org code with the Tip-Of-Tree Chromium code it won't compile since a number of necessary CLs hadn't made their way to frameworks/webview. The WebView team is aware of this and for now your best bet is to do all of your work in the upstream Chromium tree.

Q3: Those would be a good start, yes. The Chromium project Android bots run more tests than the ones you described: see the ones that run on the Android Tests and the WebKit layout tests bots. Unfortunately I don't think it's possible to have a test suite that gives you a guarantee that any code will work on a particular piece of hardware.