What technical specifications are relevant to creating an at-home UnitTest lab?

Question

I need to run tests against physical mobile devices, and need to deploy unit tests to many concurrent devices at the same time.

Conceptually, this is similar to the Xamarin Mobile Test Cloud, or Microsoft Mobile cloud... but I need an on-premise equivalent so that I can test Bluetooth and GPS features

I'm looking to test 7 to 14 physical devices all connected to the same USB hub.

Question

• How many devices can a single Mac / Windows machine talk to over a single USB port?

• What are the power considerations I need to keep in mind while selecting hardware?

Here is a visual example of what I imagine, however this device is focused on charging, not connecting a USB backplane to a computer.

Answer 1

Per USB controller you can have a 127 devices (128 minus the controller), bu I am not saying you can simultaneously have 127 devices communicating to the host and maintain any type of data throughput, that is a different story....

The core USB issue with a lot of devices is the power draw. We use special Y cables that have a data only side and a power only side even when using the so-called "powered" USB hubs.

The really good cables that do this are typically found via companies that cater to audiophiles and as such are typically very expensive and work great but are overkill for this usage. But also avoid the cheap Y-cables where the data-only side is "ungrounded" as you will have data transfer failures. Low quality cables will kill data through-put and have you running in circles chasing test failures.

Consumer external USB hubs are not created equal and wildly vary in their power output and simultaneous throughput across each port. While most will do well with one/two devices talking simultaneously, many start failing when 4-8 devices are all talking. Also trying to cascade them for increasing the number of ports can cause failures across the entire chain. Make sure you have a good return policy where ever you can buying it from.

You did not stated the testing framework, but some Appium info:

For Appium-based Android testing I have use a combination of:

• Android<-USB->Host(adb)
  • 8 devices is about the threshold for reliable simultaneously (bi-directional) testing
• Android<-Wifi->AP<-Host(adb)
  • A DHCP server handles assigning an IP to the Android device via MAC address so we can create a static mapping of ports to IPs for remapping via adb
  • 8 simultaneous Appium servers (thus devices) running per Window server/host is about what to expect
  • This works for about 32/64 devices per dedicated AP.
    • Remember the AP has to be ~4/5 feet away from the devices for proper signal attenuation
    • Mobile cloud datacenters use EMI / faraday cages to isolate each mobile device rack where multiple servers, an AP and the devices are installed within each rack
• Android<-USB2Ethernet<-Host(adb)
  • Not all devices support OTG, so this is not always an option, but works great.
  • A DHCP server is required just like the Android<-Wifi->AP setup

iOS is a totally different can of worms:

• 4 iOS devices connected to a Mac is about all you can achieve with reliable results.
• Remember a separate Xcode Instruments is spun up per Appium server on the host.
• Mac Minis work great for dedicated Appium test servers.

In terms of GPS testing, mocking is the easiest way to go. There are hardware devices designed for GPS testing that are use by the top tier app developers (mapping, traffic, avionics, etc..) but for an ad-hoc in-house device lab these are usually way to expensive to purchase and setup (EMI / faraday cages are needed enclosing the device and the GPS injector antenna).

Bluetooth can be a big problem in a device lab due to interference, pairing issues, etc..., having a number of devices in close proximity competing for a single (or more) bluetooth accessory can make the accessories fail to pair with any device. Basically bluetooth testing simultaneously does not work well, but you can use Appium to automate the turning the wifi/bluetooth/airplane mode on/off, but for Android devices this requires that they are all USB connected and not using adb to remap host ports to IP addresses over wifi...