Bulletproof HTTP Monitor for iOS

Question

I'm using Charles Proxy and Wireshark to monitor http(s) traffic from various iOS apps I'm using on my iPhone. These apps require me to set the HTTP Proxy under the iOS Wifi settings (let's call these the Proxy Settings).

My business needs to see ALL URL's that are being called from my phone. From all apps. All URL's, not some of them.

Now Charles and Wireshark both work fine and I can see a ton of traffic coming from my phone.

However, I can't help but wonder whether I might be missing some HTTP calls. Maybe calls that don't use the Cocoa Core Foundation libraries as the basis for their networking.

For instance, I could write my own HTTP library out of TCP/IP and these would bypass the Proxy Settings.

So my question is: what is the likelihood that some apps are using custom-rolled HTTP libraries and side-stepping my Proxy Settings. Or worse, they're using raw TCP/IP to communicate with a server. I know it's possible, but do any APIs work this way? Does anyone do it?

Answer 1

I found the answer: Use mitmproxy in transparent mode. proxy is not used. harder to setup because it needs work on the router, but it reliably captures every packet on port 80 and 443 regardless of proxy settings.

Answer 2

Assuming that you are able to keep your device tethered, then you may be able to use the pcap service to monitor all traffic. According to the following paper (2014) the pcap service is running on every iOS device:

"Identifying back doors, attack points, and surveillance mechanisms in iOS devices"

You should be able to connect to it via usbmuxd. I'm not sure whether there is a pre-rolled client for the pcap service. There is a list of services supported by libimobiledevice here. Pcap is not on that list.

Alternatively, you can use wireshark to capture all traffic on your wifi network.