iOS Packet Tunnel Provider with Local On-Device Server

Question

I'm using the Network Extension framework provided by Apple to build a packet sniffing/monitoring application similar to Charles Proxy and Surge 4 for iOS.

So far, I have the basic structure of the project up and running with the Main Application triggering the PacketTunnelProvider Extension where I can see packets being forwarded via the packetFlow.readPackets(completionHandler:) method. My background isn't in networking so I'm confused on the basic structure of these kinds of apps. Do they host a server on the device that act as the proxy which intercepts network requests? Could anyone provide a diagram of the general flow of the network requests? I.e. what is the relationship between the Packet Tunnel Provider, Proxy Server, Virtual Interface, and Tunnel?

If these apps do use a local on-device server, how do you configure the NEPacketTunnelNetworkSettings to allow for a connection? I have tried incorporating a local on-device server such as GCDWebServer with no luck in establishing a link between the two.

For example, if the GCDWebServer was reachable at 192.168.1.231:8080, how would I change the code below for the client to communicate with the server?

Main App:

    let proxyServer = NEProxyServer(address: "192.168.1.231", port: 8080)
    
    let proxySettings = NEProxySettings()
    proxySettings.exceptionList = []
    proxySettings.httpEnabled = true
    proxySettings.httpServer = proxyServer
    
    let providerProtocol = NETunnelProviderProtocol()
    providerProtocol.providerBundleIdentifier = self.tunnelBundleId
    providerProtocol.serverAddress = "My Server"
    providerProtocol.providerConfiguration = [:]
    providerProtocol.proxySettings = proxySettings
    
    let newManager = NETunnelProviderManager()
    newManager.localizedDescription = "Custom VPN"
    newManager.protocolConfiguration = providerProtocol
    newManager.isEnabled = true
    saveLoadManager()
    self.vpnManager = newManager

PacketTunnelProviderExtension:

func startTunnel(options: [String : NSObject]?, completionHandler: @escaping (Error?) -> Void) {
  ...
        let settings = NEPacketTunnelNetworkSettings(tunnelRemoteAddress: "127.0.0.143")
        settings.ipv4Settings = NEIPv4Settings(addresses: ["198.17.203.2"], subnetMasks: ["255.255.255.255"])
        settings.ipv4Settings?.includedRoutes = [NEIPv4Route.default()]
        settings.ipv4Settings?.excludedRoutes = []
        settings.dnsSettings = NEDNSSettings(servers: ["8.8.8.8", "8.8.4.4"])

        settings.dnsSettings?.matchDomains = [""]
        self.setTunnelNetworkSettings(settings) { error in
            if let e = error {
                NSLog("Settings error %@", e.localizedDescription)
            } else {
                completionHandler(error)
                self.readPackets()
            }
        }
  ...
}

Answer 1

I'm working on the iOS version of Proxyman and my experience can help you:

Do they host a server on the device that acts as the proxy which intercepts network requests?

Yes, you have to start a Listener on the Network Extension (not the main app) to act as a Proxy Server. You can write a simple Proxy Server by using Swift NIO or CocoaAsyncSocket.

To intercept the HTTPS traffic, it's a quite big challenge, but I won't mention here since it's out of the scope.

Could anyone provide a diagram of the general flow of the network requests?

As the Network Extension and the Main app are two different processes, so they couldn't communicate directly like normal apps.

Thus, the flow may look like:

The Internet -> iPhone -> Your Network Extension (VPN) -> Forward to your Local Proxy Server -> Intercept or monitor -> Save to a local database (in Shared Container Group) -> Forward again to the destination server.

From the main app, you can receive the data by reading the local database.

how do you configure the NEPacketTunnelNetworkSettings to allow for a connection?

In the Network extension, let start a Proxy Server at Host:Port, then init the NetworkSetting, like the sample:

    private func initTunnelSettings(proxyHost: String, proxyPort: Int) -> NEPacketTunnelNetworkSettings {
    let settings: NEPacketTunnelNetworkSettings = NEPacketTunnelNetworkSettings(tunnelRemoteAddress: "127.0.0.1")

    /* proxy settings */
    let proxySettings: NEProxySettings = NEProxySettings()
    proxySettings.httpServer = NEProxyServer(
        address: proxyHost,
        port: proxyPort
    )
    proxySettings.httpsServer = NEProxyServer(
        address: proxyHost,
        port: proxyPort
    )
    proxySettings.autoProxyConfigurationEnabled = false
    proxySettings.httpEnabled = true
    proxySettings.httpsEnabled = true
    proxySettings.excludeSimpleHostnames = true
    proxySettings.exceptionList = [
        "192.168.0.0/16",
        "10.0.0.0/8",
        "172.16.0.0/12",
        "127.0.0.1",
        "localhost",
        "*.local"
    ]
    settings.proxySettings = proxySettings

    /* ipv4 settings */
    let ipv4Settings: NEIPv4Settings = NEIPv4Settings(
        addresses: [settings.tunnelRemoteAddress],
        subnetMasks: ["255.255.255.255"]
    )
    ipv4Settings.includedRoutes = [NEIPv4Route.default()]
    ipv4Settings.excludedRoutes = [
        NEIPv4Route(destinationAddress: "192.168.0.0", subnetMask: "255.255.0.0"),
        NEIPv4Route(destinationAddress: "10.0.0.0", subnetMask: "255.0.0.0"),
        NEIPv4Route(destinationAddress: "172.16.0.0", subnetMask: "255.240.0.0")
    ]
    settings.ipv4Settings = ipv4Settings

    /* MTU */
    settings.mtu = 1500

    return settings
}

Then start a VPN,

let networkSettings = initTunnelSettings(proxyHost: ip, proxyPort: port)

// Start
setTunnelNetworkSettings(networkSettings) { // Handle success }

Then forward the package to your local proxy server:

let endpoint = NWHostEndpoint(hostname: proxyIP, port: proxyPort)
self.connection = self.createTCPConnection(to: endpoint, enableTLS: false, tlsParameters: nil, delegate: nil)

    packetFlow.readPackets {[weak self] (packets, protocols) in
        guard let strongSelf = self else { return }
        for packet in packets {
            strongSelf.connection.write(packet, completionHandler: { (error) in
            })
        }

        // Repeat
        strongSelf.readPackets()
    }

From that, your local server can receive the packages then forwarding to the destination server.