Seeminly impossible to change HTTP headers in webkit2gtk

Question

To start things off, there are no functions to change these http headers, which already seems ridiculous. I'm attempting to craft an unfingerprintable browser, and the first step was to modify the user agent. This was easily exposed via a webkit api. Next, http headers. After a lot of research I found out that webkit utilized a library called libsoup, and I would have to pull each request with a signal and manually modify it, instead of simply changing the default headers. But I went along with it. I attempted to catch resource-load-starting and resource-load-started, but while I could temporarily change the http headers, and according to libsoup they had truly changed, when testing it by loading up various websites that tell you your http headers, they did not change.

For the record, I was modifying the "accept" header using soup_message_headers_remove(); Next, I attempted to build a WebExtension, a whole separate library, simply to change the header. I was catching the send-request symbol. But still, modifying the accept header did not show any change. So now I'm wondering, is this even possible? Do the creators of WebKit simply not allow you to change this atrocious outdated header? I've seen people do work arounds on software containing webkit, but no solutions work for webkit2, at least for me. Perhaps there is an easier api to expose these headers or change the things they present to the websites, such as changing the xml accept through a webkit api? Any suggestions would be appreciated.

Answer 1

Answer tested through Python version of web extension.

WebPage::send-request signal ... it's the replacement of the resource-load-started signal in wk1.

https://narkive.com/XYgtjDeN:12.1465.275

WebKit1 was not multiprocess, the replacement for that signal is WebPage::send-request.

https://narkive.com/XYgtjDeN:12.1907.87

In WebKit1, it was possible to change HTTP headers without the use of WebExtensions but it would have made the main UI slower. For that reason, WebPage::send-request signal was introduced.

Now for how to solve your problem, you will have to create an extension and listen for WebPage::send-request signals. Here is a very nice example of how to change the headers (I tested it myself). See the following code:

#include <webkit2/webkit-web-extension.h>

static gboolean set_http_headers(WebKitWebPage * page, WebKitURIRequest * req, WebKitURIResponse * res)
{

    SoupMessageHeaders * http_headers = webkit_uri_request_get_http_headers(req);

    const gchar * type = webkit_uri_request_get_http_method(req);

    if (strncmp(type, "GET", 3) == 0)
        {
            // ** this is where the crash happens **
            soup_message_headers_replace(http_headers, "Accept-Encoding", "gzip, deflate, br");
            soup_message_headers_replace(http_headers, "Accept-Language", "en-US,en;q=0.5");
        }

    return FALSE;

}

static void wp_handle_cb(WebKitWebExtension * webext, WebKitWebPage * wp, gpointer dummy)
{
    g_signal_connect_object (wp, "send-request", G_CALLBACK(set_http_headers), NULL, 0);
}

G_MODULE_EXPORT void webkit_web_extension_initialize(WebKitWebExtension * webext)
{
    g_signal_connect (webext, "page-created", G_CALLBACK (wp_handle_cb), NULL);
}

For more documentation on SoupMessageHeaders, you can visit here. In WebKit2.WebContext, you can use set_web_extensions_directory to set the directory where your COMPILED extension is. Then initialize the extension using set_web_extensions_initialization_user_data. Pass the created context to WebKit2.WebView.new_with_context and your are done!