Use Proxy and Headers in AngleSharp.Io

Question

I am trying to get AngleSharp to use both a proxy and set header properties like this:

        var handler = new HttpClientHandler
        {
            Proxy = new WebProxy(ProxyMan.GetProxy()),
            UseProxy = true,
            PreAuthenticate = true,
            UseDefaultCredentials = false
        };
        var requester = new DefaultHttpRequester();
        requester.Headers["User-Agent"] = Tools.GetAgentString();
        requester.Headers["Accept-Language"] = "en-US";
        requester.Headers["Accept-Charset"] = "ISO-8859-1";
        requester.Headers["Content-Type"] = "text/html; charset=UTF-8";
        var config = Configuration.Default
            .WithRequesters(handler)
            .With(requester)
            .WithTemporaryCookies()
            .WithDefaultLoader();
        var context = BrowsingContext.New(config);
        var doc = await context.OpenAsync(Url);

When I added the Header requester, it stopped the proxy handler from working. I know there is some conflict between the .WithRequesters() and the .With() but I cannot locate the proper syntax for doing both in the same request.

Thanks.

Answer 1

Yeah I am not sure why you use both.

1. The DefaultHttpRequester is the requester from AngleSharp and uses WebClient underneath
2. WithRequesters comes from AngleSharp.Io and adds all available requesters (incl. HttpClientRequester)
3. WithDefaultLoader puts a loader into the config and registers the default HTTP requester, if no other requester has been registered yet

I think what you want to do should be actually done using the HttpClientRequester directly.

        var handler = new HttpClientHandler
        {
            Proxy = new WebProxy(ProxyMan.GetProxy()),
            UseProxy = true,
            PreAuthenticate = true,
            UseDefaultCredentials = false,
            UseCookies = false,
            AllowAutoRedirect = false
        };
        var client = new HttpClient(handler);
        client.DefaultRequestHeaders.Append("User-Agent", Tools.GetAgentString());
        // ... and the others if you want, even though `content-type` etc. should / will be determined by AngleSharp
        var config = Configuration.Default
            .With(new HttpClientRequester(client))
            .WithTemporaryCookies()
            .WithDefaultLoader();
        var context = BrowsingContext.New(config);
        var doc = await context.OpenAsync(Url);