WebExtensions : Can you access a JSON object that was fetched by the webpage

Question

Is there a way after a webpage has loaded to access a JSON object that was fetched by the webpage?

I am aware that I could use the webRequest api to get the url of the JSON resource and fetch it again but i'd like to know if there's a way to access it without re-downloading?

Can I hook into the pages javascript or is there a better way?

This might seem unnecessary given I could just re-download but that doesn't feel very elegant to me.

I have googled for a while but can't find much. A steer in the right direction would be appreciated.

I am using Firefox

Answer 1

In my scenario the JSON data I wish to access is loaded by an xmlhttprequest

Capturing such a JSON resource can be simply achieved with Firefox by capturing the http response using browser.webRequest.onBeforeRequest.addListener and then filtering the response within the listener using browser.webRequest.filterResponseData

Given that JSON files can be quite large the following is a minimal example built upon the generic example found here.

background.js

function listener(details) {
    const filter = browser.webRequest.filterResponseData(details.requestId);
    const decoder = new TextDecoder("utf-8");
    const encoder = new TextEncoder();
  
    const data = [];
    filter.ondata = (event) => {
      data.push(event.data);
    };
  
    filter.onstop = (event) => {
      let jsonStr = "";

      if (data.length === 1) {
        jsonStr = decoder.decode(data[0]);
      } else {
        for (let i = 0; i < data.length; i++) {
          const stream = i !== data.length - 1;
          jsonStr += decoder.decode(data[i], { stream });
        }
      }
      
      // build json object from string.
      var jsonObject = JSON.parse(jsonStr);

      filter.write(encoder.encode(jsonStr));
      filter.close();
    };
  }
  
  browser.webRequest.onBeforeRequest.addListener(
    listener,
    { urls: ["https://example.com/jsonresource/*"], types: ["xmlhttprequest"] },
    ["blocking"],
  );

Ensure "webRequestBlocking" permission is included in manifest.