I am trying to scrap a web page. Part of the data on this web page is updated using following script:
<script>
(function ($) {
$(document).ready(function () {
var timeoutID = window.setTimeout(function () {
var $promise = $.ajax({
method: 'POST',
url: 'https://www.somedomain.com/cms/wp-admin/admin-ajax.php',
data: {
action: 'get_cache_guide',
symbol_id: 6533,
_wpnonce: 'b0ce23fbe8'
}
});
$promise.always(function (data) {
Chart_Guide({charts: {guide: JSON.parse(data)}});
});
}, 500);
});
}(jQuery));
</script>
So, my question is how I can get this data (JSON) based on which some table and graphs are displayed using Chart_Guide function?
I have tried the following (parsing text of above script element):
method <- strsplit(strsplit(script, "method: '")[[1]][2], "',\n")[[1]][1]
urlOfInterest <- strsplit(strsplit(script, "url: '")[[1]][2], "',\n")[[1]][1]
action <- strsplit(strsplit(script, "action: '")[[1]][2], "',\n")[[1]][1]
symbolID <- as.numeric(strsplit(strsplit(script, "symbol_id: ")[[1]][2], ",\n")[[1]][1])
wpnonce <- strsplit(strsplit(script, "_wpnonce: '")[[1]][2], "'\n")[[1]][1]
data <- strsplit(strsplit(script, "data: ")[[1]][2], "});")[[1]][1]
data <- gsub("\\\n", "", data)
Then I used rvest and httr in the following way(s):
pageSession <- request_POST(pageSession, url=urlOfInterest, body=data)
Then I tried:
values <- list("action"=action, "symbol_id"=symbolID, "_wpnonce"=wpnonce)
pageSession <- request_POST_1(pageSession, url=urlOfInterest, body=values)
And it all fails...
Suggestions on how to accomplish this task highly appreciated.