wget not displaying all website data

225 views Asked by At

Input: wget -qO- http://runescape.com/community | grep -i playerCount

Output: <li class="header-top__right-option"><strong id="playerCount">0</strong> Online</li>

In browser: enter image description here

Using cygwin..I am trying to use wget to pull a number out of a webpage. As shown in the example above, the playerCount is 0. If you actually load the webpage up and look at the same code, it is a completely different number. How can I get the real number? I was told it may be something with cookies or a user agent. This just started not working a few weeks ago.

2

There are 2 answers

2
Etan Reisner On BEST ANSWER

That value appears to be filled in via javascript (though I can't find the request at a quick glance). If that's the case then you cannot get it with something like wget or curl in this way. You would need to find the specific request and send that.

Given the URL indicated by aadarshs (which I saw but mistested when I looked at it the first time) something like this should work.

curl -s 'http://www.runescape.com/player_count.js?varname=iPlayerCount&callback=jQuery000000000000000000000_0000000000000' | awk -F '[()]' '{print $2}'
3
aadarshsg On

This worked for me

curl http://runescape.com/community | grep -i playercount

EDIT: Adding the player count link

curl http://www.runescape.com/player_count.js\?varname\=iPlayerCount\&callback\=jQuery111004241600367240608_1434074587842\&_\=1434074587843