How can I get Meteor's HTTP.call (or get) method to modify the HTTP request Headers?

1k views Asked by At

I am trying to speed up my Meteor application by only loading enough content of a webpage to get the <head> tag of its HTML, to obtain its title, image, and description. I have a client calling a server-side method with the following code:

Meteor.call("metaGetter", url, function(err, res){...});

And on the server side, in the metaGetter method, I am using Meteor's HTTP.call:

var result = HTTP.call('GET', url, {headers: {'content-range': "bytes 0-100"}});

as written in Meteor's documentation. I am able to get the result's content, html. However, after printing the returned headers, I do not see the content-range attribute that I have tried to set.

Edit: Akshat's solution works, but only for some websites, very few in fact. Any help would be much appreciated.

2

There are 2 answers

0
Jakozaur On BEST ANSWER

In general, you can't have fixed limit if you want always fetch the title:

  1. Some HTTP servers doesn't support range header: How can I find out whether a server supports the Range header?
  2. You can't guarantee that X bytes will always contain title. E.g. it may appear after 1000 bytes.

In general I would fetch whole HTML file. On most decent servers, that should take less than 100 ms. Hardly noticeable by human. If you do that a lot, you may want to allow executing server side method in parallel (see http://docs.meteor.com/#/full/method_unblock)

If optimization is must, you can use previous method, fetch 100 bytes, but if you don't find </title> than you fall back to downloading whole HTML file.

7
Tarang On

use the range header:

var result = HTTP.call('GET', url, {headers: {'range': "bytes=0-100"}});

The response should have a content-range header if the server used supports content ranges.

Of course, this needs a host that supports request ranges. I've tried the above code and it does work on http://www.microsoft.com as the url.

Its sad to say there's nothing you can do really for websites that don't support it besides requesting the entire document.

One rather weird alternative is to manually request the webpage as a socket and cut off when you get more bytes than what you need.