Use Prerender.io on one.com server with AngularJS

Question

For my AngularJS app I try to use the service of Prerender.io on a server by one.com. The webserver used is Apache.

I already tried to use the following .htaccess file (compare https://gist.github.com/thoop/8072354)

# Change YOUR_TOKEN to your prerender token and uncomment that line if you want to cache urls and view crawl stats
# Change http://example.com (at the end of the last RewriteRule) to your website url

<IfModule mod_headers.c>
    RequestHeader set X-Prerender-Token "YOUR_TOKEN"
</IfModule>

<IfModule mod_rewrite.c>
    RewriteEngine On

    # If requested resource exists as a file or directory
    # (REQUEST_FILENAME is only relative in virtualhost context, so not usable)
    RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -f [OR]
    RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -d
    # Go to it as is
    RewriteRule ^ - [L]

    RewriteCond %{HTTP_USER_AGENT} baiduspider|facebookexternalhit|twitterbot|rogerbot|linkedinbot|embedly|quora\ link\ preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator [NC,OR]
    RewriteCond %{QUERY_STRING} _escaped_fragment_

    # Only proxy the request to Prerender if it's a request for HTML
    RewriteRule ^(?!.*?(\.js|\.css|\.xml|\.less|\.png|\.jpg|\.jpeg|\.gif|\.pdf|\.doc|\.txt|\.ico|\.rss|\.zip|\.mp3|\.rar|\.exe|\.wmv|\.doc|\.avi|\.ppt|\.mpg|\.mpeg|\.tif|\.wav|\.mov|\.psd|\.ai|\.xls|\.mp4|\.m4a|\.swf|\.dat|\.dmg|\.iso|\.flv|\.m4v|\.torrent|\.ttf|\.woff))(.*) http://service.prerender.io/http://%{HTTP_HOST}/$2 [P,L]

    # If non existent
    # Accept everything on index.html
    RewriteRule ^ /index.html [L]
</IfModule>

I tried to use Google Webmaster Tools (Fetch as Google) to check if it works. Unfortunately, the website that the Google bot is fetching is not the one that is prefetched by Prerender.io.

It might be, that the P flag in the RewriteRule is not executed. But I am not quite sure about that.

Answer 1

Fetch as Google has a known issue where it doesn't check for the escaped fragment protocol. The actual Googlebot does not have this issue and checks for the meta fragment tag and adds the escaped fragment parameter.

For either #! URLs or HTML5 push state URLs you still need to append ?_escaped_fragment_= manually.

Here's an answer from Google on that: https://productforums.google.com/forum/#!topic/webmasters/YaOZrKGYrGA

If you have a #! URL like http://www.example.com/#!/user/1 then you'll want to have Fetch as Google fetch this URL http://www.example.com/?_escaped_fragment_=/user/1

If you have a non #! URL like http://www.example.com/user/1 then you'll want to have Fetch as Google fetch this URL http://www.example.com/user/1?_escaped_fragment_=

Feel free to email me at [email protected] if you're still seeing issues!