Rails: Timeout when opening file on same server

Question

I have a Rails controller method that does nothing but opens a file on the same server:

open 'http://lvh.me:3000/feeds/madonna.rss'

About 60% of the time, when I start the server, and hit this controller method, the 'open' call times out with:

Net::ReadTimeout: Net::ReadTimeout

The rest of the time it opens just fine.

It also seems that if the first request works after starting the server, it'll continue to work on subsequent requests. And if the first request fails, it'll continue to fail.

The failures only seem to occur when the file to open is on the same server that's make the request. In other words, when I change the URL to call the same endpoint but on the staging server, it seems to work every time.

The same exact behavior occurs with both open-uri and FeedJira.

Could there be some blocking issue with the server trying to make a request to the same server? Any ideas how to get around this or debug it further?

Here is the controller method that makes the call to generate the feed:

  def add_feed
    url = params['url']
    puts "FEEDJIRA"
    open url
    # feed = Feedjira::Feed.fetch_and_parse(url)
    # eventually return response... but we timeout on the line above
  end

Here is the controller that the 'url' above hits (i.e. 'http://lvh.me:3000/feeds/madonna.rss') to generates the feed:

class FeedsController < ApplicationController

  def show
    respond_to do |format|
      format.rss  { render :layout => false }
    end
  end

end

Here's the log:

env_host: lvh.me:3000
Host: lvh.me
   (0.9ms)  BEGIN
  SQL (0.7ms)  UPDATE "users" SET "last_seen_at" = $1, "updated_at" = $2 WHERE "users"."id" = $3  [["last_seen_at", "2018-01-08 19:47:29.215953"], ["updated_at", "2018-01-08 19:47:29.218610"], ["id", 3071]]
   (1.6ms)  COMMIT
FEEDJIRA
Started GET "/feeds/madonna.rss" for 127.0.0.1 at 2018-01-08 11:47:29 -0800
Completed 500 Internal Server Error in 120051ms (ActiveRecord: 7.3ms)



Net::ReadTimeout (Net::ReadTimeout):

app/controllers/registrations_controller.rb:105:in `add_feed'

Answer 1

I have a Rails controller method that does nothing but opens a file on the same server

When you say this, I expect that action to do File.open('path/to/some/local/file.txt'). There's much more happening here. You're calling Kernel#open which does different things based on the input being passed to it. Assuming the input is something that looks like a URL, it's opening a connection to fetch that URL.

What is the value you are passing is params[:url]? Are you able to inspect the server logs at whatever that remote URL is to verify the connection is ever established? If it is established, is it just taking a long time to process the request? Are you sure params[:url] is reachable from your server?

Answer 2

It looks like it dies after taking 12+ seconds. What is in app/views/feeds/show.html.haml, which it seems is what does all the work? Probably you'll need to find a way to generate that result more quickly, e.g. by caching or by making sure you don't have n+1 problems or other inefficiencies.

Answer 3

The failure - 500 - means error in the server. The client has nothing to do with it. You are not opening a file, you perform a HTTP call. The error code means that the server decided that the call takes too much, interrupted it and returned the error. The same error shall occur if you use a Java client or a browser.

The problem is your application on the server side. Since you mention that the error only occurs at startup, can it be that the server is waiting for another connection, such as database, search, etc?