Allowing all access to robots.txt in nginx?

Question

I have a website with what I think is an uncomplicated setup. In my main location / {...} block, I have a bunch of deny directives for specific IP addresses that seem to be malicious in some way. I would like to add a block to allow anyone to reach my robots.txt file (i.e., even the people who are blocked with a deny directive), but I can't get this to work.

I tried to add, at the same level as the main location / {} block, the block location = /robots.txt { allow all; }, but when I restart nginx, no one can reach robots.txt; everyone gets a 404, which seems odd—if I screwed something up, I'd think people would just get their access denied, not a statement that the file doesn't exist.

What's the correct way to do this?

Edit: It seems pointless to show most of the code, as it feels clearly irrelevant, but the gist is:

location / {
    deny 1.2.3.4;
    deny 5.6.7.8;
    # etc.
    proxy_pass http://something/;
    # other proxy stuff here
}

location /static {
    alias /path/to/static/files;
}

# If this is here, going to http://example.com/robots.txt returns 404 for everyone
# If it's commented out, everyone can see it
# (except for 1.2.3.4 and 5.6.7.8, in the deny directive above:
# they get a 403. I want them to be able to see it.)
location = /robots.txt {
    allow all;
}

Answer 1

Since you're getting everything from backend (the proxy_pass http://something/;), you have to repeat it in your robots.txt location. Without it, NGINX will try to find it on the filesystem, and of course, return 404.

Should be:

location = /robots.txt {
    allow all;
    proxy_pass http://something/;
}