ASP.NET MVC : check if referrer from same project and specified controller and action

Question

I am working on an ASP.NET MVC project. I have an action which has static base64 of an image.

I want this action to render the correct image only if it is called from myproject/allowedController/allowedAction else show a different image.

I also want to restrict if someone is loggedin at my website and open the image url directly, but referrer is always returned as empty.

I tried with this code:

string host = Request.ServerVariables["HTTP_REFERER"].ToString().Replace("http://", "").Replace("https://", "").ToLower();   
//string host = Request.UrlReferrer.Host.ToString().Replace("http://", "").Replace("https://", "").ToLower();

 string localPath = Request.UrlReferrer.LocalPath.ToString().ToLower();

 if (host != "www.mydomain.com")
 {
       throw new Exception("unauthorized request code 1");
 }

 if (!(localPath.StartsWith("/allowedController/allowedAction")))
 {
        throw new Exception("unauthorized request code2");
 }

Host is always throwing an error

Object reference not set to an instance of an object