ASP.NET core session keeps expiring regardless of the IdealTimout setting

Question

I have asp.net core 1.1 application and i am setting the session timeout for 3 hours. So i am expecting the data in the session will remain for 3 hours.

    public void ConfigureServices(IServiceCollection services)
    {
        // authorization                      
        services.AddAuthorization();
        services.AddMvc(config =>
        {
            var policy = new AuthorizationPolicyBuilder()
            .RequireAuthenticatedUser()
            .Build();
        })
        services.AddKendo();
        services.AddSession();
    }

    public void Configure(IApplicationBuilder app, IHostingEnvironment env)
    {            
        //configure identity server 3
        app.UseIdentityServer(Configuration["Identity:Authority"],
            Configuration["Identity:ClientId"],
            Configuration["Identity:PostLogoutRedirectUri"],
            domainService);

        app.UseSession(new SessionOptions()
        {
            IdleTimeout = new TimeSpan(3, 0, 0)
        });
        app.UseMvc(routes =>
        {                
        });

        // Configure Kendo UI
        app.UseKendo(env);

    }

However after remaining ideal for 20 mins, user's session gets timeout

Based on the documentation

The app uses the IdleTimeout property to determine how long a session can be idle before its contents in the server's cache are abandoned. This property is independent of the cookie expiration.

So there is timeout for data, and a separate timeout for session cookie. However based on issue 169 you cannot set timeout for session cookie.

My application is also using IdentityServer 3 and creates auth cookie which has longer timeout (more than 3 hours). and i dont think auth cookie has anything to do with session cookie.

Questions
Is there any other additional session related setting i need to do?
How can i check particular cookie's expiration time. On server i can not see that information in Request.Cookies. its just key-value pair.