PagedList Filter

Question

I have a Product Controller that has an Index method that displays all the products available. However, I have three buttons on the home page that when clicked will pass a asp-route-category ="CategoryName" and this is checked in the Index method to display the matching products with the same category. This is my code for the Index method

   public async Task<IActionResult> Index(string searchString, string category, int? page)
          {
              var pageNumber = page ?? 1;

              var product = _context.Products.Include(p => p.Category);

              if (!String.IsNullOrEmpty(searchString))
              {
                  product = product.Where(p => p.Name.Contains(searchString)).Include(c => c.Category);
              }
              if (!String.IsNullOrEmpty(category))
              {
                  if (category == "New")
                  {
                      product = product.Where(p => p.Category.Name == "New").Include(c => c.Category);
                      return View(product.ToPagedList(pageNumber, 6));
                  }
                  else if (category == "Popular")
                  {
                      product = product.Where(p => p.Category.Name == "Popular").Include(c => c.Category);
                      return View(product.ToPagedList(pageNumber, 6));
                  }
                  else if (category == "Special")
                  {
                      product = product.Where(p => p.Category.Name == "Special").Include(c => c.Category);
                      return View(product.ToPagedList(pageNumber, 6));
                  }

              }
              //maximum of 6 on a page
              return View(product.ToPagedList(pageNumber, 6));
          }

This works however,when I try to click to page 2, it directs me to the entire list of products instead of keeping it filtered based on category. This is my PagedList code

@*display number of pages *@
@Html.PagedListPager(Model, page => Url.Action("Index",
        new { Page = page }),
        new PagedListRenderOptions
        {
            LiElementClasses = new string[] { "page-item" },
            PageClasses = new string[] { "page-link" }
    })

How should I be able to display only the matching ones?