How to configure Nginx SSL to work while 443 is already taken by other process?

Question

I'm trying to set up a server with two websites, one is my Angular+Spring Boot server, another is for BTCPay server. Each with a different domain name.
In BTCPay server, those environment variables are set:

export BTCPAY_HOST="btcpay.YourDomain.com"
export NBITCOIN_NETWORK="mainnet"
export BTCPAYGEN_CRYPTO1="btc"
export BTCPAYGEN_CRYPTO2="ltc"
export BTCPAYGEN_REVERSEPROXY="nginx"
export BTCPAYGEN_LIGHTNING="clightning"

In short, port 443 is already taken by the BTCPay server, a process I have barely configure.

At first, I thought it's not a big deal, I just need to put my other server on another port 1880 for HTTP and 3443 for HTTPS. But after switch port, the Letsencrypt certificate won't pass any verification, browsers all start to give me the warning:

The certificate is not trusted because it is self-signed.
 
Error code: MOZILLA_PKIX_ERROR_SELF_SIGNED_CERT

Based on some other online resources, SSL connection using Nginx must take 443 to verify the certificate, so my legit certificate is not verified properly, which is causing the problem.
Can someone help me to fix this?
Is there a way for me to make it through the verification process by editing Nginx configuration?

Answer 1

The official document actually already has a solution for this problem, though it's not clear enough for me: BTCPay Server Document, under the Can I use an existing Nginx server as a reverse proxy with SSL termination? section.
Config your existed Nginx configuration files, to add the new server sections inside. After the change, new problems appeared for me:
Port 80 already taken: which is because currently BTCPay Server docker is trying to use 80 as the incoming port, so environment variable REVERSEPROXY_HTTP_PORT (The public port the reverse proxy binds to for HTTP traffic (default: 80)) need to be changed. I changed it to 1880, so in the Nginx config file, proxy_pass http://127.0.0.1:10080; need to change to: proxy_pass http://127.0.0.1:1880;

After this change, most problems are solved, but when I try to access https://www.example.com it responds with error code 503.
This is because the BTCPAY_HOST environment variable has been set to "example.com". To solve this I choose to separate "www.example.com" and "example.com" in the Nginx config file's server section, so https://www.example.com will be redirected to https://example.com.

After this, the server is working for both the original server and the BTCPay Server, hopefully this will help someone else suffering from the same problem :D