How do I stream with grpc-web and use a REST API on the same port?

Question

I have a server witten in Go that uses cmux to allow multiple protocols to run on the same port, but there's a limitation specified in the README saying that

cmux matches the connection when it's accepted. For example, one connection can be either gRPC or REST, but not both. That is, we assume that a client connection is either used for gRPC or REST.

I need the browser to be able to both stream from grpc-web and call a REST API on the same port, but the browser reuses the same connection and causes the muxing to not work.

Answer 1

This is a pretty tricky problem to identify. Since browsers prefer to use an existing TCP connection for pipelining, the mux tends to send packets to the wrong protocol handler, for example it could send grpc-web packets to REST, and vice versa. Luckily, there is a pretty simple solution for this:

package main

listener := net.Listen("tcp", ":2289")
multiplexer := cmux.New(listener)
grpcListener := multiplexer.Match(cmux.HTTP2())
httpListener := multiplexer.Match(cmux.HTTP1Fast())
grpcServer := grpc.Server()
wrappedGrpc := grpcweb.WrapServer(grpcServer)
go func() {
  httpServer := echo.New()
  (&http.Server{
      Handler: http.HandlerFunc(func(resp http.ResponseWriter, req *http.Request) {
        if strings.Contains(req.Header.Get("Access-Control-Request-Headers"), "x-grpc-web") || req.Header.Get("x-grpc-web") == "1" || req.Header.Get("Sec-Websocket-Protocol") == "grpc-websockets" {
          inst.API.GrpcWebServer.ServeHTTP(resp, req)
        } else {
          httpServer.ServeHTTP(resp, req)
      }
    }),
  }).Serve(httpListener)
}()

go func() {
  grpcServer.Serve(grpcListener)
}()

go func() {
  multiplexer.Serve()
}()

How does this work?

Essentially, instead of using cmux's default muxing (which only muxes per-connection) we registered a new mini http server handler on all HTTP requests coming in, which then lets us explicitly check headers and call handlers directly.