How to create span tree in distributed tracing system using Golang

Question

I already have a microservice and I want to apply distributed tracing for it. I have written the following code to create a tracing span using jaeger and opentracing , but my problem is that it does not create parrent id and I am not able to see the span graph. It seems it only shows the default operations in UI dashboard. I used docker-compose for config jaeger.

 jaeger:
  container_name: jaeger
  image: jaegertracing/all-in-one:1.39
  restart: always
  ports:
    - 5775:5775/udp
    - 6831:6831/udp
    - 6832:6832/udp
    - 5778:5778
    - 16686:16686
    - 14268:14268
    - 9411:9411
    
  environment:
    - COLLECTOR_OTLP_ENABLED=true
    - OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:16686"
    - COLLECTOR_ZIPKIN_HTTP_PORT=9411
    - JAEGER_SERVICE_NAME=baran-endpoint
    - JAEGER_AGENT_HOST=jaeger
    - JAEGER_REPORTER_LOG_SPANS=true
    - JAEGER_SAMPLER_TYPE=const
    - JAEGER_SAMPLER_PARAM=1
    - JAEGER_SAMPLER_SERVER_URL=
  networks:
    - network-jaeger

I tried to implement a basic architecture in jaeger component,but it seems that i'm missing somthing in between.

var (
  TracingAnalysisEndpoint      = "http://localhost:26855/SaleService.svc"
)
const AgentSwitch = false

func NewJaegerTracer(service string) opentracing.Tracer {
    if AgentSwitch == true {
        return NewJaegerTracerAgent(service)
    }
    return NewJaegerTracerDirect(service)
}

func NewJaegerTracerDirect(service string) opentracing.Tracer {
    sender := transport.NewHTTPTransport(
        TracingAnalysisEndpoint,
    )
    tracer, _ := jaeger.NewTracer(service,
        jaeger.NewConstSampler(true),
        jaeger.NewRemoteReporter(sender, jaeger.ReporterOptions.Logger(jaeger.StdLogger)),
    )
    return tracer
}

func NewJaegerTracerAgent(service string) opentracing.Tracer {
    sender, _ := jaeger.NewUDPTransport("",0)
    tracer, _ := jaeger.NewTracer(service,
        jaeger.NewConstSampler(true),
        jaeger.NewRemoteReporter(sender, jaeger.ReporterOptions.Logger(jaeger.StdLogger)),
    )
    return tracer
}

func StartSpanFromRequest(tracer opentracing.Tracer, r *http.Request, funcDesc string) opentracing.Span {
 spanCtx, _ := Extract(tracer, r)
 return tracer.StartSpan(funcDesc, ext.RPCServerOption(spanCtx))
}

func Inject(span opentracing.Span, request *http.Request) error {
 return span.Tracer().Inject(
  span.Context(),
  opentracing.HTTPHeaders,
  opentracing.HTTPHeadersCarrier(request.Header))
}

func Extract(tracer opentracing.Tracer, r *http.Request) (opentracing.SpanContext, error) {
 return tracer.Extract(
  opentracing.HTTPHeaders,
  opentracing.HTTPHeadersCarrier(r.Header))
}

I created first span on one of my services.

  span:= config.StartSpanFromRequest(config.Tracer,ctx.Request(), "customer method")
  defer span.Finish()   
  span.LogKV(fmt.Sprintf("Customer not found: %s", request.Mobile))

It only shows the default operations in UI dashboard.

I can not create parent and child spans in my trace correctly.

I want to create spans in hierarchical order

Answer 1

To create a parent-child relationship between spans, you need to make sure the context of the parent span is passed to the child span creation process. That is often done by propagating the span context through the headers in the case of HTTP requests.

So, modify your StartSpanFromRequest, making sure the function correctly extracts the parent span context from the incoming request. That context is then used to create the new span.

func StartSpanFromRequest(tracer opentracing.Tracer, r *http.Request, operationName string) opentracing.Span {
    parentSpanContext, _ := tracer.Extract(opentracing.HTTPHeaders, opentracing.HTTPHeadersCarrier(r.Header))
    span := tracer.StartSpan(operationName, ext.RPCServerOption(parentSpanContext))
    return span
}

When making HTTP requests to other services, inject the current span context into the HTTP headers. That allows the called service to extract the parent span context and create a child span.

Example of creating a child span, when making an HTTP request to another service:

span := StartSpanFromRequest(tracer, r, "customer method")
defer span.Finish()

// Inject current span context into the headers of the outgoing request
outgoingRequest, _ := http.NewRequest("GET", "http://other-service-url", nil)
Inject(span, outgoingRequest)

// Make the request
// 

Example of using the span in the called service:

// In the called service
func SomeHTTPHandler(w http.ResponseWriter, r *http.Request) {
    span := StartSpanFromRequest(tracer, r, "called method")
    defer span.Finish()

    // Business logic
    // 
}

You need to propagate the span context for all inter-service calls to maintain the trace continuity.