What escapes to the heap?

Question

I have this piece of code, which is supposed to do no allocations at all, but for some reason it does. As benchmarks say, there happens 2 allocs/op.

Which lines of the functions do the allocation and why?

The functions:

func (vi *VarInt /* int32 */) Read(input io.Reader) error {
    var (
        b     byte
        buf   = unsafe.Slice(&b, 1)
        shift int
        value uint32
    )
    for {
        _, err := io.ReadFull(input, buf)
        if err != nil {
            return err
        }

        value |= (uint32(b) & 0b01111111) << shift

        if (b & 0b10000000) == 0 {
            *vi = VarInt(value)
            return nil
        }

        shift += 7
        if shift >= 32 {
            return ErrVarIntTooLong
        }
    }
}

func (vi *VarInt /* int32 */) Write(output io.Writer) error {
    var (
        varint [5]byte
        uvalue = uint32(*vi)
        x      int
    )
    for ; ; x++ {
        vb := uint8(uvalue)

        if (vb & 0b10000000) == 0 {
            varint[x] = vb
            break
        }

        varint[x] = (vb & 0b01111111) | 0b10000000

        uvalue >>= 7
    }

    _, err := output.Write(varint[:x+1])
    if err != nil {
        return err
    }

    return nil
}

The benchmark:

func BenchmarkVarInt(b *testing.B) {
    var buf bytes.Buffer
    buf.Grow(5)
    b.ResetTimer()
    for i := 0; i < b.N; i++ {
        vi := (VarInt)(i)
        vi.Write(&buf)
        vi.Read(&buf)
        buf.Reset()
    }
}

I suppose the buf slice somehow escapes, but I can't figure out how, because as far as I understand a slice is a struct allocated on the stack in this case, which would point to the variable b as its data. I tried changing the expression unsafe.Slice(&b, 1) to (*[1]byte)(unsafe.Pointer(&b))[:], but it changed nothing.

Answer 1

When a value is boxed in an interface, it's always considered to escape — even if the value is never used outside of the call stack, Go just stops the analysis at that point and considers that someone might have gotten hold of the address, and therefore the value has to go on the heap.

Since Read takes io.Reader and Write takes io.Writer, buf (which is a bytes.Buffer passed to both functions) has to escape.

Even if you made those functions take the concrete type bytes.Buffer (which you presumably don't want), that wouldn't be enough because Read calls io.ReadFull which again takes an io.Reader . You'd have to work harder than that to make this allocation-free.

As a side note, there's a much easier solution to your other issue in Read that doesn't require any unsafe.Slice shenanigans: just replace var b byte with var b [1]byte (which is exactly the same in memory), pass b[:] to ReadFull, and use b[0] in the other places you use b.