Under the Hood · Data Structures

What a Go Slice Actually Is

· 28 min read

A slice is not a growable array. It is a small value that describes part of an array it does not own. Every surprising thing slices do follows from that split: the description is copied, the array is shared.

Three words and an array somewhere else

The Go specification calls a slice a descriptor for a contiguous segment of an underlying array, with a length and a capacity. The gc compiler represents that descriptor as three machine words. The first is a pointer to the first element the slice can reach. The second counts the elements addressable through it, and the third counts the elements the array has room for from that point onwards. You can measure the size of the descriptor yourself, since unsafe.Sizeof(s) on any slice returns the same number regardless of how many elements it has.

The invariant is that 0 <= len(s) <= cap(s), and that the cap(s) elements starting at the slice’s data pointer are all valid memory. Indexing past len is a runtime panic; reslicing up to cap is legal and gives you addressable elements that were already there. That asymmetry is where the trouble lives: len is what you are allowed to read, cap is what exists, and the gap between them is memory that some other slice value may also be describing.

The array underneath is the type Go programmers almost never write by hand. [4]int is a different type from [5]int, because the length is part of the type, and arrays have value semantics: assigning one copies every element, passing one to a function copies every element, and returning one copies every element again. A function signature taking [4]int cannot accept [5]int, and generics do not help, because Go has no way to make the array length a type parameter. So arrays appear in real code mostly where a fixed width is genuinely part of the data, as in the [32]byte that sha256.Sum256 returns or a sixteen-byte identifier, and the rest of the time there is a slice over an array nobody named.

Here is the shape that causes the bug.

low data → element 0 len 2 cap 5 high data → element 2 len 3 cap 3 one backing array of five ints 1 2 3 4 5 index 0 index 1 index 2 index 3 index 4 len(low) cap(low) inside cap(low), outside len(low): an append through low lands here and high reads the result

Two headers, one array. The headers are values, so low and high each got their own copy of three words, and changing one changes nothing about the other. The array is not a value anybody holds, so a write through either header is visible through both wherever their reaches overlap.

There are four ways to get a slice and they differ in which of the three words they set. var s []int leaves all three zero, which is the nil slice. []int{1, 2, 3} allocates an array of exactly three, so length and capacity match. make([]int, 4) gives you four zero values and no spare room. make([]int, 0, 4) gives you an empty slice over an array with room for four, which is the form to reach for in a loop that appends a known number of times. Slicing an existing array or slice sets the data pointer into memory that already exists. Only that fourth way can hand you a slice somebody else is also looking at.

What it costs

Indexing is an address calculation plus a bounds check the compiler sometimes proves away. Elements sit next to each other in memory, which is why a slice of a million ints walks faster than a node-per-element structure holding the same data. The hardware prefetcher reads ahead along a straight line; a linked list stalls it at every node. Pointer chasing is the cost that dominates in practice, and it never shows up in a complexity bound.

len and cap read fields of a value the compiler usually keeps in registers. Passing a slice to a function copies three words no matter how long it is, which is why you almost never see *[]T in idiomatic Go; passing the array itself would copy every element.

Space overhead is the three-word header per slice value plus whatever the backing array allocated but has not filled. Nothing per element. A doubly-linked list carries a pointer in each direction per element, plus a separate allocation per element for the allocator to track. container/list adds an any field per element, so small values get boxed on their way in and type-asserted on their way out.

The append cost is where care is needed. The documentation for the built-in says the destination is resliced to accommodate the new elements if the slice has sufficient capacity. Otherwise a new underlying array is allocated. It does not state a complexity, and it does not say how much extra capacity the new array gets. The growth formula is an implementation choice that has changed between Go releases. Treat the factor as unspecified, and never write code whose correctness depends on a particular capacity coming back from append. What you can rely on is the branch: room, or reallocation plus a copy of everything already there.

copy moves min(len(dst), len(src)) elements and returns how many it moved. The source and destination may overlap, which is what makes the shift-left and shift-right tricks below correct. It is linear in the number of elements moved. Nothing in the language says how the move happens, and the bulk memory move the gc toolchain emits for it is an implementation choice.

Allocation is the other real cost, and it is the one you can control. If the compiler can prove a slice’s backing array does not outlive the function, it may put the array on the stack instead of the heap, and go build -gcflags=-m prints the decisions it made. Those rules are not part of the language, so they are a thing to measure rather than a thing to plan around. The reliable lever is make([]T, 0, n) when you know n, because it removes the reallocations entirely and the allocation count is visible in a benchmark.

When a slice is what you want

Reach for a slice for a run of same-typed values you will walk start to finish, append to, and index by position. That covers most collections in real programs: rows from a query, bytes read off a connection, the arguments to a command, a batch of subscriptions due for delivery tomorrow.

It stays the right answer for longer than people expect in two cases that look like they call for something cleverer. A stack is a slice: push is append, pop is read the last element and reslice to s[:len(s)-1], and the whole structure is contiguous. And linear search through a short slice beats a map. The map lookup hashes the key and then chases a pointer, while the slice scan stays in memory the cache already holds. “Short” is not a number you should take from me; it is a number you measure for your key type and your length, with a benchmark that uses your data.

A slice is also the right answer when the thing you are modelling genuinely is a window onto a larger buffer. Parsing is the clearest case: a tokeniser can return subslices of the input rather than copying each token out, and the whole parse allocates once. That is the same aliasing mechanism as the bug, used on purpose.

When not to, and what instead

Use a map when you look values up by identity rather than position and it is large enough that the hash beats the scan. Map iteration order is unspecified, and the runtime deliberately randomises it. Code that iterates a map and expects stable output is broken whether or not it has failed yet. Where you want deterministic output, collect the keys into a slice, sort them, and iterate that.

Avoid a slice for a queue with sustained traffic at both ends. Popping from the front with q = q[1:] moves the data pointer forward and leaves the elements behind it unreachable but still allocated. A long-lived queue drags the whole backing array along with it. A ring buffer over a fixed slice solves it with two indices and no reallocation.

Avoid a slice for repeated insertion or removal in the middle of a long collection, since every one of those is a linear shift of the tail. This is the case the textbook answers with a linked list. In Go the answer is usually still a slice. At the lengths most programs deal in, the shift is a bulk memory move, and the list’s per-node allocation and pointer chasing cost more. Measure before you switch. If you do switch, prefer your own generic node type over container/list. Its Element.Value is an any, so you get an interface value per element and no type checking at compile time. The same caution applies to container/heap, where the algorithm is fine and the interface predates generics. You implement five methods on your own type, and the element handling goes through any.

Do not use a slice as a set. map[T]struct{} is the idiom, and the slices package’s Contains exists for the short-collection case rather than as a licence to build set semantics out of linear scans.

A great deal of what people hand-write over slices is already in the standard library. The slices package has Clone, Grow, Clip, Insert, Delete, Compact, Reverse, Equal, Index, Contains, Sort, SortFunc and BinarySearch. The maps package has Keys and Values, so you do not write the collection loop by hand. Read both package docs before writing a helper; the version you write will have the aliasing subtleties and theirs will not.

Writing the operations out in Go

Slices are built in, so the implementation worth writing is the one that makes the invariant visible. These are append, Insert and Delete spelled out, with the capacity branch in the open.

// Package slicekit spells out the slice operations the built-ins and the
// slices package already provide. It exists to make the capacity branch
// visible; use the standard library in real code.
package slicekit

import (
	"errors"
	"fmt"
	"unsafe"
)

// Header returns the three fields a slice value carries. The data pointer is
// only meaningful when cap(s) > 0: unsafe.SliceData documents that it returns
// nil for a nil slice and an unspecified address for an empty non-nil one.
func Header[T any](s []T) (data *T, length, capacity int) {
	return unsafe.SliceData(s), len(s), cap(s)
}

// Append is the built-in, written out. The branch is the whole behaviour. On
// the first path the write lands in an array other slices may be describing;
// on the second it lands in memory nobody else has a pointer to yet.
func Append[T any](s []T, vs ...T) []T {
	n := len(s)
	if n+len(vs) <= cap(s) {
		s = s[:n+len(vs)]
		copy(s[n:], vs)
		return s
	}
	grown := make([]T, n+len(vs), nextCap(n+len(vs)))
	copy(grown, s)
	copy(grown[n:], vs)
	return grown
}

// nextCap doubles, which is this package's choice and not Go's. The runtime's
// growth formula is unspecified and has changed between releases, so no caller
// should depend on the capacity that comes back from either one.
func nextCap(need int) int {
	c := 1
	for c < need {
		c *= 2
	}
	return c
}

Insert makes room at the end, shifts the tail right, and writes into the hole. The middle copy has overlapping source and destination, which is defined to work.

// Insert puts vs at index i. It may return a slice over a new array, so the
// caller must use the return value; writing Insert(s, i, v) and ignoring the
// result is the same mistake as ignoring append's.
func Insert[T any](s []T, i int, vs ...T) []T {
	s = Append(s, vs...)       // make room, possibly reallocating
	copy(s[i+len(vs):], s[i:]) // shift the tail right, overlapping
	copy(s[i:], vs)            // write the new elements into the hole
	return s
}

// Delete removes s[i:j] by shifting the tail left, then zeroing the elements
// between the new length and the old one. The zeroing is not cosmetic: those
// elements are still inside cap(s), so a pointer left there keeps whatever it
// points at reachable by the garbage collector.
func Delete[T any](s []T, i, j int) []T {
	var zero T
	n := copy(s[i:], s[j:])
	for k := i + n; k < len(s); k++ {
		s[k] = zero
	}
	return s[:i+n]
}

// invariants is what every slice these functions return must satisfy. The
// tests call it after each mutation.
func invariants[T any](s []T) error {
	if len(s) > cap(s) {
		return fmt.Errorf("length %d exceeds capacity %d", len(s), cap(s))
	}
	if s == nil {
		if len(s) != 0 || cap(s) != 0 {
			return fmt.Errorf("nil slice with length %d, capacity %d", len(s), cap(s))
		}
		return nil
	}
	if cap(s) > 0 && unsafe.SliceData(s) == nil {
		return errors.New("slice with capacity and no data pointer")
	}
	return nil
}

The same capacity branch is what produces the bug. Here are two slices over one array, one of them with room to spare.

base := []int{1, 2, 3, 4, 5}
low := base[:2]  // len 2, cap 5
high := base[2:] // len 3, cap 3

low = append(low, 99) // cap allows it, so this writes base[2]

// low  is [1 2 99]
// high is [99 4 5]

Nothing in low’s type or in the signature of any function taking it says that appending to it can change another slice’s contents. The fix is the three-index slice expression, which sets the capacity as well as the bounds: base[i:j:k] has length j-i and capacity k-i. Writing low := base[:2:2] leaves no spare capacity, so the append must allocate, and high is safe.

The version that reaches production looks innocent, because the capacity is somewhere else:

func withUrgent(tags []string) []string {
	return append(tags, "urgent")
}

all := make([]string, 0, 4)
all = append(all, "inbox", "receipts")

first := withUrgent(all[:1]) // writes all[1]
// first is ["inbox" "urgent"]
// all   is ["inbox" "urgent"]  <- "receipts" is gone

withUrgent is correct in isolation, and nothing inside it can prevent the damage. A function that must not disturb its caller’s array either copies first with slices.Clone or is documented as appending in place. There is no third option: the capacity it would need to check is not part of any contract the compiler can see.

Keeping a long-lived small slice of a short-lived large array leaks in the same way. A slice holds its whole backing array reachable, including the elements outside len. A header := buf[:32] taken from a ten-megabyte read keeps ten megabytes alive for as long as header lives. slices.Clone, or copy into a right-sized slice, is what releases the rest. slices.Clip does the opposite job: it reduces the capacity to the length so a later append cannot reach past the end, without copying anything.

Two more mechanics worth having straight. clear(s) sets every element up to len(s) to the zero value and leaves the length alone. s = s[:0] is not the same thing: it keeps the array, keeps the values in it reachable, and gives you an empty slice to append into. And a method on a value receiver copies the struct, which copies the slice header:

type Buffer struct{ data []byte }

// Wrong. The receiver is a copy, so the new length is written to a Buffer
// that goes out of scope on return. If the append happened to fit in the
// existing capacity, the caller's array changed and its length did not.
func (b Buffer) AddBroken(p byte) { b.data = append(b.data, p) }

func (b *Buffer) Add(p byte) { b.data = append(b.data, p) }

Last, nil against empty. var s []int is nil with length and capacity zero; s := []int{} is non-nil with length zero. Both support len, cap, range, copy and append, slices compare to nothing but nil, and so for almost all code the distinction does not arise. It arises in encoding/json, which marshals a nil slice as null and an empty one as []. An API’s response shape then depends on whether a filter that matched nothing returned make([]T, 0) or never assigned at all.

How to test it

Everything below lives in slicekit_test.go alongside the code, so it can call the unexported invariants.

package slicekit

import (
	"bytes"
	"math/rand/v2"
	"slices"
	"sync"
	"testing"
)

Table-driven tests cover the cases you can name, and for slice code the named cases are the boundaries: empty range, first index, last index, whole slice, nil input. Each one calls the invariant checker, so a mutation that leaves len above cap fails even if the contents happen to be right.

func TestDelete(t *testing.T) {
	tests := []struct {
		name string
		in   []int
		i, j int
		want []int
	}{
		{"middle", []int{1, 2, 3, 4}, 1, 3, []int{1, 4}},
		{"head", []int{1, 2, 3}, 0, 1, []int{2, 3}},
		{"tail", []int{1, 2, 3}, 2, 3, []int{1, 2}},
		{"empty range", []int{1, 2, 3}, 1, 1, []int{1, 2, 3}},
		{"everything", []int{1, 2, 3}, 0, 3, []int{}},
		{"nil input", nil, 0, 0, []int{}},
	}
	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			got := Delete(slices.Clone(tt.in), tt.i, tt.j)
			if !slices.Equal(got, tt.want) {
				t.Errorf("Delete(%v, %d, %d) = %v, want %v", tt.in, tt.i, tt.j, got, tt.want)
			}
			if err := invariants(got); err != nil {
				t.Error(err)
			}
		})
	}
}

The aliasing contract needs its own tests, because it is the part a reader will get wrong and it is invisible in the function signature. Assert both branches explicitly, and construct the capacity by hand so the test does not depend on the runtime’s growth formula.

func TestAppendWritesTheSharedArrayWhenCapacityAllows(t *testing.T) {
	backing := make([]int, 3, 4)
	head := backing[:2]

	head = append(head, 99)

	if backing[2] != 99 {
		t.Fatalf("append did not write the shared array: backing = %v", backing)
	}
	if len(backing) != 3 {
		t.Errorf("the other header changed: len = %d, want 3", len(backing))
	}
}

func TestAppendReallocatesWhenCapacityRunsOut(t *testing.T) {
	backing := make([]int, 2, 2)
	head := backing[:2]

	head = append(head, 99)

	if &head[0] == &backing[0] {
		t.Fatal("expected a new backing array")
	}
	head[0] = 7
	if backing[0] != 0 {
		t.Errorf("a write through the new slice reached the old array: %v", backing)
	}
}

func TestCappedSliceCannotReachTheTail(t *testing.T) {
	backing := make([]int, 3, 8)
	head := backing[0:2:2]

	head = append(head, 99)

	if backing[2] == 99 {
		t.Error("append wrote through a capped slice into the shared array")
	}
}

Hand-written cases run out long before the input space does. The next test generates inputs and compares against a reference implementation that is correct by inspection and too slow to ship. The reference builds a new slice every time and never touches its input, which is exactly why it is the reference.

func TestInsertMatchesReference(t *testing.T) {
	rng := rand.New(rand.NewPCG(1, 2)) // math/rand/v2, fixed seed so failures replay
	for range 2000 {
		base := randomInts(rng, rng.IntN(12))
		vs := randomInts(rng, rng.IntN(4))
		i := rng.IntN(len(base) + 1)

		got := Insert(slices.Clone(base), i, vs...)
		want := referenceInsert(base, i, vs)

		if !slices.Equal(got, want) {
			t.Fatalf("Insert(%v, %d, %v) = %v, want %v", base, i, vs, got, want)
		}
		if err := invariants(got); err != nil {
			t.Fatal(err)
		}
	}
}

func referenceInsert(base []int, i int, vs []int) []int {
	out := make([]int, 0, len(base)+len(vs))
	out = append(out, base[:i]...)
	out = append(out, vs...)
	return append(out, base[i:]...)
}

func randomInts(rng *rand.Rand, n int) []int {
	out := make([]int, n)
	for i := range out {
		out[i] = rng.IntN(100)
	}
	return out
}

For anything that slices bytes, go test -fuzz reaches cases a hand-written generator tends to miss. The target below splits on a zero byte and returns subslices of its input. The two properties to assert are that the input is unchanged and that the pieces rejoin into it. The b[:i:i] in the implementation is the three-index expression doing its job: without it, a caller appending to one returned piece would overwrite the next.

// Fields splits b on 0x00 and returns subslices of b rather than copies. Each
// piece is capped at its own length so appending to one cannot reach the next.
func Fields(b []byte) [][]byte {
	var out [][]byte
	for {
		i := bytes.IndexByte(b, 0)
		if i < 0 {
			return append(out, b)
		}
		out = append(out, b[:i:i])
		b = b[i+1:]
	}
}

func FuzzFields(f *testing.F) {
	f.Add([]byte("inbox\x00receipts\x00"))
	f.Add([]byte{})
	f.Fuzz(func(t *testing.T, in []byte) {
		original := bytes.Clone(in)

		parts := Fields(in)

		if !bytes.Equal(in, original) {
			t.Fatalf("Fields mutated its input: %q became %q", original, in)
		}
		if joined := bytes.Join(parts, []byte{0}); !bytes.Equal(joined, original) {
			t.Fatalf("round trip lost data: %q became %q", original, joined)
		}
	})
}

Where the claim is about cost, the test is a benchmark, and a benchmark says something about one machine rather than something about Go. Run this with go test -bench=Append -benchmem and read the allocs/op column. That is the number the preallocation changes, and the one that does not move with the clock speed of whatever ran it.

var sink []int // package-level so the compiler cannot discard the work

func BenchmarkAppend(b *testing.B) {
	const n = 4096
	b.Run("fresh", func(b *testing.B) {
		b.ReportAllocs()
		for range b.N {
			var s []int
			for i := range n {
				s = append(s, i)
			}
			sink = s
		}
	})
	b.Run("preallocated", func(b *testing.B) {
		b.ReportAllocs()
		for range b.N {
			s := make([]int, 0, n)
			for i := range n {
				s = append(s, i)
			}
			sink = s
		}
	})
}

Shared slices need go test -race, and the distinction it enforces is worth internalising. Writing distinct elements through one slice from many goroutines is safe, because each goroutine touches different memory. Appending from two goroutines races on the header even when the elements never collide, since both write the same three words.

func TestConcurrentWritesToDistinctIndexes(t *testing.T) {
	s := make([]int, 64)
	var wg sync.WaitGroup
	for i := range s {
		wg.Add(1)
		go func(i int) { // explicit parameter; under Go 1.22 loop semantics a capture is also safe
			defer wg.Done()
			s[i] = i * i
		}(i)
	}
	wg.Wait()

	for i, v := range s {
		if v != i*i {
			t.Fatalf("s[%d] = %d, want %d", i, v, i*i)
		}
	}
}

Replace those index writes with s = append(s, i*i) and the goroutines stop sharing elements and start sharing the header. Each one reads three words, works out a new length, and writes three words back. No two of them touch the same element, the detector reports the race anyway, and a run without -race can finish the whole thing and tell you nothing.

These posts are LLM-aided. Backbone, original writing, and structure by Craig. Research and editing by Craig + LLM. Proof-reading by Craig.