How to Benchmark Go vs. Python Fairly: A Reproducible Method

If you are choosing an implementation for a service, start with the decision the benchmark must inform. Is the constraint requests per second, response latency at the busy end of the distribution, CPU time, or memory? Record more than one metric, but name the deciding one before running the test. A fast recursive calculation and a fast database-backed endpoint answer different questions.

Choose the work before choosing the stopwatch

Split the work you care about into CPU-bound processing, I/O-bound requests, and paths that combine them. Then use representative inputs, data sizes, request mixes, and concurrency. A recursive Fibonacci benchmark at n=35 is a useful narrow computation test; it does not stand in for a service waiting on storage or a network. For a mixed example, a scraping comparison separates fetching URLs from processing their text.

Make behavior equivalent before attributing a difference to a language. Match inputs, outputs, storage backend, caching, and dependency choices. Record the framework and inspect the implementations: the scraping comparison compiled a regular expression inside each Go processing call, while its Python and Go versions represented bigram keys differently. The Benchmarks Game likewise cautions readers to inspect source code, including unusually optimized programs, before interpreting a language ranking.

A small comparison with a shared correctness check

The following deliberately narrow workload counts adjacent pairs in pre-split ASCII words. It tests processing, not HTTP or database performance. In an existing Go module, save this as pairs_test.go:

package bench

import (
    "strings"
    "testing"
)

type pair struct{ first, second string }

func countPairs(words []string) map[pair]int {
    counts := make(map[pair]int)
    for i := 0; i+1 < len(words); i++ {
        counts[pair{words[i], words[i+1]}]++
    }
    return counts
}

const input = "go go fast go"

var expected = map[pair]int{
    {"go", "go"}:     1,
    {"go", "fast"}:   1,
    {"fast", "go"}:   1,
}

func check(t testing.TB, got map[pair]int) {
    t.Helper()
    if len(got) != len(expected) {
        t.Fatalf("got %v; want %v", got, expected)
    }
    for key, want := range expected {
        if got[key] != want {
            t.Fatalf("pair %v: got %d; want %d", key, got[key], want)
        }
    }
}

func TestCountPairs(t *testing.T) {
    check(t, countPairs(strings.Fields(input)))
}

var result map[pair]int

func BenchmarkCountPairs(b *testing.B) {
    words := strings.Fields(input)
    check(b, countPairs(words))
    b.ResetTimer()
    for i := 0; i < b.N; i++ {
        result = countPairs(words)
    }
}

Save the matching Python implementation as pairs.py. It checks the same counts before calling timeit; neither timed loop includes splitting the input.

from timeit import Timer

WORDS = "go go fast go".split()
EXPECTED = {
    ("go", "go"): 1,
    ("go", "fast"): 1,
    ("fast", "go"): 1,
}


def count_pairs(words):
    counts = {}
    for i in range(len(words) - 1):
        key = (words[i], words[i + 1])
        counts[key] = counts.get(key, 0) + 1
    return counts


if __name__ == "__main__":
    actual = count_pairs(WORDS)
    if actual != EXPECTED:
        raise ValueError(f"got {actual}; want {EXPECTED}")

    seconds = Timer(lambda: count_pairs(WORDS)).repeat(repeat=5, number=1000)
    print("seconds per call:", [elapsed / 1000 for elapsed in seconds])

Run these on the same host, with other substantial work stopped:

go test -bench=. -count=5 > go-time.txt
python3 pairs.py

The Go test checks correctness before its benchmark. The Python script does likewise before its five timing repetitions. Keep each repetition rather than selecting the nicest number. These harnesses have different overheads and iteration strategies, which matter particularly for a function this small; use this example to establish a procedure, not a service-speed ratio.

Collect allocation and memory observations in separate runs, not from the timings above:

go test -bench=. -benchmem -count=5 > go-alloc.txt

-benchmem reports Go allocation statistics. For Python, run the same prepared function under memory_profiler in a separate session and record its reported peak alongside an idle-process baseline. A process peak and Go allocations per operation are different quantities; do not put them in one column labeled “memory per call.” When comparing revisions of the Go implementation, keep their repeated benchmark outputs and use benchstat to compare those sets. Profiling is the next step when a result needs explaining: collect Go CPU or memory profiles, or profile the Python path, but keep instrumented runs out of the final timing set.

Move from functions to a service load test

For an API decision, hold the endpoint’s response, database, row count, and caching policy fixed. A concrete scenario is a GET that reads 20 products and returns JSON, without caching. A published REST comparison used that shape of workload with 200 parallel requests for one minute; its Pocketbase configuration supported SQLite, while its Python frameworks also supported PostgreSQL. Those are different database comparisons, not interchangeable language results.

Run the application on one host and wrk on a separate host with enough spare CPU and network capacity. For example, from the generator host, targeting the application host’s product endpoint:

wrk -t2 -c200 -d60s --latency http://192.0.2.10:8080/products

Replace the example address with your endpoint. This specifies two generator threads, 200 connections, and a 60-second run; record the actual request rate and latency distribution. Watch both hosts, and repeat at different generator capacity before concluding the application has reached its throughput limit. If the generator cannot sustain the intended load, the number describes the generator too.

That distinction changed the interpretation of a URL-shortener comparison: Locust initially produced similar throughput, while a subsequent wrk run reported roughly 280 requests per second for Python and 2,500 for Go on one retrieval endpoint. The author suspected a limit in the initial Locust setup; the later figures still describe only that endpoint and setup. The same account changed the Go storage implementation to Redis after testing its write endpoint, so results across that change cannot isolate language effects. Background work matters as well: the REST comparison traced inconsistent runs after Docker image builds to Docker Desktop vulnerability scans consuming CPU and disk.

Keep a report another developer can repeat

Use one record per workload and configuration. Investigate anomalous repetitions rather than silently dropping them, and label any ratio with the endpoint, storage, and machine setup that produced it.

Question and deciding metric:
Workload: input sizes or endpoint; request mix; concurrency; expected output
Go: version; build settings; framework and dependencies; storage; caching
Python: version; framework and dependencies; storage; caching
Hosts: CPU, RAM, OS; application and generator placement; resource limits
Runs: duration or iterations; repetitions; background activity; anomalies
Results: throughput; latency including tail; CPU use/time; memory;
         per-run values and variation; generator-capacity check
Interpretation: workload-specific ratio; implementation complexity or tradeoffs

Only calculate a speed ratio after both programs pass equivalent correctness checks and the conditions behind the measurements remain comparable.