A Makefile for your Go project

Vincent Bernat

My most loathed fea­ture of Go was the manda­tory use of GOPATH: I do not want to put my own code next to its de­pen­den­cies. I was not alone and peo­ple de­vised tools or crafted their own Makefile to avoid or­ga­niz­ing their code around GOPATH.

Hope­fully, since Go 1.11, it is pos­si­ble to use Go’s mod­ules to man­age de­pen­den­cies with­out re­ly­ing on GOPATH. First, you need to con­vert your project to a module:1

$ go mod init hellogopher
go: creating new go.mod: module hellogopher
$ cat go.mod
module hellogopher

Then, you can in­voke the usual com­mands, like go build or go test. The go com­mand re­solves im­ports by us­ing ver­sions listed in go.​mod. When it runs into an im­port of a pack­age not present in go.​mod, it au­to­mat­i­cally looks up the mod­ule con­tain­ing that pack­age us­ing the lat­est ver­sion and adds it.

$ go test ./...
go: finding github.com/spf13/cobra v0.0.5
go: downloading github.com/spf13/cobra v0.0.5
?       hellogopher     [no test files]
?       hellogopher/cmd [no test files]
ok      hellogopher/hello       0.001s
$ cat go.mod
module hellogopher

require github.com/spf13/cobra v0.0.5

If you want a spe­cific ver­sion, you can ei­ther edit go.​mod or in­voke go get:

$ go get github.com/spf13/cobra@v0.0.4
go: finding github.com/spf13/cobra v0.0.4
go: downloading github.com/spf13/cobra v0.0.4
$ cat go.mod
module hellogopher

require github.com/spf13/cobra v0.0.4

Add go.​mod to your ver­sion con­trol sys­tem. Op­tion­ally,2 you can also add go.​sum as a safety net against over­rid­den tags. If you re­ally want to ven­dor the de­pen­den­cies, you can in­voke go mod vendor and add the vendor/ di­rec­tory to your ver­sion con­trol system.

Thanks to the mod­ules, in my opin­ion, Go’s de­pen­dency man­age­ment is now on a par with other lan­guages, like Ruby. While it is pos­si­ble to run day-to-day op­er­a­tions—build­ing and test­ing—with only the go com­mand, a Makefile can still be use­ful to or­ga­nize com­mon tasks, a bit like Python’s setup.py or Ruby’s Rakefile. Let me de­scribe mine.

Using third-party tools#

Most projects need some third-party tools for test­ing or build­ing. We can ei­ther ex­pect them to be al­ready in­stalled or com­pile them on the fly. For ex­am­ple, here is how code lint­ing is done with Golint:

BIN = $(CURDIR)/bin
$(BIN):
    @mkdir -p $@
$(BIN)/%: | $(BIN)
    @tmp=$$(mktemp -d); \
       env GO111MODULE=off GOPATH=$$tmp GOBIN=$(BIN) go get $(PACKAGE) \
        || ret=$$?; \
       rm -rf $$tmp ; exit $$ret

$(BIN)/golint: PACKAGE=golang.org/x/lint/golint

GOLINT = $(BIN)/golint
lint: | $(GOLINT)
    $(GOLINT) -set_exit_status ./...

The first block de­fines how a third-party tool is built: go get is in­voked with the pack­age name match­ing the tool we want to in­stall. We do not want to pol­lute our de­pen­dency man­age­ment and there­fore, we are work­ing in an empty GOPATH. The gen­er­ated bi­na­ries are put in bin/.

The sec­ond block ex­tends the pat­tern rule de­fined in the first block by pro­vid­ing the pack­age name for golint. Ad­di­tional tools can be added by just adding an­other line like this.

The last block de­fines the recipe to lint the code. The de­fault lint­ing tool is the golint built us­ing the first block but it can be over­rid­den with make GOLINT=/​usr/​bin/golint.

Up­date (2025-03)

Start­ing with Go 1.24 (re­leased in Feb­ru­ary 2025), the go.​mod file in­cludes a new tool di­rec­tive to han­dle tool de­pen­den­cies na­tively. You can see this method im­ple­mented in the up­dated ex­am­ple repos­i­tory. The pre­vi­ously rec­om­mended ap­proach was to cre­ate a file tools.go that im­ports pack­ages con­tain­ing the tools to man­age de­pen­den­cies through go.​mod. This method en­sures all users work with the same ver­sion of tools, which is bet­ter than the pre­vi­ous approach.

Tests#

Here are some rules to help run­ning tests:

TIMEOUT  = 20
PKGS     = $(or $(PKG),$(shell env GO111MODULE=on $(GO) list ./...))
TESTPKGS = $(shell env GO111MODULE=on $(GO) list -f \
            '{{ if or .TestGoFiles .XTestGoFiles }}{{ .ImportPath }}{{ end }}' \
            $(PKGS))

TEST_TARGETS := test-default test-bench test-short test-verbose test-race
test-bench:   ARGS=-run=__absolutelynothing__ -bench=.
test-short:   ARGS=-short
test-verbose: ARGS=-v
test-race:    ARGS=-race
$(TEST_TARGETS): test
check test tests: fmt lint
    go test -timeout $(TIMEOUT)s $(ARGS) $(TESTPKGS)

A user can in­voke tests in dif­fer­ent ways:

  • make test runs all tests;
  • make test TIMEOUT=10 runs all tests with a time­out of 10 seconds;
  • make test PKG=​hellogopher/​cmd only runs tests for the cmd package;
  • make test ARGS="-v -short" runs tests with the spec­i­fied ar­guments;
  • make test-race runs tests with race de­tec­tor enabled.

go test in­cludes a test cov­er­age tool. Un­for­tu­nately, it only han­dles one pack­age at a time and you have to ex­plic­itly list the pack­ages to be in­stru­mented, oth­er­wise the in­stru­men­ta­tion is lim­ited to the cur­rently tested pack­age. If you pro­vide too many pack­ages, the com­pi­la­tion time will sky­rocket. More­over, if you want an out­put com­pat­i­ble with Jenk­ins, you need some ad­di­tional tools.

COVERAGE_MODE    = atomic
COVERAGE_PROFILE = $(COVERAGE_DIR)/profile.out
COVERAGE_XML     = $(COVERAGE_DIR)/coverage.xml
COVERAGE_HTML    = $(COVERAGE_DIR)/index.html
test-coverage-tools: | $(GOCOVMERGE) $(GOCOV) $(GOCOVXML) # ❶
test-coverage: COVERAGE_DIR := $(CURDIR)/test/coverage.$(shell date -u +"%Y-%m-%dT%H:%M:%SZ")
test-coverage: fmt lint test-coverage-tools
    @mkdir -p $(COVERAGE_DIR)/coverage
    @for pkg in $(TESTPKGS); do \ # ❷
        go test \
            -coverpkg=$$(go list -f '{{ join .Deps "\n" }}' $$pkg | \
                    grep '^$(MODULE)/' | \
                    tr '\n' ',')$$pkg \
            -covermode=$(COVERAGE_MODE) \
            -coverprofile="$(COVERAGE_DIR)/coverage/`echo $$pkg | tr "/" "-"`.cover" $$pkg ;\
     done
    @$(GOCOVMERGE) $(COVERAGE_DIR)/coverage/*.cover > $(COVERAGE_PROFILE)
    @go tool cover -html=$(COVERAGE_PROFILE) -o $(COVERAGE_HTML)
    @$(GOCOV) convert $(COVERAGE_PROFILE) | $(GOCOVXML) > $(COVERAGE_XML)

First, we de­fine some vari­ables to let the user over­ride them. In ❶, we re­quire the fol­low­ing tools—built like golint pre­viously:

  • gocovmerge merges pro­files from dif­fer­ent runs into a single one;
  • gocov-xml con­verts a cov­er­age pro­file to the Cober­tura for­mat, for Jenkins;
  • gocov is needed to con­vert a cov­er­age pro­file to a for­mat han­dled by gocov-xml.

In ❷, for each pack­age to test, we run go test with the -coverprofile ar­gu­ment. We also ex­plic­itly pro­vide the list of pack­ages to in­stru­ment to -coverpkg by us­ing go list to get a list of de­pen­den­cies for the tested pack­age and keep­ing only our own.

Up­date (2019-09)

As men­tioned in the com­ments, since Go 1.10, it is pos­si­ble to test sev­eral pack­ages while still us­ing -coverprofile. There­fore, the test-coverage recipe can be sim­pli­fied a bit and we can drop gocovmerge.

Build#

An­other use­ful recipe is to build the pro­gram. While this could be done with just go build, it is not un­com­mon to have to spec­ify build tags, ad­di­tional flags, or to ex­e­cute sup­ple­men­tary build steps. In the fol­low­ing ex­am­ple, the ver­sion is ex­tracted from Git tags. It will re­place the value of the Version vari­able in the hellogopher/​cmd package.

VERSION ?= $(shell git describe --tags --always --dirty --match=v* 2> /dev/null || \
            echo v0)
all: fmt lint | $(BIN)
    go build \
        -tags release \
        -ldflags '-X hellogopher/cmd.Version=$(VERSION)' \
        -o $(BIN)/hellogopher main.go

The recipe also runs code for­mat­ting and linting.


The ex­cerpts pro­vided in this post are a bit sim­pli­fied. Have a look at the fi­nal re­sult for more perks, in­clud­ing fancy out­put and in­te­grated help!

Up­date (2019-09)

There is an in­ter­est­ing thread about this ar­ti­cle on Red­dit. It con­tains a clue on how to lock tools needed for build as well. Sev­eral peo­ple also pre­fer to use Mage, a build tool us­ing Go. Iron­i­cally, it re­quires a non-triv­ial build step.