Better testing for golang http handlers

I’m writing this up as it doesn’t seem to be a common testing pattern for Go projects that I’ve seen, so might prove useful to someone somewhere as it did for me in a recent project.

One of the things that bugs me about the typical golang http server setup is that it relies on hidden globals. You typically write something like:

package main

import (
    "net/http"
    "fmt"
)

func myHandler(w http.ResponseWriter, r *http.Request) {
     fmt.Fprintf(w, "Hello, world!")
}

func main() {
     http.HandleFunc("/", myHandler)
     http.ListenAndServe(":8080", nil)
}

This is all lovely and simple, but there’s some serious hidden work going on here. The bit that’s always made me uncomfortable is that I set up all this state without any way to track it, which makes it very hard to test, particularly as the http library in golang doesn’t allow for any introspection on the handlers you’ve set up. This means I need to write integration tests rather than unittests to have confidence that my URL handlers are set up correctly. The best I’ve seen done test wise normally with this setup is to test each handler function.

But there is a very easy solution to this, just it’s not really considered something you’d ever do in the golang docs - they explicitly state no one would ever really do this. Clearly their attitude to testing is somewhat different to mine :)

The solution is in that nil parameter in the last line, which the golang documents state:

“ListenAndServe starts an HTTP server with a given address and handler. The handler is usually nil, which means to use DefaultServeMux.”

That handler is a global variable, http.DefaultServeMux, which is the request multiplexer that takes the incoming requests, looks at the paths, and then works out which handler to call (including the default built in handlers if there’s no match to return 404s etc.). This is all documented extrememly well in this article by Amit Saha, which I can highly recommend.

But you don’t need to use the global, you can just instantiate your own multiplexer object and use that. If you do this suddenly your code stops using side effects to set up the http server and suddenly becomes a lot nicer to reason about and test.

package main

import (
    "net/http"
    "fmt"
)

func myHandler(w http.ResponseWriter, r *http.Request) {
     fmt.Fprintf(w, "Hello, world!")
}

func main() {
     mymux := http.NewServeMux()
     mymux.HandleFunc("/", myHandler)
     http.ListenAndServe(":8080", mymux)
}

The above is functionally the same as our first example, but no longer takes advantage of the hidden global state. This in itself may seem not to buy us much, but in reality you’ll have lots of handlers to set up, and so your code can be made to look something more like:

func SetupMyHandlers() *http.ServeMux {
     mux := http.NewServeMux()

    // setup dynamic handlers
     mux.HandleFunc("/", MyIndexHandler)
     mux.HandleFunx("/login/", MyLoginHandler)
    // etc.

    // set up static handlers
     http.Handle("/static/", http.StripPrefix("/static/", http.FileServer(http.Dir("/static/"))))
    // etc.

     return mux
}

func main() {
     mymux := SetupMyHandlers()
     http.ListenAndServe(":8080", mymux)
}

At this point you can start using setupHandlers in your unit tests. Without this the common pattern I’d seen was:

package main

import (
    "net/http"
    "net/http/httptest"
    "testing"
)

func TestLoginHandler(t *testing.T) {

     r, err := http.NewRequest("GET", "/login", nil)
     if err != nil {
          t.Fatal(err)
     }
     w := httptest.NewRecorder()
     handler := http.HandlerFunc(MyLoginHandler)
     handler.ServeHTTP(w, r)

     resp := w.Result()

     if resp.StatusCode != http.StatusOK {
          t.Errorf("Unexpected status code %d", resp.StatusCode)
     }
}

Here you just wrap your specific handler function directly and call that in your tests. Which is very good for testing that the handler function works, but not so good for checking that someone hasn’t botched the series of handler registration calls in your server. Instead, you can now change one line and get that additional coverage:

package main

import (
    "net/http"
    "net/http/httptest"
    "testing"
)

func TestLoginHandler(t *testing.T) {

     r, err := http.NewRequest("GET", "/login", nil)
     if err != nil {
          t.Fatal(err)
     }
     w := httptest.NewRecorder()
     handler := SetupMyHandlers()  // <---- this is the change :)
     handler.ServeHTTP(w, r)

     resp := w.Result()

     if resp.StatusCode != http.StatusOK {
          t.Errorf("Unexpected status code %d", resp.StatusCode)
     }
}

Same test as before, but now I’m checking the actual multiplexer used by the HTTP server works too, without having to write an integration test for that. Technically if someone forgets to pass the multiplexer to the server then that will not be picked up by my unit tests, so they’re not perfect; but that’s a single line mistake and all your URL handlers won’t work, so I’m less concerned about that being not picked up by the developer than someone forgetting one handler in dozens. You also will automatically be testing any new http wrapper functions people insert into the chain. This could be a mixed blessing perhaps, but I’d argue it’s better to make sure the wrappers are test friendly than have less overall coverage.

The other win of this approach is you can also unittest that your static content is is being mapped correctly, which you can’t do using the common approach. You can happily test that requests to the static path I set up in SetupMyHandlers returns something sensible. Again, that may seem more like an integration style test, rather than a unit test, but if I add a unit test to check that then I’m more likely to find a fix bugs earlier in the dev cycle, rather than wasting time waiting for CI to pick up my mistake.

In general, if you have global state, you have a testing problem, so I’m surprised this approach isn’t more common. It’s hardly any code complexity increase to do what I suggest, but your test coverage grows a lot as a result.