Recently an interesting series of commits landed in Prometheus with the goal of reducing the size of the Prometheus binary by allowing the Go linker to remove more unused code ( something it's quite good at in general , although the linker is also deliberately limited in this ). The commit with the message that's most informative about what is going on and why is discovery/gce: keep [Google Cloud] Compute SD client from defeating dead-code elimination , and you can read the full details in it. The short version is that if you're using certain sorts of reflection anywhere in your program, the Go linker won't remove exported (public) methods of any concrete type that's reachable through an interface. It doesn't matter how narrow the interface is (it can be the famous and minimal fmt.Stringer ); the moment you combine reflection and a concrete type in an interface, the Go linker more or less stops throwing out unused functions and methods. Well, sort of, as the commit explains.
Unlike the standard Go toolchain not doing dead code elimination for package level variables with constant values , this isn't merely the linker deciding it's too much work to do this dead code elimination optimization. Instead it's at least partly a correctness issue. The problem for the Go linker is that reflect allows you to reach through any retained interface value to use any and all exported methods on the underlying concrete type of the value (and any types it contains), using things like
Value.MethodByName()
and
Value.Call()
. This makes it hard or impossible for the Go linker to know which exported methods are really dead and can never be reached at runtime.
(This has to apply to concrete types contained in top level concrete types because reflect can reach through structs, channels, maps, arrays, and so on to retrieve underlying types and values, and thus methods on those types.)
The current Go linker is actually doing more work and eliminating more dead code than the documentation requires it to. The documentation for
Value.MethodByName()
and friends say that they apply to all exported methods (possibly only of a given name), but apparently the linker will skip this for types that are never directly or indirectly boxed into an interface, because such types aren't reachable through reflect . Since all reflect functions that create a
Type
or a
Value
take an
any
(ie, '
interface{}
') as their argument, you can't go from a value of a concrete type to either without putting the concrete type in an interface and triggering this. What this means in practice in a program where there's any use of reflect (including in some sub-dependency off in a corner) is that if you put a 'big' type with a lot of direct and indirect exported methods into an interface, all of those methods and all of their dependencies will have to be retained in the binary (and increase its size, possibly a lot), even if you only use a tiny subset of them.
(I believe this includes innocent looking things like merely printing such a 'big' concrete struct, which you might do for debugging purposes or because it has a
String()
method that does useful stuff. And of course JSON serialization uses interface values;
json.Marshall()
takes an '
any
' as an argument, so there's your interface. While the json package uses reflect internally, it doesn't currently call any of the reflect methods that triggers this linker behavior.)
There are at least two ways around this, visible in the Compute service discovery commit and a similar Kubernetes commit . In the Kubernetes commit, a concrete top level Kubernetes struct was not retained in full in a Prometheus service discovery struct that would then be boxed into an interface; instead, only the methods on the Kubernetes struct that were actually needed were extracted and embedded into a new struct, so the Go linker only had to retained those methods and their code dependencies. In the more complex Compute commit, some processing had to be done dynamically using concrete types that had to be retained, so instead of putting the concrete types in a Prometheus struct (that would then be boxed as an interface inside the Prometheus code), the values of the concrete types were made inaccessible to reflect by putting them inside a function closure, and only the function closure was stored in the Prometheus struct.
One thing I take away from this is that one should avoid using the various reflect method-getting methods if at all possible, both in a program and especially in a package that you expect other people to use. If your package uses these internally, you're creating spooky action at a distance effects on the whole program (and you should probably mention this in your documentation).
PS: The Go linker's dead code elimination is (currently) discussed in general in a big comment in cmd/link/internal/ld/deadcode.go , which is worth reading for some details that I hadn't thought about until now, such as needing to retain all methods that might be reached through interfaces (which is necessary because you might wind up casting an interface value to another interface entirely, eg , also ).
PPS: As mentioned in the Prometheus commits, one of the packages that uses reflect this way is go.yaml.in/yaml/v4 . For the actual code and usage involved, see here and here , which seem like reasonably sensible uses to me, even if they have awkward consequences.