Pass setup/interpolated variables through as run arguments instead of closing over them in the harness

[jrevels]

cc @maleadt
cc @KristofferC (this approach could help with the optimization-brittleness issues you've said run into)

See my comment here: https://github.com/JuliaGPU/CuArrays.jl/issues/210#issuecomment-442860840

Right now, interpolated variables get closed over in the benchmarking harness here.

We should probably change this so that BenchmarkTools.run accepts a (splatted?) tuple of arguments that gets forwarded into the harness/kernel and bound to the interpolated variables. This way arguments can be passed in from top-level, which would likely yield better results than the current implementation (less brittle w.r.t. compiler optimizations) + make the implementation nicer (we could have a single harness definition instead of generating benchmark-specific ones, and probably factor out some other stuff as well).
For @btime/@benchmark, this would not be a breaking change. It would probably need to be breaking for @benchmarkable, though - that would have to now e.g. return the Benchmark + the tuple/NamedTuple for its run arguments.

I'm not sure this would've been possible to implement easily in the pre-NamedTuple era, but now that we have them, we might be able to use them to forward variables through to the benchmark kernel.

E.g. the whole setup expression could be rendered into a thunk that gets called with the interpolated variable arguments, and returns a NamedTuple of all the resulting setup variables.

[KristofferC]

Yeah, I noticed this when reading https://discourse.julialang.org/t/when-does-the-garbage-collector-runs/15142/3 (which made me make #117 but that, of course, doesn't change that values get captured in the generated function).

Is there any way to use anonymous functions which gets garbage collected and then release the resources they close over. Perhaps that would give nasty closure Box problems.

[willow-ahrens]

This was fixed in #265