Interesting, I don’t use python day-to-day, so I had assumed that the delayed capturing of the iteration variable was a lambda thing. Looks like this is still broken:
for i in range(10):
def mul(x):
return i^x
powers_of_x.append(mul)
That to me coming from a JS background is totally wild
By definition, a closures closes over its definition context, it doesn't capture the values at definition time but instead keeps referring to said definition context. If the definition context is mutable and modified, the closure reflects that change when it's finally invoked.
It’s actually due to python late binding. The variables are looked up at call time, rather than function define time. One could argue that your example shouldn’t work anymore in python3 since variables in list comprehensions go out of scope when they finish now. I haven’t tried it myself. I fill it under, “things I never need to do” :-)
It's not "python late binding", closures behave this way in any language with mutable bindings[0]. The entire point of a closure is to close over its lexical context, if the lexical context is mutable and mutated before the invocation of the closure, the closure is going to reflect that change.
will behave the same way because it does the same thing, so will e.g.
for i := 0; i<10; i++ {
powers_of_x = append(powers_of_x, func(x int) int { return x ^ i })
}
in Go.
One of the confounding factors is that many languages have block-scoped for loops especially when using iterators instead of low-level C-style loops; that's also the case in Javascript when using `let` bindings (and is in fact one of the major reason to use `let` instead of `var` if that's possible).
The underlying concern is still there[1], but this very common failure case is avoided: rather than update a single binding, each iteration creates a brand new binding for the closure to close over.
Alternatively, a common mitigation technique is to emulate that using e.g. immediately invoked function expressions.
In Python you can also use the "default parameter" trick to shadow the closed-over binding (though most of the time this is used for performance reasons) without the overhead (both syntactic and runtime) of lambdas in lambdas in lambdas:
for i in range(10):
powers_of_x.append(lambda x, i=i: x^i)
[0] excluding the special case of low-level languages with capture clauses, as well as languages like Java where closing over mutable bindings is specifically forbidden (a lambda or anonymous class can only close over `final` bindings)
[1] and remains a regular issue in async code fighting over closed-over context
You're right. I remembered the reason after reading your sibling comment and almost deleted my comment to avoid the confusion. Thanks for going into the details.
std::vector<std::function<int(int)>> powers_of_x;
for (int i = 0; i < 10; ++1)
power_of_x.push_back([=](int x) { return std::pow(x, i); });
works as one would intuitively expect (a different i is captured for each iteration). The issue is with those languages that conflate values with references.
> excluding the special case of low-level languages with capture clauses
Of course it works, you're specifically capturing `i` by value. It's not exactly surprising that doing things completely differently yields a different result.
To be fair I missed the note at the end of your commend on my first read. Still, in the specific C++ example, no other capture clause is valid.
Also, python could have chosen a slightly different closure semantics and preserved sanity: instead of closing over the binding itself, it could close over each object reference separately (exactly in the same way the default parameter hack works).
package main
import (
"fmt"
)
// return a^n
func Power(a, n int) int {
var i, result int
result = 1
for i = 0; i < n; i++ {
result *= a
}
return result
}
func main() {
fmt.Println("Hello, playground")
x := 2
var powers_of_x []int
for i := 0; i < 10; i++ {
powers_of_x = append(powers_of_x, func(x int) int { return Power(x, i) }(x))
}
fmt.Println(powers_of_x)
}
//Output:
//Hello, playground
//[1 2 4 8 16 32 64 128 256 512]
I know nothing about go, but it seems to me in your example powers_of_x is a list of n integers, while in the example being discussed is an list of n functions (which can be used to compute the n'th power).
for i in range(10):
That to me coming from a JS background is totally wild