Why doesn't my closure see the new value of the variable it's closing over?

[balnbibarbi]

it seems that closures over a variable defined in a scope, and closures over function parameters, work differently: the former closes over the variable by reference, the latter by value.

Is this intentional?
The wording in the language spec around variables and bindings is not clear to me.

Example code demonstrating the difference:

def assert_fail(message):
  fail("Assertion failed: " + message)
end

def assert_equal(val1, val2):
  if val1 != val2:
    assert_fail("Values should be equal: " + repr((val1, val2)))
  end
end

def test_closures():
  # This function creates and returns a function that closes over the parameter
  # to the first function, which is a scalar value.
  def make_closure_over_scalar_param(passedval):
    def closed_over_scalar_param():
      return passedval
    end
    return closed_over_scalar_param
  end
  # This function creates and returns a function that closes over the parameter
  # to the first function, which is a compound value.
  def make_closure_over_compound_param(compound_param):
    def closed_over_compound_param():
      passedval = compound_param[0]
      return passedval
    end
    return closed_over_compound_param
  end
  def test_closed_over_enclosing():
    myval = "old value"
    # This function closes over a variable that is defined in an enclosing scope.
    def closed_over_enclosing():
      return myval
    end
    # This case makes sense. The value hasn't changed yet.
    assert_equal(closed_over_enclosing(), "old value")
    myval = "new value"
    # I do not understand this case. The nested function appears to have a closure by reference,
    # not by value.
    assert_equal(closed_over_enclosing(), "new value")
  end
  def test_closed_over_scalar_param():
    myval = "old value"
    closed_over_scalar_param = make_closure_over_scalar_param(myval)
    # This case makes sense. The value hasn't changed yet.
    assert_equal(closed_over_scalar_param(), "old value")
    myval = "new value"
    # I expect this to show the old value, because the value is passed by value not by reference.
    assert_equal(closed_over_scalar_param(), "old value")
  end
  def test_closed_over_compound_param():
    myval = "old value"
    compound_param = [ myval ]
    closed_over_compound_param = make_closure_over_compound_param(compound_param)
    # This case makes sense. The value hasn't changed yet.
    assert_equal(closed_over_compound_param(), "old value")
    compound_param[0] = "new value"
    # I expect this to show the new value, because although the value is passed by value,
    # it is a compound value that is later modified in place.
    # So the value being modified is being passed by reference.
    assert_equal(closed_over_compound_param(), "new value")
  end
  test_closed_over_scalar_param()
  test_closed_over_compound_param()
  test_closed_over_enclosing()
end
test_closures()

[Alan Donovan]

It's really very simple: if a function refers to a variable, each evaluation of that reference yields the current value of the variable. (In your terminology, it's always "by reference".)

(BTW, what are all those ends? That's not valid Starlark)

[balnbibarbi]

I'm using the version of Starlark vendored into ytt, which I've only just realised has several changes relative to your original.
I should go open an issue over there. Sorry for the noise.

Nevertheless, Alan, if it's not too much work, would you mind please testing whether the code above runs with your build of Starlark?

It would be interesting to know whether the problem I'm seeing is specific to ytt's modified starlark or not.

Thanks,

Matt

[balnbibarbi]

Obviously you will have to egrep -v '^[ ]+end$', sorry :-)

[balnbibarbi]

Ok, I tried this with starlark@latest. The following code fails, but according to my understanding should work:

def assert_fail(message):
  fail("Assertion failed: " + message)

def assert_equal(val1, val2):
  if val1 != val2:
    assert_fail("Values should be equal: " + repr((val1, val2)))

# This function creates and returns a function that closes over the parameter
# to the first function, which is a scalar value.
def make_closure_over_scalar_param(passedval):
  def closed_over_scalar_param():
    return passedval
  return closed_over_scalar_param

def test_closed_over_scalar_param():
  myval = "old value"
  closed_over_scalar_param = make_closure_over_scalar_param(myval)
  # This case makes sense. The value hasn't changed yet.
  assert_equal(closed_over_scalar_param(), "old value")
  myval = "new value"
  # The new value should be visible in the closure
  assert_equal(closed_over_scalar_param(), "new value")

test_closed_over_scalar_param()

Output is:

Traceback (most recent call last):
  test-closures.star:24:30: in <toplevel>
  test-closures.star:22:15: in test_closed_over_scalar_param
  test-closures.star:6:16: in assert_equal
  test-closures.star:2:7: in assert_fail
Error in fail: fail: Assertion failed: Values should be equal: ("old value", "new value")

[balnbibarbi]

Do you mind having another look please @adonovan ?

[Alan Donovan]

The call to make_closure is passed the value of myval, which is "old value". When the returned closure is called, it returns "old value". The variable named myval is not referenced by any closure, so changing its value is not going to have any effect.