Recent commit breaks modifying jsObject from brython code

[Ben Levitt]

Starting with this commit: 7ad09ad

My code no longer runs correctly. I'm still trying to figure out exactly what is going wrong, but here is an example:

I call into a js library to get a menubar definition (a map/dict of names to menu items, and sub-items). I change the sub-items list of one of the menu items, and call the library's update() method to rebuild the menubar. This works before the above commit, but afterwards the library does not see the changes to the menu sub-items. Maybe my brython code is receiving a copy of the dict, so changes to it aren't reflected in the js library's copy?

I'll try to put together a minimal test case, but I don't have much time right now.

[Ben Levitt]

Ok, I finally have an example / test case for this one. In recent brython versions, since the commit referenced above:

I can run the following JS:

window.jsObj = {item: {}};
window.GetTest = function () { return window.jsObj; };
window.PrintTest = function () { console.log("JS:", window.jsObj.item['sub']); };

and then the following brython:

from browser import window

window.GetTest().item = {'sub': []}
window.GetTest().item['sub'] = [1, 2]
print("Py:", window.GetTest().item['sub'])
window.PrintTest()

and I get:

Py: [1, 2]
JS: []

but they should both show [1, 2], which worked in older bython versions. The issue seems to only happen when I assign a brython dict to a js variable, and then set a field of that js dict from brython code.

[Ben Levitt]

This issue is fixed when I remove these two newly added lines:

pyobj[JSOBJ] = jsobj
jsobj[PYOBJ] = pyobj

from here.

But I imagine that breaks the issue that they were added to fix.

[Ben Levitt]

This is still broken for me using the latest brython code.

[Pierre Quentel]

Sorry about that. It should be ok now.

[Ben Levitt]

This test case was fixed, but I still have a related problem. I'm using a js library function that takes a JSObject, and fails if it contains unknown fields. When I pass a brython dict into this js function, it fails because although it still sees the intended fields, it also finds an additional, and unexpected 'Symbol(PYOBJ)' field.

I used to be able to bypass this issue by wrapping my dict in window.Object.new() as I passed it into the JS function, but this no longer works. The 'Symbol(PYOBJ)' field is still added.

[Pierre Quentel]

Interesting... Usually symbols don't appear when iterating on an object's keys. Can you tell me which library it is so that I can reproduce the issue ?

[Ben Levitt]

This is coming up when using the ace js text editor, when I send back results to the provided callback after I generate results from a request for an autocomplete list. But it's easy to reproduce without using that library:

I can run the following JS:

window.PrintObj = function(o) {
    console.log(o);
}

and then the following brython:

from browser import window

o = window.Object.new({"value": 1})
window.PrintObj(o)

and I get:

{value: 1, Symbol(PYOBJ): {…}}

whereas using previous versions of brython, I get:

{value: 1}

[Ben Levitt]

Or if this new behavior of including this symbol when passing a dict to a js function is intended, I can work around it with a js function to clean the object of symbol fields in the rare case that the js function cares.

[Pierre Quentel]

With the commit above, no extra attribute is added to the Javascript object. Hopefully this will fix the issue.

The solution is not optimal since it uses a JS Map as a mapping between the JS object and its Python corresponding object, which might lead to memory issues. But I hope that the use case will not be too frequent. If there is a better solution I am all ears !

[Ben Levitt]

This is very interesting. This fix did not help, so I dug more into what's happening inside the ace js library. After the commit that this issue is originally discussing (7ad09ad), this js library callback receives an array of brython dicts, instead of an array of js objects. Before that commit, it correctly receives js objects.

This means that the problem I'm seeing now is not due to the Symbol(PYOBJ) field in the js object, it's that these js objects are being converted back to brython dicts while calling the js callback, so I think you can safely revert your last change adding the JS Map. (Yay!) And I'll try to figure out what is so special about this library that is causing the problem. I haven't been able to put together a simple test case that shows this problem yet.

In my brython code, I run:

# acList is a list of stings
# callback is a js function, to which I must send a list of objects, each with a "value" field.
compData = [window.Object.new({"value": w}) for w in acList]
# at this point in both older and newer brython, compData is a nice clean list of js objects
callback(javascript.NULL, compData)

But ever since the problem commit, the callback() function receives a js Array of brython dicts, so brython must be converting the js objects back to brython dicts during this call -- but only the copy passed into the callback function. Locally, compData is still a list of js objects. And when I try this by calling my own js function instead of the callback, my js code receives a js array of js objects.

But if instead, I create and pass in a js Array of js Objects, the Objects are not converted back to dicts, and everhthing works correctly. This workaround is actually fine for me, although I feel that some edge case is still very slightly broken.

compData = window.Array.new(*[window.Object.new({"value": w}) for w in acList])
callback(javascript.NULL, compData)  # Works as expected

[Pierre Quentel]

Another attempt in commit 239647b

[Ben Levitt]

Yes! This fixed the issue for me. Thank you again!

[Pierre Quentel]

Yippee !