ArgumentException: Don't know how to create ISeq
kahunamoore opened this issue · comments
Myself and @Pi-3Orionis (from gitter) found an exception when importing Arcadia (as of commit a44bea7) into a fresh project generated by 2018.3.0.f2:
ArgumentException: Don't know how to create ISeq from: arcadia.internal.map_utils$valsr$reify3643536445
arcadia$internal$filewatcher$watch_threadwatch_loop47096fn4710147114.invoke () (at <bcce9265ed47429c8ada151ec0c32a14>:0)
arcadia$internal$filewatcher$watch_threadwatch_loop47096__47117.invoke () (at <bcce9265ed47429c8ada151ec0c32a14>:0) [snipped for brevity]
Original post on gitter:
https://gitter.im/arcadia-unity/Arcadia?at=5c20e726d945b96882eb3b0c
What OS are you on?
On my system where I observed this behavior, I'm running Windows 10.
This is a somewhat mysterious (and difficult to reproduce) issue that's come up before. I think it has something to do with the AOT machinery, perhaps a stale version of an AOT'd type generated by reify. Try deleting Library and re-AOTing Arcadia, see if that helps.
When I reproduced the issue it was with Windows 7.
FWIW, I have a backup of a project where the message shows up -- when I tested what might make the error go away, I worked off of a copy. So it is reproducible here.
Deleting Library and Arcadia/Compiled folders, then opening the project in Unity and running AOT Compile again worked for both of you? It didn't help for me, I'm still getting the same error after trying that.
Follow Up: I ran AOT Compile a second time (and a third time for good measure). The error stops showing up on the second compile onward, so maybe it's a race condition in compilation that disappears once there is an existing 'copy' of the compiled code. I've seen something like this in Clojure before, though in an entirely different context. It may even have more to do with the existence of a file, rather than the content of such a file.
For clarification, I did the directory deletion while Unity wasn't running, then started Unity. I didn't have to do an explicit AOT Compile.
We've seen reify, which is used in arcadia.internal.map-utils/valsr, act janky in other contexts (#321), and making it reliable is a priority for MAGIC. It is possible this bug is related to some incorrect bytecode being emitted by reify itself.
Another possibility is that the version of clojure.core.protocols.CollReduce valsr reifies over is not the same referred to by a function attempting to process a value produced by valsr. In particular, reduce is defined for values implementing clojure.core.protocols.CollReduce. If another function uses reduce if a value implements that protocol, and otherwise attempts to convert it to a seq, the error here could be thrown if that function has a different idea of what the type clojure.core.protocols.CollReduce is than the value. To hit this problem would require reevaluating the definition of clojure.core.protocols.CollReduce, however, or perhaps having some subtly inconsistent version of it baked into the dlls that ship with Arcadia.
The fact that this problem doesn't seem to be happening on Mac or Linux makes we wonder if our Windows version is sometimes evaluating the source for Clojure code we mean to ship as dlls, due to discrepancies in file lookup or something. For those of you hitting this bug, how fast are Clojure load times on Windows? If they're slower than expected it may mean code is being compiled rather than dlls loaded.
Since this is breaking for some Windows users, and I don't have a good Windows dev environment, my inclination is to sidestep the underlying issue by just rewriting the code in question to not use valsr. I'm not going to be able to really test if that fixed it or not though, since I'm not on Windows myself and can't reproduce the issue. I could push that to another branch and then perhaps a Windows user could confirm, then we could merge it into master.
Another possibility is that some source file that is supposed to be baked into the shipped dll got marked dirty somehow, causing it to be evaluated rather than the corresponding dll namespace loaded. One way to test this would be by recloning Arcadia and seeing if it helps.
Not sure if it was clear, but one place I see the problem here is on Mojave (macOS).
I just checked my Windows 7 environment and confirmed there is still a project where I see those messages via the console, so I've backed up that project. Hence I should be able to try a variety of things under Windows 7 as well.
For both the Mojave and Windows 7 environments, after recloning to master (488e64b), the message didn't appear in the console.
Great, thanks for the confirmation @sogaiu . This strongly suggests the problem is unwanted evaluation of source code resulting in type references that are inconsistent with the shipped dlls.
Next steps are to replicate the issue on my side by touching source files and seeing if it yields the same problem. If this confirms it, we'll need to come up with some policy for fixing it – possibly just not shipping source code for Clojure.
Another data point -- I copied the project to a Manjaro Linux environment and replicated the issue.
On a side note, I'm not very knowledgeable regarding what info is stored in the project folder (e.g. whether local info such as ethernet addresses, license info, etc. is stored). It seems possible that it's relatively "sanitary". Anyone know? If it's clean or can be easily cleaned, may be figuring out how to get you folks the project and doing so would expedite investigations.
I'd love to understand more of what's going on, but I don't want to slow down this process unnecessarily.
On the Manjaro Linux box, with the problematic project folder in question, doing a git checkout master wasn't sufficient to cause the error to go away, but subsequently choosing the Arcadia -> Clean -> Clean All menu item and restarting lead to no error messages in the console.
Windows 10, Unity 2018.3, same error after "AOT Compile", there are no errors without it, but inadmissible delays in Unity editor.
BTW Using "Arcadia -> Clean -> Clean All" displays error
UnauthorizedAccessException: Access to the path ...\Arcadia\Compiled\arcadia.data.clj.dll' is denied.
Also trying to get Arcadia to work on Windows 10. First I got the ISeq error twice (the second time after removing /Arcadia and /Assets/Arcadia and then recloning without restarting Unity). Then I quit Unity, removed the folders, recloned. Started Unity, clicked "AOT Compile". Still the same error. :(
Did some more poking around. I can reproduce the error (not the stack that leads to it) as follows:
- From the application menu select
Arcadia > Clean > Clean All. - At the bottom of
Arcadia/Source/arcadia/internal/map_utils.clj, put the following:
(defonce old-valsr-instance
(valsr {:a :A :b :B}))
(try
(UnityEngine.Debug/Log
(str
"running old-valsr-instance. vectorified: "
(into [] (map identity) old-valsr-instance)))
(catch Exception e
(UnityEngine.Debug/Log
(str "!! Exception encountered while trying to run old-valsr-instance !!"))
(throw e)))- Exit and reopen Unity.
- Select
Arcadia > AOT Compile.
This reliably throws
ArgumentException: Don't know how to create ISeq from: arcadia.internal.map_utils$valsr$reify__2007__2017
arcadia$internal$map_utils$fn__9850__9854.invokeStatic () (at <e6855988074d484d85c710f9b6ebf07d>:0)
arcadia$internal$map_utils$fn__9850__9854.invoke () (at <e6855988074d484d85c710f9b6ebf07d>:0)
clojure.lang.AFn.ApplyToHelper (clojure.lang.IFn fn, clojure.lang.ISeq argList) (at <fb1204dbf0b848beaab1559f81f1b22c>:0)
clojure.lang.AFn.applyTo (clojure.lang.ISeq arglist) (at <fb1204dbf0b848beaab1559f81f1b22c>:0)
clojure.lang.CljCompiler.Ast.InvokeExpr.Eval () (at <fb1204dbf0b848beaab1559f81f1b22c>:0)
Rethrow as CompilerException: System.ArgumentException: Don't know how to create ISeq from: arcadia.internal.map_utils$valsr$reify__2007__2017
arcadia$internal$map_utils$fn__9850__9854.invokeStatic () (at <e6855988074d484d85c710f9b6ebf07d>:0)
arcadia$internal$map_utils$fn__9850__9854.invoke () (at <e6855988074d484d85c710f9b6ebf07d>:0)
clojure.lang.AFn.ApplyToHelper (clojure.lang.IFn fn, clojure.lang.ISeq argList) (at <fb1204dbf0b848beaab1559f81f1b22c>:0)
clojure.lang.AFn.applyTo (clojure.lang.ISeq arglist) (at <fb1204dbf0b848beaab1559f81f1b22c>:0)
clojure.lang.CljCompiler.Ast.InvokeExpr.Eval () (at <fb1204dbf0b848beaab1559f81f1b22c>:0 , compiling: (map_utils.clj:256:2))
clojure$core$load_libfn__25538__25542.invoke () (at <75aedcbf6ad14e4293bea1888c10ab1d>:0)
clojure$core$load_lib__25549.invokeStatic (System.Object , System.Object , clojure.lang.ISeq ) (at <75aedcbf6ad14e4293bea1888c10ab1d>:0)
clojure$core$load_lib__25549.doInvoke (System.Object , System.Object , System.Object ) (at <75aedcbf6ad14e4293bea1888c10ab1d>:0)
clojure.lang.RestFn.applyTo (clojure.lang.ISeq arglist) (at <fb1204dbf0b848beaab1559f81f1b22c>:0)
clojure$core$apply__19534.invokeStatic (System.Object , System.Object , System.Object ) (at <75aedcbf6ad14e4293bea1888c10ab1d>:0)
clojure$core$apply__19534.invoke (System.Object , System.Object , System.Object ) (at <75aedcbf6ad14e4293bea1888c10ab1d>:0)
clojure$core$load_libs__25574.invokeStatic (clojure.lang.ISeq ) (at <75aedcbf6ad14e4293bea1888c10ab1d>:0)
clojure$core$load_libs__25574.doInvoke (System.Object ) (at <75aedcbf6ad14e4293bea1888c10ab1d>:0)
clojure.lang.RestFn.applyTo (clojure.lang.ISeq arglist) (at <fb1204dbf0b848beaab1559f81f1b22c>:0)
clojure$core$apply__19534.invokeStatic (System.Object , System.Object , System.Object ) (at <75aedcbf6ad14e4293bea1888c10ab1d>:0)
clojure$core$apply__19534.invoke (System.Object , System.Object , System.Object ) (at <75aedcbf6ad14e4293bea1888c10ab1d>:0)
clojure$core$require__25596.invokeStatic (clojure.lang.ISeq ) (at <75aedcbf6ad14e4293bea1888c10ab1d>:0)
clojure$core$require__25596.doInvoke (System.Object ) (at <75aedcbf6ad14e4293bea1888c10ab1d>:0)
clojure.lang.RestFn.invoke (System.Object arg1) (at <fb1204dbf0b848beaab1559f81f1b22c>:0)
arcadia$internal$compiler$aot_namespacesfn__6076__6084.invoke () (at <95c22a79ec4449e88fa42d471083359b>:0)
arcadia$internal$compiler$aot_namespaces__6094.invokeStatic (System.Object , System.Object , System.Object ) (at <95c22a79ec4449e88fa42d471083359b>:0)
arcadia$internal$compiler$aot_namespaces__6094.invoke (System.Object , System.Object , System.Object ) (at <95c22a79ec4449e88fa42d471083359b>:0)
Arcadia.Util.Invoke (clojure.lang.Var v, System.Object a, System.Object b, System.Object c) (at Assets/Arcadia/Helpers/Util.cs:71)
Arcadia.BuildPipeline.CompileNamespacesToFolder (System.Collections.IEnumerable userNameSpaces, System.String targetFolder) (at Assets/Arcadia/Editor/BuildPipeline.cs:130)
Arcadia.BuildPipeline.BuildInternal () (at Assets/Arcadia/Editor/BuildPipeline.cs:148)
Arcadia.Menu.AOTCompile () (at Assets/Arcadia/Editor/Menu.cs:10)
and sometimes throws
ArgumentException: Don't know how to create ISeq from: arcadia.internal.map_utils$valsr$reify__2007__2017
arcadia$internal$filewatcher$watch_threadwatch_loop__4374fn__4379__4392.invoke () (at <95c22a79ec4449e88fa42d471083359b>:0)
arcadia$internal$filewatcher$watch_threadwatch_loop__4374__4395.invoke () (at <95c22a79ec4449e88fa42d471083359b>:0)
arcadia$internal$thread$start_threadfn__2503__2507.invoke () (at <95c22a79ec4449e88fa42d471083359b>:0)
(wrapper dynamic-method) System.Object.lambda_method(System.Runtime.CompilerServices.Closure)
System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) (at <ac823e2bb42b41bda67924a45a0173c3>:0)
System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) (at <ac823e2bb42b41bda67924a45a0173c3>:0)
System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) (at <ac823e2bb42b41bda67924a45a0173c3>:0)
System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) (at <ac823e2bb42b41bda67924a45a0173c3>:0)
System.Threading.ThreadHelper.ThreadStart () (at <ac823e2bb42b41bda67924a45a0173c3>:0)
UnityEngine.UnhandledExceptionHandler:<RegisterUECatcher>m__0(Object, UnhandledExceptionEventArgs)
Pushed a possible fix to c792a91. If people could please confirm the fix, I'll merge it into master.
I just tried this branch w Unity 2018.3.6f1 with the socket repl via telnet/socket server and didn't encounter this error. I don't exactly remember which client I was using previously but it was using the socket server so it is possible I'm not repro'ing the original conditions. Also, this is a different version of Unity than I used previously.
So AFAIK this works! Thanks for all your hard work on this!!! I'm excited to get back into Arcadia after a long hiatus. Hopefully I'll learn enough to be able to contribute back.
Looks good here too.
For Manjaro Linux, Windows 7, and Mojave, I basically followed the steps below:
- opened project with problem,
- observed error in console,
- did arcadia > clean > clean all,
- exited unity,
- git pull of Arcadia,
- git reset --hard,
- git checkout issue-336,
- started unity,
- observed no error in console
There was a possibly unrelated oddity with doing the clean part for Windows 7 (UnauthorizedAccessException), but a manual deletion worked there.
Also tried creating a fresh project under Windows 7 with a fresh clone of the issue-336 branch. Didn't observe the error in the console :)
Strange... I'm getting connection errors in some cases. My guess is that they are environmental/system issues and not related to this #336 issue, but I thought I'd include some info here in case someone else runs across these weird symptoms:
What works:
>telnet localhost 37220
user=> :repl/quit
What doesn't work:
>telnet 127.0.0.1 37220
Connecting To 127.0.0.1...Could not open connection to the host, on port 37220: Connect failed
>clj -m tubular.core --host localhost --port 37220
Exception in thread "main" java.net.ConnectException: Connection refused: connect
>clj -m tubular.core --host 127.0.0.1 --port 37220
Exception in thread "main" java.net.ConnectException: Connection refused: connect
FYI: the tubular socket client can be found here https://github.com/mfikes/tubular
I verified that I can connect to a standalone socket server using both of the clients using port 5555 for both localhost and the loopback address 127.0.0.1.
In one shell:
>clj -J-Dclojure.server.repl="{:port 5555 :accept clojure.core.server/repl}"
In another:
>telnet localhost 5555
user=> :reply/quit
>telnet 127.0.0.1 5555
user=> :reply/quit
>clj -m tubular.core --host localhost --port 5555
user=> :repl/quit
>clj -m tubular.core --host 127.0.0.1 --port 5555
user=> :repl/quit
I suspect that there is something about the way, or context in which, the socket is being opened on either the client or server side. I've looked at all the client and server code for clojure, Arcadia and tubular to see if anything stands out. The handling of "localhost" as a default seems to be in order in all cases but I could be missing something.
Next I investigated the server side of the socket and found something that may be relevant. I can bind to the port used by Arcadia (37220) using the standalone clojure server socket repl.
With Unity/Arcadia already running:
clj -J-Dclojure.server.repl="{:port 37220 :accept clojure.core.server/repl}"
Clojure 1.10.0
user=> (arcadia.core/create-primitive :cube)
Execution error (ClassNotFoundException) at java.net.URLClassLoader/findClass (REPL:-1).
arcadia.core
user=>
It claims to have created a standalone server at that same address that is unrelated to Arcadia.
When I connect to that port I get the Arcadia server:
>telnet localhost 37220
user=> (arcadia.core/create-primitive :cube)
#<Cube (UnityEngine.GameObject)>
user=> :repl/quit
As a counter example, if I try to bind two clojure server socket repls to the same port (5555 in this case) I get a different result:
In one shell:
clj -J-Dclojure.server.repl="{:port 5555 :accept clojure.core.server/repl}"
Clojure 1.10.0
user=>
In another:
clj -J-Dclojure.server.repl="{:port 5555 :accept clojure.core.server/repl}"
Exception in thread "main" java.lang.ExceptionInInitializerError at clojure.main.<clinit>(main.java:20)
Caused by: java.net.BindException: Address already in use: NET_Bind
As expected, a bind error.
In theory the server socket code in Arcadia and the command line server invocation both use clojure.core.server/repl but I suspect something about the clj command line app (java) and the ClojureCLR used by Arcadia could account for the difference.
Or maybe I'm just ignorant of something about how Windows 10 bind sockets. My excuse is that I'm used to Linux/Posix sockets. Is WINSOCK still a thing?
It smells like something involving SO_REUSEADDR, SO_REUSEPORT and/or INADDR_ANY.
Also, there are lots of moving parts in my environment since I'm also using:
https://github.com/frericksm/clj-windows
As the clj launcher in the examples above. I'm not exactly sure what Clojure version it is running or what the default config for it are.
Hope this helps. Let me know if you need any more info...
Don't know if this will be of any help, but related to network issues...
I've been so used to localhost == 127.0.0.1 (and disabling IPv6 on most of my machines) that I was surprised to find that on at least some machines where I haven't disabled IPv6, localhost gets resolved as ::1 (at least under certain circumstances). On such a machine, it is typical (here at least) that Arcadia is listening at ::1 port 37220 and NOT 127.0.0.1 port 37220.
Does this have any relevance to your situation?
Regarding start-server, note that for JVM Clojure:
address (InetAddress/getByName address) ;; nil returns loopbackand for CLR Clojure:
address (if (instance? System.Net.IPAddress address) address (aget (.AddressList (let [^String a (or address "localhost")] (Dns/GetHostEntry a))) 0)) ;;; Add IPAddress check, also: (InetAddress/getByName address) ;; nil returns loopback Added let to get type infoI experienced unexpected behavior with ClojureCLR when trying to use an IPv4 address (well, actually a string like "192.168.1.8") for a socket REPL. At least on Windows 7, the result was listening on an IPv6 address :( It appears possible to hand an appropriately created System.Net.IPAddress in for address though.
Works for me! I had to remove all old stuff (/Arcadia and /Assets/Arcadia) with unity shut down.
Strange, in another project I still get the same error, even after removing everything and restarting unity etc.
Hm, having a hard time reproducing this on my Windows machine. If anyone is still seeing this, can you provide the steps you take to reproduce it?
I just tried it again and it seems to work.
Win10-64, 2019.2.17f1, fresh project, Arcadia@master: 7f068c4 , patched with c792a91(WIP-possible-fix-for-336-pending-confirmation), none of my own code, choosing "prepare for export" yields:
https://gist.github.com/kamisori/1a56453eabb4f68e8c08c424ec51e5b2
For comparison with my initial problem:
Win10-64, 2019.2.17f1, fresh project, Arcadia@master: 7f068c4 , none of my own code, choosing "prepare for export" yields:
https://gist.github.com/kamisori/1d7e5050b8ccf384aef9658f961ed705