load performance issue with concatenate

[Brannon King]

See the attached image of the profiler output. You can see that most of the import time is np.concatenate. Can we not figure out the correct dimensions early on, by the time we're adding the column data? If not, we might need to append to lists and then generate the final array at the end. The rest of the time is in the list indexer because you're relying on exceptions. Don't ever rely on an exception. They tend to be two orders of magnitude slower than an if statement.

[Julian Märte]

Hi!

This code was not originally meant for large problems and thus the reliances on imperformant instructions.

However I am well aware of the problem and of the fact that this is used for larger problems now. The numpy_free branch has a version that fixes this issue.

Take a look at it and let me know what you think.

Best regards!