It's too slow. Is There anyway get fast?
liuwanfu opened this issue · comments
Large DXF files will run very slow. The example runs on a single thread which can even freeze a tab in a browser like chrome if there is too much data. Additionally, we rely on three js to the do the rendering. I'm not sure if there are any optimizations we can make for rendering as I'm not an expert in this stuff. I just wrote it as a minimal library to display DXF drawings. So the answer is... maybe?
Thanks,I will try Multithreading.I hope it will be better.
This implementation is too naive to work with big files. Main reason is using three.js scene objects for each dxf entity. Geometry batching is the way to go (everything sharing one material should be drawn by a single draw call). Also there are many places with incorrect handling of DXF entities attributes and thus incorrect rendering result.
Check my viewer for high-performance implementation. Tested with real-world files. Here is demo page: https://vagran.github.io/dxf-viewer-example/
Awesome work! I'm happy to have someone take the reigns with a much better implementation of the viewer. I'll direct people to dxf-viewer. Yeah, I always knew it was too naïve for anything complex and I've had little time to keep contributing meaningful stuff to it. I most recently was toying with Bitmap fonts that could help with multiline calculations, but never got anything worth committing finished.
I'm happy to take pull requests to publish under this package. I can probably do that under GDS Estimating's time. I wouldn't be hurt if you created your own package though. Totally understand and happy to have gotten you a decent start.
Thanks! I also have plans to use bitmap fonts in the future, but not because of multiline (I just did not have example file where MTEXT is used, seems to be quite rare case in real docs), but because of performance. Vectorized text is taking quite much resources for rendering, comparing with bitmap fonts. However, in practice I found that my current approach works well in drawings with a lot of text, so this is the reason why bitmap font was not implemented initially. Also I tried to stick to look-and-feel of Autodesk official online viewer, and they use vectorized text as well, most probably because it can be scaled freely without visible quality loss, and a typical drawing usually has significantly differing text sizes. I have an idea to use instanced drawing for a particular characters based on some usage analysis in a drawing. So this is quite complex topic after all.
Actually, we tried to use your package in our product, and besides performance problem (in terms of rendering latency) we firstly encountered out-of-memory problem. With a typical construction site drawing browser quickly consumes several gigs of RAM and becomes irresponsive. So in my package I also tried to optimize for memory space as much as possible. BTW, since this is 2D viewer, I do not store Z-coordinates, and use custom shaders to handle 2D-coordinates. There are many other tricky areas, for example, coordinates normalization - i.e. you should not operate coordinate values which store significant data in its mantissa last bits which happens when whole the drawing is defined in some small region far away from coordinate system origin (region is small relatively to distance from origin). And this is typical case for construction site drawings - they are usually defined in some EPSG projection. This becomes way more complex when taking into account DXF blocks concept. They have own coordinates space, origin, and insertion point which also are often jumping between global origin and drawing region. After that we have 64-bit to 32-bit float coordinates conversion when defining vertex attributes and transformation matrices for WebGL. Even Autodesk guys did not get it fully - here is how their viewer displays some small details on such drawings (due to FP numbers precision loss):
Here is mine:
I also forked your parser (currently just inside the dxf-viewer source). Main concern was missing streaming parsing - to save some memory on huge drawings it would be nice to feed some text chunks and continuously update the result being built. But the first attempt was unsuccessful - I made all parsing functions async so that they can await on next dxf line, but it became too slow, most probably because these calls are transformed by babel to some synthetic generator polyfills which seems to be very ineffective. I did not try with native async because we are stuck to babel in the project where this viewer is used. Without using async for each input line, it requires full rewrite because of complete different processing flow. I probably will try it one day.
I doubt I will have time to make pull requests to your package but I indeed can consult on GL-related topics. I also tried to write well structured commented code, hoping it can be used as reference by anyone for many questions.
Very nice! Yes, most of the issues you have mentioned have also come up for me in the issues of this repository. I too tried to do an async streaming parser but never finished. Large files have millions of lines making splitting on a line very slow. I then tried to iterate a thousand lines at a time but ran out of time writing a complex parser that kept track of the state of the current object between calls. Another problem is how to continue parsing in the middle of a deeply nested object like block inserts in block inserts, all while not filling up memory. When everything is at the root of the document, it is easy. But I came across some dxf files where everything is in a single block which had block inserts within that.
I actually know very little about working in 3D space and three.js so I'm happy to see someone who knows more about things take a good swing at it.
Since this isn't really and issue an more about making another project known, I'm going to close this. Note: some improvements have been made since this discussion. Splines and text are now handled in three-dxf