failed to load big model
flower4wine opened this issue · comments
Hi Theo,
It seems viewer isn't happy with big model (about 150M). I used Jeremy's Revit vA3C exporter to output JSON file of rac_advanced_sample_project.rvt (Revit sample model). Of course, there're some user data in my JSON.
While loading, FireFox complains that va3c-viewer.js is running timeout and inform me to choose wait or abort va3c-viewer.js process. Either way causes nothing rendered.
Any thing I can do to overcome this? Thanks a lot!
Best Regards,
David Tan
David
Jeremy's exporter exports data in a very human readable format. This means
a ton of spaces and returns in the file - all of which can serve to create
files too large for the browser.
If you can get the data read - try the Chrome browser - by the Three.js
Editor or the vA3C Viewer and export, then you may possibly obtain a much
smaller file size which could load then in FireFox.
Also can you make this - or other files - available on the web for testing
- and as demos?
Thanks,
Theo
On Oct 11, 2014 7:38 PM, "flower4wine" notifications@github.com wrote:
Hi Theo,
It seems viewer isn't happy with big model (about 150M). I used Jeremy's
Revit vA3C exporter to output JSON file of rac_advanced_sample_project.rvt
(Revit sample model). Of course, there're some user data in my JSON.While loading, FireFox complains that va3c-viewer.js is running timeout
and inform me to choose wait or abort va3c-viewer.js process. Either way
causes nothing rendered.Any thing I can do to overcome this? Thanks a lot!
Best Regards,
David Tan—
Reply to this email directly or view it on GitHub
#6.
Theo,
** If you can get the data read - try the Chrome browser - by the Three.js Editor or the vA3C Viewer
** and export, then you may possibly obtain a much smaller file size which could load then in FireFox.
Many thanks for your tips! I found that three.js has a build tool which might be helpful. Anyway, I'll let you know what I get.
** Also can you make this - or other files - available on the web for testing and as demos?
Sure, I'll setup a wseb server and send link to you afterwards. Per current problem, actually I'm using Revit out-of-box sample "rac_advanced_sample_project.rvt". You can find RVT model in the samples folder of Revit installation.
David
Theo,
Bad news!
I use Chrome to load rac_advanced_sample_project.rvt.js in Chrome. The loading completes, but there's nothing I can do anymore (rotation, drag, or export). Chrome pops up error message saying it crashes.
Three.js build tool is for three.js script, instead of json file.
It seems that I have to rewrite Jeremy's RvtVa3c and output compact json by myself. Now my question is : is there any reference about compact json file format I can read?
Thanks a lot!
David
Theo,
I think three.js can read the big json file, in other words, the loading is OK. But it seems there's much runtime work killing three.js (or webgl engine?).
As you know, Autodesk published its own webgl service last month, which is also based on three.js. I read a dialog in Autodesk's developer blog and an Autodesk engineer said that their service can read model of about 180M (it should be the file size of Revit model, not json file).
My question is: what's the biggest "json" file you ever load? Maybe I should ask, how many faces is the limit of vA3C viewer?
David
Theo,
Here's the link of json file "rac_advanced_sample_project.rvt.js"
http://url.cn/UcNUqW (password：AnPh)
David
dear david,
if you discover a way to create a more compact json output from RvtVa3c, i will happily cooperate in integrating it into the master code stream.
yes, the autodesk view and data api team definitely put in a lot of effort optimising the three.js viewer and enabling streaming and large model support.
here are hints at some of the research and optimisations made:
cheers,
jeremy
Jeremy,
OK, giant always has enough resouce, dwarf has to work on his own hand for everything :-)
You link is every helpful, thanks a lot!
Actually I have no idea of what the compact json file looks like now. But I'll keep you posted of my progress. Cheers!
David
David
Thanks for sending the link to the file.
I can enter the password, but after that the app seems to be asking for
another user name and password. So unable to proceed.
Further thoughts:
From past experience, I have had trouble loading more than about 100 to 120
MB into a single browser tab. So your file's size may be over some inherent
browser limit.
In any case 100 MB files are not much fun to deal with anyway. if you start
playing with the vA3C Viewer you will note that it can load data
dynamically, saving any data set you create using permalinks.
This type of operation will lend itself - as it improves - to emulating
layers, external references, level of detail and the like.
Would it be possible for you to export various portions of your Revit files
- which the vA3C viewer could reassemble using just the bits you need to
see now on an as needed basis.
If we are going to be able to view data on phones and tablets then its
going to have to be based upon real-time streaming of just-in-time data
rather than huge hard disk loads into 16 GB of RAM...
Regarding reducing the size of JSON files, the issue is mostly about
removing the 'air' from files - spaces, new lines and replacing
0.00000000034 with 0.
Also if you look at the Three.js Editor you will see that a sphere is being
exported as object with radius and position and no longer as hundreds of
vertices and faces.
How about joining the vA3C project on GitHub and uploading your files there?
We dwarfs have to keep trying (or find shoulders of giants to atand on)...
Theo
On Thu, Oct 16, 2014 at 12:33 AM, flower4wine notifications@github.com
wrote:
Jeremy,
OK, giant always has enough resouce, dwarf has to work on his own hand for
everything :-)Actually I have no idea of what the compact json file looks like now. But
I'll keep you posted of my progress. Cheers!David
—
Reply to this email directly or view it on GitHub
#6 (comment).
Theo,
Many thanks for your advices!
** I can enter the password, but after that the app seems to be asking for another user name and
** password. So unable to proceed.
Please try this one without password: http://url.cn/UcNUqW
The link is from a cloud storage located in China. I had no chance to try it in other countries.
** Would it be possible for you to export various portions of your Revit files - which the vA3C viewer
** could reassemble using just the bits you need to see now on an as needed basis.
Sure, I'll try.
** How about joining the vA3C project on GitHub and uploading your files there?
I would say it's a great honor. May thanks for your invitation!
I spent the last 3 years on BIM, which was based on Revit. I have implemented a whole set of BIM use cases cover stages of design, construction and operation. Now I'm trying to work out my web BIM system with all those use cases.
I'm definitely a newbie of vA3C, so I guess I need more time to get familiar with it before I can do actual contribution. At this moment, I plan to firstly minify json file output from RvtVa3c, following your and Jeremy's suggestions. After that, I would like to discuss with you guys about the real-time streaming of vA3C. I believe it will bring vA3C to much more application scenarios.
David
Theo, Jeremy,
I tried to minify json output of RvtVa3c and figure out 2 approaches so far:
- leave RvtVa3c untouched and minify its output json file.
- modify RvtVa3c and skip output of any space and new line.
As approache #1 seemed easier, I tried it yesterday. I found a java version of json minifier (https://github.com/getify/JSON.minify) and translate it to c# version. The new json minifier works fine on json file of small size (say less than 1M). However, as it will load the whole json file into memory and processes char one by one, the performance is unacceptable on big json file.
So I gave up approach #1 and will try approach #2. However, I didn't find complete settings of Newton JSON library. It only proviedes property to set indent within a line, but no property to prohibit new line. So I guess I have to modify Newton JSON source code to achieve the perfect minification.
I'll keep you posted.
David
Dear David,
the JSON file output from RvtVa3c is generated automatically by the following line in
File.WriteAllText( _filename,
JsonConvert.SerializeObject( _container,
Formatting.Indented, settings ) );
If you replace Formatting.Indented by Formatting.None, the unnecessary spaces will be removed.
Since I am using integers and a limiting precision when collecting the object coordinates, I do not see much more compression possibility.
Cheers,
Jeremy
David
It looks like I let your link to the large file expire. Sorry.
In any case I feel that a long term solution is not dealing with huge JSON files.
A much better approach is many small JSON files assembled at runtime.
R6 of the viewer does this easily:
http://va3c.github.io/viewer/va3c-viewer-html5/latest/
For example, see the permalinks tab and click 'generate autocrapdoodle'
Also, would you kindly send an email to https://groups.google.com/forum/#!forum/va3c and I will invite you to join the mailing list.
Jeremy
I think your tip will make the files a lot smaller. Would it be possible to add a toggle to the exporter that would enable/disable human readable versions? Thanks.
Theo
hi theo,
sure i can add such a switch.
cheers
jeremy
if one really wants to get aggressive about json compression, here is one possible approach:
https://code.google.com/p/rewriting-compressor-for-json/
david, did you experiment with my suggestion yet? does it make any useful difference at all?
Theo,
Thanks for your knid guide!
It would be a reasonable approach to take splitted parts firstly, and assemble them on the fly. I'm using the same approach to get around the performance issue in a Unity3D case.
Jeremy,
Sorry, I didn't get much time recently. But I did a quick test just now, and your suggestion is perfect.
I thought Formatting property of JSON.NET only process space within a line, but obviously I was wrong. After setting Formatting from Indent to None, the output JSON doesn't have any space or line break anymore.
Thanks a lot!
David
dear david, thank you for testing! i can add a runtime end user switch to control that setting. cheers, jeremy.
implemented, uploaded and released:
updated again and described in more detail as well: