Conversion vrml->x3d and vice versa: nasty decimal point/comma problem!
elmkni opened this issue · comments
Hi Michalis,
it's me again Elmar.
I've tested the latest View3DScene and ToVRMLX3D in view3dscene-4.3.0-win64-x86_64 from 07-May-2023 23:06.
Consider the code from issue #54 again:
#X3D V4.0 utf8
PROFILE Full
Viewpoint {
fieldOfView .4
centerOfRotation 0 1 0
position -5.875 5.34 6.9
orientation -.64157 -.76 -.1 .8758
}
Shape {
appearance Appearance {
material Material {}
}
geometry Box {}
}
DEF CR Transform {
children Transform {
translation .056 -.037 -.2 children Shape {
geometry DEF COPYRIGHT Text {
string "Box (C) 28.04.2023 Elmar Knittel"
fontStyle FontStyle { size 0.00213 family "SANS" justify "END" }
}
}
}
}
DEF PS ProximitySensor { size 1e4 1e4 1e4 }
ROUTE PS.position_changed TO CR.set_translation
ROUTE PS.orientation_changed TO CR.set_rotation
If I convert it to X3D (either with tovrmlx3d or view3dscene), I get this:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE X3D PUBLIC "ISO//Web3D//DTD X3D 4.0//EN" "http://www.web3d.org/specifications/x3d-4.0.dtd">
<X3D profile="Full" version="4.0"
xmlns:xsd="http://www.w3.org/2001/XMLSchema-instance"
xsd:noNamespaceSchemaLocation="http://www.web3d.org/specifications/x3d-4.0.xsd">
<head>
<meta name="generator" content="tovrmlx3d, https://castle-engine.io/view3dscene.php#section_converting" />
<meta name="source" content="view3dscene_issue_55.x3dv" />
</head>
<Scene>
<Viewpoint
fieldOfView="0,400000005960464"
centerOfRotation="0 1 0"
position="-5.875 5.34000015258789 6.90000009536743"
orientation="-0.641569972038269 -0.759999990463257 -0.100000001490116 0,875800013542175" />
<Shape>
<Appearance>
<Material />
</Appearance>
<Box />
</Shape>
<Transform DEF="CR">
<Transform
translation="0.0560000017285347 -0.0370000004768372 -0.200000002980232">
<Shape>
<Text DEF="COPYRIGHT"
string='"Box (C) 28.04.2023 Elmar Knittel"'>
<FontStyle
size="0,00212999992072582"
family='"SANS"'
justify='"END"' />
</Text>
</Shape>
</Transform>
</Transform>
<ProximitySensor DEF="PS"
size="10000 10000 10000" />
<ROUTE fromNode="PS" fromField="position_changed" toNode="CR" toField="set_translation" />
<ROUTE fromNode="PS" fromField="orientation_changed" toNode="CR" toField="set_rotation" />
</Scene>
</X3D>
At a first glance, it looks correct (and mostly is), but if you look close at fieldOfView, orientation and size, you'll find that some (but strangely not all) decimal points are exchanged by a comma -- which results in an empty scene without any error shown!
This might be a Windows problem, because the german version of Windows (e.g. the calculator) does the exchange too (which is very uncomfortable, because I can not simply copy/paste results from calculations to other apps, that always use the US/UK convention of a decimal point!).
But it also might be a FPC problem, because tovrmlx3d/view3dscene only do the exchange on single floats like size and angles ...
BTW: The problem also occurs in the opposite direction, X3D -> VRML ...
I would be interested, if this problem occurs only on my system or if it is a general problem!
If it is a general problem, it should be fixed, because it renders converted files unusable!
I hope you can help!
With best regards,
Elmar
P.S.: here are the zipped test files: view3dscene_issue_55.zip
Hi Michalis,
I've tested the old View3DScene and ToVRMLX3D in view3dscene-4.3.0-win64-x86_64 from 24-Oct-2022 12:32:
No problems! -- All values are converted correctly.
So it is neither a Windows nor a FPC problem.
What did you change for the new View3DScene/ToVRMLX3D versions?
I hope you can fix this!
With best regards,
Elmar
Thank you for this very important bug-report (I know that many people use view3dscene/tovrmlx3d to perform conversions, so it's important to have it reliable).
Fixed!
I have also added automatic tests -- to make sure the problem will never ever come back again. The output VRML/X3D should always use dots, regardless of on which system/locale the view3dscene or tovrmlx3d have been used.
The new build will be available on https://jenkins.castle-engine.io/public/builds/view3dscene/ . It requires rebuilding CGE so, as before, give it some time. You can keep observing the page castle-engine/castle-engine@snapshot...master : when it will no longer show the commit "Fix VRML/X3D conversion making sometimes comma (need to use FormatDot……" then this CGE commit is in snapshots, and latest view3dscene will include it in next ~15 minutes.
Background:
The problem was caused by a change in Castle Game Engine code some time ago. On some systems and locales (like German Windows or Polish Windows) some Pascal routines (of both FPC and Delphi) generate comma by default. We have a code in CGE to force them to use dot of course, regardless of locale, but it didn't cover all cases as you found out. If you're interested in more technical details, they are on https://castle-engine.io/coding_conventions#float_string_dot .
Hi Michalis,
I have just tested the latest View3DSsene from 10-May-2023 04:49.
The decimal-point/comma issues in conversions from VRML to X3D and vice versa are indeed gone!
But now I found the same problem in View3DScenes's logfile:
Log for "view3dscene".
Version: 4.3.0.
Started on 2023-05-10 at 09:35:42.
Castle Game Engine version: 7.0-alpha.snapshot.
Compiled with FPC 3.2.2.
Platform: Desktop, OS: Win64, CPU: x86_64 (this exe is using 64-bit architecture).
ZLib detected (version 1.2.3).
LibPng detected (version 1.4.0).
Warning: Using ApplicationConfig('view3dscene.conf') before the Application.OnInitialize was called. This is not reliable on mobile platforms (Android, iOS). This usually happens if you open a file from the "initialization" section of a unit. You should do it in Application.OnInitialize instead.
Warning: Opening file "file:///C:/Users/admin/AppData/Local/view3dscene/view3dscene.conf" before the Application.OnInitialize was called. This is not reliable on mobile platforms (Android, iOS). This usually happens if you open a file from the "initialization" section of a unit. You should do it in Application.OnInitialize instead.
Config: Loaded configuration from "file:///C:/Users/admin/AppData/Local/view3dscene/view3dscene.conf"
Auto-calculated Radius 0,00500000, ProjectionNear 0,00300000
MultiSampling: WGL_ARB_multisample supported, using multisampling
Rendering Initialized: OpenGL 4.6 (Intel) (modern rendering: True) (for more info: LogGLInformationVerbose:=true)
Path: Program data path detected as "file:///D:/Binaries/View3DScene/"
Auto-calculated Radius 0,03465270, ProjectionNear 0,02079162
-------------------- Loading begin
Loaded "D:\Temp\view3dscene_issue_54-2.x3dv" with dependencies in 0,28 seconds:
0,00 to load from disk (create X3D graph)
0,06 to initialize scene (initialize shapes tree, collisions...)
0,22 to prepare resources (load textures, prepare OpenGL resources...)
-------------------- Loading enConfig: Saving configuration to "file:///C:/Users/elmkni/AppData/Local/view3dscene/view3dscene.conf"
Especially have a look at the loading time values, auto-calculated Radius and ProjectionNear ...
This may not be critical, but it's a bit annoying, isn't it?
It might be a good idea to fix this too!
But anyway, thank you for fixing the conversion problem!
With best regards,
Elmar
This may not be critical, but it's a bit annoying, isn't it?
Thank you, I fixed more of them to be dots.
Admittedly this comma/dot difference is a bit annoying, but I have to do this one-by-one. In the past, we had a simple routine setting "all decimal separators to dot", but it was causing other issues (namely, some people in some contexts (like number inputs in editor) actually wanted to see or write the other separator, like a comma on German Windows). And as an engine (that can be used by various applications), we have to leave the default setting of decimal separator to whatever it is (even if it is rather weird in programming or X3D context, where dot is standard) and just use a different output method (that forces a dot) in specific places.
I see you noted Jenkins slowness to update lately. Yes, our Jenkins server was just really busy recently -- both CGE and view3dscene builds had a long queue. There's lots of development (in branches and PR) and we keep increasing how many tests are done automatically. The builds of https://jenkins.castle-engine.io/public/builds/view3dscene/ we're up-to-date at one point earlier today (but now they again lag, but Jenkins will catch up again :) ).
Hi Michalis,
I have just tested the latest View3DSsene from 11-May-2023 01:13.
Seconds in view3dscene.log are now printed correctly, but Radius and ProjectionNear are still odd ...
Auto-calculated Radius 0,10000000, ProjectionNear 0,06000000
-------------------- Loading begin
Loaded "D:\Temp\print\v4_twing-063.x3dv" with dependencies in 0.39 seconds:
0.06 to load from disk (create X3D graph)
0.06 to initialize scene (initialize shapes tree, collisions...)
0.27 to prepare resources (load textures, prepare OpenGL resources...)
-------------------- Loading end
Anyway, thanks for the correction!
With best regards,
Elmar
Seconds in view3dscene.log are now printed correctly, but Radius and ProjectionNear are still odd ...
Jenkins didn't get to that change yet, it is in CGE. These logs should be fixed (in build) once commit "Format floats with dot (in more places)" disappears from castle-engine/castle-engine@snapshot...master .
Hi Michalis,
I have just tested the latest View3DSsene from 11-May-2023 15.39.
Seconds, Radius and ProjectionNear in view3dscene.log are now printed correctly!
(even though Format floats with dot (in more places) is not even gone from castle-engine/castle-engine@snapshot...master)
Auto-calculated Radius 0.10000000, ProjectionNear 0.06000000
-------------------- Loading begin
Loaded "D:\Temp\all\v4_twing-064.x3dv" with dependencies in 0.30 seconds:
0.00 to load from disk (create X3D graph)
0.06 to initialize scene (initialize shapes tree, collisions...)
0.24 to prepare resources (load textures, prepare OpenGL resources...)
-------------------- Loading end
Thanks a lot, Michalis!
With best regards,
Elmar
P.S.: How do I activate LogGLInformationVerbose:=true from the commandline?
(even though Format floats with dot (in more places) is not even gone from castle-engine/castle-engine@snapshot...master)
Ah, I see why: the Docker container of CGE (with the commit castle-engine/castle-engine@52f628a ) was actually already generated and uploaded by Jenkins. And view3dscene build depends on this container only.
Usually the Docker container is uploaded after the CGE downloads on https://castle-engine.io/download are updated ( and castle-engine/castle-engine@snapshot...master is updated)... but in this case they were provoked by another job.
Oh well, nothing bad happened, just a the order in which Jenkins did it got a bit non-standard. I guess this is OK :)
P.S.: How do I activate LogGLInformationVerbose:=true from the commandline?
There is no option to do this from command-line for now.
That said, you possibly don't need it: the information produced by LogGLInformationVerbose:=true is exactly the same thing you see if you enter view3dscene menu "View -> OpenGL Information" (and you can copy it with Ctrl+C). It shows literally the same string that LogGLInformationVerbose:=true would add to log.
Hi Michalis,
thanks for the information about LogGLInformationVerbose:=true!
I only wanted to see, if there might be further problems with the decimal separator -- but there aren't any!
Thanks a lot again.
With regards,
Elmar