Diff containing "differences" for identical lines
justvanrossum opened this issue · comments
I'm finding an issue where fdiff reports lines as being different, even though they aren't. In the fragment below, only the EndSize lines are actually different. I've included a "manual" diff fragment as well.
The full diff of this GPOS table is pretty large in both cases, but the fdiff one is a lot larger than the manual diff:
fdiff: 84459 linesdiff -u: 25151 lines
$ fdiff font_A.ttf font_B.ttf --include=GPOS:
--- font_A.ttf 2020-04-30T07:14:55.087104+02:00
+++ font_B.ttf 2020-05-03T22:51:25.100047+02:00
@@ -285,57 +285,57 @@
<Value1 XAdvance="-72">
<XAdvDevice>
<StartSize value="8"/>
- <EndSize value="654"/>
- <DeltaFormat value="32768"/>
- </XAdvDevice>
- </Value1>
- </PairValueRecord>
- <PairValueRecord index="1">
- <SecondGlyph value="g"/>
- <Value1 XAdvance="-51">
- <XAdvDevice>
- <StartSize value="8"/>
- <EndSize value="749"/>
- <DeltaFormat value="32768"/>
- </XAdvDevice>
- </Value1>
[ many lines snipped ]
+ <EndSize value="656"/>
+ <DeltaFormat value="32768"/>
+ </XAdvDevice>
+ </Value1>
+ </PairValueRecord>
+ <PairValueRecord index="1">
+ <SecondGlyph value="g"/>
+ <Value1 XAdvance="-51">
+ <XAdvDevice>
+ <StartSize value="8"/>
+ <EndSize value="751"/>
+ <DeltaFormat value="32768"/>
+ </XAdvDevice>
+ </Value1>
[ many lines snipped ]$ diff -u gpos_old.ttx gpos_new.ttx:
--- gpos_old.ttx 2020-05-04 06:39:29.000000000 +0200
+++ gpos_new.ttx 2020-05-04 06:39:45.000000000 +0200
@@ -285,7 +285,7 @@
<Value1 XAdvance="-72">
<XAdvDevice>
<StartSize value="8"/>
- <EndSize value="654"/>
+ <EndSize value="656"/>
<DeltaFormat value="32768"/>
</XAdvDevice>
</Value1>
@@ -295,7 +295,7 @@
<Value1 XAdvance="-51">
<XAdvDevice>
<StartSize value="8"/>
- <EndSize value="749"/>
+ <EndSize value="751"/>
<DeltaFormat value="32768"/>
</XAdvDevice>
</Value1>I can't publicly share this font, though :(
Hmmm... Are you able to mail me the ttx dumps from the GPOS table only for both the left and right side of the comparisons? I will need to create a ~min example for a test and go from there. Not even sure where to begin with ~60k lines of differences!
And FWIW, you can use fdiff as a diff runner with the --external option. It supports the ttx dumps (and remote pull if the files are not local). Just enter the executable and any options from the diff command that should receive the two file path args for the ttx dumps as the final two positional args.
$ fdiff --external="diff -u" [FONT 1] [FONT 2]
Ah, the external option is very cool, thanks! Emailing you two GPOS ttx files.
Received! Will take a look. Thanks for reporting this Just.
I found the problem. It is a bug in the "autojunk" heuristic in large files. I introduced this problem in an attempt at performance gains in larger files...
I'll revert to the upstream difflib and push a fix.
There is still a bit of a diff across the cPython and diff -u unified diff implementations. This is the diff of the diff :)
Diff
--- diffu.txt 2020-05-06 10:17:10.000000000 -0400
+++ pydiff.txt 2020-05-06 10:46:02.000000000 -0400
@@ -1,5 +1,5 @@
---- gpos_before.ttx 2020-05-04 10:00:33.000000000 -0400
-+++ gpos_after.ttx 2020-05-04 10:00:52.000000000 -0400
+---
++++
@@ -285,7 +285,7 @@
<Value1 XAdvance="-72">
<XAdvDevice>
@@ -14760,21 +14760,29 @@
<DeltaFormat value="32768"/>
</XAdvDevice>
</Value1>
-@@ -44897,7 +44897,7 @@
+@@ -44897,16 +44897,16 @@
<Value1 XAdvance="20">
<XAdvDevice>
<StartSize value="8"/>
-- <EndSize value="234"/>
+ <EndSize value="235"/>
- <DeltaFormat value="32768"/>
- </XAdvDevice>
- </Value1>
-@@ -44906,7 +44906,7 @@
- <Value1 XAdvance="20">
- <XAdvDevice>
- <StartSize value="8"/>
++ <DeltaFormat value="32768"/>
++ </XAdvDevice>
++ </Value1>
++ </Class2Record>
++ <Class2Record index="38">
++ <Value1 XAdvance="20">
++ <XAdvDevice>
++ <StartSize value="8"/>
+ <EndSize value="234"/>
+- <DeltaFormat value="32768"/>
+- </XAdvDevice>
+- </Value1>
+- </Class2Record>
+- <Class2Record index="38">
+- <Value1 XAdvance="20">
+- <XAdvDevice>
+- <StartSize value="8"/>
- <EndSize value="233"/>
-+ <EndSize value="234"/>
<DeltaFormat value="32768"/>
</XAdvDevice>
</Value1>
@@ -24732,10 +24740,11 @@
<DeltaFormat value="32768"/>
</YDeviceTable>
</BaseAnchor>
-@@ -62466,6 +62467,40 @@
+@@ -62465,6 +62466,40 @@
+ </XDeviceTable>
</BaseAnchor>
<BaseAnchor index="2" empty="1"/>
- </BaseRecord>
++ </BaseRecord>
+ <BaseRecord index="243">
+ <BaseAnchor index="0" Format="3">
+ <XCoordinate value="632"/>
@@ -24769,10 +24778,9 @@
+ <DeltaFormat value="32768"/>
+ </XDeviceTable>
+ </BaseAnchor>
-+ </BaseRecord>
+ </BaseRecord>
</BaseArray>
</MarkBasePos>
- </Lookup>
@@ -62522,7 +62557,7 @@
</XDeviceTable>
<YDeviceTable>@justvanrossum Mind confirming that the changes in #55 (fix-udiff branch) fix this issue? If so, I will merge and push a new release. I think that this should address the problem here based on direct tests of your GPOS XML files.
Hi Chris, sorry for my slow reply: works like a charm!
And indeed slower than before, which is expected. I don't mind to wait a little longer for a more usable diff. Thanks so much!
No worries at all. Good to hear this. I will push a release today. Thanks for reporting it Just!
Pushed these changes to PyPI as v2.0.0 release.
https://pypi.org/project/fdiff/2.0.0/
Thanks again for reporting this Just!