Tests failing locally

[Andy Tran]

I tried to pull down the repo and I ran into three issues:

• the code doesn't seem to be formatted according to mix format
• test fail (examples pasted below)
• test writing images fail (examples below)

I am willing to help fix the first one (simple fix but large PR, about 40 files need formatting). I'm unsure of how to help for the others, but I'm willing.

Test failure examples

I ran mix test test/text_image_test.exs. My output was:

Compiling 31 files (.ex)
Generated image app
Excluding tags: [full: true]

.

  1) test Autofit text composed over a base image (TextImage.Test)
     test/text_image_test.exs:104
     ** (Image.Error) Failed to write VipsImage to file: /Users/andy/dev/elixir/image/test/support/did_not_match/text/autofit_default_composed_on_base_image.tif
     code: assert_images_equal(final_image, validate_path)
     stacktrace:
       (image 0.15.0) lib/image.ex:1017: Image.write!/3
       (image 0.15.0) test/support/image_test_helpers.ex:59: Image.TestSupport.compare_images/2
       test/text_image_test.exs:110: (test)

..

  2) test Autofit text default colors (TextImage.Test)
     test/text_image_test.exs:78
     ** (Image.Error) Failed to write VipsImage to file: /Users/andy/dev/elixir/image/test/support/did_not_match/text/autofit_default.tif
     code: assert_images_equal(image, validate_path)
     stacktrace:
       (image 0.15.0) lib/image.ex:1017: Image.write!/3
       (image 0.15.0) test/support/image_test_helpers.ex:59: Image.TestSupport.compare_images/2
       test/text_image_test.exs:83: (test)

.

  3) test Autofit text default colors and justification (TextImage.Test)
     test/text_image_test.exs:86
     ** (Image.Error) Failed to write VipsImage to file: /Users/andy/dev/elixir/image/test/support/did_not_match/text/autofit_default_with_justify.tif
     code: assert_images_equal(image, validate_path)
     stacktrace:
       (image 0.15.0) lib/image.ex:1017: Image.write!/3
       (image 0.15.0) test/support/image_test_helpers.ex:59: Image.TestSupport.compare_images/2
       test/text_image_test.exs:92: (test)



  4) test Autofit text alternate colors (TextImage.Test)
     test/text_image_test.exs:95
     ** (Image.Error) Failed to write VipsImage to file: /Users/andy/dev/elixir/image/test/support/did_not_match/text/autofit_default_with_alternate_color.tif
     code: assert_images_equal(image, validate_path)
     stacktrace:
       (image 0.15.0) lib/image.ex:1017: Image.write!/3
       (image 0.15.0) test/support/image_test_helpers.ex:59: Image.TestSupport.compare_images/2
       test/text_image_test.exs:101: (test)



Finished in 9.3 seconds (9.3s async, 0.00s sync)
8 tests, 4 failures

As you can see, some of them are failing in two ways. First that in when assert_images_equal, the images are coming back not equal (test/support/image_test_helpers.ex:50). Then, when writing them to the did_not_match directory, Image writing is failing.

If I'm doing something wrong, please let me know.

Versions:

Erlang/OTP 25 [erts-13.1.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit]

Elixir 1.14.2 (compiled with Erlang/OTP 23)

MacOs 11.3.1

[Kip Cole]

@a8t thanks for the report. I am inconsistent with my use of the formatter - because sometimes I don't like what it does. Eventually I will get over it.

Images being equal is quite brittle due to version differences of the various libs that are included with libvips. I need to find a more "tolerant" image equality test but haven't had the time to do that.

I am more surprised at the file write errors - maybe I need to add a .gitkeep to the relevant directory, I'll check and report back.

[Kip Cole]

OK, I've run the formatter and now keeping the did_not_match folders in the repo. So the test should run without that error - although the images won't always match. I wouldn't worry about that for now. I will work on the bug you reported now and I'll close this issue tomorrow unless something else is unresolved.

[Andy Tran]

Thanks Kip, sounds good to me. Appreciate the quick feedback. I'm interested in this area, so if you find you're in need of some help, let me know - I'd be happy to contribute after I learn, lol.

[Andy Tran]

tests run and write expected files. the produced diffs are very minimal, you're right. i'm curious now, so i can DM you with some options i find for a tolerant image equality.

[Kip Cole]

Andy, collaboration is very welcome and a more robust image similarity assertion would be great if you're interested in digging into that. I also have a branch where I'm testing out image classification with Bumblebee in case that's interesting to you as well.