Giters

cchesser / java-perf-workshop

Guided walkthrough to understand the performance aspects of a Java web service

Home Page:https://jvmperf.net/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Random test failures in SearcherTest#testSearch from text.png file access

AnEmortalKid opened this issue · comments

commented

The SearcherTest#testSearch suffers from intermittent failures, I've been able to reproduce it a bunch and have some symptoms:

When the test takes less than a second, things fail. When it doesn't, things pass. I'm guessing there's some synchronization / concurrency state involved in the failure.

I've been able to get a consistent exception in the console when this fails:

On thread:wkshp-conf-loader-1
Process finished with exit code -1
java.io.FileNotFoundException: text.png (Access is denied)
	at java.io.RandomAccessFile.open0(Native Method)
	at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
	at javax.imageio.stream.FileImageOutputStream.<init>(FileImageOutputStream.java:69)
	at com.sun.imageio.spi.FileImageOutputStreamSpi.createOutputStreamInstance(FileImageOutputStreamSpi.java:55)
	at javax.imageio.ImageIO.createImageOutputStream(ImageIO.java:419)
	at javax.imageio.ImageIO.write(ImageIO.java:1530)
	at cchesser.javaperf.workshop.data.AsciiArtConverter.convert(AsciiArtConverter.java:32)
	at cchesser.javaperf.workshop.data.ConferenceSessionLoader$SessionLoadCallable.call(ConferenceSessionLoader.java:128)
	at cchesser.javaperf.workshop.data.ConferenceSessionLoader$SessionLoadCallable.call(ConferenceSessionLoader.java:105)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: java.lang.NullPointerException

	at cchesser.javaperf.workshop.data.ConferenceSessionLoader.load(ConferenceSessionLoader.java:98)
	at cchesser.javaperf.workshop.data.Searcher.search(Searcher.java:44)
	at cchesser.javaperf.workshop.data.SearcherTest.testSearch(SearcherTest.java:19)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
	at com.github.tomakehurst.wiremock.junit.WireMockRule$1.evaluate(WireMockRule.java:72)
	at org.junit.rules.RunRules.evaluate(RunRules.java:18)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: java.lang.NullPointerException
	at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
	at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
	at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
	at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:137)
	at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2348)
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2320)
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
	at com.google.common.cache.LocalCache.get(LocalCache.java:3937)
	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)
	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)
	at cchesser.javaperf.workshop.data.ConferenceSessionLoader.load(ConferenceSessionLoader.java:96)
	... 26 more
Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at cchesser.javaperf.workshop.data.ConferenceSessionLoader$1.load(ConferenceSessionLoader.java:58)
	at cchesser.javaperf.workshop.data.ConferenceSessionLoader$1.load(ConferenceSessionLoader.java:49)
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
	... 32 more
Caused by: java.lang.NullPointerException
	at javax.imageio.ImageIO.write(ImageIO.java:1538)
	at cchesser.javaperf.workshop.data.AsciiArtConverter.convert(AsciiArtConverter.java:32)
	at cchesser.javaperf.workshop.data.ConferenceSessionLoader$SessionLoadCallable.call(ConferenceSessionLoader.java:128)
	at cchesser.javaperf.workshop.data.ConferenceSessionLoader$SessionLoadCallable.call(ConferenceSessionLoader.java:105)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

I thought it had to do with where the threads are launched from (and somehow not having access to write, so I added some naive prints for the thread name:

On thread:wkshp-conf-loader-0
java.io.FileNotFoundException: text.png (Access is denied)
	at java.io.RandomAccessFile.open0(Native Method)
	at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
	at javax.imageio.stream.FileImageOutputStream.<init>(FileImageOutputStream.java:69)
	at com.sun.imageio.spi.FileImageOutputStreamSpi.createOutputStreamInstance(FileImageOutputStreamSpi.java:55)
	at javax.imageio.ImageIO.createImageOutputStream(ImageIO.java:419)
	at javax.imageio.ImageIO.write(ImageIO.java:1530)
	at cchesser.javaperf.workshop.data.AsciiArtConverter.convert(AsciiArtConverter.java:32)
	at cchesser.javaperf.workshop.data.ConferenceSessionLoader$SessionLoadCallable.call(ConferenceSessionLoader.java:128)
	at cchesser.javaperf.workshop.data.ConferenceSessionLoader$SessionLoadCallable.call(ConferenceSessionLoader.java:105)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
On thread:wkshp-conf-loader-0

However, this also happens with wkshp-conf-loader-1 , so no dice there :(

I've found a way to fix it (if the issue is not being able to create a file at a certain spot, concurrency problems are always tough).

Path tempFilePath = Files.createTempFile("text", "png");
File tempFile = tempFilePath.toFile();
tempFile.deleteOnExit();

ImageIO.write(image, "png", tempFile);

Was the intermittent failure an intended behavior for the purpose of the workshop? If so, we can keep the failures!

commented

Was the intermittent failure an intended behavior for the purpose of the workshop?

Ha, I wish I could use this reasoning on more of the problems I introduce.

As you pointed out, the AsciiArtConvertor, which uses this text.png as a buffer; however, it is a single file supporting the buffer (which is generally, always a bad thing when it comes to concurrency).

Yes, feel free to change the file utilization - which should avoid this problem. Also, we can yank the ASCII art component out of this too - it was just a contrived example earlier that exposed more inefficient usage of things when it came to profiling.

commented

Ah, well I did start working on a gatling sim (https://github.com/AnEmortalKid/java-perf-workshop/blob/CleverCache/java-perf-workshop-tester/src/test/scala/cchesser/javaperf/workshop/WorkshopSimulation.scala) and making full use of the ascii art contrived example.

I'm also adding a cache to intentionally introduce a memory hog (playing around with settings atm), so the profiling / heap consumption is a bit more interesting for the workshop :)