pemistahl / lingua

The most accurate natural language detection library for Java and the JVM, suitable for long and short text alike

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

run with gradle ok, but error with maven

LeeMH opened this issue · comments

hello.

I am working on changing the tika library to linuga.
An unknown error occurred during operation, so please contact us.

Problem: Samples made with gradle work fine. But the sample written with maven throws an error

Environment

  1. ide: intelij 2022.1.4
  2. java: jdk11

Error Detail:
java.lang.NoClassDefFoundError: Could not initialize class com.github.pemistahl.lingua.internal.util.extension.CharExtensionsKt
at com.github.pemistahl.lingua.api.LanguageDetector.splitTextIntoWords$lingua(LanguageDetector.kt:245) ~[lingua-1.2.2.jar:na]
at com.github.pemistahl.lingua.api.LanguageDetector.computeLanguageConfidenceValues(LanguageDetector.kt:114) ~[lingua-1.2.2.jar:na]
at com.github.pemistahl.lingua.api.LanguageDetector.detectLanguageOf(LanguageDetector.kt:74) ~[lingua-1.2.2.jar:na]
at com.example.demo.DetectController.subMethod(DetectController.java:49) ~[classes/:na]
at com.example.demo.DetectController.lambda$detect$0(DetectController.java:41) ~[classes/:na]
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[na:na]
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[na:na]
at java.base/java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:952) ~[na:na]
at java.base/java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:926) ~[na:na]
at java.base/java.util.stream.AbstractTask.compute(AbstractTask.java:327) ~[na:na]
at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746) ~[na:na]
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) ~[na:na]
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) ~[na:na]
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) ~[na:na]
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) ~[na:na]
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) ~[na:na]

Sample Code:
`
@RestController
public class DetectController {

final LanguageDetector detector = LanguageDetectorBuilder
        .fromLanguages(Language.ENGLISH, Language.KOREAN, Language.JAPANESE, Language.CHINESE, Language.SPANISH, Language.PORTUGUESE, Language.ARABIC)
        .withMinimumRelativeDistance(0.8)
        .build();

@GetMapping("/detect")
public String detect() {
    List<String> words = new ArrayList<>();
    words.add("lol");
    words.add("hallo");
    words.add("hallo Jane");
    words.add("hello");
    words.add("hello Jane. Nice to meet you.");
    words.add("hello Sofía, I'm fine thank you.");
    words.add("こんにちは");
    words.add("Πόσο είναι το εμφιαλωμένο νερό;");
    words.add("Quanto costa l'acqua in bottiglia?");
    words.add("你好简");
    words.add("中國的***廣場有什麼特別之處嗎?");
    words.add("안녕하세요 jane");

    List<String> result = new ArrayList<>();

    var res = words.parallelStream()
            .map(it -> subMethod(it))
            .collect(Collectors.joining("<br>"));

    return res;
}

public String subMethod(String word) {
    try {
        System.out.println(Thread.currentThread().getName() + " :: 호출됨 = " + word);
        var res = detector.detectLanguageOf(word);
        Thread.sleep(1000L);
        return "org = " + word + " ==> " + res.getIsoCode639_1().name();
    } catch (InterruptedException e) {
        throw new RuntimeException(e);
    }
}

}`

hello.

We share information by solving problems ourselves.

Estimating the cause of the problem
It seems that there are cases where the kotlin library is not supported depending on the spring boot version.

Resolution

  1. Upgrade the boot version to the latest one.

  2. Add the kotlin-stdlib dependency.

     <dependency>
     	<groupId>org.jetbrains.kotlin</groupId>
     	<artifactId>kotlin-stdlib</artifactId>
     	<version>1.7.20</version>
     </dependency>
    

Hi @LeeMH, I just wanted to write you the same. I'm glad that you could solve the problem yourselves. This bug seems to be related to Spring projects only, it's similar to #153. Adding the Kotlin standard library as a dependency solves it.