Unicode support
dlauc opened this issue · comments
Fetching of annotations breaks on non-ASCII chars, I suppose UTF-8 encoding is missing somewhere.
Error log:`Jun 11, 2016 6:18:32 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [Search] in context with path [/anot] threw exception
java.io.CharConversionException: Not an ISO 8859-1 character: Ć
at javax.servlet.ServletOutputStream.print(ServletOutputStream.java:76)
at javax.servlet.ServletOutputStream.println(ServletOutputStream.java:186)
at uk.org.llgc.annotation.store.Search.doGet(Search.java:41)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at uk.org.llgc.annotation.store.filters.CorsFilter.doFilter(CorsFilter.java:19)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
`
Thanks will investigate. I hadn't tested UTF-8 characters but obviously they are causing a few issues. Thanks for reporting.
Glen
I wasn't able to re-create the UTF-8 error. When I added a UTF-8 character in mirador it HTML encoded it rather than UTF-8. I suspect the error your getting was because I was writing to an Servlet output stream which doesn't do UTF-8 conversion. I've changed all the outputs to use the Servlet writer which does encode things correctly as UTF-8.
I would appreciate if you could test this fix and let me know if it works with UTF-8 characters.
Cheers
Glen
Closing as no further reports but feel free to re-open if you are still having issues.