[BUG] Some character converted to diamond shaped question mark
HanpoomAnj opened this issue · comments
Describe the bug
Issue with the Korean character converting to a diamond symbol. It only happens on pages with very long text content such as terms of use etc. The issue doesn't happen on specific text consistently, it will happen from a different text.
Expected behaviour
Korean text should be rendered correctly and should not convert to a diamond question mark symbol
Screenshots
Additional context
Note that we already apply either of these two in our index.html file
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta charset=UTF-8">
Also, we cannot replicate it in our local branch
I would start by correcting the html syntax, the attribute and value delimiters are not being declared correctly, avoiding duplicate declarations the character set, and testing with a complete html header. The server header is also able to handle language related metadata declarations.
<!DOCTYPE html>
<html lang="ko">
<title>Example Header Korean Language</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<body>
<p dir="ltr">이 HTML 문서 언어 속성은 한국어입니다.</p>
<p dir="ltr" lang="en">This is a different example of if
you wanted to switch languages inside the document</p>
</body>
</html>
Seems to be working for me, if this does not help feel free to post the characters not showing up in the thread here and I'll see if I can figure out what is happening
The editor and debugging software you are using might not be configured for your language, I would also start by telling us what software you are using and then we can see if the characters are not rendering due to software configurations
Hi @RaymondCrandall
thanks for your help appreciate it! This is still an issue on our site...
Reply to your 1st comment:
This is what we currently have on our header.
<html lang="ko"><head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
_<more code here>_
</head>
We've also tried this one <meta charset=UTF-8"> instead of <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> but that didn't fix.
Reply to your 2nd comment:
Software: WebStorm by JetBrains
Software file encoding: UTF8
I want to add, I saw a ticket with a similar issue: https://github.com/Shopify/hydrogen/issues/2302
Do you think it might be related to this?
I'm not familiar with this problem, so we may have to try different things to narrow down what the issue is.
Does Korean show up in your browser on most websites without any issues? If so, assuming your http encoding headers are correctly formatted, find the character(s)/symbols(s) which fail to render properly, and use an analysis tool to see if they are supported, and maybe we will find an answer there.
I would definitely run the html syntax through a markup validator tool to make sure it's not just a typing error
the ticket with a similar issue (link above) goes to a no results found page...so I looked up the issue by its ID
It looks like you made a mistake in your markdown formatting.
Here is the corrected link:
https://github.com/Shopify/hydrogen/issues/2302
I also found this which may be worth checking out:
https://github.com/Shopify/hydrogen/pull/2351
I can look into seeing what is being talked about in these links later today, I'm working on a few things currently
hope this helps
@RaymondCrandall Yes Korean characters are showing correctly in my browser on all Korean websites I visited. This is our first time to encounter an issue after switching from Woocommerce to Shopify hydrogen.
@HanpoomAnj did you run the generated html response through a markup validator?
@RaymondCrandall here's the result of the markup validator
@RaymondCrandall Let me know if you have check on this again, thank you!! 🙏
It occurs in Japanese long text too, only when production deploy (at netlfy, vercel, cloudflare worker).
yarn dev works fine.
So, our team deploy by docker with yarn build.
Is there any update on this? It still happens on our site randomly. @RaymondCrandall