Ambiguous characters map to full-width characters in East Asian locale

[Paalon]

Description of the problem or steps to reproduce

The following script

let Δt = 0
let α = 0

echo Δt, α

results

Greek characters occupy fullwidth but should be halfwidth.

Specifications

Version: 1.4.0
Commit hash: af520cf
Compiled on January 26, 2018

OS: macOS
Terminal: iTerm2

[zyedidia]

This is not happening for me even with the same setup (macOS, iTerm2):

I'm not quite sure what the issue could be. What filetype is micro detecting?

[Paalon]

That was .nim file (Nim programming language code) but also in other filetype, I can see the behavior.

[ei1chi]

It can be "Ambiguous Width Characters" related problems. See http://www.unicode.org/reports/tr11/tr11-33.html#Ambiguous . Greek characters are a kind of ambiguous width characters.
Package github.com/mattn/go-runewidth decides whether ambiguous width is half or full by IsEastAsian(). So there are differences between environments.
I'm not sure about solutions, but for example, vim has an "ambiwidth" option that fixes it to full width or half width.
( Sorry for bad English. )

[Paalon]

How can I force to the non-east-Asian context?

[Paalon]

My environment's locale is:

$ locale
LANG="ja_JP.UTF-8"
LC_COLLATE="ja_JP.UTF-8"
LC_CTYPE="ja_JP.UTF-8"
LC_MESSAGES="ja_JP.UTF-8"
LC_MONETARY="ja_JP.UTF-8"
LC_NUMERIC="ja_JP.UTF-8"
LC_TIME="ja_JP.UTF-8"
LC_ALL=

Temporarily changing and setting locale variable LANG to "en_US.UTF-8"

# fish
set LANG "en_US.UTF-8"

resolves the problem. Reverting the locale to "ja_JP.UTF-8" or other East Asian locale like "zh_CN.UTF-8" reproduces the problem.

[dmaluka]

Just checked this in both vim and nano with LANG set to ja_JP.UTF-8 and the result is even worse than in micro.

So I'm wondering if this issue is possible/needed to fix at all?

[Andriamanitra]

Just checked this in both vim and nano with LANG set to ja_JP.UTF-8 and the result is even worse than in micro.

Weird, I only have the problem in micro and not in any other editor I tried (nvim, helix, flow, nano). Maybe it's a glibc vs musl thing, I know they differ in how they do localization (I'm using a musl-based variant of Void Linux).

[Andriamanitra]

I think there are two separate issues here:

1. The background color doesn't work properly for the double wide ambiguous width character (eg. Δ) in CJK locales
2. The ambiguous width characters are rendered 2-wide on CJK locales

The first issue is fixed by upgrading tcell (#4174). It's debatable if the second one is even a bug (rendering ambiguous characters double wide is a valid choice), but it might be preferable for us to set runewidth.DefaultCondition.EastAsianWidth = false to render ambiguous width characters consistently regardless of the locale. UAX #11 recommends that

Ambiguous characters behave like wide or narrow characters depending on the context (language tag, script identification, associated font, source of data, or explicit markup; all can provide the context). If the context cannot be established reliably, they should be treated as narrow characters by default.

diff --git a/internal/util/util.go b/internal/util/util.go
index 9838b025..dda31cb8 100644
--- a/internal/util/util.go
+++ b/internal/util/util.go
@@ -90,6 +90,7 @@ func init() {
 		fmt.Println("Invalid version: ", Version, err)
 	}
 
+	runewidth.DefaultCondition.EastAsianWidth = false
 	Stdout = new(bytes.Buffer)
 }