Skip to content

Commit 4065fc1

Browse files
ramalhoramalho
committed
tofus anotados ou removidos
1 parent 68195ff commit 4065fc1

14 files changed

Lines changed: 358 additions & 22 deletions

‎glifos/README.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Tofu, ou glifos fantando
2+
3+
Glifos são as figuras dos caracteres. Na falta de um glifo,
4+
o software usado para exibir o texto mostra um caractere alternativo,
5+
geralmente um retângulo branco apelidado de "tofu".
6+
7+
Nem todos os caracteres que usei no *Python Fluente Segunda Edição* têm
8+
os glifos necessários nas fontes usadas por padrão pelo `asciidoctor-pdf`,
9+
então o PDF exibe vários tofu, e o PDF foi a base do livro impresso.
10+
11+
O problema não acontece nos navegadores exibindo o livro em HTML:
12+
https://pythonfluente.com.
13+
14+
## Diagnosticar o problema
15+
16+
Para diagnosticar o problema, escrevi o script `list_symbols.py` que lê
17+
`stdin` ou arquivos de uma lista de argumentos na linha de comando,
18+
e gera no `stdout` um arquivo `.adoc` com cada um dos caracteres
19+
não-ASCII que aparecem nos arquivos de entrada, bem como sua contagem.
20+
21+
## Solução
22+
23+
### Plano A
24+
25+
O ideal seria configurar o `asciidoctor-pdf` para usar o conceito
26+
de *fallback font*: uma ou mais fonte alternativas onde encontrar
27+
os glifos faltando.
28+
Em tese, todos os glifos necessários existem na coleção de fontes Noto.
29+
30+
Mas a fonte Noto usada pelo `asciidoctor-pdf` não tem todos os glifos,
31+
e minhas tentativas de usar fallback fracassaram.
32+
33+
### Plano B
34+
35+
Eu poderia editar a fonte Noto que vem com o `asciidoctor-pdf` para
36+
incluir algumas dúzias de glifos necessários para o *Python Fluente*,
37+
mas a documentação do Asciidoctor alerta que há vários truques necessários
38+
para que uma fonte moderna como a Noto funcionem com a biblioteca
39+
`prawn` em Ruby, usada pelo `asciidoctor-pdf`.
40+
41+
Como não sou especialista em fontes, esse plano ficou inviável.
42+
43+
### Plano C
44+
45+
Descobri que a maioria dos tofus estão no capítulo 4 cujo tema é Unicode.
46+
Alguns símbolos de operações de conjunto aparecem no capítulo 3, sobre
47+
dicts e sets.
48+
49+
Com o prazo acabando, removi os caracteres que apareciam como tofu
50+
no capítulo 2, e coloquei notas explicando o problema no capítulo 4.

‎glifos/find-tofus.ipynb‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "code",
5+
"execution_count": 3,
6+
"id": "9fd118ac-e880-43ab-98c8-89ca369530e1",
7+
"metadata": {},
8+
"outputs": [
9+
{
10+
"name": "stdout",
11+
"output_type": "stream",
12+
"text": [
13+
"\"Character encoding and Unicode in Python: How to (╯°□°)╯︵ ┻━┻ with dignity\"\n",
14+
"\n",
15+
"(_Codificação de caracteres e o Unicode no Python: como (╯°□°)╯︵ ┻━┻ com dignidade_)\n",
16+
"\n"
17+
]
18+
}
19+
],
20+
"source": [
21+
"tofus = [\n",
22+
" '\\N{PRESENTATION FORM FOR VERTICAL LEFT PARENTHESIS}',\n",
23+
" '\\N{BOX DRAWINGS HEAVY HORIZONTAL}',\n",
24+
" '\\N{BOX DRAWINGS LIGHT ARC UP AND LEFT}',\n",
25+
" '\\N{BOX DRAWINGS HEAVY UP AND HORIZONTAL}',\n",
26+
"]\n",
27+
"\n",
28+
"def find_tofus():\n",
29+
" with open('../vol1/cap04.adoc') as fp:\n",
30+
" lines = fp.readlines()\n",
31+
"\n",
32+
" found = set()\n",
33+
" for line in lines:\n",
34+
" for tofu in tofus:\n",
35+
" if tofu in line and line not in found:\n",
36+
" print(line)\n",
37+
" found.add(line)\n",
38+
"\n",
39+
"find_tofus()\n",
40+
" "
41+
]
42+
},
43+
{
44+
"cell_type": "code",
45+
"execution_count": null,
46+
"id": "9083fc4f-711f-4f80-91c4-75602dcb7e68",
47+
"metadata": {},
48+
"outputs": [],
49+
"source": []
50+
}
51+
],
52+
"metadata": {
53+
"kernelspec": {
54+
"display_name": "Python 3 (ipykernel)",
55+
"language": "python",
56+
"name": "python3"
57+
},
58+
"language_info": {
59+
"codemirror_mode": {
60+
"name": "ipython",
61+
"version": 3
62+
},
63+
"file_extension": ".py",
64+
"mimetype": "text/x-python",
65+
"name": "python",
66+
"nbconvert_exporter": "python",
67+
"pygments_lexer": "ipython3",
68+
"version": "3.14.0rc1"
69+
}
70+
},
71+
"nbformat": 4,
72+
"nbformat_minor": 5
73+
}

‎glifos/list_symbols.py‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,7 @@ def compact_display(characters, row_width):
6161
print('|'+cell, end='')
6262
print()
6363

64-
def detail_display(counter, key=None):
64+
def detail_display(counter, sortkey=None):
6565
uchars: list[UniChar] = []
6666

6767
for char, count in ((c, n) for c, n in counter.items()
@@ -70,8 +70,8 @@ def detail_display(counter, key=None):
7070
categ = unicodedata.category(char)
7171
uchars.append(UniChar(char, name, categ, count))
7272

73-
if key is not None:
74-
uchars.sort(key=key)
73+
if sortkey is not None:
74+
uchars.sort(key=sortkey)
7575

7676
print('[cols=">2,^1,11,1,>1"]')
7777
print('|====')
@@ -98,13 +98,13 @@ def main():
9898
# compact_display(used.keys(), num_cols)
9999
# print('|====')
100100

101-
detail_display(used, key=lambda uc: -uc.count)
101+
detail_display(used, sortkey=lambda uc: -uc.count)
102102

103103
print('\n## Other')
104104

105105
detail_display({c : n for c, n in non_ascii.items()
106106
if (ord(c) >= 256) and c not in cp1252},
107-
key=attrgetter('categ', 'count'))
107+
sortkey=attrgetter('categ', 'count'))
108108

109109

110110
if __name__ == '__main__':

‎glifos/tofu-cap04.adoc‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
Generated 19:03:44
2+
3+
## CP1252
4+
5+
[cols=">2,^1,11,1,>1"]
6+
|====
7+
|`U+fffd`|�|REPLACEMENT CHARACTER|So|2
8+
|====
9+
10+
## Other
11+
[cols=">2,^1,11,1,>1"]
12+
|====
13+
|`U+1F006`|🀆|MAHJONG TILE WHITE DRAGON|??|0
14+
|`U+6587`|文|CJK UNIFIED IDEOGRAPH-6587|Lo|2
15+
|`U+5b57`|字|CJK UNIFIED IDEOGRAPH-5B57|Lo|2
16+
|`U+5316`|化|CJK UNIFIED IDEOGRAPH-5316|Lo|2
17+
|`U+3051`|け|HIRAGANA LETTER KE|Lo|2
18+
|`U+32b7`|㊷|CIRCLED NUMBER FORTY TWO|No|1
19+
|`U+2265`|≥|GREATER-THAN OR EQUAL TO|Sm|13
20+
|`U+fe35`|︵|PRESENTATION FORM FOR VERTICAL LEFT PARENTHESIS|Ps|2
21+
|`U+2044`|⁄|FRACTION SLASH|Sm|4
22+
|`U+2501`|━|BOX DRAWINGS HEAVY HORIZONTAL|So|2
23+
|`U+256f`|╯|BOX DRAWINGS LIGHT ARC UP AND LEFT|So|4
24+
|`U+253b`|┻|BOX DRAWINGS HEAVY UP AND HORIZONTAL|So|4
25+
|====

‎glifos/tofu-cap04.pdf‎

67.5 KB
Binary file not shown.

‎glifos/vol1-cap01-08-SEM-cap04.adoc‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
Generated 19:06:44
2+
3+
## Latin 1
4+
5+
|====
6+
|é|í|ó|«|»|á|ê|ô|É|ç|ã|õ|à|ú|â|Á|¢|£|¥|¤|²|×|§|µ|¶|©|°|÷|±|¬|®|Ã
7+
|ª|Ô|ý|ü|À|Ú| | | | | | | | | | | | | | | | | | | | | | | | | |
8+
|====
9+
10+
## CP1252
11+
12+
[cols=">2,^1,11,1,>1"]
13+
|====
14+
|`U+2014`|—|EM DASH|Pd|273
15+
|`U+2026`|…|HORIZONTAL ELLIPSIS|Po|44
16+
|`U+201c`|“|LEFT DOUBLE QUOTATION MARK|Pi|17
17+
|`U+201d`|”|RIGHT DOUBLE QUOTATION MARK|Pf|16
18+
|`U+20ac`|€|EURO SIGN|Sc|4
19+
|`U+2019`|’|RIGHT SINGLE QUOTATION MARK|Pf|4
20+
|`U+2018`|‘|LEFT SINGLE QUOTATION MARK|Pi|3
21+
|`U+2013`|–|EN DASH|Pd|2
22+
|`U+0160`|Š|LATIN CAPITAL LETTER S WITH CARON|Lu|1
23+
|====
24+
25+
## Other
26+
[cols=">2,^1,11,1,>1"]
27+
|====
28+
|`U+0141`|Ł|LATIN CAPITAL LETTER L WITH STROKE|Lu|1
29+
|`U+2264`|≤|LESS-THAN OR EQUAL TO|Sm|1
30+
|`U+2265`|≥|GREATER-THAN OR EQUAL TO|Sm|13
31+
|`U+25cf`|●|BLACK CIRCLE|So|249
32+
|====

‎glifos/vol1-cap01-08-SEM-cap04.pdf‎

103 KB
Binary file not shown.

‎glifos/vol1-cap01-08.adoc‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
Generated 19:03:44
2+
3+
## Latin 1
4+
5+
|====
6+
|é|í|ó|«|»|á|ê|ô|É|ç|ã|õ|à|ú|â|Á|¢|£|¥|¤|²|×|§|µ|¶|©|°|÷|±|¬|®|ñ
7+
|³|Ã|½|ß|è|ü|ý|Ä|Ú|ª|Ô|À| | | | | | | | | | | | | | | | | | | |
8+
|====
9+
10+
## CP1252
11+
12+
[cols=">2,^1,11,1,>1"]
13+
|====
14+
|`U+2014`|—|EM DASH|Pd|324
15+
|`U+2026`|…|HORIZONTAL ELLIPSIS|Po|47
16+
|`U+201c`|“|LEFT DOUBLE QUOTATION MARK|Pi|20
17+
|`U+201d`|”|RIGHT DOUBLE QUOTATION MARK|Pf|19
18+
|`U+2022`|•|BULLET|Po|6
19+
|`U+20ac`|€|EURO SIGN|Sc|5
20+
|`U+2019`|’|RIGHT SINGLE QUOTATION MARK|Pf|5
21+
|`U+2122`|™|TRADE MARK SIGN|So|4
22+
|`U+0152`|Œ|LATIN CAPITAL LIGATURE OE|Lu|3
23+
|`U+2018`|‘|LEFT SINGLE QUOTATION MARK|Pi|3
24+
|`U+2013`|–|EN DASH|Pd|2
25+
|`U+fffd`|�|REPLACEMENT CHARACTER|So|2
26+
|`U+0160`|Š|LATIN CAPITAL LETTER S WITH CARON|Lu|2
27+
|====
28+
29+
## Other
30+
[cols=">2,^1,11,1,>1"]
31+
|====
32+
|`U+03b9`|ι|GREEK SMALL LETTER IOTA|Ll|1
33+
|`U+0131`|ı|LATIN SMALL LETTER DOTLESS I|Ll|1
34+
|`U+03ad`|έ|GREEK SMALL LETTER EPSILON WITH TONOS|Ll|2
35+
|`U+03c6`|φ|GREEK SMALL LETTER PHI|Ll|2
36+
|`U+03c5`|υ|GREEK SMALL LETTER UPSILON|Ll|2
37+
|`U+03c1`|ρ|GREEK SMALL LETTER RHO|Ll|2
38+
|`U+03bf`|ο|GREEK SMALL LETTER OMICRON|Ll|2
39+
|`U+03c2`|ς|GREEK SMALL LETTER FINAL SIGMA|Ll|2
40+
|`U+03c0`|π|GREEK SMALL LETTER PI|Ll|2
41+
|`U+03bc`|μ|GREEK SMALL LETTER MU|Ll|3
42+
|`U+03b5`|ε|GREEK SMALL LETTER EPSILON|Ll|3
43+
|`U+6587`|文|CJK UNIFIED IDEOGRAPH-6587|Lo|2
44+
|`U+5b57`|字|CJK UNIFIED IDEOGRAPH-5B57|Lo|2
45+
|`U+5316`|化|CJK UNIFIED IDEOGRAPH-5316|Lo|2
46+
|`U+3051`|け|HIRAGANA LETTER KE|Lo|2
47+
|`U+03a9`|Ω|GREEK CAPITAL LETTER OMEGA|Lu|1
48+
|`U+0130`|İ|LATIN CAPITAL LETTER I WITH DOT ABOVE|Lu|1
49+
|`U+0141`|Ł|LATIN CAPITAL LETTER L WITH STROKE|Lu|1
50+
|`U+0418`|И|CYRILLIC CAPITAL LETTER I|Lu|2
51+
|`U+0396`|Ζ|GREEK CAPITAL LETTER ZETA|Lu|2
52+
|`U+32b7`|㊷|CIRCLED NUMBER FORTY TWO|No|1
53+
|`U+fe35`|︵|PRESENTATION FORM FOR VERTICAL LEFT PARENTHESIS|Ps|2
54+
|`U+2264`|≤|LESS-THAN OR EQUAL TO|Sm|1
55+
|`U+221e`|∞|INFINITY|Sm|1
56+
|`U+2044`|⁄|FRACTION SLASH|Sm|4
57+
|`U+2265`|≥|GREATER-THAN OR EQUAL TO|Sm|13
58+
|`U+25a1`|□|WHITE SQUARE|So|2
59+
|`U+2501`|━|BOX DRAWINGS HEAVY HORIZONTAL|So|2
60+
|`U+256f`|╯|BOX DRAWINGS LIGHT ARC UP AND LEFT|So|4
61+
|`U+253b`|┻|BOX DRAWINGS HEAVY UP AND HORIZONTAL|So|4
62+
|`U+25cf`|●|BLACK CIRCLE|So|249
63+
|====

‎glifos/vol1-cap01-08.pdf‎

189 KB
Binary file not shown.

‎glifos/vol1-cap04.adoc‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
Generated 19:21:30
2+
3+
## Latin 1
4+
5+
|====
6+
|ç|ã|«|»|í|ê|á|õ|é|ô|ó|ú|É|â|ñ|³|Ã|©|à|µ|½|²|ß|è|ü|ý|Ä|°|Ú| | |
7+
|====
8+
9+
## CP1252
10+
11+
[cols=">2,^1,11,1,>1"]
12+
|====
13+
|`U+2014`|—|EM DASH|Pd|51
14+
|`U+2022`|•|BULLET|Po|6
15+
|`U+2122`|™|TRADE MARK SIGN|So|4
16+
|`U+2026`|…|HORIZONTAL ELLIPSIS|Po|3
17+
|`U+201c`|“|LEFT DOUBLE QUOTATION MARK|Pi|3
18+
|`U+0152`|Œ|LATIN CAPITAL LIGATURE OE|Lu|3
19+
|`U+201d`|”|RIGHT DOUBLE QUOTATION MARK|Pf|3
20+
|`U+fffd`|�|REPLACEMENT CHARACTER|So|2
21+
|`U+20ac`|€|EURO SIGN|Sc|1
22+
|`U+0160`|Š|LATIN CAPITAL LETTER S WITH CARON|Lu|1
23+
|`U+2019`|’|RIGHT SINGLE QUOTATION MARK|Pf|1
24+
|====
25+
26+
## Other
27+
[cols=">2,^1,11,1,>1"]
28+
|====
29+
|`U+03b9`|ι|GREEK SMALL LETTER IOTA|Ll|1
30+
|`U+0131`|ı|LATIN SMALL LETTER DOTLESS I|Ll|1
31+
|`U+03ad`|έ|GREEK SMALL LETTER EPSILON WITH TONOS|Ll|2
32+
|`U+03c6`|φ|GREEK SMALL LETTER PHI|Ll|2
33+
|`U+03c5`|υ|GREEK SMALL LETTER UPSILON|Ll|2
34+
|`U+03c1`|ρ|GREEK SMALL LETTER RHO|Ll|2
35+
|`U+03bf`|ο|GREEK SMALL LETTER OMICRON|Ll|2
36+
|`U+03c2`|ς|GREEK SMALL LETTER FINAL SIGMA|Ll|2
37+
|`U+03c0`|π|GREEK SMALL LETTER PI|Ll|2
38+
|`U+03bc`|μ|GREEK SMALL LETTER MU|Ll|3
39+
|`U+03b5`|ε|GREEK SMALL LETTER EPSILON|Ll|3
40+
|`U+6587`|文|CJK UNIFIED IDEOGRAPH-6587|Lo|2
41+
|`U+5b57`|字|CJK UNIFIED IDEOGRAPH-5B57|Lo|2
42+
|`U+5316`|化|CJK UNIFIED IDEOGRAPH-5316|Lo|2
43+
|`U+3051`|け|HIRAGANA LETTER KE|Lo|2
44+
|`U+03a9`|Ω|GREEK CAPITAL LETTER OMEGA|Lu|1
45+
|`U+0130`|İ|LATIN CAPITAL LETTER I WITH DOT ABOVE|Lu|1
46+
|`U+0418`|И|CYRILLIC CAPITAL LETTER I|Lu|2
47+
|`U+0396`|Ζ|GREEK CAPITAL LETTER ZETA|Lu|2
48+
|`U+32b7`|㊷|CIRCLED NUMBER FORTY TWO|No|1
49+
|`U+fe35`|︵|PRESENTATION FORM FOR VERTICAL LEFT PARENTHESIS|Ps|2
50+
|`U+221e`|∞|INFINITY|Sm|1
51+
|`U+2044`|⁄|FRACTION SLASH|Sm|4
52+
|`U+25a1`|□|WHITE SQUARE|So|2
53+
|`U+2501`|━|BOX DRAWINGS HEAVY HORIZONTAL|So|2
54+
|`U+256f`|╯|BOX DRAWINGS LIGHT ARC UP AND LEFT|So|4
55+
|`U+253b`|┻|BOX DRAWINGS HEAVY UP AND HORIZONTAL|So|4
56+
|====

0 commit comments

Comments
 (0)