Greek circumflex accent with ascii characters using mapping does not work

Question

I want to input Greek letters with polytonic accents (ancient Greek) using ASCII characters in my tex document (so I don't need to change the keyboard layout). I am using polyglossia.

I have been following the method lined out in the top answer here:

Stackexchange: How to insert Greek with "ascii keyboard" and XeTeX, Polyglossia

I created a mapping file asciitogreek.map and put it in the same directory as the tex file:

; TECkit mapping for ASCII Greek <-> Unicode characters

LHSName "TeX-text"
RHSName "UNICODE"

pass(Unicode)

; Class "letter" consists of letters and characters denoting accents
; in order to cope with "sigma finalis", i.e., A-Z a-z ` ' > < ~
; we don't need the double quote because it goes only after a vowel
UniClass[letter] = ( U+0041..U+005A U+0061..U+007A U+0060 U+0027 U+003E U+003C U+007E )


; ligatures from Knuth's original CMR fonts
U+002D U+002D <> U+2013 ; -- -> en dash
U+002D U+002D U+002D <> U+2014 ; --- -> em dash

; Greek (according to C. Beccari conventions)

; (-1)
U+0022 <> U+2019 ; APOSTROPHE
U+0060 U+0060 <> U+201C ; `` -> LEFT DOUBLE QUOTATION MARK
U+0027 U+0027 <> U+201D ; '' -> RIGHT DOUBLE QUOTATION MARK
U+003C U+003C <> U+00AB ; << -> LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
U+003E U+003E <> U+00BB ; >> -> RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
U+0028 U+0028 <> U+00AB ; (( -> LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
U+0029 U+0029 <> U+00BB ; )) -> RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK

; (0) basic letters
; uppercase
U+0041 <> U+0391 
U+0042 <> U+0392 
U+0044 <> U+0394 
U+0045 <> U+0395 
U+0046 <> U+03A6 
U+0047 <> U+0393 
U+0048 <> U+0397 
U+0049 <> U+0399 
U+004A <> U+0398 
U+004B <> U+039A 
U+004C <> U+039B 
U+004D <> U+039C 
U+004E <> U+039D 
U+004F <> U+039F 
U+0050 <> U+03A0 
U+0051 <> U+03A7 
U+0052 <> U+03A1 
U+0053 <> U+03A3 
U+0054 <> U+03A4 
U+0055 <> U+03A5 
U+0057 <> U+03A9 
U+0058 <> U+039E 
U+0059 <> U+03A8 
U+005A <> U+0396 
; lowercase
U+0061 <> U+03B1 
U+0062 <> U+03B2 
U+0063 <> U+03C2 
U+0064 <> U+03B4 
U+0065 <> U+03B5 
U+0066 <> U+03C6 
U+0067 <> U+03B3 
U+0068 <> U+03B7 
U+0069 <> U+03B9 
U+006A <> U+03B8 
U+006B <> U+03BA 
U+006C <> U+03BB 
U+006D <> U+03BC 
U+006E <> U+03BD 
U+006F <> U+03BF 
U+0070 <> U+03C0 
U+0071 <> U+03C7 
U+0072 <> U+03C1 
U+0073 <> U+03C3 
U+0073 / _ ^[letter] <> U+03C2 
U+0074 <> U+03C4 
U+0075 <> U+03C5 
U+0077 <> U+03C9 
U+0078 <> U+03BE 
U+0079 <> U+03C8 
U+007A <> U+03B6 

; (1) ` + letter -> varia
U+0060 U+0041 <> U+1FBA 
U+0060 U+0061 <> U+1F70 
U+0060 U+0045 <> U+1FC8 
U+0060 U+0065 <> U+1F72 
U+0060 U+0048 <> U+1FCA 
U+0060 U+0068 <> U+1F74 
U+0060 U+0049 <> U+1FDA 
U+0060 U+0069 <> U+1F76 
U+0060 U+004F <> U+1FF8 
U+0060 U+006F <> U+1F78 
U+0060 U+0055 <> U+1FEA 
U+0060 U+0075 <> U+1F7A 
U+0060 U+0057 <> U+1FFA 
U+0060 U+0077 <> U+1F7C 

; (2) ' + letter -> oxia
U+0027 U+0041 <> U+1FBB 
U+0027 U+0061 <> U+1F71 
U+0027 U+0045 <> U+1FC9 
U+0027 U+0065 <> U+1F73 
U+0027 U+0048 <> U+1FCB 
U+0027 U+0068 <> U+1F75 
U+0027 U+0049 <> U+1FDB 
U+0027 U+0069 <> U+1F77 
U+0027 U+004F <> U+1FF9 
U+0027 U+006F <> U+1F79 
U+0027 U+0055 <> U+1FEB 
U+0027 U+0075 <> U+1F7B 
U+0027 U+0057 <> U+1FFB 
U+0027 U+0077 <> U+1F7D 

; (3) > + letter -> psili
U+003E U+0041 <> U+1F08 
U+003E U+0061 <> U+1F00 
U+003E U+0045 <> U+1F18 
U+003E U+0065 <> U+1F10 
U+003E U+0048 <> U+1F28 
U+003E U+0068 <> U+1F20 
U+003E U+0049 <> U+1F38 
U+003E U+0069 <> U+1F30 
U+003E U+004F <> U+1F48 
U+003E U+006F <> U+1F40 
U+003E U+0055 <> U+1FBF U+03A5 
U+003E U+0075 <> U+1F50 
U+003E U+0057 <> U+1F68 
U+003E U+0077 <> U+1F60 

; (4) < + letter -> dasia
U+003C U+0041 <> U+1F09 
U+003C U+0061 <> U+1F01 
U+003C U+0045 <> U+1F19 
U+003C U+0065 <> U+1F11 
U+003C U+0048 <> U+1F29 
U+003C U+0068 <> U+1F21 
U+003C U+0049 <> U+1F39 
U+003C U+0069 <> U+1F31 
U+003C U+004F <> U+1F49 
U+003C U+006F <> U+1F41 
U+003C U+0055 <> U+1F59 
U+003C U+0075 <> U+1F51 
U+003C U+0057 <> U+1F69 
U+003C U+0077 <> U+1F61 

; (5) >` + letter -> psili and varia
U+003E U+0060 U+0041 <> U+1F0A 
U+003E U+0060 U+0061 <> U+1F02 
U+003E U+0060 U+0045 <> U+1F1A 
U+003E U+0060 U+0065 <> U+1F12 
U+003E U+0060 U+0048 <> U+1F2A 
U+003E U+0060 U+0068 <> U+1F22 
U+003E U+0060 U+0049 <> U+1F3A 
U+003E U+0060 U+0069 <> U+1F32 
U+003E U+0060 U+004F <> U+1F4A 
U+003E U+0060 U+006F <> U+1F42 
U+003E U+0060 U+0055 <> U+1FCD U+03A5 
U+003E U+0060 U+0075 <> U+1F52 
U+003E U+0060 U+0057 <> U+1F6A 
U+003E U+0060 U+0077 <> U+1F62 

; (6) `> + letter -> psili and varia
U+0060 U+003E U+0041 <> U+1F0A 
U+0060 U+003E U+0061 <> U+1F02 
U+0060 U+003E U+0045 <> U+1F1A 
U+0060 U+003E U+0065 <> U+1F12 
U+0060 U+003E U+0048 <> U+1F2A 
U+0060 U+003E U+0068 <> U+1F22 
U+0060 U+003E U+0049 <> U+1F3A 
U+0060 U+003E U+0069 <> U+1F32 
U+0060 U+003E U+004F <> U+1F4A 
U+0060 U+003E U+006F <> U+1F42 
U+0060 U+003E U+0055 <> U+1FCD U+03A5 
U+0060 U+003E U+0075 <> U+1F52 
U+0060 U+003E U+0057 <> U+1F6A 
U+0060 U+003E U+0077 <> U+1F62 

; (7) >' + letter -> psili and oxia
U+003E U+0027 U+0041 <> U+1F0C 
U+003E U+0027 U+0061 <> U+1F04 
U+003E U+0027 U+0045 <> U+1F1C 
U+003E U+0027 U+0065 <> U+1F14 
U+003E U+0027 U+0048 <> U+1F2C 
U+003E U+0027 U+0068 <> U+1F24 
U+003E U+0027 U+0049 <> U+1F3C 
U+003E U+0027 U+0069 <> U+1F34 
U+003E U+0027 U+004F <> U+1F4C 
U+003E U+0027 U+006F <> U+1F44 
U+003E U+0027 U+0055 <> U+1FCE U+03A5 
U+003E U+0027 U+0075 <> U+1F54 
U+003E U+0027 U+0057 <> U+1F6C 
U+003E U+0027 U+0077 <> U+1F64 

; (8) '> + letter -> psili and oxia
U+0027 U+003E U+0041 <> U+1F0C 
U+0027 U+003E U+0061 <> U+1F04 
U+0027 U+003E U+0045 <> U+1F1C 
U+0027 U+003E U+0065 <> U+1F14 
U+0027 U+003E U+0048 <> U+1F2C 
U+0027 U+003E U+0068 <> U+1F24 
U+0027 U+003E U+0049 <> U+1F3C 
U+0027 U+003E U+0069 <> U+1F34 
U+0027 U+003E U+004F <> U+1F4C 
U+0027 U+003E U+006F <> U+1F44 
U+0027 U+003E U+0055 <> U+1FCE U+03A5 
U+0027 U+003E U+0075 <> U+1F54 
U+0027 U+003E U+0057 <> U+1F6C 
U+0027 U+003E U+0077 <> U+1F64 

; (9) <` + letter -> dasia and varia
U+003C U+0060 U+0041 <> U+1F0B 
U+003C U+0060 U+0061 <> U+1F03 
U+003C U+0060 U+0045 <> U+1F1B 
U+003C U+0060 U+0065 <> U+1F13 
U+003C U+0060 U+0048 <> U+1F2B 
U+003C U+0060 U+0068 <> U+1F23 
U+003C U+0060 U+0049 <> U+1F3B 
U+003C U+0060 U+0069 <> U+1F33 
U+003C U+0060 U+004F <> U+1F4B 
U+003C U+0060 U+006F <> U+1F43 
U+003C U+0060 U+0055 <> U+1F5B 
U+003C U+0060 U+0075 <> U+1F53 
U+003C U+0060 U+0057 <> U+1F6B 
U+003C U+0060 U+0077 <> U+1F63 

; (10) `< + letter -> dasia and varia
U+0060 U+003C U+0041 <> U+1F0B 
U+0060 U+003C U+0061 <> U+1F03 
U+0060 U+003C U+0045 <> U+1F1B 
U+0060 U+003C U+0065 <> U+1F13 
U+0060 U+003C U+0048 <> U+1F2B 
U+0060 U+003C U+0068 <> U+1F23 
U+0060 U+003C U+0049 <> U+1F3B 
U+0060 U+003C U+0069 <> U+1F33 
U+0060 U+003C U+004F <> U+1F4B 
U+0060 U+003C U+006F <> U+1F43 
U+0060 U+003C U+0055 <> U+1F5B 
U+0060 U+003C U+0075 <> U+1F53 
U+0060 U+003C U+0057 <> U+1F6B 
U+0060 U+003C U+0077 <> U+1F63 

; (11) ~ + letter -> perispomeni
U+007E U+0041 <> U+0391 U+0342 
U+007E U+0061 <> U+1FB6 
U+007E U+0048 <> U+0397 U+0342 
U+007E U+0068 <> U+1FC6 
U+007E U+0049 <> U+0399 U+0342 
U+007E U+0069 <> U+1FD6 
U+007E U+0055 <> U+03A5 U+0342 
U+007E U+0075 <> U+1FE6 
U+007E U+0057 <> U+03A9 U+0342 
U+007E U+0077 <> U+1FF6 

; (12) >~ + letter -> psili and perispomeni
U+003E U+007E U+0041 <> U+1F0E 
U+003E U+007E U+0061 <> U+1F06 
U+003E U+007E U+0048 <> U+1F2E 
U+003E U+007E U+0068 <> U+1F26 
U+003E U+007E U+0049 <> U+1F3E 
U+003E U+007E U+0069 <> U+1F36 
U+003E U+007E U+0055 <> U+1FCF U+03A5 
U+003E U+007E U+0075 <> U+1F56 
U+003E U+007E U+0057 <> U+1F6E 
U+003E U+007E U+0077 <> U+1F66 

; (13) ~> + letter -> psili and perispomeni
U+007E U+003E U+0041 <> U+1F0E 
U+007E U+003E U+0061 <> U+1F06 
U+007E U+003E U+0048 <> U+1F2E 
U+007E U+003E U+0068 <> U+1F26 
U+007E U+003E U+0049 <> U+1F3E 
U+007E U+003E U+0069 <> U+1F36 
U+007E U+003E U+0055 <> U+1FCF U+03A5 
U+007E U+003E U+0075 <> U+1F56 
U+007E U+003E U+0057 <> U+1F6E 
U+007E U+003E U+0077 <> U+1F66 

; (14) <~ + letter -> dasia and perispomeni
U+003C U+007E U+0041 <> U+1F0F 
U+003C U+007E U+0061 <> U+1F07 
U+003C U+007E U+0048 <> U+1F2F 
U+003C U+007E U+0068 <> U+1F27 
U+003C U+007E U+0049 <> U+1F3F 
U+003C U+007E U+0069 <> U+1F37 
U+003C U+007E U+0055 <> U+1F5F 
U+003C U+007E U+0075 <> U+1F57 
U+003C U+007E U+0057 <> U+1F6F 
U+003C U+007E U+0077 <> U+1F67 

; (15) ~< + letter -> dasia and perispomeni
U+007E U+003C U+0041 <> U+1F0F 
U+007E U+003C U+0061 <> U+1F07 
U+007E U+003C U+0048 <> U+1F2F 
U+007E U+003C U+0068 <> U+1F27 
U+007E U+003C U+0049 <> U+1F3F 
U+007E U+003C U+0069 <> U+1F37 
U+007E U+003C U+0055 <> U+1F5F 
U+007E U+003C U+0075 <> U+1F57 
U+007E U+003C U+0057 <> U+1F6F 
U+007E U+003C U+0077 <> U+1F67 

; (16) letter + | -> ypogegrammeni
U+0041 U+007C <> U+1FBC 
U+0061 U+007C <> U+1FB3 
U+0048 U+007C <> U+1FCC 
U+0068 U+007C <> U+1FC3 
U+0057 U+007C <> U+1FFC 
U+0077 U+007C <> U+1FF3 

; (17) ` + letter + | -> varia and ypogegrammeni
U+0060 U+0061 U+007C <> U+1FB2 
U+0060 U+0068 U+007C <> U+1FC2 
U+0060 U+0077 U+007C <> U+1FF2 

; (18) ' + letter + | -> oxia and ypogegrammeni
U+0027 U+0061 U+007C <> U+1FB2 
U+0027 U+0068 U+007C <> U+1FC2 
U+0027 U+0077 U+007C <> U+1FF2 

; (19) ~ + letter + | -> perispomeni and ypogegrammeni
U+007E U+0061 U+007C <> U+1FB7 
U+007E U+0068 U+007C <> U+1FC7 
U+007E U+0077 U+007C <> U+1FF7 

; (20) > + letter + | -> psili and ypogegrammeni
U+003E U+0041 U+007C <> U+1F88 
U+003E U+0061 U+007C <> U+1F80 
U+003E U+0048 U+007C <> U+1F98 
U+003E U+0068 U+007C <> U+1F90 
U+003E U+0057 U+007C <> U+1FA8 
U+003E U+0077 U+007C <> U+1FA0 

; (21) >` + letter + | -> psili and varia and ypogegrammeni
U+003E U+0060 U+0041 U+007C <> U+1F8A 
U+003E U+0060 U+0061 U+007C <> U+1F82 
U+003E U+0060 U+0048 U+007C <> U+1F9A 
U+003E U+0060 U+0068 U+007C <> U+1F92 
U+003E U+0060 U+0057 U+007C <> U+1FAA 
U+003E U+0060 U+0077 U+007C <> U+1FA2 

; (22) `> + letter + | -> psili and varia and ypogegrammeni
U+0060 U+003E U+0041 U+007C <> U+1F8A 
U+0060 U+003E U+0061 U+007C <> U+1F82 
U+0060 U+003E U+0048 U+007C <> U+1F9A 
U+0060 U+003E U+0068 U+007C <> U+1F92 
U+0060 U+003E U+0057 U+007C <> U+1FAA 
U+0060 U+003E U+0077 U+007C <> U+1FA2 

; (23) >' + letter + | -> psili and oxia and ypogegrammeni
U+003E U+0027 U+0041 U+007C <> U+1F8C 
U+003E U+0027 U+0061 U+007C <> U+1F84 
U+003E U+0027 U+0048 U+007C <> U+1F9C 
U+003E U+0027 U+0068 U+007C <> U+1F94 
U+003E U+0027 U+0057 U+007C <> U+1FAC 
U+003E U+0027 U+0077 U+007C <> U+1FA4 

; (24) '> + letter + | -> psili and oxia and ypogegrammeni
U+0027 U+003E U+0041 U+007C <> U+1F8C 
U+0027 U+003E U+0061 U+007C <> U+1F84 
U+0027 U+003E U+0048 U+007C <> U+1F9C 
U+0027 U+003E U+0068 U+007C <> U+1F94 
U+0027 U+003E U+0057 U+007C <> U+1FAC 
U+0027 U+003E U+0077 U+007C <> U+1FA4 

; (25) < + letter + | -> dasia and ypogegrammeni
U+003C U+0041 U+007C <> U+1F89 
U+003C U+0061 U+007C <> U+1F81 
U+003C U+0048 U+007C <> U+1F99 
U+003C U+0068 U+007C <> U+1F91 
U+003C U+0057 U+007C <> U+1FA9 
U+003C U+0077 U+007C <> U+1FA1 

; (26) <` + letter + | -> dasia and varia and ypogegrammeni
U+003C U+0060 U+0041 U+007C <> U+1F8B 
U+003C U+0060 U+0061 U+007C <> U+1F83 
U+003C U+0060 U+0048 U+007C <> U+1F9B 
U+003C U+0060 U+0068 U+007C <> U+1F93 
U+003C U+0060 U+0057 U+007C <> U+1FAB 
U+003C U+0060 U+0077 U+007C <> U+1FA3 

; (27) `< + letter + | -> dasia and varia and ypogegrammeni
U+0060 U+003C U+0041 U+007C <> U+1F8B 
U+0060 U+003C U+0061 U+007C <> U+1F83 
U+0060 U+003C U+0048 U+007C <> U+1F9B 
U+0060 U+003C U+0068 U+007C <> U+1F93 
U+0060 U+003C U+0057 U+007C <> U+1FAB 
U+0060 U+003C U+0077 U+007C <> U+1FA3 

; (28) <' + letter + | -> dasia and oxia and ypogegrammeni
U+003C U+0027 U+0041 U+007C <> U+1F8D 
U+003C U+0027 U+0061 U+007C <> U+1F85 
U+003C U+0027 U+0048 U+007C <> U+1F9D 
U+003C U+0027 U+0068 U+007C <> U+1F95 
U+003C U+0027 U+0057 U+007C <> U+1FAD 
U+003C U+0027 U+0077 U+007C <> U+1FA5 

; (29) '< + letter + | -> dasia and oxia and ypogegrammeni
U+0027 U+003C U+0041 U+007C <> U+1F8D 
U+0027 U+003C U+0061 U+007C <> U+1F85 
U+0027 U+003C U+0048 U+007C <> U+1F9D 
U+0027 U+003C U+0068 U+007C <> U+1F95 
U+0027 U+003C U+0057 U+007C <> U+1FAD 
U+0027 U+003C U+0077 U+007C <> U+1FA5 

; (30) " + letter -> dialytika
U+0022 U+0049 <> U+03AA 
U+0022 U+0069 <> U+03CA 
U+0022 U+0055 <> U+03AB 
U+0022 U+0075 <> U+03CB 

; (31) "' + letter -> dialytika and oxia
U+0022 U+0027 U+0069 <> U+1FD3 
U+0022 U+0027 U+0075 <> U+1FE3 

; (32) '" + letter -> dialytika and oxia
U+0027 U+0022 U+0069 <> U+1FD3 
U+0027 U+0022 U+0075 <> U+1FE3 

; (33) "` + letter -> dialytika and varia
U+0022 U+0060 U+0069 <> U+1FD2 
U+0022 U+0060 U+0075 <> U+1FE2 

; (34) `" + letter -> dialytika and varia
U+0060 U+0022 U+0069 <> U+1FD2 
U+0060 U+0022 U+0075 <> U+1FE2 

; (35) "~ + letter -> dialytika and perispomeni
U+0022 U+007E U+0069 <> U+1FD7 
U+0022 U+007E U+0075 <> U+1FE7 

; (36) ~" + letter -> dialytika and perispomeni
U+007E U+0022 U+0069 <> U+1FD7 
U+007E U+0022 U+0075 <> U+1FE7 

; (37) rho, medial sigma and punctuation
; >r
U+003E U+0072 <> U+1FE4 
; <R
U+003C U+0052 <> U+1FEC 
; <r
U+003C U+0072 <> U+1FE5 
; medial sigma (sv)
U+0073 U+0076 <> U+03C3 
; question mark
U+003F <> U+037E 
; semicolon
U+003B <> U+0387 

Then I ran the following command:

teckit_compile asciitogreek.map 

and created the minimal working sample tex file to show you the problem:

\documentclass{scrreprt}
%\usepackage[cm-default]{fontspec}
%\setmainfont{Liberation Serif}
%\setromanfont{Liberation Serif}
%\usepackage{xunicode}
%\usepackage{fixltx2e}
\usepackage{polyglossia}
\setdefaultlanguage[spelling=new,latesthyphen=true]{german}
\setotherlanguage[variant=ancient]{greek}
\newfontfamily{\greekfont}[Mapping=asciitogreek]{FreeSerif}
\newfontfamily{\greekfontsf}[Mapping=asciitogreek]{FreeSans}
\newfontfamily{\greekfonttt}[Mapping=asciitogreek]{FreeMono}
\newcommand{\gk}[1]{\textgreek[variant=ancient]{#1}}

\begin{document}
 \gk{d~wr>'a}

 \gk{δῶρα}
\end{document}

Compilation worked fine. However, the ~ before the w is not displayed in the document. Instead, there is a little gap. This is apparently not a font issue since entering the unicode key directly works, as the second example shows.

See this screenshot from the pdf:

However, I checked the mapping, and the unicode keys in there seem to be correct. Though probably something with the mapping must be wrong. Changing the Greek variant from "ancient" to "polytonic" does not help either.

In case it matters, I am using texlive 2013.20140215-1 on kubuntu 14.04 trusty (LTS).

Does anyone have a good idea?

Answer 1

You also need to change the category code of ~:

\newcommand{\gk}{\begingroup\catcode`~=12 \dogk}
\newcommand{\dogk}[1]{\textgreek[variant=ancient]{#1}\endgroup}

Another possibility is to exploit the fact that ~ is active and locally define it to a ~ with the correct category code.

\documentclass{scrreprt}
\usepackage{fontspec}
\usepackage{polyglossia}
\setdefaultlanguage[spelling=new,latesthyphen=true]{german}
\setotherlanguage[variant=ancient]{greek}
\newfontfamily{\greekfont}[Mapping=asciitogreek]{FreeSerif}
\newfontfamily{\greekfontsf}[Mapping=asciitogreek]{FreeSans}
\newfontfamily{\greekfonttt}[Mapping=asciitogreek]{FreeMono}

\begingroup\lccode`?=`~
\lowercase{\endgroup\def\gktilde{\def~{?}}}
\protected\def\gk{\begingroup\gktilde\dogk}
\newcommand{\dogk}[1]{\textgreek[variant=ancient]{#1}\endgroup}

\begin{document}
\tableofcontents
\section{Section \gk{d~wr>'a}}

 \gk{d~wr>'a}

 \gk{δῶρα}
\end{document}