Where to find documentation for XeTeX's line breaking behavior?

Question

I am trying to find out more about how XeTeX implements line breaking for different languages using \XeTeXlinebreaklocale. From the XeTeX reference guide:

\XeTeXlinebreaklocale⟨Locale ID⟩: Defines how to break lines for multilingual text.

That's all the information I could find about it, leaving me with several questions:

1. What are acceptable values for ⟨Locale ID⟩? In this question the OP arrives at the conclusion that ISO 639-1 language codes are used, but where can I find out which one of these are actually implemented and lead to changes in XeTeX's behavior?
2. Where in my TeX Live system would I find a file containing the line breaking information specific to each locale? It seems like it must be baked in into XeTeX the engine directly as it works even if I only load what xetex.ini loads, which as far as I can see does not include or \input any file containing such information.
3. Is there any way to find documentation on how exactly the line breaking algorithm changes when a \XeTeXlinebreaklocale command is issued?

Now let me add that I do know that certain XeLaTeX packages handle multilingual line breaking for me, but this is not what I am looking for; I am interested in getting to know what XeTeX does exactly.

Answer 1

well in xetex.web you end up here

procedure do_locale_linebreaks(s: integer; len: integer);
var
  offs, prevOffs, i: integer;
  use_penalty, use_skip: boolean;
begin
  if (XeTeX_linebreak_locale = 0) or (len = 1) then begin
    link(tail):=new_native_word_node(main_f, len);
    tail:=link(tail);
    for i:=0 to len - 1 do
      set_native_char(tail, i, native_text[s + i]);
    set_native_metrics(tail, XeTeX_use_glyph_metrics);
  end else begin
    use_skip:=XeTeX_linebreak_skip <> zero_glue;
    use_penalty:=XeTeX_linebreak_penalty <> 0 or not use_skip;
    linebreak_start(main_f, XeTeX_linebreak_locale, native_text + s, len);
    offs:=0;
    repeat
      prevOffs:=offs;
      offs:=linebreak_next;
      if offs > 0 then begin
        if prevOffs <> 0 then begin
          if use_penalty then
            tail_append(new_penalty(XeTeX_linebreak_penalty));
          if use_skip then
            tail_append(new_param_glue(XeTeX_linebreak_skip_code));
        end;
        link(tail):=new_native_word_node(main_f, offs - prevOffs);
        tail:=link(tail);
        for i:=prevOffs to offs - 1 do
          set_native_char(tail, i - prevOffs, native_text[s + i]);
        set_native_metrics(tail, XeTeX_use_glyph_metrics);
      end;
    until offs < 0;
  end
end;

so the locale ends up being passed to

linebreak_start(main_f, XeTeX_linebreak_locale, native_text + s, len);

As Marcel points out in comments below this function is defined in xetex_ext.c where it is called linebreakstart (general C-web name mapping rules)

linebreakstart(int f, integer localeStrNum, uint16_t* text, integer textLength)
...

Which in turn basically calls the ICU library ubrk_open function which implements the locale-specific linebreaking from UAX #14 with the locale data available in the ICU tree. Actual changes occur for all locales for which the file there contains a line:process line like zh

see also

http://userguide.icu-project.org/boundaryanalysis