0

I am truly desperate. I need accented characters in the MATLAB script. I wouldn't mind with comments, but accents are also in Figures, which does actually matter. Both .tex file and .m file are saved in UTF-8 encoding (both natively, as I configured MATLAB to use UTF-8 encoding from startup). Yet pdflatex still somehow thinks there are invalid byte sequences, which I in fact verified is false.

This is my preamble:

\documentclass[12pt,twoside,a4paper]{article}
\usepackage[unicode]{hyperref}
\usepackage[czech]{babel}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{float}
\usepackage{lmodern}
\usepackage{graphicx}
\usepackage{attachfile}
\usepackage{subcaption}
\usepackage{cleveref}
\usepackage[document]{ragged2e}

\usepackage{siunitx}
\sisetup{
    detect-mode,
    detect-family,
    detect-inline-family=math,
    group-separator={ },
    group-minimum-digits={3},
    output-decimal-marker={,}
}

%\usepackage{listings}
\usepackage{listingsutf8}
\usepackage{xcolor}

\definecolor{codegreen}{rgb}{0,0.6,0}
\definecolor{codegray}{rgb}{0.5,0.5,0.5}
\definecolor{codepurple}{rgb}{0.58,0,0.82}
\definecolor{backcolour}{rgb}{0.95,0.95,0.92}

\lstset{
    extendedchars=\true,                    % Use extended charset
    inputencoding=utf8,                     % Use input encoding UTF-8
    backgroundcolor=\color{white},          % choose the background color; you must add \usepackage{color} or \usepackage{xcolor}; should come as last argument
    basicstyle=\tt\scriptsize,              % the size of the fonts that are used for the code
    breakatwhitespace=false,                % sets if automatic breaks should only happen at whitespace
    breaklines=true,                        % sets automatic line breaking
    captionpos=b,                           % sets the caption-position to bottom
    commentstyle=\color{codegreen},         % comment style
    deletekeywords={...},                   % if you want to delete keywords from the given language
    escapeinside={\%*}{*)},                 % if you want to add LaTeX within your code
    extendedchars=true,                     % lets you use non-ASCII characters; for 8-bits encodings only, does not work with UTF-8
    firstnumber=1000,                       % start line enumeration with line 1000
    frame=single,                           % adds a frame around the code
    keepspaces=true,                        % keeps spaces in text, useful for keeping indentation of code (possibly needs columns=flexible)
    keywordstyle=\color{blue},              % keyword style
    language=MATLAB,                        % the language of the code
    morekeywords={*,...},                   % if you want to add more keywords to the set
    numbers=left,                           % where to put the line-numbers; possible values are (none, left, right)
    numbersep=5pt,                          % how far the line-numbers are from the code
    numberstyle=\tiny\color{codegray},      % the style that is used for the line-numbers
    rulecolor=\color{black},                % if not set, the frame-color may be changed on line-breaks within not-black text (e.g. comments (green here))
    showspaces=false,                       % show spaces everywhere adding particular underscores; it overrides 'showstringspaces'
    showstringspaces=false,                 % underline spaces within strings only
    showtabs=false,                         % show tabs within strings adding particular underscores
    stepnumber=2,                           % the step between two line-numbers. If it's 1, each line will be numbered
    stringstyle=\color{codepurple},         % string literal style
    tabsize=2,                              % sets default tabsize to 2 spaces
    caption=\texttt{\protect\filename@parse{\lstname}\protect\filename@base.\protect\filename@ext}
                                            % show the filename of files included with \lstinputlisting; also try caption instead of title
}

\captionsetup[subfigure]{subrefformat=simple,labelformat=simple}
\renewcommand\thesubfigure{(\alph{subfigure})}

I am trying to import the MATLAB script file contents with this line:

\lstinputlisting{files/source/cast1.m}

This is what my LaTeX compiler says about the MATLAB script:

Package inputenc: Invalid UTF-8 byte sequence.
Package inputenc: Invalid UTF-8 byte "8C.
Package inputenc: Invalid UTF-8 byte sequence.
Package inputenc: Invalid UTF-8 byte sequence.
Package inputenc: Invalid UTF-8 byte "8D.
Package inputenc: Invalid UTF-8 byte sequence.
Package inputenc: Invalid UTF-8 byte "A1.
Package inputenc: Invalid UTF-8 byte sequence.
Package inputenc: Invalid UTF-8 byte "9B.
Package inputenc: Invalid UTF-8 byte sequence.
Package inputenc: Invalid UTF-8 byte "AD.
Package inputenc: Invalid UTF-8 byte sequence.
Package inputenc: Invalid UTF-8 byte "AD.
...

Screenshot of the actual error messages: MATLAB seems to have encoding error

However file encoding shows no UTF-8 errors in validation:

$ iconv -f UTF-8 files/source/cast1.m -o /dev/null; echo $?
0

And this is contents of the MATLAB script:

%% Projekt: POL0423 [Část I]
clc; close all; clear all;          % Vyčištění pracovní plochy

%% 01: Část I.A - Zadané parametry

% Průřezy
A_0 = 15.9;                         % [m^2]             průřez nádoby
A_2 = 2.595;                        % [m^2]             průřez odtoku

% Koeficienty přítoku
Q_A = 15;                           % [m^3 * s^(-1)]    přítok poč.
Q_B = 29;                           % [m^3 * s^(-1)]    přítok mod.
Q_C = 11.5;                         % [m^3 * s^(-1)]    přítok ust.

% Časové koficienty
t_A = 36;                           % [s]               čas skok. změny
t_B = 81;                           % [s]               čas snižování
t_C = 176;                          % [s]               čas ustálení

% Fyzikální konstanty
g = 9.81;                           % [m * s^(-2)]      tíhové zrychlení

% Diferenciální rovnice
% A_0 * dh(t)/dt = Q_1(t) - Q_2(t) = Q_1(t) - A_2 * sqrt(2gh(t))

%% 02: Část I.B - Sestavení modelu
modelB = sim('cast1_model_B');          % Simulink: simulace modelu

% Načtení dat a uložení do samostatných proměnných

% Vstupní proměnná
Scope_Q1_tB = modelB.Scope_Q1_modelB.time;              % Čas
Scope_Q1_wB = modelB.Scope_Q1_modelB.signals.values;    % Přítok

% Výstupní proměnná
Scope_h_tB = modelB.Scope_h_modelB.time;                % Čas
Scope_h_wB = modelB.Scope_h_modelB.signals.values;      % Hladina

%% 03: Část I.C - Linearizace modelu
u_0 = Q_C                           % Ustálený přítok
h_0 = (Q_C / A_2)^2 * 1/(2*g)       % Ustálená hladina

% Matice linearizovaného modelu
A = [ (-sqrt(2) * A_2 * sqrt(g))/(2 * A_0 * sqrt(h_0)) ]
B = [ 1/A_0 ]
C = [ 1 ]
D = [ 0 ]

[lin_nom,lin_den] = ss2tf(A,B,C,D);     % Převod do vnějšího popisu
G_lin = tf(lin_nom,lin_den);

modelC = sim('cast1_model_C');          % Simulink: simulace modelu

% Výstupní proměnná
Scope_h_tC = modelC.Scope_h_modelC.time;                % Čas
Scope_h_wC = modelC.Scope_h_modelC.signals.values;      % Hladina

% Vykreslení grafů
figure
plot(Scope_Q1_tB, Scope_Q1_wB)
grid on
range=ylim();
ylim([0 range(2)])
title('Přítok Q_1(t)')
xlabel('$t$ [s]','interpreter','latex')
ylabel('$Q_1(t)$ [$\mathrm{m}^3 \cdot \mathrm{s}^{-1}$]',...
    'interpreter','latex')

figure
plot(Scope_h_tB, Scope_h_wB, Scope_h_tC, Scope_h_wC)
grid on
title('Hladina h(t)')
legend('Nelineární model','Linearizovaný model')
xlabel('$t$ [s]','interpreter','latex')
ylabel('$h(t)$ [m]','interpreter','latex')

%% 04: Část I.D - Zvýšení přítoku a porovnání s linearizovaným modelem
modelD = sim('cast1_model_D');          % Simulink: simulace modelu

% Vstupní proměnná
Scope_Q1_tD = modelD.Scope_Q1_modelD.time;              % Čas
Scope_Q1_wD = modelD.Scope_Q1_modelD.signals.values;    % Přítok

% Výstupní proměnná
Scope_h_tD_nelin = modelD.Scope_h_modelD_nelin.time;    % Čas
Scope_h_wD_nelin = modelD.Scope_h_modelD_nelin.signals.values;
                                                        % Hladina

Scope_h_tD_lin = modelD.Scope_h_modelD_lin.time;        % Čas
Scope_h_wD_lin = modelD.Scope_h_modelD_lin.signals.values;
                                                        % Hladina

% Vykreslení grafů
figure
plot(Scope_Q1_tD, Scope_Q1_wD)
grid on
range=ylim();
ylim([0 range(2)])
title('Přítok Q_1(t)')
xlabel('$t$ [s]','interpreter','latex')
ylabel('$Q_1(t)$ [$\mathrm{m}^3 \cdot \mathrm{s}^{-1}$]',...
    'interpreter','latex')

figure
plot(Scope_h_tD_nelin, Scope_h_wD_nelin, Scope_h_tD_lin, Scope_h_wD_lin)
grid on
title('Hladina h(t)')
legend('Nelineární model','Linearizovaný model')
xlabel('$t$ [s]','interpreter','latex')
ylabel('$h(t)$ [m]','interpreter','latex')

%% 05: Uložení parametrů a výpočtů
save('save_param',...
    't_A','t_B','t_C',...
    'Q_A','Q_B','Q_C',...
    'A_0','A_2',...
    'g',...
    'A','B','C','D',...
    'u_0','h_0')

Does anybody have any idea why this is hapenning? Did I overlook something?

17
  • listings splits utf8 chars in bytes while processing the code and this breaks them. You can try listingsutf8, but normally it is better to define the output of the accented chars with the literate option of listings. Apr 7 at 18:34
  • @UlrikeFischer Well, neighter seems to work. If you look into the preamble, you'll see that I'm already using listingsutf8 package. The original listings package is commented out.
    – Polda18
    Apr 7 at 18:44
  • Well as I wrote: the literate option normally works better. Apr 7 at 18:53
  • @UlrikeFischer So what do you suggest me to do with the MATLAB script file? If the package splits it up to individual bytes, then the literate option won't work eighter, because the file is in UTF-8 encoding. I would have to convert it to Windows 1250 encoding (Central European encoding), specify it's in this encoding, and then specify literate option, right? Not only that, I would actually have to delete UTF-8 configuration statup script because MATLAB will overwrite the file with UTF-8 encoding again otherwise. Do you have an idea how can I solve the issue while remaining fully in UTF-8?
    – Polda18
    Apr 7 at 19:04
  • Is there a specific reason why you need to use pdflatex? I just tried your example in xelatex, and it works just fine there.
    – chsk
    Apr 7 at 19:46
0

Okay, so this is what I ended up doing, finally solving my issue. I added extendedchars=true option back to the listings settings, and I realised that I forgot to include some of the characters. As soon as I included them, the issue was gone.

This is now my listingsutf8 settings in preamble:

\documentclass[12pt,twoside,a4paper]{article}
\usepackage[unicode]{hyperref}
\usepackage[czech]{babel}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{float}
\usepackage{lmodern}
\usepackage{graphicx}
\usepackage{attachfile}
\usepackage{subcaption}
\usepackage{cleveref}
\usepackage[document]{ragged2e}

% ...

\usepackage{listingsutf8}
\usepackage{xcolor}

\definecolor{codegreen}{rgb}{0,0.6,0}
\definecolor{codegray}{rgb}{0.5,0.5,0.5}
\definecolor{codepurple}{rgb}{0.58,0,0.82}
\definecolor{backcolour}{rgb}{0.95,0.95,0.92}

\lstset{
    %inputencoding=utf8,                    % Use input encoding UTF-8
    extendedchars=true,                     % Use extended character set
    backgroundcolor=\color{white},          % choose the background color; you must add \usepackage{color} or \usepackage{xcolor}; should come as last argument
    basicstyle=\tt\scriptsize,              % the size of the fonts that are used for the code
    breakatwhitespace=false,                % sets if automatic breaks should only happen at whitespace
    breaklines=true,                        % sets automatic line breaking
    captionpos=t,                           % sets the caption-position to bottom
    commentstyle=\color{codegreen},         % comment style
    escapeinside={\%*}{*)},                 % if you want to add LaTeX within your code
    firstnumber=1,                          % start line enumeration with line 1000
    frame=single,                           % adds a frame around the code
    keepspaces=true,                        % keeps spaces in text, useful for keeping indentation of code (possibly needs columns=flexible)
    keywordstyle=\color{blue},              % keyword style
    language=MATLAB,                        % the language of the code
    morekeywords={*,...},                   % if you want to add more keywords to the set
    numbers=left,                           % where to put the line-numbers; possible values are (none, left, right)
    numbersep=5pt,                          % how far the line-numbers are from the code
    numberstyle=\tiny\color{codegray},      % the style that is used for the line-numbers
    rulecolor=\color{black},                % if not set, the frame-color may be changed on line-breaks within not-black text (e.g. comments (green here))
    showspaces=false,                       % show spaces everywhere adding particular underscores; it overrides 'showstringspaces'
    showstringspaces=false,                 % underline spaces within strings only
    showtabs=false,                         % show tabs within strings adding particular underscores
    stepnumber=1,                           % the step between two line-numbers. If it's 1, each line will be numbered
    stringstyle=\color{codepurple},         % string literal style
    tabsize=4,                              % sets default tabsize to 2 spaces
    caption=\texttt{\protect\filename@parse{\lstname}\protect\filename@base.\protect\filename@ext},
                                            % show the filename of files included with \lstinputlisting; also try caption instead of title
    literate=                               % Replace Czech characters with composite letters counterpart
                                            % áÁ čČ ďĎ éÉ ěĚ íÍ ňŇ óÓ řŘ šŠ ťŤ úÚ ůŮ
    {á}{{\'{a}}}1 {Á}{{\'{A}}}1 {č}{{\v{c}}}1 {Č}{{\v{C}}}1
    {ď}{{\v{d}}}1 {Ď}{{\v{D}}}1 {é}{{\'{e}}}1 {É}{{\'{E}}}1
    {ě}{{\v{e}}}1 {Ě}{{\v{E}}}1 {í}{{\'{i}}}1 {Í}{{\'{I}}}1
    {ň}{{\v{n}}}1 {Ň}{{\v{N}}}1 {ó}{{\'{o}}}1 {Ó}{{\'{O}}}1
    {ř}{{\v{r}}}1 {Ř}{{\v{R}}}1 {š}{{\'{s}}}1 {Š}{{\'{S}}}1
    {ť}{{\v{t}}}1 {Ť}{{\v{T}}}1 {ú}{{\'{u}}}1 {Ú}{{\'{U}}}1
    {ů}{{\r{u}}}1 {Ů}{{\r{U}}}1 {ž}{{\v{z}}}1 {Ž}{{\v{Z}}}1
    {ý}{{\'{y}}}1 {Ý}{{\'{Y}}}1
}

% ...

And the listings in my document now look like this: enter image description here

Thank you and sorry for my rather stupid behaviour. I missed what the compiler actually told me, or rather said didn't quite understand it. Now I know I need to replace Unicode characters with their composite counterparts. Not sure how that would work with let's say Arabic or Chinese, but that's a whole different issue and I don't think I'll ever encounter that issue with these thousands characters languages. This is quite bulky workaround, that however works for my needs. Doesn't need to be pretty. 🙃

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.