Writing pseudocode for parallel programming

Question

"How do you write pseudo-code for parallel programming? Especially, how do you differentiate local and shared variables? How do you represent operations like scatter, gather, reduce, broadcast, and point-to-point communications? Is there some standards about that?"

Please, note the original question was asked here but it still does NOT have a good acceptable answer.

Re-elaborating other questions, summarizing and keeping the things easy: is there a way of saying in pseudocode "DO ALL THESE THINGS IN PARALLEL"?

I am using the package algorithm2e with TeXStudio. How to typeset the parallel code using them?

Answer 1

After some web research, I have realized that a kind of "standard" still does not exits. So here is my personal solution using algorithm2e:

\usepackage[linesnumbered,ruled,vlined]{algorithm2e}
...
\begin{algorithm}
    \DontPrintSemicolon 
    \SetKwBlock{DoParallel}{do in parallel}{end}
    \KwIn{Some inputs}
    \KwOut{The ouput}
    \DoParallel{
        Compute a \;
        Compute b \;
        Compute c \;
    }
    \DoParallel{
        a1\;
        b1\;
        c1\;
    }
    \Return{the solution}\;
    \caption{Parallel Algo}
    \label{algo:parallelAlgorithm}
\end{algorithm}

The result is:

It is based on defining a new command using the expression \SetKwBlock. The manual of the package can be found here. Also, I have just added almost the same solution to a similar question on StackOverflow. The answer and the question can be found here.

Using the strategy of defining your keywords in order to describe your algorithm with the details you prefer, it should be always possible. Take into consideration that:

1. more details → more you will be close to your programming languages.
2. fewer details → more it can be seen as a pseudo-code.

Concluding: it is always a matter of trade-offs: you decide where is the limit (taking into account the target people you refer to).

The same strategy has been used in journal papers (for instance, see Algorithm 3 and 4 of this IEEE journal paper).