As @Bernard commented this is possible with the
titlesec package. However, it is also easy to do without packages, which might be preferred if you don't want to use
titlesec for some reason.
Here a new command called
\optionalsection is defined that first changes the section number format to a *, the chapter number, a dot, and the section number, then creates a section with this format, then resets the section number to the default format.