This is what I get when calling \showlists after the formula:
LuaTeX
### horizontal mode entered at line 6
\whatsit
.\localinterlinepenalty=0
.\localbrokenpenalty=0
.\localleftbox=null
.\localrightbox=null
\hbox(0.0+0.0)x20.0, direction TLT
\mathon
\vbox(8.20554+0.0)x7.95831, direction TLT
.\hbox(5.67776+0.0)x0.0, direction TLT
..\OT1/cmr/m/n/10 ^^V
.\kern-4.30554
.\hbox(6.83331+0.0)x7.95831, direction TLT
..\OML/cmm/m/it/10 E
\mathoff
XeTeX
### horizontal mode entered at line 6
\hbox(0.0+0.0)x20.0
\mathon
\vbox(8.20554+0.0)x7.95831
.\hbox(5.67776+0.0)x0.0, shifted 2.31252
..\OT1/cmr/m/n/10 ^^V
.\kern-4.30554
.\hbox(6.83331+0.0)x7.95831
..\OML/cmm/m/it/10 E
\mathoff
There is clearly a problem with LuaTeX; it can be due to fontspec as, when not loading it or loading a different font than Linux Libertine O (I tested Old Standard and TeX Gyre Termes) I get the following:
LuaTeX (without Linux Libertine O)
### horizontal mode entered at line 6
\whatsit
.\localinterlinepenalty=0
.\localbrokenpenalty=0
.\localleftbox=null
.\localrightbox=null
\hbox(0.0+0.0)x20.0, direction TLT
\mathon
\vbox(8.20554+0.0)x7.95831, direction TLT
.\hbox(5.67776+0.0)x0.0, shifted 2.31252, direction TLT
..\OT1/cmr/m/n/10 ^^V
.\kern-4.30554
.\hbox(6.83331+0.0)x7.95831, direction TLT
..\OML/cmm/m/it/10 E
\mathoff
fontspecwith theno-mathoption. – Andrey Vihrov Nov 4 '11 at 15:01