What you describe seems similar to the nonmonotonic inference relation symbol used in this paper. The main difference being that this a small space between the two characters, which I think actually looks better than having them touch.
\documentclass{article}
\newcommand*{\nc}[2]{#1\mathbin{\left| \sim \vphantom{#1#2} \right.}#2}%
\begin{document}
$\nc{A}{B} \qquad \nc{\frac{A}{C}}{B}$
\end{document}
Update: Here is a version where both the straight and wavy line are re-sized and the two symbols touch:
I have used \scalebox
from the \graphicx
package to resize the symbols (resize horizontally for the wavy line and vertically for the vertical bar) based on the height of the operands on either side. The \mathrlap
is obtained from a TUGboat 22 article entitled a complement to \smash, \llap, and \rlap. I am not an expert in the use of the rlap
type command, so perhaps some of the resize code could be simplified. I also used pgf
to do the math, and this is probably overkill but I am more familiar with that then doing math in plain TeX.
\documentclass[border=2pt]{standalone}
\usepackage{amsmath}% only needed for \dfrac
\usepackage{graphicx}% needed for \scalebox
\usepackage{pgf}% needed for the math calculations
% http://math.arizona.edu/~aprl/publications/mathclap/
\def\mathrlap{\mathpalette\mathrlapinternal}%
\def\mathrlapinternal#1#2{\rlap{$\mathsurround=0pt#1{#2}$}}%
\makeatletter
\newdimen\@mydimen%
\newdimen\@myHeightOfBar%
\settoheight{\@myHeightOfBar}{$|$}%
\newcommand{\SetScaleFactor}[1]{%
\settoheight{\@mydimen}{#1}%
\pgfmathsetmacro{\scaleFactor}{\@mydimen/\@myHeightOfBar}%
}%
\newcommand*{\Scale}[2][3]{\scalebox{#1}{\ensuremath{#2}}}%
\newcommand*{\nct}[2]{%
\SetScaleFactor{\vphantom{\ensuremath{#1#2}}}% Compute scale to be applied
#1%
\mathrel{\Scale[\scaleFactor]{|\mathrlap{\kern-0.48ex\sim}\hphantom{\kern-0.41ex\sim}}}%
#2%
}%
\begin{document}
$\nct{a}{b} \qquad \nct{\frac{A}{C}}{B} \qquad \nct{\dfrac{A}{C}}{B}$
\end{document}
An enhanced version of this solution that allows you to specify a character to be placed above and below the wavy line is available at the follow-up question: Symbol for skeptical consequence that matches the turnstile package
$|\!\!\!\sim$
. This might not scale well...