Covert communication (also known as steganography) is the practice of
concealing a secret inside an innocuous-looking public object (cover) so that
the modified public object (covert code) makes sense to everyone but only
someone who knows the code can extract the secret (message). Linguistic
steganography is the practice of encoding a secret message in natural language
text such as spoken conversation or short public communications such as
tweets.. While ad hoc methods for covert communications in specific domains
exist ( JPEG images, Chinese poetry, etc), there is no general model for
linguistic steganography specifically. We present a novel mathematical
formalism for creating linguistic steganographic codes, with three parameters:
Decodability (probability that the receiver of the coded message will decode
the cover correctly), density (frequency of code words in a cover code), and
detectability (probability that an attacker can tell the difference between an
untampered cover compared to its steganized version). Verbal or linguistic
steganography is most challenging because of its lack of artifacts to hide the
secret message in. We detail a practical construction in Python of a
steganographic code for Tweets using inserted words to encode hidden digits
while using n-gram frequency distortion as the measure of detectability of the
insertions. Using the publicly accessible Stanford Sentiment Analysis dataset
we implemented the tweet steganization scheme -- a codeword (an existing word
in the data set) inserted in random positions in random existing tweets to find
the tweet that has the least possible n-gram distortion. We argue that this
approximates KL distance in a localized manner at low cost and thus we get a
linguistic steganography scheme that is both formal and practical and permits a
tradeoff between codeword density and detectability of the covert message.

使用 Python 编写了一个用于推文的隐写码实现，采用插入词以隐藏数字，并使用 n-gram 频率扭曲作为插入检测度量，从而获得一种既形式化又实用的语言隐写方案，允许在编码字密度和隐秘信息的可检测性之间进行权衡。

隐匿在眼前：走向语言隐写科学

Hiding in Plain Sight: Towards the Science of Linguistic Steganography

We develop a new Low-level, First-order Probabilistic Programming Language
(LF-PPL) suited for models containing a mix of continuous, discrete, and/or
piecewise-continuous variables. The key success of this language and its
compilation scheme is in its ability to automatically distinguish parameters
the density function is discontinuous with respect to, while further providing
runtime checks for boundary crossings. This enables the introduction of new
inference engines that are able to exploit gradient information, while
remaining efficient for models which are not everywhere differentiable. We
demonstrate this ability by incorporating a discontinuous Hamiltonian Monte
Carlo (DHMC) inference engine that is able to deliver automated and efficient
inference for non-differentiable models. Our system is backed up by a
mathematical formalism that ensures that any model expressed in this language
has a density with measure zero discontinuities to maintain the validity of the
inference engine.

我们开发了一种新的低级、一阶概率编程语言（LF-PPL），适用于包含连续、离散和 / 或分段连续变量的模型，在能够自动区分与密度函数不连续有关的参数的同时，提供运行时检查边界交叉的编译方案。通过将不连续的哈密尔顿蒙特卡洛（DHMC）推断引擎整合进去，我们证明了该语言的能力，可以提供自动和高效的非可微模型推断。同时，我们的系统还有数学形式主义来确保任何表达在该语言中的模型具有密度，且在测度上具有零间断，以维护推断引擎的有效性。