This paper provides a novel characterization of the max-margin distribution in input space. The use of the statistical Extreme Value Theory (EVT) is introduced for modeling margin distances, allowing us to derive a scalable non-linear model called the Extreme Value Machine (EVM). Without the need for a kernel, the EVM leverages a margin model to estimate the