We introduce a new $5$-parameter family of distributions, the Asymmetric Exponential Power (AEP), able to cope with asymmetries and leptokurtosis and, at the same time, allowing for a continuous variation from non-normality to normality. We prove that the Maximum Likelihood (ML) estimates of the AEP parameters are consistent on the whole parameter space, and when sufficiently large values of the shape parameters are considered, they are also asymptotically efficient and normal. We derive the Fisher information matrix for the AEP and we show that it can be continuously extended also to the region of small shape parameters. Through numerical simulations, we find that this extension can be used to obtain a reliable value for the errors associated to ML estimates also for samples of relatively small size ($100$ observations). Moreover we show that around this sample size, the bias associated with ML estimates, although present, becomes negligible. Finally, we present a few empirical investigations, using diverse data from economics and finance, to compare the performance of AEP with respect to other, commonly used, families of distributions.