Asymmetric dark matter: residual annihilations and self-interactions