Large-$N$ $SU(N)$ Yang-Mills theories with milder topological freezing