Spectral analysis of transfer operators associated to Farey fractions