Maybe it's because the girls were generally not placed in positions where they were likely to come into frequent contact with priests in isolated settings, and vice-versa.
Other options:
no subject
Well, duh -- altar boys are generally, well, male.