In a number of schools, athletic departments are self-sufficient, so it's certainly possible the parent poster drew no general funds for his varsity sport participation.
Further, your argument about recreational activities for a few can be applied to pretty much every non-class activity at a university, from theater to quiz bowl to student radio. Why single out athletics?
Actually, for the most part I do extend it to pretty much every non-class activity at a university [1]. I only singled out athletics since that is what we were discussing. As a rule, I believe the cost of non-class activities should be paid for in full by participating students, not by taxpayers or other students.
Further, your argument about recreational activities for a few can be applied to pretty much every non-class activity at a university, from theater to quiz bowl to student radio. Why single out athletics?