That's correct, you do lose out on that training time if you send the coach away. For that reason, I typically specialize each of the three coaches for recruiting (high charisma), scouting (high scouting) and teaching (high teaching). That way you can send your recruiter or scout away and not lose out on too much teaching. I know some people like to max out the their own charisma skill first because it helps with recruiting, but you can have one of your assistants do all that so you can train the available time each week to maximize the abilities of the players I have. High charisma is a nice to have, but most of the recruits that are going to be interested in your school are going to sign regardless of whether you have an A+ charisma yourself or you send your assistant with a B+ charisma.
Here's the priorities I use for coaches ability:
1. Teaching
2. Offense
3. Defense
4. Discipline
5. Charisma (Assistant Coach)
6. Scouting (Assistant Coach)