I’ve had that problem. I’ve missed practices to do my own workouts (without my coaches knowledge) and they do get pretty upset about things like that. Maybe it wouldn’t have been much of a problem if you told him that you were doing your own workouts. It’s probably not an issue of you practicing as much as it is about accountability and being present because if you’re on the relay, it’s obvious he thinks you’re fast.
