@Sick Boy Buddhism suggests something called "The Middle Way" of moderation, between the extremes of sensual indulgence and self-mortification. Getting too attached to friends/girls/the gym will only lead to eventual disappointment, and so will restricting activities entirely. That restriction is actually attachment in disguise.
Just keep gently following the clues. You'll find your balance.