How much one can retain from ones experience mostly relies on ones ability to "integrate" the experience. If you have done much substance but little work, there may be a long while until you fully integrate the teachings by "doing the work". In a sense we must live and "apply" what the teachers show/teach us, to then integrate and generate a higher vibrational state naturally. Its like reading a book without applying the knowledge.
The Wisdom comes from the action, application, experience. You will only remember what is relevant for you at the time being.
I feel that a rest from substance may serve you well for now, or try something else, like mushrooms.
Blessings