The modern statements that weed was used for religious reasons in ancient times lack context and are very exaggerated.
Egypt shows the HEMP plant in many contexts, but it was used medicinally.
Hindu rituals existed in the Vedic period where individuals would drink Bhang, a sort of milk concoction that did contain cannabis.
But the cannabis of ancient times was incredibly weak, and these "potions" only existed to slightly relax, not to get zonked.
Nowadays, not only is all cannabis upwards of 20-30 times stronger than what existed at that time, but we also have a lot more knowledge about its adverse effect that, had the ancients had, they would've been even more wary of the plant.
At the end of the day, ToZ does not force you to do or not do something, you can do as you wish. But what ToZ does do is frown upon the use of mind altering substances. Nothing is sacred about it, and it can end up getting you in a lot of trouble.