Yes, your spiritual progress will be affected in some manner that might be not beneficial towards you if you do full isolation from the common man. How else will you understand the current world, specifically, your surroundings.
No, do not be involved with the lowest of humans that are the...