Can strangers interact with me without my consent?

No.
All interactions require clear intent and confirmation. You only collaborate with others by actively joining or starting a quest together.