If your download folder is located outside of your user profile, for instance, you put it on a different partition, then VS at default settings will not monitor it, when VS toggles to OFF. The sign of this state is when the icon turns red (ON is blue). In this state, only high-risk areas are monitored.
What's the danger? Let's say you close your browser, and now you have no web-facing apps open. VS will toggle to OFF after a couple seconds. Now you go to your download folder, and click on a baddie. It will not be blocked unless it starts running LOL bins.
But you could run VS in always-on mode, as a workaround. Many people do that anyway, for increased security. The downside of always-on mode is that theoretically, there are certain background processes might get continually blocked. I don't know how much this actually happens in reality, though.