Most recreational activies are productive and serve a purpose other than mindless pleasure, be it reading, exercising, creative projects or other such activites. Recreational drug use serves no such purpose and only exists to numb one's mind from the outside world, and likely their troubles, this fact just is covered by pretty words about "relaxation" and such.
Banning a substance (with strategic eye of course, for example a total immediate ban on alcohol would cause more problems than it would solve, a incremental ban is better in this case) that complements this behaviour also discourages this behaviour, as no-one who isn't addicted to the substance will take the risk of procuring it illegally. And the ones who are addicted obviously require medical care.
The decrease of organised crime in drug trade is no argument, one could argue that murder should be legal since the amount of convicted murderers goes down if it is legalised. Not to mention how the ban on certain drugs in America isn't designed to lower the usage of drugs to begin with, rather simply profit off it in various ways.
