If they are producing problematic exposure to a large number of people, for the benefit of only a few, yes I would.
It gets a lot more difficult however to figure out where to draw the line. In terms of noise, smoking is like someone letting off flashbangs. It's obviously antisocial and problematic, particularly in a close environment. However , how should we handle loud music? Obviously playing music loud enough to vibrate windows at 3am is a problem, but below that is a murky zone where it's difficult to agree on what's problematic or not.
The equivalent to this, in smoking terms is vaping etc. It still produces something with a negative effect, but with a far lower problem potential. While I would personally prefer not to be exposed to vaping smells either, the balance is a lot less obvious. I accept that it would do more harm (to our personal freedoms etc) than good adding vapes to a ban.
There should be some rules on smell production, but it's the sort of thing that is difficult to write into regulations. It's currently impossible to write a quantitative test into law. All would be subjective, and so prone to problems.