Selfhosted @lemmy.world aesir @lemmy.world 2y ago

Another good reason not to open port 22

In the past two weeks I set up a new VPS, and I run a small experiment. I share the results for those who are curious.

Consider that this is a backup server only, meaning that there is no outgoing traffic unless a backup is actually to be recovered, or as we will see, because of sshd.

I initially left the standard "port 22 open to the world" for 4-5 days, I then moved sshd to a different port (still open to the whole world), and finally I closed everything and turned on tailscale. You find a visualization of the resulting egress traffic in the image. Different colors are different areas of the world. Ignore the orange spikes which were my own ssh connections to set up stuff.

Main points:

there were about 10 Mb of egress per day due just to sshd answering to scanners. Not to mention the cluttering of access logs.
moving to a non standard port is reasonably sufficient to avoid traffic and log cluttering even without IP restrictions
Tailscale causes a bit of traffic, negligible of course, but continuous.

120

120 comments

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

Fewer Letters	More Letters
DNS	Domain Name Service/System
HTTP	Hypertext Transfer Protocol, the Web
IP	Internet Protocol
SSH	Secure Shell for remote terminal access
UDP	User Datagram Protocol, for real-time communications
VPN	Virtual Private Network
VPS	Virtual Private Server (opposed to shared hosting)
nginx	Popular HTTP server

7 acronyms in this thread; the most compressed thread commented on today has 8 acronyms.

[Thread #42 for this sub, first seen 14th Aug 2023, 15:55] [FAQ] [Full list] [Contact] [Source code]

Public key auth, and fail2ban on an extremely strict mode with scaling bantime works well enough for me to leave 22 open.

Fail2ban will ban people for even checking if the port is open.
- Honest question, is there a good default config available somewhere or is what apt install fail2ban does good to go? All the tutorials I’ve found have left it to the reader to configure their own rules.
  
  Honestly the default config is good enough to prevent brute force attacks on ssh. Just installing it and forgetting about it is a definite option.
  
  I think the default block time is 10 minutes after 5 failed login attempts in 10 minutes. Not enough to ever be in your way but enough to fustrate any automated attacks. And it's got default config for a ton of services by default. Check your /etc/fail2ban/jail.conf for an overview.
  
  I see that a recidive filter that bans repeat offenders for a week after 10 fail2ban bans in one day is also default now. So I'd say that the results are perfect unless you have some exotic or own service you need fail2ban for.
- Yeah fail2ban has worked great for me
- If Fail2Ban is so important, why the h*** does it not come installed and enabled as standard?!
  
  Security is the number-1 priority for any OS, and yet stock SSHD apparently does not have Fail2Ban-level security built in. My conclusion is that Fail2Ban cannot therefore be that vital.
You really shouldn't have something removed SSHD open to the world, that's just an unnecessary atrack surface. Instead, run a VPN on the server (or even one for a network if you have several servers on one subnet), connect to that then ssh to your server. The advantage is that a well setup VPN simply won't respond to an invalid connection, and to an attacker, looks just like the firewall dropping the packet. Wireguard is good for this, and easy to configure. OpenVPN is pretty solid too.
- You say this and are downvoted.
  
  While we are coming off the tail of Def Con where there where a plethora or small talks and live examples of taking advantage and abusing just this.
  
  Just trying to parse your comment, I assume your first "this" and second "this" are referring to different things, right?
  
  I don't understand your comment, what you are saying. Could you elaborate a bit, please? I'm interested why it's a bad idea what previous comment suggested.
  
  Of course I can dig into DefCon videos and probably would do if needed, but perhaps you know what exactly the issue is
- I usually just run a ZeroTier client on my Pi connected to a private P2P network to solve this issue, and then have ProtonVPN over Wireguard for all internet traffic in and out of the Pi.
I’ll take that tiny amount of traffic telling scanners there’s no password auth over having to remember port settings for ssh, scp and rsync any day.
- vim ~/.ssh/config
- My configs remember stuff for me.
- For me it's not about the traffic, more the log spam.
  
  Generally I'll have :22 enabled internally, and anything non-standard is defined in ~/.ssh/config and shared out so I don't have to remember things.
- Fair point. These logs are only useless chatter anyway for everyone with proper key auth.
Or, you know, just use key auth only and fail2ban. Putting sshd behind another port only buys you a little time.
- Yeah but the majority of bots out there are going after easy prey. Honestly, if you use public key authentication with ssh you should be fine, even if it is on port 22. But it does of course clog up access logs.
  
  The majority of bots out there are stopped by just using a hard to guess password. It's not them that you should be worried about.
- I've noticed that a lot of the scans these days almost always switch IPs after 2-3 attempts, making IP blocking a lot more difficult.
  
  Let's say that you could ban for an indefinitely large amount of time after a single failure to authenticate, that'd make them run out of IPs much quicker than you'd run out of CPU/BW, so I don't really see the issue
I opened a raw text channel on the Telnet port for a personal game engine project and someone tried to enable commands and do some shady stuff. Unfortunately for them, that's not a valid chess move.
Just do it properly and configure sshd securely. When you have a machine exposed to the internet, you should expect it to be attacked. If you really want to give the finger to bots, run endlessh on port 22 and keep sshd on a non-standard port. Stay safe.
- endlessh
  
  Lmao thanks for this
  
  endlessh
  
  yep... https://nullprogram.com/blog/2019/03/22/
  
  reminds me of this
  
  https://www.vice.com/en/article/d3b7na/the-story-of-lenny-the-internets-favorite-telemarketing-troll
I get what you say, and you're definitely not wrong to do it. But as I see it, you only saved ~80Kib of ingress and a few lines of logs in the end. From my monitoring I get ~5000 failed auth per day, which account for less than 1Mbps average bandwidth for the day.

It's not like it's consuming my 1Gbps bandwidth or threatening me as I enforce ssh key login. I like to keep things simple, and ssh on port 22 over internet makes it easy to access my boxes from anywhere.
- ssh -p 12345 would leave your boxes accessible from anywhere too. Other blocks of IPs receive 10 times or more requests, as scanners can focus on blocks of ips from major providers.
  
  Yeah I know, I just don't really care about that traffic to bother changing it :) Also, I'm talking about a server hosted on Hetzner, so I feel like it's scanned a lot.
- I don't get why people leave interfaces the public doesn't need access to open to the public -- especially SSH.
  
  Use a VPN if you need access to those interfaces from the "outside". They're stupidly easy to set up these days, particularly with Wireguard.
  
  A VPN is easy to setup (and I have it setup by the way), but no VPN is even easier. SSH by itself is sufficiently secure if you keep it up to date with a sane configuration. Bots poking at my ssh port is not something that bother me at all, and not part of any attack vector I want to be secure against.
  
  Out of all the services I expose to the clear web, SSH is probably the one I trust the most.
I'd favour own VPN instead of relying on an additional third party
- Right? PiVPN is easy AF and uses WireGuard. No reason not to set up something yourself if you're already selfhosting.
  
  Please ELI5: How does this solution work? You tunnel yourself in your home network with a VPN on your PI? How is that safer, isn't there still a port open?
  
  Mine is quick enough to run remote desktop over
10mb is pretty much nothing. May as well just use Fail2Ban.
If you do want to open 22, and there are plenty of good reasons to want to, just implement something called port knocking and you can do it safely.

Note with this you still need good authentication. That means no passwords, key based auth only.
- I have read elsewhere that port knocking is just security through obscurity and isn’t worth considering. I found it when searching for ways to set it up and that put me off.
  
  It is and it isn't. It prevents random scans from opening 22 and attempting to authenticate, that's basically the entire purpose. You still need good authentication after because you're right, it's not a security measure, it's just a way to keep your logs useful and to keep botnets from beating the hell out of 22.
  
  By "good authentication" I mean a key pair based authentication. That is impossible to brute force. If you use a password on 22 you shouldn't open it at all and you should rethink allowing any remote access.
  
  Put another way: You're the doorman at a speak easy. You can answer the little window with "what's the password?" to every jack ass that approaches, and you'd be asking all the time. But if they don't know they have to knock "shave and a haircut" first, your job gets a lot easier and you're dealing with a lot fewer nuisance password promptings.
  
  You can also use it to blacklist. If someone tries to hit 22 without knocking you can blacklist that IP entirely because you know it's nuisance.
  
  Security through obscurity is a very valid way to secure something. It shouldn't be the only way of securing something, but it can be a valid additional layer.
  
  The knock sequence is a secret, just like a password. It may not be a particularly strong secret, but is is strong enough to keep out casual attackers. You'll still need additional security, but sshd is well equipped to provide that.
  
  If you want to go down that path, a password is only security by obscurity.
  
  Port knocking is an extra layer of security, and one that can stop attackers from ever knowing your private server even exists. A random scanner won’t even see any open ports.
  
  Always bear in mind that any random guy advising people not to use port knocking may be doing it with malicious intent. I’m sure there’s someone out there advising that random passwords are a waste of time, and everyone should just use monkey123.
I am not in IT, what does this mean ?
- Computers communicate across networks using ports. Port 22 is a commonly used remote administration port called ssh. Bots go around probing computers with an open port 22 hoping to find badly secured or outside misconfigured ssh servers to turn them into bots and crypto miners, etc.
  
  Its crazy people can do all these stuff. I can't even edit my word document. Being said that I want to learn IT but looks hell stressful for $100-$200k job. I was in content & marketing making $130k and we used to discuss about shades of color to use in a font for 3-4 weeks. Its crazy how you guys have to fix issues within minutes on those tickets. The more i learn about IT, I feel like I should stay away from it.
Now try IPv6 only :P
- Next time
ITT: People who don't understand Tailscale or are allergic to it for 'reasons'
As others have already said, set up a VPN like wireguard, connect to the VPN and then SSH to the server. No need to open ports for SSH.

I do have port 22 open on my network, but it's forwarded to an SSH tarpit: https://github.com/skeeto/endlessh
- But Tailscale is Wireguard under the hood.
  
  Yeah, but worse cause it's company owned and not really open source. Why do people use tailscale? Are you so desperate to pay money for it?
- I have wireguard for other purposes but I also have ssh open on a different port. I don't much understand the argument of exchanging ssh for wireguard. In the end, we're just trading an attack vector for another.
  
  My ssh only allows connections from my user. If I'm using password auth, I also request a 2FA.
  
  Tail scale is also a good idea but I don't like having my control plane under someone else's control.
  
  There is quite a significant difference. An ssh server - even when running on a non-default port - is easily detectable by scanning for it. With a properly configured Wireguard setup this is not the case. As someone scanning from the outside, it is impossible to tell if there is Wireguard listening or not, since it simply won't send any reply to you if you don't have the correct key. Since it uses UDP it isn't even possible to tell if there is any service running on a given UDP port.
  
  The reason a VPN is better to expose than SSH, is the feedback.
  
  If someone tries connecting to your SSH with the wrong key or password, they get a nice and clear permission denied. They now know that you have SSH, and which version. Which might allow them to find a vulnerability.
  
  If someone connects to your wireguard with the wrong key, they get zero response. Exactly as if the port had not been open in the first place. They have no additional information, and they don't even know that the port was even open.
  
  Try running your public IP through shodan.io, and see what ports and services are discovered.
  
  So just run headscale then.
  
  If someone finds a 0day in your SSH server and goes on drive-by attacking the whole internet you're toast.
  
  Already moving off port 22 reduces much of the risk, essentially reducing the attack surface for drive-by attacks to zero while still being susceptible to targeted attacks -- that is, still susceptible to attackers bothering to scan the whole range. Anything that makes you unscannable (VPN, portknockd, doesn't matter) mitigates that. Even state-level actors would have to be quite determined to get through that one.
  
  Yes it's security through obscurity. Yes it's a good idea: There's a difference between hiding your unlocked front door and hiding your military-grade front door lock, one of them is silly the other isn't.
I don't get tailscale? In-kernel Wireguard is easy to setup. What does it add to this?
- Tailscale is a nice way to set up a private network between your machines. It’s perfectly fine.
- I believe it adds a third party server to help facilitate communication between clients.
- I think it’s just the ease of GUI for people. This isn’t to shit on anyone, btw. A lot of people don’t like dealing with the keys and IPs involved, few as there may be, with setting up wireguard.
  
  If someone else has a compelling difference or reason to use tailscale then I’d be happy to hear it. I tried it once and it worked fine enough. But wireguard works just as fine and takes the same time to setup if you already know what to do. Like wireguard seriously takes 2 minutes.
  
  Didn't see it has GUI. Valid argument, thanks.
  
  I'm not using Tailscale, but I'd imagine that if you wanted to form a private network that involves devices controlled by non-technical people, the GUI becomes less of a "don't like to deal with keys/IPs" and more of a "can't deal".
And yet it is more likely that tailscale get owned since the reward is much higher. I take my chances with my secured openssh server at port 22 vs a 3rd party company who controlls the access.
- Just setup a wireguard system that’s pretty bulletproof
  
  So is SSH
For management ports, I set up a firewall on the VPS to only respond to connections from known IPs.
I don't know if it was mentioned already but how could I check if my ports are under "attack"? OPs graphic looks really nice
- Hi, to check attacks you should look at the logs. In this case auth.log. Being attacked on port 22 is not surprising neither really troublesome if you connect via key pair.
  
  My graph was showing egress traffic, on any kind of server the traffic due to these attacks would have been invisible but on a backup server which has (hopefully) only ingress you can clearly see the volume of connections from attackers from bytes teansmitted
Throw CrowdSec on there to stop the bots before they can do anything
The best reason is reducing attack surface. It's such an easy thing to do. I don't know why people still expose services they don't need to.
- Moving the port doesn't reduce attack surface. It's the same amount of surface.
  
  Tailscale is a bit controversial because it requires a 3rd party to validate connections, a 3rd party that is a large target for threat actors, and is reliant on profitability to stay online.
  
  I would recommend a client VPN like wireguard, or SSH being validated using signed keys against a certificate authority your control, with fail2ban.
  
  You can run headscale to do your own validation.
  
  It reduces the probability that a drive by scanner is going to detect a vulnerable service. Camouflage isn't a guarantee that you aren't going to be sighted on a battlefield, but it's still a good idea to reduce the probability of becoming a target in whatever ways you can.
Prevent password auth and setup sshguard. Wireguard is very nice in that it doesn't support password auth.
Is public key authentication not good enough? Tailscale is cool but can be tedious if you also use other VPNs
- Yeah that's what I have too. One of my servers is exposed with key auth and I just tunnel to other servers from there. A few MB egress is nothing compared with the amount of spam my webserver needs to deal with
Whys this a problem disable password auth and wish em good luck lol.
What software did you use you monitor this? I am newish to self hosting and would like to secure my connection better.
- Sorry, it's the built-in console of Google Cloud. But there are so many monitoring solution around that you can probably find one of your liking. Look on awesome-selfhosted for "monitoring"
Exploring Go High Level:

What’s the Enthusiasm About?
Hello there! Are you familiar with Go High Level? If not, let's break it down. Imagine managing multiple apps for marketing. Sounds busy. That's where Go High Level comes in. It's like that friend who has a solution for everything. Whether you are working on creating an outstanding sales funnel or sending out an email campaign, this platform has your back. The best part? They let you test drive everything with a 14-day free trial. It's like trying out a new car but for your business. And if you ever need assistance, their support team is just a click away. Pretty exciting, huh?

So, How Much Does It Cost?
Alright, let's talk about the financial side. Go High Level offers 3 main plans. The Agency Starter Plan is perfect if you're just starting or have a small business. It's loaded with all the essential tools, and it's quite cost-effective. But if you're looking to scale up, the Agency Unlimited Plan is your best choice. It's like the VIP pass at a concert, giving you access to everything without any limitations. Not sure about committing? Remember that 14-day free trial I mentioned? It's a great way to give it a try without any obligations. Lastly, they offer a Pro plan that includes "SaaS mode," where you can white label the product under your brand. Fantastic!

Why Everyone's Raving About It:
In a world full of sophisticated digital tools, Go High Level is like that all-in-one Swiss Army knife. There's no need to switch between apps because it has everything under one roof. Whether you're a newcomer or an experienced marketer, it's super user-friendly. It's not just about launching impressive campaigns; they ensure you understand their performance with top-notch analytics. There's even a white-label feature for marketing agencies, so you can add your brand and impress your clients.

Here is a link to their 14-day free trial:
https://www.gohighlevel.com/?fp_ref=get-started-now.
Exploring Go High Level:

What’s the Enthusiasm About?
Hello there! Are you familiar with Go High Level? If not, let's break it down. Imagine managing multiple apps for marketing. Sounds busy. That's where Go High Level comes in. It's like that friend who has a solution for everything. Whether you are working on creating an outstanding sales funnel or sending out an email campaign, this platform has your back. The best part? They let you test drive everything with a 14-day free trial. It's like trying out a new car but for your business. And if you ever need assistance, their support team is just a click away. Pretty exciting, huh?

So, How Much Does It Cost?
Alright, let's talk about the financial side. Go High Level offers 3 main plans. The Agency Starter Plan is perfect if you're just starting or have a small business. It's loaded with all the essential tools, and it's quite cost-effective. But if you're looking to scale up, the Agency Unlimited Plan is your best choice. It's like the VIP pass at a concert, giving you access to everything without any limitations. Not sure about committing? Remember that 14-day free trial I mentioned? It's a great way to give it a try without any obligations. Lastly, they offer a Pro plan that includes "SaaS mode," where you can white label the product under your brand. Fantastic!

Why Everyone's Raving About It:
In a world full of sophisticated digital tools, Go High Level is like that all-in-one Swiss Army knife. There's no need to switch between apps because it has everything under one roof. Whether you're a newcomer or an experienced marketer, it's super user-friendly. It's not just about launching impressive campaigns; they ensure you understand their performance with top-notch analytics. There's even a white-label feature for marketing agencies, so you can add your brand and impress your clients.

Here is a link to their 14-day free trial:
https://www.gohighlevel.com/?fp_ref=get-started-now.

120 comments