People comment a lot on performance, but I think Caddy can (and should) hold up perfectly fine. It might be worth it to experiment with running servers half on Caddy and half on NGINX, then see how the traffic is being handled by both to compare.
I do think the much cleaner config makes up for the maybe slight performance loss, though. It's just so much less work to set up and maintain compared to NGINX. The last time I've used NGINX was years ago, when I decided to drop it entirely in favor of Caddy. I do think NGINX is only "standard" because it came before Caddy, and that most applications should not prefer it over Caddy.