2y ago

Why are we stuck with bash programming language in the shell?

Bash-like scripting has become ubiquitous in operating systems, and it makes me wonder about its widespread adoption despite lacking certain programming conveniences found in other languages. While it may not be the ideal choice for large-scale software development, the bash shell possesses unique features that make it well-suited for interactive command-line processing, including pipelining, text manipulation, and file handling. Why isn't there an alternative that combines the best of bash's command-line capabilities with the robustness and scalability of traditional programming languages. Why do even new operating systems, such as Redox OS, opt for a similar syntax rather than a completely different programming language?

Here are some of the limitations I find in Bash compared to other programming languages:

Syntax and Expressiveness:
- Bash has a relatively simple syntax compared to other programming languages. It lacks some advanced language features such as object-oriented programming, complex data structures, and advanced control flow constructs.
- The syntax of Bash can be less intuitive and more error-prone, especially for complex tasks or larger projects.
Performance:
- Bash scripts can be slower compared to compiled languages like C or Java. This is because Bash is an interpreted language, and each line of code is interpreted at runtime.
- Bash may not be the best choice for computationally intensive tasks or applications that require high performance.
Error Handling and Debugging:
- Error handling and debugging in Bash can be challenging. Bash does not provide robust error handling mechanisms, and error messages can be cryptic and difficult to interpret.
- Debugging Bash scripts can be cumbersome, as there is limited tooling and debugging support compared to other programming languages.
Portability:
- While Bash is available on most Unix-like systems, it may not be available on all platforms or versions. This can limit the portability of Bash scripts.
- Bash scripts may not work as expected on non-Unix systems or require modifications to run on different platforms.
Limited Standard Library:
- Bash has a limited standard library compared to other programming languages. It lacks comprehensive libraries for tasks such as networking, database access, or advanced data manipulation.
- Bash often relies on external tools or utilities to perform complex operations, which can introduce dependencies and compatibility issues.
Lack of Modularity and Reusability:
- Bash scripts can become monolithic and difficult to maintain as they grow in size. Bash does not provide strong mechanisms for modularization or code reuse.
- Reusing code or creating libraries in Bash can be challenging, leading to code duplication and decreased maintainability.

70 comments

It’s here, it’s there, it’s everywhere. The problem with replacing things that work with something “better” is that “better” is subjective, so you end up with a new “better” way every few years, and maintaining existing systems becomes a god awful slog. See the JavaScript ecosystem.
The bash I wrote 10 years ago still works today, and it will still work in 10 more years. The same bash will very likely work on your computer, on a remote server, etc. This is the power of not chasing “better” all the time.
Try running a Ruby or Node program from 10 years ago today on your computer. Now, try running it on a random Linux server.
Please do not take this as a slight against Ruby or Node, or any other high level programming language. Bash compared to those is simply apples and oranges, they are not the same thing.
By all means, if you have a project that requires a Ruby runtime anyway, write operational scripts with Ruby, run them with Rake, etc.
Want a portable script that doesn’t depend on a complex runtime? Use bash.
If bash is too limiting, use Perl. No, seriously. Perl is fine. It is about as ubiquitously available as bash, and the standard library likely has what you need to get the job done. People blindly dismiss Perl because some blog post told them to, usually in the context of writing application code. You’re not writing application code, you’re writing scripts. Would you write an application with bash? No.

What features does bash have that make it so suitable for shells?
You mean, except being automatically available in basically every Linux distribution without having to install any additional software?
- They're asking why it became available everywhere.
  Bash-like scripting has become ubiquitous in operating systems, and it makes me wonder about its widespread adoption despite lacking certain programming conveniences found in other languages.
  
  Because other languages available at that time lacked certain programming conveniences found in bash :) Despite its shortcomings, it's still a very convenient language for running other programs, working with files, and piping output from one program to another, or to a file. Bash was first released in 1989, and I don't know exactly when it passed the threshold for widespread adoption, but I can't think of anything that would've been a better alternative.
  I think OP is also asking "why aren't people switching to something else now" which is a completely different question.
  
  Because it's automatically available in basically every Linux distribution without having to install any additional software.

Bash is fine. Now powershell, that's utter garbage.
- On behalf of garbage, I loudly protest on this attempt to assimilate it to Powershell.
- Bah! I find it hard to believe that if you ever truly used ps that you'd think it was garbage.
  I'm rather fond of it myself.
- I upvoted because fuck microsoft but i was very surprised at how easy it was to script in ps. Outputs are formatted similarly and you can quite easily have exactly the info you want ,how you want, while in bash its a lot more loose. That comes with the open source territory though.
  
  That's about why I don't like it. Commands are too verbose and specific.
  
  This, powershell is pretty nice albeit verbose and TTY in Windows suck and slow (haven’t scripted in PS since 4 years ago so not sure about current state of affairs). Python is my de-facto shell now if I want to do anything shell related.
- .
  
  50k lines is, IMHO, way too large for a bash script. I'd switch to python long before before that. Bash is primarily an *interactive shell. Bash has a lot of nice scripting capabilities (that few are aware of, let alone use) but its primary use is not as a programing language. As far as standard libraries go: sed, awk, grep, curl, netcat, etc... provide plenty of advanced capabilities. That's the Unix philosophy, lots of small utilities that each do one thing well and that work with streams of bytes as i/o. Tie them together with a powerful shell, and an ordinary user can do quite a lot without "programming". Is the Unix philosophy perfect? No. Has it proved to be the most flexible and successful compute environment developed for over 50 years? Yes.
  
  Engineering is tradeoffs.
  A command shell is focused on file operations and starting/stopping applications. So it makes it easy to do those things.
  You can use scripting languages (e.g. Node.js/Python) to do everything bash does but they are for general purpose computing and so what and how you perform a task becomes more complicated.
  This is why its important to know multiple languages, since each one will make specific tasks easier and a community forms around them as a result.
  If I want to mess with the file system/configuration I will use Bash, if I want to build a website I will use Typescript, if I want to train a machine learning model I will use Python, if I am data engineering I will use Java, etc .
  
  I personally wouldn't, because the Unix philosophy should still apply. If you need 50k lines of bash to do something that a collection of existing command line tools already can do, you may need to re-evaluate your needs.
  As @nous@programming.dev said here, POSIX compliance is extremely important. Much of the "real world" infrastructure is still UNIX based, especially in finance. It isn't easy to replace those systems at all, especially a legacy codebase that literally the entire world runs on. COBOL and Fortran applications in banking are still being updated today, despite efforts to modernise systems because they just work and the code is pretty much hardened at this point.
  As always, in every industry, there is a "right tool" for the job. The great thing about the Unix philosophy is, if correctly applied across your stack, it doesn't matter what language you write the tools in. Your bash script is only going to be forwarding that output to something that is suited to handle it. This person sped up their python application by using Rust for one set of functions.

why are you stuck with bash? just write a shebang and then your interpreter in the first line of your script.
i can use bash, python, lua and al kinds of stuff...
i don't understand the question. if you mean, why does my shell only accept bash syntax, if i set my shell to bash, idk. use another shell?
'sh' is kind of the smallest common thing that's available everywhere. so when you got to script something that needs to run somewhere not under your control, you use 'sh'. and that's kind of it works. you'll find something, that's been around for some time, otherwise it won't have spread everywhere. and now you can't replace it in newer products, because there is so much stuff using it.
if that isn't one of your problems, go with my first suggestions and just use python or something like that as your scripting language.
- I'm gonna get crucified for saying this, but... I write a lot of my scripts in PHP. It's just a language that I'm very familiar with.
  
  This isn't reddit, if that tool does the jobs effectively then great.
  
  crucified?! You're awesome mate!
  Plus, on the fediverse, PHP's kinda cool again (I think) ... or at least should be.
  
  ( my oppinion: i personally don't like php and js for being too complicated. they are very easy to begin with, which is a good thing. but too difficult to master. there are soo many weird things going on, so many edge cases to remember, and all the pitfalls that are there to help the beginner but require a professional to pay close attention. i like something strict, maybe with type safety and a compiler that yells at you if you're forgetting something or trying to do something stupid. not silently convert it. )
  Other than that: Whatever floats your boat... I don't think it's wrong to use something like php. i have used it and still use it from time to time. heck, i've even used lua where people start counting with '1'. sometimes it is the correct tool. sometimes it's more important to get a job done, than having it done in a certain way. and of course people like to work with something that they're familiar with. you're probably better off and faster this way.
  
  I've used Js for scripts, especially when JSON is involved.
  My logic is to use whichever language is best for the job out of those I'm familiar with, and can be understood by people who might read it. Then I forget logic and try to use lisps wherever possible.

Bash isn’t the only shell. Most systems can use zsh or fish or tcsh or whatever shell you prefer, if you like that better than bash. You’re gonna have to run a shell if you want to use the terminal, though, you understand that?
Shell scripts are very good for specific tasks. Don’t use them for tasks that are unsuitable. Use python or go or node or c if those are better for your needs. Use the right tool for the job. But also, learn to understand why the industry has been using shell scripts for decades.
- Bash isn’t the only shell. Most systems can use zsh or fish or tcsh or whatever shell you prefer, if you like that better than bash. You’re gonna have to run a shell if you want to use the terminal, though, you understand that?
  Defaults matter. There are a lot of situations where you aren't allowed to install whatever you want. It would be less of an issue if other shells or language run-times came preinstalled. The problem isn't really Bash itself. It's the fact that it's often the only dependable option. But thankfully that's starting to change and you can usually find python preinstalled at minimum.
- As a shell, tcsh was OK if only because it included GNU Readline (or something like it) and, despite being a couple of years younger, kind of beat bash to the punch for that user-friendliness on account of being a largely unchanged drop-in replacement for its predecessor.
  Unfortunately, that also means it's as problematic as its predecessor too: Csh programming considered harmful.
  Not to be used for anything serious.

Why did you write this post in English instead of Esperanto? English is a pretty shit language, especially as the Lingua Franca...
As for your reasoning, it's obvious you don't know what bash is used for. It's purpose is to get shit done quickly and to make working with cli tools easy. For tasks where you need to write more than a few lines of code, or need a feature like cross platform compilation, you should switch to a proper language.
As for why that's good, try writing something along the lines of "curl | jq | cat >> file" in C. For extra points write it out in a single line, in under a minute, while also keeping it perfectly readable.

POSIX compliance for the most part. Which is meant to give you a consistent API you can write software against that will work on any UNIX system. At least that is the theory behind it. It basically acts as a lowest common denominator for UNIX systems. Want to write a script that will work on all systems - bash or a similar shell will almost certainly be present.
A new system can change this if they wanted to. But then they won't be able to take advantage of all the existing bash scripts out there that everyone is writing. Which adds a lot of friction to adoption.
So in a way we are locked into a bash like shell now. Unless you can get a lot of distros to change this shell it is very hard to change it for a single one. And changing something so fundamental across lots of distros will upset a lot of people that are used to bash.
And shells overall are preferred to general purpose programming languages as their main focus is executing other commands and piping input/output from one to another or to/from files. Something most programming languages make a bit awkward as that is not their focus.
There are a bunch of non-POSIX shells, like fish, nushell, ion etc but these will likely not become the default anytime soon due to the inertia that bash has. But luckjly you can install what ever shell or language you want in addition to bash and switch to that for your interactive shell or even scripting if you want to. They just won't become the defaults for a long time - if ever.

I wouldn't use the word "stuck", Bash is a result of over 50 years of technological advancement and experience. It works on servers, remotely, is lean and powerful. It's not that user friendly, because it doesn't need to be.
ZSH and Fish are available on nix systems for a little bit more sugar. I personally don't use raw Bash because I just love what ZSH can do, but I love writing Bash scripts.

Let's see here. We're talking about a thirty-plus-year-old language that was deliberately written as a superset of sh from 1979. Its main purpose is to glue together other command-line programs on 'NIXish systems to automate complex procedures for sysadmins. Using it as a general-purpose programming language, while not impossible, is kinda dumb if you have any other options. Within its original niche, it ain't broke, so there's no need to fix it.
It's a contemporary of Tcl and Perl, not of Python and Java (and sh is from the era where microcomputers mostly used BASIC dialects). Unlike Perl, it didn't bolt on object orientation or other more modern features afterwards—and really, Perl was intended as just the kind of "shell script killer" you seem to be thinking of in your introduction. However, it never completely displaced the shell built-in scripting languages even in the days before it fell out of favour itself. Might be that there's a reason for that.

You're not. Your shell is set in /etc/passwd. Set it to iPython if you want.

Check out xonsh.
- This is so cool. It's exactly what I had in mind when thinking of a modern bash alternative.

Bash adheres to the POSIX standard for a shell, which limits what it can have in terms of UX. If you really want something with a different UX you can try something like fish. Just be warned that if you do, you'll quickly learn how wrong your bullet point about bash not being portable is.

You aren’t. Even if you want to stick with bash, you can run scripts written in other languages. I have a few Ruby scripts. I can run them just like I do bash scripts.

You're missing the point.
Bash is the ducktape of programming languages.
- And *nix shells a perfect example of the KISS principle.

Everyone talks about the fact that Bash is what it is because it is first and foremost an interactive shell, but nowadays some design decisions are just inexcusable in my opinion, like the awful syntax of common programming constructs, the if in particular, that would only benefit from following how every other language works even if they aren't meant as shells.
Some also argue against the non-modularity with the fact that you should use it for only quick and easy stuff, but that's just an excuse, if the language runtime that comes preinstalled in your system had modern features and sane syntax you would stick to that and save yourself from installing Python/Ruby if they're not needed; and it is clear that there is a need for modularity, otherwise plugin managers wouldn't exist, many swear by downloading the scripts directly and sourcing them in the name of "KISS", but that is just silly when there is a good system set in place that makes it actually easier to manage it all.
Then there's the issue of the holy pipelining, that has more or less been overcome by some languages already, this example in Rust shows that it can be easy, so there's no reason why a terse scripting language couldn't achieve the same.
In the end I don't know what's holding the landscape back, I noticed Xonsh that looks very interesting, but I never tried it, I wonder if it is POSIX compliant and if that aspect even is so fundamental to the success of a shell

I think you just need to use the right tool for the job.
Personally bash scripts are fine for any basic comparison operations or just running stuff together like a windows batch file. Maybe checking result codes, searching for processes, selectively killing etc.
Beyond that, but where I expect it to be still a convenience/automation script I use perl (which is where probably most people would now use python, but I'm old). It can still be run from command line, it can access databases, can be OO if you want it to (but generally if I am going that far I move to a faster language) and in general for moderately complex automation I use perl.
If it gets complicated (250+ lines as a general rule) or needs speed, then I'd move onto a proper programming language because now it has become a project.
But, that's just me.

How standard is Python on Unix machines nowadays? I’m guessing some relatively recent version of Python 2 is standard on many machines.
Not sure how many would prefer it, but I’ve always wondered if retiring as a shell language would be a good home for Python. Not sure how well it’d work as a shell without some major changes, but it feels like something could have been done to get to a sweet spot and we could have slowly moved beyond bash to something nicer …?
- I’m guessing some relatively recent version of Python 2 is standard on many machines.
  I hope not. Python2 has been EOL for a long time now and should not be used any more. And I really hate pythons dependency management.
  
  Except this is about Bash ... not bashing Python.
  First, Python2 need not be EOL. I'm clearly talking about some parallel timeline where python as a *nix shell became a project and got traction. Somewhere around ~2000, when python2 was first released, some project works on making the necessary adaptations for a shell and presumably gets that working well.
  Second, we're talking about Bash and Bash scripts, any serious dependency management or package management doesn't really factor into this, it seems you're bashing Python as a fully fledged programming language.
  The weakest point in my imaginary proposal, IMO, is getting python to work as a shell language. As a scripting language it's fine and many are happy using it where Bash might be used. But as a shell, and in scripting too, it's interop with core utils and commands is pretty suboptimal. Ergonomics would at some point require something to be done about that and I don't know what a successful version of that would look like. If we're talking about a POSIX standard, maybe all that'd be necessary would be a standard library that wraps everything needed? Or a nicer object/function/syntax for calling subprocesses, which various third party libs have tried obviously. The Xonsh project also comes to mind ... a superset of python, which is a great idea IMO but maybe undermines the idea that python2 is everywhere already.

Because for what it was made to do and what I want to use it for, it's utterly ideal. It's easy, it's direct, it works seamlessly with any program's command line, and I can run anything network-wide on any linux machine on my network out of box with no fiddling around. No check for version, no missing packages to hunt up, no libraries to download and verify; I type, I save, it runs, I'm done. If I need to integrate command line tools on six separate programs and/or five to eight scripts in two languages to do a stat/resource/network check on my Linux machines, I can do them all from one script and I can do it to six separate machines over ssh in a loop in under 200 lines of code and throw the results up on a webpage in apache with another thirty if I want to make it pretty in html. Then I set it to a cron job to run once an hour and forget it for months; it keeps on keeping on, I just check that webpage to see everything is fine, in separate tabs even. And I can do all that very very very fast and literally out of box; if I add a brand new machine, all I do is copy my base bash library over and set permissions and it's ready to go.
Those scripts will always work, on every linux machine, every time, in the same way; they will run in ubuntu, solus, fedora, arch, debian, raspberry pi, probably slackware I haven't checked, the scripts do not care. Ones I wrote ten years ago are still running just fine.
Bash is kind of like the general of my script and cli army; she does not need to know everything herself, she just needs to organize the troops to do their jobs, and tell me if someone's slacking off because python decided to be a dick about a package or php is being cranky or apache just won't speak to anyone no idea wtf is going on there or otbr vanished into the ether or all my wifi drivers are in revolt after an update. She does not stress me at all; she is the finder of my stresses before the drama hits critical, and this is why she is my favorite.

I know I am reviving an old thread, but my philosophy is that posts are timeless and age should not be a reason not to respond.
Currently I am in the project of learning Rust and Raku, because I am interested in becoming a better systems programmer and I want to be able to do things for my computer without hitting a wall when a solution does not exist, or simply to master my second home.
This is a mindset issue. There is a lot of legacy opinions on how to use your computer, but never forget it is YOUR computer. I say never worry about something being portable to others. What you make will be portable to you, and that is all that matters. Make your computer yours. If someone wants to use your computer but can't, isn't that a win?

For the same reason GNU is still using lisp.
Software conservatives and software elitists. They refuse to evolve and for that reason we're stuck with bash.

70 comments