Choosing Shell script vs Python vs Node.js vs PHP for Server-side Command Line Scripting

After decades of improvements in graphical user interfaces, the Unix command line interface still remains the most powerful way to get things done for a lot of workflows. This is due to many reasons: the difficulty of quickly and precisely moving the mouse cursor compared to the ability of touch typists to select exactly what they want (even when it's via hotkey combinations), but mainly (IMO) the expressive power of composing separate Unix commands into one pipe-separated command operating on standardized files (usually text).

It's also easy to write your own Unix commands which can then be part of this Unix command ecosystem. Dealing with input and output is trivial and therefore inclusive to all users. And today there are a number of popular languages available to Unix users for writing their command line scripts.

I will consider 4 commonly known languages and when each one is well-suited for writing Unix/Linux/OSX command-line scripts. Thus you can decide when to use Shell scripting and when to use Python/Node/PHP.

Shell script (sh, bash, ksh, etc.)

  • 1a. Advantages: If you are already comfortable using your shell, then with a bit of knowledge of conditionals (if - then - fi, while - done, for - in - do - done, switch - case - esac) and variables (arguments passed in as $0, $1-$9, variables set in your environment, etc.) you can write full programs. When arranging some shell commands you've already run, shell scripting is the simplest approach to a working MVP. So this is great for those who're often in a shell already anyways and can comfortably pipe commands through other commands, and have long one-liners which can even include conditionals (despite becoming hard to read on one line) or the implicit if/then using return-value short-circuiting with two commands separated by && or || (e.g. s foo || echo 'foo not found'). Much of Unix and Linux systems are patched together with shell scripts.
  • 1b. Disadvantages: Shell relies on Unix commands or a handful of "builtins" (many are replacements of Unix commands) as its "functions" and it's hard to do much in the language itself, like manipulating complex structures of data, without Python's dicts or JavaScript's objects, and even multidimensional arrays are a mess in shell. The language itself won't help in manipulating any kind of binary data and it doesn't expose access to Unix system calls. Doing lots of heavy data manipulation, which might also need to call out to other programs many times, will perform poorly compared to doing it in a single "real" language. You can still use some other Unix tools for basic hex editing, just like you'll have to call on sed & awk (themselves powerful command line programming languages which are worth investing time in learning to supercharge your command line abilities) to do more than basic text processing of files. But if you are comfortable with basic Unix commands (or their GNU versions) then they are a big advantage when writing shell scripts.

There are notable differences between shell syntaxes. Before, people aimed for compatibility with Bourne or POSIX sh or ksh but bash is now ubiquitous on Linux distributions and OSX. Not only ubiquitously available, bash is also the default shell.

Python

  • 2a. Python as a language can elegantly do so many things which require hackish workarounds in a shell script. Python can do things like async i/o which bash can't do at all. There are modules for most needs, from manipulating common file types to doing advanced scientific programming. And ipython is a nice interactive "shell" for writing and running some bits of python on the fly. You can easily go from the command line to building GUI desktop apps or web apps with database backends. It's a versatile and easy to write language.

  • 2b. A downside to Python is a dependency on modules for heavy programming. But you can still do a lot of basic i/o and file munging with the core Python modules. There's always a bit of boilerplate to import things like "sys". Python is fairly ubiquitous but you are most likely to find Python 2.7 (or even older) rather than Python 3 pre-installed. It's recommended to use pip to manage installing Python modules but pip itself isn't ubiquitous. These are problems if you want to write scripts that can be run elsewhere.

Node.js (JavaScript)

  • 3a. Many people who are web developers are already familiar with JavaScript. Many are also comfortable opening a JavaScript console (e.g. Firebug or Chrome Develoepr Tools) and executing a line of JavaScript ad hoc. Now they can use that knowledge on the command line with node.js and the command line interactive node JavaScript interpreter. There is a large and active community developing node modules (npm), mostly helpers for building more node stuff or for building servers and web apps. And node itself exposes apis to do things which the browser version of JavaScript wouldn't let you - like manipulate files and create network servers. You can connect directly to databases in order to build db-backed content websites and a server to run the site. Quite powerful, no?

  • 3b. The downside to doing file manipulation with node.js is that all those i/o calls are async. That's good if you want to keep doing things in the main thread while your file reads/writes are blocked or processing. But you are probably not used to coding this way. And although the JavaScript language (or ECMAScript) in Node.js and in your browser are very similar (or the same if Chrome) there are some differences in how some JavaScript libraries work and in the environment itself - it's not DOM-based. Node's strength is in writing asynchronous network servers which don't block while serving many connections. Overall, for writing scripts to do stuff from the command line node.js probably isn't your best option unless the only language you know is JavaScript. Like Python, there will also be some boilerplate for even simple input-processing programs, and dependency-management problems with Node modules and their versions.

PHP

  • 4a. PHP started life with very limited abilities, and wasn't designed to be a general purpose programming language at all. But at some point, this Apache web server module separated into a command line version. Some PHP developers are including PHP-based build scripts along with their PHP software.

  • 4b. Unfortunately, unless you are on a web server and the web server runs PHP, you might not have php-cli. And PHP as a tool for cli still suffers from being designed first for the web as part of a web server. Nowadays, PHP developers are distributed PHP modules and using tools like composer to bring them into their projects. And so you'll have the same problems as Node/Python if you need to include package management in your simple script. There should be less boilerplate though if just using the many main PHP functions.

Example of using PHP on the command line: php -r 'echo "Hello World\n";'

You can also have a .php file with #!/path/to/php as the first line (shebang, the convention to execute the file using the given interpreter).

Others:

  • Ruby: Most people know Ruby from Ruby on Rails. Some then become comfortable with using it for everything. One problem with Ruby command line scripts is the dependency on specific versions of ruby gems, and dealing with the resulting Ruby Gem Hell (similar to DLL Hell). But basically, same arguments apply as for Python.

  • TCL: Most kids have never heard of tcl and they are lucky and not missing anything. Tcl in the past had 2 uses: scripting Expect, and making toy GUI apps for Linux+X11 with tcl/tk. Like PHP, it suffers from not having designed as a proper language in the beginning. It is possible to make things like websites with Tcl but nobody in the 21st century should do this.

  • Perl: Perl used to be popular among sysadmins-cum-developers. There are some legacy software packages still around written in Perl. It's more powerful than Shell, but nobody in the modern age should be learning it except to try and understand preexisting Perl scripts, which is something notoriously difficult even for experienced Perl developers.