I’ve been using ClusterSSH (cssh at the command line) to simultaneously interact with multiple remote systems for years. If you’re not familiar with why you might want to do this, have a quick glance at Kyle Rankin’s Hack and / article on ClusterSSH. ClusterSSH has been a very useful tool for me and it has some handy features such as being able to select and toggle groups of systems to send input to but I’ve always found the UI clunky and I wanted something that didn’t rely on using a GUI.

Enter tmux-cssh – it’s a wrapper that accomplishes the same thing but using tmux panes. The easiest way to get it is to git clone https://github.com/dennishafemann/tmux-cssh.git and run ./tmux-cssh (or copy the file to a location in your path). Here is an execution using the same username on each host, as hijacked from the documentation:

tmux-cssh -u my-user-name my_server 1.2.3.4 11.22.33.44 my_second_server my_third_server my_and_so_on_server

In my case, I’m using a custom port and have a couple servers that require a different username, so I’m launching it with the -sc option with quoted ssh arguments, such as:

tmux-cssh -sc "-p 1234 my-server-name" "-p 4321 differntuser@differenthost.com"

With either method, a tmux session will start in tiled mode with the number of remote servers you’ve connected to. At this point, anything you type will go to every server simultaneously. Once you’ve logged in to all of the systems you can now do fancy things like vim ~/.ssh/authorized_keys (which would select the home directory on all remote servers even if they have different usernames) and paste input to all of the remote servers at the same time. If one of the remote servers is out of sync or you need to send text to only one of them, you can hit your tmux escape sequence such as C-b (Ctrl+b) followed by : and type:

set-window-option synchronize-panes

This will change tmux to its normal behavior of typing in only one window at a time. It gets cumbersome typing the command out, so add the following to your .tmux.conf file:

# Toggle tmux-cssh all with C-b e
bind e set-window-option synchronize-panes

You can substitute a different key instead of lowercase e, but this one is free by default and I remember it with whether I want to send input to (e)very window or not. If you need to do something a bit more fancy you can also use tmux’s built-in window and pane movement commands to move panes to another window and operate on a group of systems at the same time. The synchronize-panes setting is unique to each window you’re looking at, meaning whatever you type only affects the panes / systems you can currently see.

This is already extremely powerful, but I’ve come up with another unusual use case – running the same commands on multiple local and remote sessions. The use case is when you need multiple instances of things that aren’t all remote systems – in my case, I’m using ipmitool to control multiple remote systems. My execution looks like this:

./tmux-cssh -sc "-p 1234 remotehost1" "-p 4321 remotehost2"

This will launch two remote SSH sessions. Create a new split pane with C-b % or C-b ” (these will be additional local panes on the system you’re running on). Once you have enough panes for your needs, hit C-b and the spacebar a few times to re-tile the windows. At this point, you can do something like this:

ipmitool -I lanplus -U root -P root -H 10.0.10.10 shell

Before you hit enter, back up to the last octet of the IP address and toggle synchronize panes, then type in the final digit of the individual remote hosts you want to connect to (say, 10.0.10.101, or rewrite the entire host as needed). This example is a bit convoluted, but can be of use if you have the same tool you need to run from two potentially separate networks. If all of your hosts are local you can just run tmux as normal and split panes then toggle synchronization. Once you are at an ipmitool prompt you can do things like send power reset to reset all of the IPMI-connected servers, or type sol activate to open up a Serial-Over-LAN interface on all of them.

There are a lot of other potential uses I can think of for some of these concepts and methods and I’m sure I’ll come up with more uses over time.