SSH Agenting and other niceities
Jul. 18th, 2007 05:34 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
tugrikThis post is for folks who use SSH and SCP a lot in their daily 'net travels. It's seriously old-news and old-hat for decent Unix folks, but I find it nifty enough to share. For those who don't shell around a lot, just skip on to the next post. :)
I manage six servers as part of the personal hosting business that![[livejournal.com profile]](https://www.dreamwidth.org/img/external/lj-userinfo.gif)
revar and I are in. At work, far more. Most all of it is done via command-line, using SSH (secure shell) to bounce from machine to machine. Additionally, I use SCP (secure copy) to toss files between boxen while working on them. As an OSX (mac) user, I use the built in shell to do this, avoiding most of the weird GUI clients. It's very rare that I have a need to 'drag and drop' files; command-line is just quicker for me. Each time I go to a new box, I have to type a password specific to that box and remember them all. Either I'm stuck with keeping a bunch of passwords in my head or I have to make sure all the servers have the same password; either method has its own pitfalls. There's a third way, though: using public-key encryption.
This involves creating a key with a password and distributing the public half of the key to all the servers you want to attach to. After this one-time annoyance, when you attempt to attach you will get asked for your key's password instead of the account on the server. All authentication is done on your end by the ssh client, giving you a form of 'single sign-on' for all the boxes. Now combine this with something called an "ssh agent" and the OSX keychain services, and it suddenly becomes very handy.
Once set up correctly, you can ssh or scp to any of your servers without username or password. The system will authenticate securely all the way back to your main OSX account's keychain. As long as your keychain is unlocked (the normal state of things when you are logged in and active on your mac's desktop) everything 'just works'. Even better, if you go from one server to another (a 2nd hop), it will still authenticate all the way back. Using secure tunnels for shell and copy becomes completely transparent.
The setup is a bit annoying, I'll warn you. But once done, it's soooooo nice. There are three major steps: generating a key, distributing that key and installing the agent. I've written them up below, along with the extra bits needed to make the config work. You can also add some DNS lookup trickery to make life easier, which I'll put in at the end.
Step 1: Generating a key
It's best that for any group of servers you generate a key specific to them. I have one key for my personal servers, one for my production servers at work, and one for my QA servers at work. This way if a key ever gets compromised, I only have to fix that particular range of machines and not every one of them at the same time. It's up to you how (if at all) you segregate these.
Do this for as many keys as you think you will need. Keep a copy of the private key somewhere safe/offline, just like you would any important password or account number. An attacker would need this key *and* your passphrase to break in to your account. so it's decently secure.
Step 2: Distribute the public key
Very important: make sure you only do this part with the public half of the key -- the file that ends in .pub. :) Do this once on every server you wish to access.
The permissions are very important. Many times an ssh daemon will barf up a security violation if the file and directory aren't set exactly to 400 and 700 respectively.
Step 3: set up your local ssh config file
Back into your local terminal you go. Now we tell your computer what machines to use what keys on.
For advanced users, if your machines follow a pattern of names you can use wildcards instead of individual Host so-and-so entries. Just type 'man ssh_config' to get details on wildcarding if you want it.
Step 4: Testing!
At this point basic SSH/SCP via keys should be up and running. To test this, first completely exit all copies of Terminal and then run it again. Then just type "ssh username@servername.domainname.com" for your server.
If it doesn't work, it will ask you for your password at the server as it always has. If it does, the password prompt will ask for your key's password instead. In my example:
Now just enter the passphrase for your key and you should be logged into your server. Test this and make sure it works for all the servers you want to attach to. If any of them don't work, check to make sure you copied the public key to the server properly and the authorized_key file and .ssh directories are properly chown'd/chmod'd appropriately, as shown in the previous step.
As a side note: if your server account names are the same as your desktop's account name, you don't have to put in the 'username@'. Just 'ssh servername.domainname.com' will do. Because of this, I keep my account name the same on most of my boxes, so that I don't have to constantly remember which account name I use on which.
Step 5: SSH Agent setup
Now comes the fun part. We're going to load an OSX-compatible ssh key agent that will bind your new ssh keys into the OSX keychain. This way your server logins will be tied to your desktop's login, making ssh/scp use a lot easier.
For advanced users, you can fuss with settings in the Security tab to your liking. For most folks, the defaults are just fine. Now you need to do something you almost never have to do on a Mac: reboot! This will let the SSH agent come up as you log in and be able to talk to both the keychain and the SSH daemon.
Step 6: Use
Once rebooted and logged in, you should see a little icon of grey keys on your title bar. That's the agent running, doin' it's thing. Open up a terminal window and attempt to ssh to one of your servers.
Instead of the shell asking you for a server password or a key passphrase, the SSHKeychain app should now pop up a window and ask you for the key passphrase instead. Turn on the checkbox for "Add key to apple keychain" before pressing OK. The agent will do this only once for each key you created - don't be surprised if it asks you for all of them the first time. Once they've been entered into the Apple Keychain, you won't have to do it again.
If you put the password in correctly, you should be connected to the server via ssh now. Log off and try it again -- this time it should work without asking you anything. If it does, congratulations! You've successfully got your keys and agent set up!
You can now try things like "scp localfile username@remotehost:/remotefile" and it should just run without asking you for passwords. You can do it between multiple machines as well. The nifty part is the agent, though. Try this stunt:
It should just work, no questions asked, as long as both servers are within your keyspace. What's happening is "ssh agenting": when the sshd on the second server asks the ssh client on the first server for authentication, it knows to pass that request back to the 'agent' -- there on your laptop. The agent hears the request and asks the mac keychain for the proper private key. As long as the mac keychain is unlocked, authentication is approved and passed back up the chain. Your local passwords and public key never leave your machine, but server2, two hops out, gets told you're OK to log in. Nifty, no?
To lock off access, just lock your screen, log off, or manually lock the Apple keychain using the "Lock Apple Keychain" item from the little keys icon on the title bar. None of the ssh agent functionality will work until you manually unlock, just like you would for any secure Apple service. You'll get a gui pop-up for your main login, and that's it. As a good network user, you DO lock your computer when you're away from it, right? Right? :)
As a side advantage, this system has two levels of fallback. If the SSHKeychain agent isn't running, the shell will ask you for your key's passphrase. Answering it will log you in just fine. If the keys are messed up, compromised, deleted or revoked for any reason -- or if you're using a machine that simply doesn't have your private key half on it -- then ssh will ask you for your server username's password as it always has. Putting in this infrastructure, therefore, won't bar you from using the older methods. It's pretty safe.
Bonus tip: DNS search domains
All of my personal servers end in "fbrtech.com". server1.fbrtech.com, server2.fbrtech.com , etc. In the System Preferences, Network Preferences, (pick your network adapter) TCP/IP prefs, I just added "fbrtech.com" to the "Search Domains" box. Now I can just "scp server1:file server2:file" instead of "scp server1.fbrtech.com:file server2.fbrtech.com:file". This saves a lot of typing when you're handling a ton of different copy events.
~~~
Hope this helps some of the other shell-heads I know out there. Personally it's made my job a ton easier. With the agent and synchronized account names, it's so much easier to just "scp machine1:file machine2:file" than have to put in usernames and passwords for each bit, each time. It also lets me do automated scripting that I can launch with a single password instance even if there are thousands of individual scp commands involved.
I've not banged out a silly tutorial like this in ages. Let me know what you think. Thanks!
I manage six servers as part of the personal hosting business that
revar and I are in. At work, far more. Most all of it is done via command-line, using SSH (secure shell) to bounce from machine to machine. Additionally, I use SCP (secure copy) to toss files between boxen while working on them. As an OSX (mac) user, I use the built in shell to do this, avoiding most of the weird GUI clients. It's very rare that I have a need to 'drag and drop' files; command-line is just quicker for me. Each time I go to a new box, I have to type a password specific to that box and remember them all. Either I'm stuck with keeping a bunch of passwords in my head or I have to make sure all the servers have the same password; either method has its own pitfalls. There's a third way, though: using public-key encryption. This involves creating a key with a password and distributing the public half of the key to all the servers you want to attach to. After this one-time annoyance, when you attempt to attach you will get asked for your key's password instead of the account on the server. All authentication is done on your end by the ssh client, giving you a form of 'single sign-on' for all the boxes. Now combine this with something called an "ssh agent" and the OSX keychain services, and it suddenly becomes very handy.
Once set up correctly, you can ssh or scp to any of your servers without username or password. The system will authenticate securely all the way back to your main OSX account's keychain. As long as your keychain is unlocked (the normal state of things when you are logged in and active on your mac's desktop) everything 'just works'. Even better, if you go from one server to another (a 2nd hop), it will still authenticate all the way back. Using secure tunnels for shell and copy becomes completely transparent.
The setup is a bit annoying, I'll warn you. But once done, it's soooooo nice. There are three major steps: generating a key, distributing that key and installing the agent. I've written them up below, along with the extra bits needed to make the config work. You can also add some DNS lookup trickery to make life easier, which I'll put in at the end.
Step 1: Generating a key
It's best that for any group of servers you generate a key specific to them. I have one key for my personal servers, one for my production servers at work, and one for my QA servers at work. This way if a key ever gets compromised, I only have to fix that particular range of machines and not every one of them at the same time. It's up to you how (if at all) you segregate these.
- Open a terminal window.
- type "cd ~/.ssh" . This puts you in your ssh key storage for your account.
- type "ssh-keygen -t dsa" to start the key generation process with a 1024 bit DSA key.
- Put in a name you'll remember. I'll use "myservers_dsa_key" for this example.
- It will ask you for a passphrase. The longer, the better... but at least 8 characters, I think it is. Don't forget this passphrase or the key will be useless!
- It will finish and write two files to your directory, a public and a private key. The public key will be "myservers_dsa_key.pub" and the private is just "myservers_dsa_key" in my example.
Do this for as many keys as you think you will need. Keep a copy of the private key somewhere safe/offline, just like you would any important password or account number. An attacker would need this key *and* your passphrase to break in to your account. so it's decently secure.
Step 2: Distribute the public key
Very important: make sure you only do this part with the public half of the key -- the file that ends in .pub. :) Do this once on every server you wish to access.
- Log into the unix server using your account.
- Change directory to .ssh as before (cd ~/.ssh)
- Look for a file called 'authorized_keys'.
- If the file is there, we will append onto it. If not, we will create it. Simply copy your public key (cut-paste works, as does scp'ing it over to the box first) into that file, starting on its own line. Use "vi" or whatever you normally use. You're only doing this tutorial if you're a shell person, right? :)
- Make sure the file is chown'd to your user and group (chown authorized_keys username.groupname -- like for me, it's usually "tugrik.tugrik"). Do the same for the parent .ssh directory (chown ../.ssh username.groupname) if it's not already.
- Make sure the file is set to the right permissions (chmod 400 authorized_keys)
- Make sure the directory itself is set to the right permissions (chmod 700 ../.ssh)
- Log out of the box.
The permissions are very important. Many times an ssh daemon will barf up a security violation if the file and directory aren't set exactly to 400 and 700 respectively.
Step 3: set up your local ssh config file
Back into your local terminal you go. Now we tell your computer what machines to use what keys on.
- Re-open terminal and change directory to ~/.ssh again.
- Edit or create the file "config" - all lower case. I just use "vi". (vi config) but you can use textedit or whatever you want.
- For each server you want to use a key on, make a 'host' entry containing an IdentityFile keyword, like so:
Host myserver1 myserver1.mydomain.com
IdentityFile ~/.ssh/myservers_dsa_key
Host myserver2 myserver2.mydomain.com
IdentityFile ~/.ssh/myservers_dsa_key
Continue on for every server you want covered. If you created multiple keys for different boxes, just make sure the right key name is in the IdentityFile line. - Finish the file with a catch-all that turns on agent forwarding:
Host *
ForwardAgent yes
This will allow us to use an SSH agent, shown lower in this tutorial.
For advanced users, if your machines follow a pattern of names you can use wildcards instead of individual Host so-and-so entries. Just type 'man ssh_config' to get details on wildcarding if you want it.
Step 4: Testing!
At this point basic SSH/SCP via keys should be up and running. To test this, first completely exit all copies of Terminal and then run it again. Then just type "ssh username@servername.domainname.com" for your server.
If it doesn't work, it will ask you for your password at the server as it always has. If it does, the password prompt will ask for your key's password instead. In my example:
Enter passphrase for key '/Users/username/.ssh/myservers_dsa_key':
Now just enter the passphrase for your key and you should be logged into your server. Test this and make sure it works for all the servers you want to attach to. If any of them don't work, check to make sure you copied the public key to the server properly and the authorized_key file and .ssh directories are properly chown'd/chmod'd appropriately, as shown in the previous step.
As a side note: if your server account names are the same as your desktop's account name, you don't have to put in the 'username@'. Just 'ssh servername.domainname.com' will do. Because of this, I keep my account name the same on most of my boxes, so that I don't have to constantly remember which account name I use on which.
Step 5: SSH Agent setup
Now comes the fun part. We're going to load an OSX-compatible ssh key agent that will bind your new ssh keys into the OSX keychain. This way your server logins will be tied to your desktop's login, making ssh/scp use a lot easier.
- Download the agent from the SSHKeychain site. For the lazy, here's a direct link to the .dmg.
- Drag the application icon (which looks like a puffer-fish) into your applications directory. Close the .dmg image and eject it.
- Run the application by double-clicking on it (from the app directory, not the .dmg)
- Open up Preferences for the application (command-comma or pick it from the drop-down menu)
- Turn on 'check updates on startup' if you wish; that part's your call.
- Under "display", set it to "display SSHKeychain: Status Bar" so it won't annoy you as much.
- Under "SSH Keys", press the + button and add in all the keys you made in the first step. Navigate to the .ssh dir (if it doesn't already put you there) and select the private keys (the ones that do NOT end in .pub!). Add each one into the application.
- Under "Environment", turn on the checkbox that says "Manage (and modify) global environment variables
- Find the pufferfish application icon on the dock. Right-click (command-click) the icon and select "Open at Login"
For advanced users, you can fuss with settings in the Security tab to your liking. For most folks, the defaults are just fine. Now you need to do something you almost never have to do on a Mac: reboot! This will let the SSH agent come up as you log in and be able to talk to both the keychain and the SSH daemon.
Step 6: Use
Once rebooted and logged in, you should see a little icon of grey keys on your title bar. That's the agent running, doin' it's thing. Open up a terminal window and attempt to ssh to one of your servers.
Instead of the shell asking you for a server password or a key passphrase, the SSHKeychain app should now pop up a window and ask you for the key passphrase instead. Turn on the checkbox for "Add key to apple keychain" before pressing OK. The agent will do this only once for each key you created - don't be surprised if it asks you for all of them the first time. Once they've been entered into the Apple Keychain, you won't have to do it again.
If you put the password in correctly, you should be connected to the server via ssh now. Log off and try it again -- this time it should work without asking you anything. If it does, congratulations! You've successfully got your keys and agent set up!
You can now try things like "scp localfile username@remotehost:/remotefile" and it should just run without asking you for passwords. You can do it between multiple machines as well. The nifty part is the agent, though. Try this stunt:
- Log on to your first server (ssh server1)
- from within the first server's shell, log on to your second server (ssh server2)
It should just work, no questions asked, as long as both servers are within your keyspace. What's happening is "ssh agenting": when the sshd on the second server asks the ssh client on the first server for authentication, it knows to pass that request back to the 'agent' -- there on your laptop. The agent hears the request and asks the mac keychain for the proper private key. As long as the mac keychain is unlocked, authentication is approved and passed back up the chain. Your local passwords and public key never leave your machine, but server2, two hops out, gets told you're OK to log in. Nifty, no?
To lock off access, just lock your screen, log off, or manually lock the Apple keychain using the "Lock Apple Keychain" item from the little keys icon on the title bar. None of the ssh agent functionality will work until you manually unlock, just like you would for any secure Apple service. You'll get a gui pop-up for your main login, and that's it. As a good network user, you DO lock your computer when you're away from it, right? Right? :)
As a side advantage, this system has two levels of fallback. If the SSHKeychain agent isn't running, the shell will ask you for your key's passphrase. Answering it will log you in just fine. If the keys are messed up, compromised, deleted or revoked for any reason -- or if you're using a machine that simply doesn't have your private key half on it -- then ssh will ask you for your server username's password as it always has. Putting in this infrastructure, therefore, won't bar you from using the older methods. It's pretty safe.
Bonus tip: DNS search domains
All of my personal servers end in "fbrtech.com". server1.fbrtech.com, server2.fbrtech.com , etc. In the System Preferences, Network Preferences, (pick your network adapter) TCP/IP prefs, I just added "fbrtech.com" to the "Search Domains" box. Now I can just "scp server1:file server2:file" instead of "scp server1.fbrtech.com:file server2.fbrtech.com:file". This saves a lot of typing when you're handling a ton of different copy events.
Hope this helps some of the other shell-heads I know out there. Personally it's made my job a ton easier. With the agent and synchronized account names, it's so much easier to just "scp machine1:file machine2:file" than have to put in usernames and passwords for each bit, each time. It also lets me do automated scripting that I can launch with a single password instance even if there are thousands of individual scp commands involved.
I've not banged out a silly tutorial like this in ages. Let me know what you think. Thanks!
![[identity profile]](https://www.dreamwidth.org/img/silk/identity/openid.png){kind=small}
no subject
Date: 2007-07-19 12:59 am (UTC)If only windows had a decent login-related keychain.. boo microsoft.
no subject
Date: 2007-07-19 01:12 am (UTC)no subject
Date: 2007-07-19 01:26 am (UTC)no subject
Date: 2007-07-19 01:15 am (UTC)no subject
Date: 2007-07-19 01:15 am (UTC)no subject
Date: 2007-07-19 01:26 am (UTC)no subject
Date: 2007-07-19 01:32 am (UTC)no subject
Date: 2007-07-19 01:30 am (UTC)Or something to that effect. Worked like a charm. This is just a simple example, but it's something I've found very handy now and then. I use it in a similar fashion to pipe text from TF on the Sun machine to a TTS program on the Mac so I can have incoming text from MUCKs read aloud in real time- handy if I'm working on a model and can't always divert my eyes to the screen.
no subject
Date: 2007-07-19 04:17 am (UTC)cd /hda1
ssh -x -c blowfish root@masterserver "cd / && tar -cl --numeric-owner -f - . " |tar -xpf -
chroot /hda1
grub-install /dev/hda
rm -f /etc/ssh/*key*
sync
/sbin/reboot
Whee! New server! Much faster than installing from scratch. ;)
no subject
Date: 2007-07-19 01:46 am (UTC)no subject
Date: 2007-07-19 06:05 am (UTC)I've got a few dozen machines at work, all with root needing to be SU'ed to, and accounts that are not only random strings, but passwords are purely random as well. (Take -THAT- dictionary attackers.) I've done this as well as the passwordless SSH links between servers. (Doing the same above, but not assigning a passphrase key to the private key.) Pretty good for userland accounts who's sole reason for existance is to pass files from server to server.
I have written up (With screenshots) how to do basically that whole deal with PuTTY SSH under Windows as the client end, and Linux/FreeBSD on the server end. IF anyone is looking for that, let me know.
no subject
Date: 2007-07-19 09:49 am (UTC)I run freeBSD and windows boxes on my site
ssh agent
Date: 2007-07-19 10:36 am (UTC)You mean:
chown username.groupname authorized_keys
Since the username, group name has to go first. (And if you want to be portable, you want "username:groupname", since the "." approach is a BSD specific thing.) And BTW, the "chmod 700 ../.ssh" can actually be written as "chmod 700 ." since you're already in the right directory -- "." means "the current directory". Also from memory you don't have to reboot -- just log out, and then log in again. But it's been ages since I set up SSH agent on my Mac so I'm not certain.
But yes ssh agent authentication is insanely useful. I don't remember how I coped without it (I've been using it for well over 10 years). The main thing to remember is that when your key is unlocked, anyone with access to your account/keyboard can log into the other machines as you, without any prompting. Which can be bad. So be sure to lock your screen (or door, if that's sufficient) when leaving your computer alone anywhere that someone you don't completely trust can get at it.
Ewen
no subject
Date: 2007-07-21 12:05 am (UTC)Strangely, this isn't entirely true. I just spent 15 months watching people smarter than I (and at least one with better Unix skills than I, and certainly much more raw experience) struggle with day-to-day usage of SSH without an agent. They all refused repeated offers to help them out of their misery.
I've gotten the distinct impression that this is pretty much the norm elsewhere, to...