Gfrktyl

I’m connecting to an Amazon Web Service (AWS) EC2 instance in a virtual private server’s private subnet via a NAT instance and getting the following error:

Permission denied (publickey)

This is happened after I have connected to the NAT and am trying to SSH to the private subnet EC2 instance.

Procedure:

1. 
   

   Define host in ~/.ssh/config with the following :

   
   
   

   Host my_aws_nat
   
   Hostname xx.xx.xx.xx
   
   User ec2-user
   
   IdentityFile /location/of/my/aws/key_pair.pem
   
   ForwardAgent yes
   
   

   
   



2. SSH to NAT instance via ssh my_aws_nat (which is successful)




3. SSH to instance in private subnet ssh ec2-user@10.0.X.X - which is when I get the error

I’m able to ping my private instance from my NAT with ping 10.0.X.X. So I’m fairly sure it’s not security groups issue. It looks like it’s an agent forwarding issue.

Currently the instance I’m connecting to uses the same key pair as the NAT instance (in learning mode).

The other way I’ve tried is to connect to the NAT with:

ssh -A ec2-user@XX.XX.XX.XX -i key_pair.pem

Which again, connects correctly to the NAT but gives the same error when connecting to the private instance.

Do I have to use ssh-agent on Mac OS X?

Or shouldn’t specifying ForwardAgent yes in /.ssh/config do the same thing ?

As per this answer and this guideline

I needed to add the key_pair.pem to the OSX ssh agent as follows:

ssh-add -K /path/to/key_pair.pem

(in my case it didn't ask for a passphrase)

After this, everything worked fine using both methodologies outlined above.

So to answer the question:

Q: Do I have to use ssh-agent on Mac OS X to log in to a private subnet instance through a NAT/bastion host?
A: YES

For me it helped to remove the bastion's (NAT instance's) public key from ~/.ssh/known_hosts.
If the key of the remote bastion host changed (which may happen quite often if you allocate EIPs to new instances) and in addition have StrictHostKeyChecking no set in your ssh_config, AgentForwarding will be disabled on the bastion host.
You'll also find a warning saying so if you login to the bastion host. For my part, I've just not read it -.-

So, delete the key and connect again, the current key will be added to the known_hosts file and you'll be able to connect to the instance in the private subnet.