You sit down to push code to your work github account. Moments later, you need to update a personal github account repository. The terminal returns “Permission denied (publickey).” This frustration is common when managing different git accounts from one laptop.

You might manually swap keys or use separate machines. These workarounds waste time and create errors. A better solution exists. You can configure multiple ssh keys to work together. Generate a unique new ssh-key for each service. Then, create a centralized config file. This file tells your system which key to use for each host. The setup eliminates manual switching. You gain a permanent, streamlined workflow for all your remote connections.

How to Configure Multiple SSH Keys on Your System

All your authentication files must live in the ~/.ssh directory on your local machine. This folder stores both private and public key pairs. Keeping everything in one place simplifies management and ensures your client can locate the right credentials automatically.

Generating Unique Keys for Each Service

Start by creating a distinct key pair for every remote service you use. The command below generates a new ssh-key with a custom filename:

ssh-keygen -t ed25519 -C "work@email.com" -f ~/.ssh/gitlab_work

The -f flag specifies the output file name. Without it, the system defaults to id_ed25519, which would overwrite any existing key. Choose descriptive names like github_personal or gitlab_work to identify each key’s purpose at a glance.

Ed25519 stands as the recommended algorithm for new keys. It offers superior security, faster performance, and smaller key sizes compared to RSA. Most modern Linux distributions and OpenSSH versions (6.5 and later) support it fully. However, some older systems or specific cloud providers may require RSA instead. If you need RSA for compatibility, generate it with at least 3072 bits, preferably 4096.

When the command prompts you for a passphrase, enter one. This extra layer protects your private key if someone gains access to your machine. You will enter this passphrase when connecting, unless you configure an agent later.

Adding Public Keys to Remote Hosts

After generating each key pair, you must register the public key with its corresponding service. For your work github account, navigate to Settings → SSH keys and paste the contents of the .pub file. For a work project repo hosted on a company server, you might append the key to ~/.ssh/authorized_keys on that machine.

The ssh-copy-id utility simplifies this process for remote servers. Run ssh-copy-id -i ~/.ssh/gitlab_work.pub user@server.com to append your public key without overwriting existing entries. This command works seamlessly when you manage multiple keys across different machines.

Before configuring the central config file, test each key individually using the -i option. A success message confirms the key works correctly. This quick check isolates any issues before you build the full configuration.

Managing Multiple SSH Keys with the Config File

The ~/.ssh/config file serves as the command center for your entire setup. This plain-text file tells your system exactly which key to present for each remote host. You create rules that match hostnames, then assign specific identity files to those rules. The file eliminates guesswork and manual key selection.

Setting Up Host Aliases and Identity Files

Open the config file with any text editor. You will create blocks that define connection parameters. Each block begins with a Host line, which creates an alias you type when connecting. The HostName line specifies the actual domain. The User line defines the account name. The IdentityFile line points to your private key.

Consider a scenario where you manage both a work github account and a personal github account. Your config file might look like this:

# --- For Work GitHub ---
Host github.com-work
HostName github.com
User git
IdentityFile ~/.ssh/github_work
IdentitiesOnly yes

# --- For Personal GitHub ---
Host github.com-personal
HostName github.com
User git
IdentityFile ~/.ssh/github_personal
IdentitiesOnly yes

The IdentitiesOnly yes directive plays a critical role. When multiple SSH keys sit in your agent, a connection might offer several keys before reaching the correct one. Servers reject wrong keys, and repeated failures can trigger security blocks. This directive ensures your system offers only the single specified key. It prevents unintended key offers and avoids authentication failures.

You can also list multiple IdentityFile entries for one host. SSH tries each key in order until the server accepts one. This approach helps when you rotate keys or maintain access through different credentials.

One caution: avoid overriding changed hosts further down the configuration. If you create a Host github.com-work alias that changes the hostname, and later add a Host github.com entry with a different key, the agent may override your first rule. Keep your aliases organized and specific.

Using Wildcards for Default Configurations

Wildcards simplify your ssh config setup when you manage many servers. The asterisk (*) matches zero or more characters. The question mark (?) matches exactly one character. These patterns let you apply settings to entire groups of hosts without writing separate blocks.

Pattern syntax: A pattern consists of zero or more non-whitespace characters, * (a wildcard that matches zero or more characters), or ? (a wildcard that matches exactly one character).

For example, you might set default parameters for any host in a specific domain:

Host *.company.com
User deploy
IdentityFile ~/.ssh/company_deploy
IdentitiesOnly yes

This rule applies to staging.company.com, production.company.com, and any other subdomain. You can also match IP ranges. The pattern Host 192.168.0.? matches any host in the 192.168.0.0 through 192.168.0.9 range.

Wildcards support inheritance. A hostname like acmereplica might match a general Host acme* block for user settings, then match a more specific Host acmereplica* block for its hostname. SSH reads the config top to bottom and uses the first matching value for each parameter.

This approach scales beautifully. Adding a new server requires only two lines with wildcards. You create host aliases automatically without repeating common settings. The wildcard pattern handles the rest.

The Host * block serves as your default configuration. Place it at the bottom of the file. Any host that lacks a specific rule inherits these settings. You might set AddKeysToAgent yes here to automate key loading.

Your ssh config file transforms how you manage remote connections. You configure multiple ssh keys once, then forget about manual selection. The system routes each connection correctly. Test your setup after creating the file to confirm each host uses the intended key.

Adding Multiple SSH Key Pairs to the Agent

The SSH agent is a background program that holds your decrypted private keys in memory. It eliminates the need to type your passphrase every time you connect to a remote host. When you use multiple ssh key pairs, the agent becomes essential. It manages all your keys in one place and supplies the correct one automatically during authentication.

Using ssh-add for Automatic Authentication

Start the agent in your terminal session with eval "$(ssh-agent -s)". This command launches the agent and sets the environment variables it needs. Then, add your private keys using the ssh-add command.

The agent prompts you for each key’s passphrase once. After that, it stores the decrypted key in memory. You can verify which keys are loaded by running ssh-add -l. This command lists all fingerprints currently held by the agent.

Security guidance on SSH agent forwarding: Agent forwarding lets you use a local key through a bastion server. However, if the forwarding destination is compromised, an attacker can hijack your key. The risk grows when you manage multiple keys, because a compromised destination could expose all forwarded keys. Only use agent forwarding toward trusted servers. A safer alternative is ProxyJump (-J), which avoids forwarding the key entirely: ssh -J bastion.example.com user_x@internal.example.com.

Persisting Keys Across Reboots

The agent stores keys only in memory. When you reboot your machine, the agent clears all loaded keys. You must re-add them manually unless you configure persistence.

On macOS, you can store passphrases in the Keychain. Run ssh-add --apple-use-keychain ~/.ssh/github_personal to add the key and save its passphrase securely. Then, edit your ~/.ssh/config file to include these lines under a Host * section:

UseKeychain yes
AddKeysToAgent yes

The UseKeychain yes directive tells SSH to retrieve passphrases from the Keychain automatically. After a reboot, the agent loads your keys without prompting you again.

On Linux, the approach differs. You can use ssh-add -K on some distributions, though this flag is not universal. A more reliable method involves a keychain manager like gnome-keyring or KWallet. These tools integrate with your desktop environment and unlock your keys when you log in. Alternatively, you can create a startup script that runs with your keys after each boot.

The ssh-agent transforms your workflow. You configure your keys once, and the agent handles authentication silently. This setup saves time and reduces frustration when you switch between different Git hosts throughout the day.

Testing Your Multiple SSH-Key Setup

You have generated unique keys, built your config file, and loaded everything into the agent. Now comes the moment of verification. Testing each connection confirms your configuration works exactly as intended. This step catches errors before they interrupt your workflow.

Verifying Connections with the -T Flag

The -T flag tells SSH to disable pseudo-terminal allocation. This flag suits Git hosts perfectly because you only need authentication confirmation, not an interactive shell. Run a test command for each host you configured.

For your personal GitHub account, execute:

ssh -T git@github.com-personal

Notice the host alias matches your config file entry. SSH reads the alias, locates the matching block, and presents the correct identity file. A successful response resembles: “Hi username! You’ve successfully authenticated, but GitHub does not provide shell access.” This message confirms GitHub recognized your key and linked it to the correct account.

For your work GitLab instance, run:

ssh -T git@gitlab.com-work

GitLab responds with a welcome message that includes your username. Each success message validates that your config file routes connections properly. The system selected the right key without any manual intervention.

Test every host you defined. Create a checklist of aliases from your config file. Work through each one systematically. This process reveals typos, incorrect paths, or mismatched settings immediately.

If a test fails, examine the error message carefully. “Permission denied (publickey)” indicates SSH could not authenticate. Check your IdentityFile path for accuracy. Confirm the public key exists on the remote host. Verify the alias spelling matches your config exactly.

Cloning a Repository to Confirm the Configuration

The -T tests prove authentication works. However, real-world usage involves cloning repositories. This practical test demonstrates your entire setup functioning under normal conditions.

Navigate to a temporary directory on your machine. Clone a private repository from your personal GitHub account:

git clone git@github.com-personal:username/private-repo.git

Git uses the host alias to locate your config entry. The system selects your personal key automatically. The repository downloads without any passphrase prompt or key selection. This seamless process confirms your personal configuration operates correctly.

Next, test your work environment. Navigate to a different directory and clone the work project repo from your company’s GitLab server:

git clone git@gitlab.com-work:company/work-project.git

Again, the alias directs SSH to the proper identity file. The work project repo downloads without conflict. You have now verified both key paths work simultaneously. The system distinguishes between hosts and applies the correct credentials each time.

Repeat this process for any additional services you configured. Clone a test repository from each host. This thorough approach ensures every alias functions correctly. You eliminate surprises before they disrupt your daily work.

Successful clones confirm your ability to configure multiple ssh keys once and use them indefinitely. The setup handles authentication silently in the background. You can switch between personal and professional projects freely. No manual key swapping. No permission errors. No wasted time.

Your multiple ssh-keys now work together seamlessly. The config file routes each connection precisely. The agent supplies credentials without prompting. Your workflow becomes frictionless across all Git services.

Troubleshooting Multiple SSH-Key Conflicts

Even careful configurations encounter problems. Two issues appear most often: permission errors and the wrong key being offered. Both have straightforward fixes once you understand their causes.

Fixing “Permission Denied (publickey)” Errors

OpenSSH enforces strict permission checks on your local machine. If your private key files have overly open permissions, SSH refuses to use them. You might see messages like “UNPROTECTED PRIVATE KEY” or “Permissions are too open.” These errors protect you from insecure configurations.

On macOS or Linux, run chmod 600 ~/.ssh/your-key-file to correct private key permissions. Windows users must adjust properties through the Security tab. Right-click the key file, select Properties, then Security, then Advanced. Disable inheritance and remove permissions for users other than yourself.

Resolving Issues with the Wrong Key Being Used

Sometimes SSH presents the wrong key to a remote host. The system defaults to ~/.ssh/id_rsa or id_ed25519 when no other instructions exist. Your config file should override this behavior, but mistakes happen.

Use verbose mode to diagnose the problem. Run ssh -v -T git@github.com-personal to see exactly which key your system offers. The output shows each authentication attempt in sequence. You can identify mismatches immediately.

You can also verify fingerprints. GitHub publishes its RSA key fingerprints publicly. Compare the fingerprint shown during connection with the published value. A match confirms you reached the correct server.

Finally, confirm your public key exists on the remote service. Navigate to your Git host’s SSH settings and verify the correct public key appears there. Remember that the private key never leaves your machine. The ssh-agent holds it locally and presents it only during authentication. Never share or copy your private key to any server.

You completed the journey. Unique keys serve each service. A structured ~/.ssh/config file routes every connection. All tests confirmed your setup works correctly. This process taught you how to configure multiple keys for simultaneous use.

Manual key swapping stops here. Correct key always matches its host. ssh-agent manages all credentials silently in background. This same configuration applies to any remote server, not just Git services.

Keep private keys secure. Use a passphrase on every key. Maintain a clean directory. Remove unused entries. Update config file as needs change. These habits ensure long-term efficiency for your entire workflow. You save time every day.

FAQ

Can I use one SSH key for multiple services?

Yes, technically possible but not recommended. Separate keys limit damage if one leaks. You gain better control and easier revocation.

What happens when I add a new SSH key later?

Generate the key with a descriptive name. Add the public key to your remote host. Create a new block in your config file. Then run ssh-add to load it.

Why does SSH keep asking for my passphrase?

The agent clears keys after reboot. Add AddKeysToAgent yes to your config file. On macOS, use ssh-add --apple-use-keychain. On Linux, configure a keychain manager.

Does this setup work on Windows?

Yes. OpenSSH for Windows supports the same config file structure. Use %USERPROFILE%\.ssh\ as your directory path. Commands work in PowerShell without modification.