From Theory to Practice: mTLS in Action on AWS ALB – Part 1

Introduction

Today, applications often communicate across different environments, such as public clouds, private data centers, mobile devices and edge locations. Much of this traffic passes through networks that are not fully controlled by the organization. Every point along the way is a possible risk for someone to intercept, modify or fake the data. That’s why security standards like GDPR, HIPAA and PCI-DSS require strong encryption and trusted identities when data moves between systems.

Transport Layer Security (TLS) helps meet these requirements by encrypting the connection and verifying the identity of the server. But it assumes that the client is trusted. In some cases, both the server and the client must prove who they are before sharing any sensitive data. This is where mutual TLS (mTLS) is used. mTLS adds an extra step to the TLS handshake – both sides share and verify certificates, so each one knows exactly who they’re talking to.

In this article, we’ll start by explaining how the regular TLS handshake works. Then we’ll look at how mTLS improves security and walk through a real example of setting up mTLS using an AWS Application Load Balancer (ALB).

PKI Infrastructure

Before we examine the inner workings of TLS and mTLS, we need to establish how trust is established on a network. Initial trust can take many forms, such as pre-shared secrets, Kerberos tickets, SAML assertions, OAuth tokens or even static allow-lists. However, for servers that must be reachable over the public Internet the most widely adopted and scalable mechanism is Public Key Infrastructure (PKI).

PKI is a framework which purpose is to facilitate the secure trustworthy electronic communication for a range of network activities.. It encompasses everything used to establish and manage public key encryption, including software, hardware, policies and procedures that are used to create, distribute, manage, store and revoke digital certificates.

A digital certificate works like an ID card for a server, user or device. It binds a public key to a verifiable identity and is digitally signed by a Certificate Authority (CA) that the client already trusts. Possessing the matching private key lets the holder prove that identity without exposing the key itself.

Well-known public CAs( DigiCert, Sectigo, GlobalSign and others) embed their root certificates in all major operating systems and browsers, however many organisations also operate private CAs for internal use.

Transport Layer Security (TLS)

So – what is Transport Layer Security (TLS)?

TLS is the de‑facto protocol for protecting data in transit on today’s Internet. It delivers two fundamental guarantees:

• Authentication – the client can confirm that the server presenting a digital certificate really owns the identity stated in that certificate.
• Confidentiality & Integrity – once a session key has been agreed, no unauthorised party can read or tamper with the traffic.

At the core of TLS (Transport Layer Security) is asymmetric cryptography, which involves a private key and a public key.

• The private key is confidential and stays with the server.
• The public key is shared with anyone who needs to communicate securely with the owner.
• Data encrypted with the public key can only be decrypted with the private key (encryption for confidentiality).
• Data signed with the private key can be verified with the public key (used for authentication and integrity checks).

How TLS Provides Authentication

Imagine a client and a server trying to connect. The client needs to be sure it’s talking to the correct server, not an attacker sitting in the middle.

To make this possible, the server first obtains a digital certificate, as explained in the PKI section. This certificate is issued by a trusted Certificate Authority (CA). The server starts by creating a Certificate Signing Request (CSR) and sends it to the CA. The CA signs it with its private key and returns the certificate, which includes the server’s public key and identity information.

When the client connects, the server presents its certificate during the TLS handshake. The client checks:

• whether the certificate is signed by a trusted CA (using a built-in trust store),
• whether the certificate is still valid (expiration date),
• and whether the domain name matches the certificate’s hostname

If all checks pass, the client accepts the server’s identity and from that moment trust is now established.

How TLS Provides Confidentiality & Integrity

After authentication, the two parties must agree on a shared symmetric key, which will be used to encrypt and decrypt messages during the session.

There are two main ways this happens:

• RSA-based key exchange (used in older TLS 1.2 suites):
  The client generates a random pre-master secret, encrypts it using the server’s public key (from its certificate) and sends it to the server. The server decrypts it using its private key.
• ECDHE-based key exchange (used in modern TLS 1.2 and TLS 1.3):
  Both the client and server exchange temporary (ephemeral) Diffie-Hellman public keys, then each side calculates the shared secret using its own private key and the other party’s public key. This method provides forward secrecy, so the server’s certificate is only used for authentication (not for encryption).

In both approaches, the pre-master secret is passed through a key derivation function (KDF) to generate identical session keys on both sides. These symmetric keys are then used to encrypt and verify all data exchanged in the session.

Typical TLS 1.2 Handshake (High-Level)

Here’s a simplified version of how a typical TLS 1.2 handshake works:

1. ClientHello – The client initiates the handshake, offering supported TLS versions, cipher suites and a session ID.
2. ServerHello – The server responds with its selected protocol version and cipher suite, along with its digital certificate.
3. ServerKeyExchange / ServerHelloDone – The server may send additional key exchange parameters, then signals it’s done.
4. ClientKeyExchange – The client sends the pre-master secret, encrypted with the server’s public key.
5. ChangeCipherSpec / Finished – Both sides derive the session keys, switch to encrypted mode, and confirm handshake integrity using hash values.

As already was mentioned, TLS 1.3 simplifies the handshake by removing unnecessary steps and encrypting more of the handshake itself, but the overall goal is still the same: establish a secure, encrypted session using a shared secret.

Once the handshake is complete, all application data is encrypted using the agreed symmetric cipher. For most public websites, authenticating the server is enough. But if the server also needs to authenticate the client – we step into mutual TLS (mTLS) territory.

Mutual Transport Layer Security (mTLS)

While regular TLS only verifies the identity of the server, some environments also require the client to prove who it is. This is common in API meshes, B2B integrations, regulated systems (like in healthcare or finance) and IoT devices. In these cases, mutual TLS (mTLS) is used. It extends the standard TLS handshake to include client authentication, so both sides present and validate certificates, establishing two-way trust.

How mTLS Adds Client Authentication

Here’s how the mTLS handshake differs from standard TLS:

1. CertificateRequest – During the handshake, the server sends a Certificate Request message, asking the client to provide its certificate.
2. Client Certificate – The client responds with its X.509 certificate chain, signed by a Certificate Authority (CA) that the server trusts (i.e. the server has the CA’s root certificate in its trust store).
3. CertificateVerify – The client proves ownership of the certificate by signing a random value with its private key.
4. Server Validation – The server validates the client’s certificate chain, checking its authenticity, expiration date and whether it was issued by a trusted CA.
5. Key Agreement – Both parties complete the same key exchange process used in TLS (e.g., ECDHE) and derive shared symmetric session keys.

After these steps, both the server and client are cryptographically sure of each other’s identities. All data exchanged after the handshake is encrypted and protected from tampering, just like in standard TLS, but now with mutual assurance.

By Pavel Luksha, Senior DevOps Engineer, Klika Tech, Inc.