How does signing a message work, and what does the sender versus the receiver do?
The sender hashes the data and transforms the hash with the private key; the receiver recomputes the hash and checks it against the signature using the sender's public key.
Sender (signer):
- Runs a hash function (e.g. SHA-256) over the data, giving a short fixed-size fingerprint.
- Applies the signature operation to that hash with the private key. In the RSA case this is often described as "encrypting the hash with the private key". The result is the signature, attached to the data.
Receiver (verifier):
- Computes the hash of the received data with the same algorithm.
- Uses the signer's public key to check the signature (for RSA: "decrypts" it back to the hash the signer computed).
- Compares the two hashes. Equal means the data was not changed since signing (integrity) and the signature was made with the matching private key (authenticity, and because only the signer has that key, non-repudiation).
Why hash first? Public-key operations are slow and work on small inputs; the hash is short, and signing it is as good as signing the whole document because finding a second document with the same hash is infeasible.
Signing does not provide confidentiality: the data stays readable. And the check only proves "whoever holds this private key signed it". Linking that key to a real person or server is the certificate's job.
* Sender hashes and signs, receiver re-hashes and compares. *
Go deeper:
Computerphile: What are Digital Signatures? — Mike Pound walks through hash-then-sign and why it proves origin.
Wikipedia: Digital signature — definitions, properties and the signing/verification diagram.