Why is there an inherent tension between transparency and privacy, and how can a system get some of both?

Full transparency means no privacy and full privacy means no transparency — but a system can make the state-transition process publicly verifiable while hiding each participant's individual data, using cryptography (at an efficiency cost).

Transparency and privacy pull in opposite directions: a fully transparent system exposes everything, a fully private one exposes nothing. But the trade-off isn't binary. A distributed ledger can provide public verifiability of the overall state without leaking each participant's individual state. The tool for this is cryptography — e.g. Zerocash uses computationally expensive cryptography to give full anonymity while still letting anyone publicly verify the ledger. The catch: privacy in a public system typically comes at the cost of lower efficiency. (This is why privacy is easier in a central system — it doesn't need transparency or public verifiability to function at all.)

Go deeper:

• Zcash (Wikipedia) — the Zerocash system in practice: zero-knowledge proofs give full privacy while the ledger stays publicly verifiable.