What is PAM, and why add pam_mkhomedir with umask=0077 when joining Linux to AD?
PAM (Pluggable Authentication Modules) separates programs like login and ssh from the way users are authenticated; pam_mkhomedir creates a domain user's home directory at first login, and umask=0077 makes it private (700).
* Every entry point runs the same PAM stack; pam_mkhomedir runs in the session phase. *
A system can be reached in many ways: console login, SSH, RDP, su, FTP. PAM gives all of them one central configuration in /etc/pam.d/. Each service lists the modules it runs (check the password, check that the account may log in, set up the session), so authentication is changed in one place instead of in every program.
AD users come from SSSD and have no home directory on the host, so the first login ends with "cannot change directory to /home/…". The line
session required pam_mkhomedir.so skel=/etc/skel/ umask=0077
in /etc/pam.d/common-session creates it in the session phase and copies the default files from /etc/skel.
The resulting rights: umask=0077 removes all rights for group and others, so the directory is drwx------ (700). Only the user (and root) can enter it. Without it, the usual default (755) would let every other user on the host read through everyone's home.
Go deeper:
Wikipedia: Pluggable authentication module — the idea and the four module types.
man7.org: pam_mkhomedir(8) — the
skelandumaskoptions.Wikipedia: umask — how a umask turns 777 into the final mode.