What does a smart IoT device have to be able to do, and what features distinguish the current generation?

Question

Accepted Answer

Sense, compute and communicate within an energy budget it manages itself — and increasingly, do analytics and AI inference on-device rather than shipping raw data to the cloud.
Core requirements — the non-negotiables for a device to function as an endpoint at all:

Sensing, computing, communication — the three primitives.
Balance of computation and communication for energy efficiency — the design discipline that makes the budget work.
Energy harvesting, plus efficient energy management, storage and battery.
Local intelligence at the edge of the cloud — enough local processing that not everything has to be sent.
Securely collect, store, process and connect data — security stated as a requirement, not an addition.
Memory, non-volatile memory (NVM) and storage.

Features of the current and future generation — what the hardware is growing towards:

Cost-effective, small-form-factor, reliable advanced computing systems.
Analytics, visual recognition and secure transmission at extremely low power.
Advanced communication protocols.
Low-cost system integration and multi-die interconnect solutions.
Demands for more compute and more memory, in a secure system.
On-device AI/ML inference and embedded AI accelerators, with hardware–software co-design optimised for AI workloads.

Read together, the two lists show the direction of travel: the requirement was always "process a bit locally so you transmit less", and the feature list is that requirement being pushed as far as silicon allows — from averaging a sensor stream, to recognising an image, without the data ever leaving the device. That is attractive for latency and for privacy, and it puts the pressure back on the two competing design goals, performance and low power.
Tip: every feature in the second list is a way of buying more capability without spending more energy. That is the single design constraint the whole category is organised around.
Go deeper:

Wikipedia — TinyML — the on-device inference item from the feature list as a field of its own: running models on microcontroller-class hardware with minimal reliance on the cloud.