The name “entropy” chosen by Shannon is no coincidence: his formula has the same structure as the one in thermodynamics, and the bridge between the two worlds is one of the most fascinating threads of applied mathematics.
Remark — From thermodynamic disorder to information disorder
Shannon’s formula has the same structure as Boltzmann’s formula (with = number of equiprobable microstates). For this reason Shannon, on von Neumann’s suggestion, called it “entropy” with the celebrated justification: “you will always win debates because nobody really knows what entropy is” (Shannon, 1948). It is the guiding thread of Borghi (Entropioni): the same quantity describes reversible transfers of energy, the optimality of compression codes, and the uncertainty of a random experiment.
Links
Topics: Distribuzioni probabilita
Concepts: Entropia
People: Boltzmann · Shannon · Von Neumann