Entropy and Information

Sorry for abruptly pausing the journey of Han. I lacked motivation to continue the story. (Describing dirty politics withing the militaristic institution is surely interesting but is simply not charming enough for me to invest a chunk of my life in it.)

I've been working on piano pieces since then. Should write about it sooner or later.

Today's shower thought:

von Neumann entropy is simple and beautiful, and tells a lot about the total information of the system. The famous BH entropy is also calculated from von Neumann entropy(Casini, 2009). However, something is deeply unsatisfying here. I feel that most theories just skim over conservation of "information" through von Neumann entropy when we could make far much out of it. Recall what Maxwell's Demon was capable of!

That being said, Maxwell's Demon just shows us how confusing the argument relating entropy and information could be. Let's focus on classical Maxwell's Demon. It's trying to separate a mixture of Helium and Argon by only allowing Helium atoms pass through a small door.

Despite being classical, the demon should somehow measure the mass of incoming atoms, preferably through colliding photons against them. Be noted that this process is what makes the concept of information relevant; otherwise all the variables are determined classically and the 'all-knowing' demon could indeed separate the mixture, decreasing the total entropy of the system. (In this setup we are using Gibbs entropy.) The demon shoots a photon toward an incoming particle. Before It loses the photon, the demon should store several real numbers in his memory to remember the four-vector p_0 of the photon. (It never has 'a priori' access to any of the observables!) What does its memory look like? It is simply a system with finite number(N) of states. In case of spontaneous emission, the emitter atom itself functions as the storage. To make the argument simple, let's say all the states are the same before anything is stored. When the demon writes something to the storage, all states except one of them become unavailable, again in sake of simpliciy. The only available state would be labeled a single four-vector such as '5.8 3.4 7.1 8.1', but details are up to the demon. Anyway,  the entropy of the storage decreases from ln N to zero. (Remember, the number of possible states is a measure of the lack of information. More information = More macroscopic variables = Less microstates corresponding to.)

One small problem is that a storage that could store an arbitrary real number should consist of infinite number of states. QFT predicts that storage of finite space could only have finite number of states(Bekenstein bound), but this is a classical theory, so we could simply assume that the demon has infinite storage(In classical theory even a single free electron carries infinite amount of states, consider its phase space). Be noted however that in quantum theory the informational constraint prevents the demon from knowing exact momentum and position of a particle. This 'precision error' is likely independent of what we call uncertainty principle, but I haven't thought deeply. 

After the particle collides with the photon, the demon measures the momentum of the photon, and calculates whether the particle is Helium or Argon, using p_0 previously stored. It need not be a complex process. Imagine the photon reflected interacting with the p_0 storage, resulting in one of the two states, either 'Helium' or 'Argon'. Through the process, the information of p_0 is lost. (This is a red flag in quantum mechanics, unitarity means no information should be lost. However, I think the information used in this context is quite different from that used in QM.) Net entropy change is zero, since it was zero before the operation and still zero after it.

Now the demon should actually open or close the gate, reading from his storage. One should also consider the information flow through this process, but I think it is quite well established that this 'gate operating' process is reversible, resulting in zero information flow. 

The crucial point of above setup is that it is the lack of information restricts the 'dynamics' of the demon, whatever it is. In other words, informational constraints restrict real world dynamics, which will restrict available theories. Quite weirdly, this means that theories should consider not only measureable quantities but which storage in the spacetime of the theory are they stored. 

Obviously, the above setup lacks details yet, not a quantum theory yet, and might logically fall apart soon, but I still think that it is interesting.