I still don't understand what entropy is

I realize that I can probably give a short explanation about the second law here.

Zerothly and most importantly, throw out any ideas about entropy having anything to do with order or disorder.

Firstly and very importantly, entropy is not a quantity of any physical state. Instead it is a quantity a description of a physical state. A description of a physical state specifies a set of possible physical states that meets that description. For example, I can describe a box that contains a given volume of an ideal gas at given pressure. There are lots of states of molecules moving with various momenta that can satisfy this description.

Given a description of a physical state we can look at the subset of the phase space that satisfies this description. This set will occupy a certain (hyper-)volume of phase space. The logarithm of this volume is the entropy of the description. Actually the entropy is the logarithm of this volume times Boltzmann's constant, but the constant isn't very important and is only there for historical reasons.

Liouville's theorem says that the volume of a subset of phase space is preserved under time evolution of the system. Therefore if you want to build a system that reliably takes you from any state satisfying description A to some state satisfying description B, the volume of phase space of states satisfying B had better be larger than the volume of phase space of states satisfying A. In otherwords, the entropy of B has to be greater than the entropy of A.

As an aside, it may be possible for a state with description A to evolve into a state with description B where the entropy of B is less than the entropy of A. However, this process will necessarily be unreliable. And if you keep trying again and again with random starting states satisfying description A, the probability that you will end up in a state satisfying description B is at most (and likely much smaller than) exp((S(B) - S(A))/k) where S(A) is the entropy of A and S(B) is the entropy of B and k is Boltzmann's constant. Jaynes, in "The Evolution of Carnot's Principle" notes that if S(B) is only smaller than S(A) by (1 microcalorie / room temperature) then the probability of reaching a state satisfying B from a random state satisfying A is less than exp(-10^15) which is approximately 10^(-400000000000000), which is an astronomically low probability. Stated differently, this is an upper bound on the probability of being able to extract an extra 1 microcalorie of energy than the second law says you are allowed from a heat engine operating in an environment at room temperature.

Hmm, I'll probably turn this into a blog post.