• 38 Posts
  • 646 Comments
Joined 1 year ago
cake
Cake day: June 27th, 2023

help-circle







  • I will try to have some more comments about the physics when I have time and energy. In the meanwhile:

    Entropy in thermodynamics is not actually a hard concept. It’s the ratio of the size of a heat flow to the temperature at which that flow is happening. (So, joules per kelvin, if you’re using SI units.) See episodes 46 and 47 of The Mechanical Universe for the old-school PBS treatment of the story. The last time I taught thermodynamics for undergraduates, we used Finn’s Thermal Physics, for the sophisticated reason that the previous professor used Finn’s Thermal Physics.

    Entropy in information theory is also not actually that hard of a concept. It’s a numerical measure of how spread-out a probability distribution is.

    It’s relating the two meanings that is tricky and subtle. The big picture is something like this: A microstate is a complete specification of the positions and momenta of all the pieces of a system. We can consider a probability distribution over all the possible microstates, and then do information theory to that. This bridges the two definitions, if we are very careful about it. One thing that trips people up (particularly if they got poisoned by pop-science oversimplifications about “disorder” first) is forgetting the momentum part. We have to consider probabilities, not just for where the pieces are, but also for how they are moving. I suspect that this is among Vopson’s many problems. Either he doesn’t get it, or he’s not capable of writing clearly enough to explain it.

    So these two were published in American Institute of Physics Advances, which looks like a serious journal about physics. Does anyone know about it? It occupies a space where I can’t easily find any obvious issues, but I also can’t find anyone saying “ye this is legit”. It claims to be peer-reviewed, and at least isn’t just a place where you dump a PDF and get a DOI in return.

    I have never heard of anything important being published there. I think it’s the kind of journal where one submits a paper after it has been rejected by one’s first and second (and possibly third) choices.

    However, after skimming, I can at least say that it doesn’t seem outlandish?

    Oh, it’s worse than “outlandish”. It’s nonsensical. He’s basically operating at a level of “there’s an E in this formula and an E in this other formula, so I will set them equal and declare it revolutionary new physics”.

    Here’s a passage from the second paragraph of the 2023 paper:

    The physical entropy of a given system is a measure of all its possible physical microstates compatible with the macrostate, SPhys. This is a characteristic of the non-information bearing microstates within the system. Assuming the same system, and assuming that one is able to create N information states within the same physical system (for example, by writing digital bits in it), the effect of creating a number of N information states is to form N additional information microstates superimposed onto the existing physical microstates. These additional microstates are information bearing states, and the additional entropy associated with them is called the entropy of information, SInfo. We can now define the total entropy of the system as the sum of the initial physical entropy and the newly created entropy of information, Stot = SPhys + SInfo, showing that the information creation increases the entropy of a given system.

    wat

    Storing a message in a system doesn’t make new microstates. How could it? You’re just rearranging the pieces to spell out a message — selecting those microstates that are consistent with that message. Choosing from a list of available options doesn’t magically add new options to the list.




  • From the Wired story:

    As a comparison, Cui cited another analysis that GPTZero ran on Wikipedia earlier this year, which estimated that around one in 20 articles on the site are likely AI-generated—about half the frequency of the posts GPTZero looked at on Substack.

    That should be one in 20 new articles, per the story they cite, which is ultimately based on arXiv:2410.08044.

    David Skilling, a sports agency CEO who runs the popular soccer newsletter Original Football (over 630,000 subscribers), told WIRED he sees AI as a substitute editor. “I proudly use modern tools for productivity in my businesses,” says Skilling.

    Babe wake up, a new insufferable prick just dropped.

    Edit to add: There’s an interesting example here of a dubious claim being laundered into truthiness. That arXiv preprint says this in the conclusion section.

    Shao et al. (2024) have even designed a retrieval-based LLM workflow for writing Wikipedia-like articles and gathered perspectives from experienced Wikipedia editors on using it—the editors unanimously agreed that it would be helpful in their pre-writing stage.

    But if we dig up arXiv:2402.14207, we find that the “unanimous” agreement depends upon lumping together “somewhat” and “strongly agree” on their Likert scale. Moreover, this grand claim rests upon a survey of a grand total of ten people. Ten people, we hasten to add, who agreed to the study in the first place, practically guaranteeing a response bias against those Wikipedians who find “AI” morally repugnant.