Imagine an AGI (Artificial General Intelligence) that could perform any task a human can do on a computer, but at a much faster pace. This AGI could create an operating system, produce a movie better than anything you’ve ever seen, and much more, all while being limited to SFW (Safe For Work) content. What are the first things you would ask it to do?
I think a lot of things that are proposed here could not be done by an AGI on an computer, no matter how intelligent. Consider this alternative scenario: You have an exceptionally intelligent young human adult with a computer locked in a room. They have no specialized education or anything. They are just extremely intelligent. What could you achieve through such a person?
Discovery of new physics is out of the question. That would need experiments.
Locked in a room with an internet connection? A lot. But without any contact with the outside world? Not nearly as much. It could have other people running experiments for it with an internet connection, but not without one.
Anyway, whether or not the AGI can interact with the real world undermines the purpose of my explicit statement in the question. I specifically mentioned that it only operates as a human on a computer. I didn’t mention it could acquire a physical body, so let’s just assume it can’t and can’t use other people to do physical labor either.