Re the no representative agent theorem with efficient markets, suggests that agents can not be eu maximisers, but still be unpumpable, so long as their preferences(?) Are path dependent in the right way. See a better explantion. This suggests having revisable preferences might let us be unpumpable and violate coherence theorems, e.g. in population ethics hopefully.
How do markets solve for general equilibrium?
Some similar stuff in voting theory for dealing with intransitive horn of arrow’s impossibility theorem, Schulze/beatpath just works with the pairwise comparisons of all outcomes, Tideman’s alternative starts from the whole option set, then restricts it based on Smith set membership, then does IRV to narrow it down.
Stuff from bargaining theory is probably another interesting angle for dealing with intransitivity/aggregating values in non-vNM rational systems
Linking Callard, distributed systems, markets, shame/social technologies, bounded rationality, past selves and other people, you cant punish a leg, Tool AI, relevant to John Wentsworth wentworths ideas somehow
- Also look at Sloterdijks theory of anthropotechnics, if that has a social aspect which idk.
You say VNM rational agents always win. But will one of those beat a market?
- could be framed as a critique of Gwerns tool AI
- maybe similar to crotch’s picture of multi-agent systems being competitively optimal
- could also frame it as how age of em would really go/Molochy stuff
So an agent is something like a set that models the world, acts upon it, and has goals of some kind. But why should these 3 types of thing need to be spatially or temporally local? If theyre not, what kinds of channels of communication or influence might exist between them, (thats where dist. Systems comes in)
You can look at MAM selves, person-stages with different preferences, markets, relational agents using commitment and other social technology as types of distributed agents, distributed across time, space, or even possible worlds/large universes (as in superrationality, evidential cooperation in large worlds, etc.)
Must see key concepts from distributed systems. Maybe systems thinking? Sources of the self. Voting and bargaining theory for multi-agent decision making. Gane theory/mechanism design for multi-agent cooperation, maybe also phil of collective action.
How could such an agent, or its parts, model itself and other similar agents?
How do winning and multiplying relate to each other, (as meta/mesa goals?)
Remember to check MSCP AI course notes on capitalism
One way of thinking about this is that a temporally unstable agent is similar to a group of agents that exist at the same time, and are fighting over resources.
In the case where a group of agents exist at the same time, each with different utility functions, there will be a tendency (once the agents become strong enough and have a varied enough option space) for the strongest agent to try to seize control from the other agents, so that the strongest agent can get everything it wants.
A similar dynamic exists for (sufficiently capable) temporally unstable agents. Alice turns into a werewolf every time the moon is full; since human-Alice and werewolf-Alice have very different goals, human-Alice will tend (once she’s strong enough) to want to chain up werewolf-Alice, or cure herself of lycanthropy, or brainwash her werewolf self, or otherwise ensure that human-Alice’s goals are met more reliably.
https://www.convergenceanalysis.org/publications/investigating-the-role-of-agency-in-ai-x-risk
The sociality of personhood, moral knowledge, and responsibilityContinuity of Responsibility Account of Personal Identity