Complexity and chaos. In complex and critical social mechanisms, such as pricing, agent-based systems could dramatically increase the instability and chaotic behavior that already characterize today's market economies.
In fact, some observers have claimed that the rigid rationality of an agent-mediated economy might provide more economic stability, but their claims are backed up by neither experience nor realistic simulations. Nor do they account for the unpredictable behaviors that can emerge in a collective. In the café scenario, the price differences in Mario and Andrea's music files may have emerged from the global agent-based economy having reached a strange—and possibly chaotic—attractor, regardless of any actual change in the demand for such goods.
In general, multitudes of interacting autonomous components executing in a dynamic environment suggest an interactive system in which the global state evolves as a function more of environmental dynamics and interactions than of internal component-level intelligence and rationality. Thus, as software agents begin to populate everyday networks and environments, global behaviors will become increasingly important in all agent-based activity.
Unfortunately, the state of the art in complexity science is still very far from offering constructive methods for controlling global state in interactive systems. Without such methods, skeptics like Mario could easily reject agent systems as demonic entities under the control of an esoteric secret society.
Matters of trust. Delegating work to agents requires trusting them, yet software agent technology is unlikely to achieve the complex human decision-making capabilities that numerous tasks require. Franco's story is a possibly naive and extreme example of how the lack of these capabilities in a message agent could ruin someone's life.
Even with much more intelligent agents, trust is a difficult issue. While we do not argue that trusting agents is and will always be wrong, we do contend that trust must be achieved gradually. Potential advantages must be carefully evaluated against potential drawbacks.
Consumer and developer enthusiasm for advanced technologies already characterizes the software market. However, this enthusiasm can lead to shortcutting best practices in product development and test. Because agents are autonomous, deploying them with poor testing and documentation—in the tradition of some large software companies—could yield disastrous results. Instead, software agents should undergo exhaustive tests defining their characteristics and limitations, learning processes (if any), and behavior in relation to environmental dynamics and uncertainty—all carefully documented.