Members of high-performing human teams can often anticipate information needs of teammates and offer relevant information to them proactively. Such capabilities are highly desirable for agent teams to achieve better teamwork processes for supporting information gathering, information fusion, and decision makings of teammates. This paper presents a framework for formally specifying proactive agent behaviors based on the SharedPlan theory. Axioms that specify an agent's anticipations of a teammate's information needs are proposed, which enable the teammate to perform an action or protect the teammate from a threat. We also introduce the semantics of two proactive communicative actions (i.e., proactive inform and third party subscribe) that are driven by information needs of teammates. Based on these semantics, we show that suitable proactive information delivery actions can be derived in the proposed framework. This paper thus establishes a formal foundation for proactive information delivery behavior that not only provides better understanding of the underlying assumptions required to justify the behavior, but also provides a coherent basis for the specification and design of agent teams with proactive information delivery capabilities.