An interaction diagram is a graphical view of computation processes and communication between entities. In concurrent object oriented systems and multi-agent systems, it can be used as a graphical language in designing the communication between concurrent objects or agents. By formalizing the diagrams, automated translation could be used to generate code, to produce reports and to test a complete system. In this paper, we present a graph algebra in which the diagrams are expressed as compositions of interconnected components according to their intended interpretations. We show how a diagram is converted into its algebraic form or a fragmentation. The conversion can also be reversed, so that we can graphically represent a fragmentation by an interaction diagram. There are two equivalent forms of fragmentations that are used for different purposes. Computation flow fragmentation is used to describe the computation processes of one particular object or agent. Message flow fragmentation can capture the communication between objects or agents. We show that they are equivalent and describe steps to translate one form to another so that we can manipulate and analyze fragmentations in either form. With the formalism of interaction diagrams and fragmentations, we can develop automated procedures and tools for using interaction diagrams in the development of concurrent object oriented systems and multi-agent systems.