Health monitoring applications often require that the patient maintain a diary of activities so that the physiological data can be correlated to what the user was doing. However, patients are notoriously bad at self-reporting. Consequently, it would be beneficial to automatically generate an activity diary. This paper presents a proof-of-concept prototype electronic textile system for recording both physiological data and context information. It also presents some of the issues that arise in the design and use of a health monitoring and activity annotation system.