In a VO (virtual organisation) environment where services are provided and shared by dissimilar organisations from different administrative domains and are protected with dissimilar security policies and measures, there is a need for a flexible authentication framework that supports the use of various authentication tokens. The authentication strengths derived from these tokens should be fed into an access control decision making process. This paper reports our ongoing efforts in designing and implementing such a framework to facilitate multi-level and multi-factor authentication and authentication strength linked fine-grained access control in Shibboleth. The proof-of-concept prototype using a Java smart card is reported.