Multi-Factor Authentication Solution Evaluation Criteria
This document outlines criteria that should be consider when evaluating multi-factor authentication products and services. Much of the content in this document is based on material from The Quest to Replace Passwords: A Framework for Comparative Evaluation of Web Authentication Schemes by Joseph Bonneau, Cormac Herley, Paul C. van Oorschot, and Frank Stajano, March, 2012.
Deployment Environment
- Is the solution compatible with existing server and client platforms? Examples for higher education include:
- Operating Systems
- Windows
- Macintosh
- Linux
- Android
- iOS
- Web/browser applications
- Firefox
- Chrome
- Safari
- Internet Explorer
- Microsoft AD
- Globus
- Unix shell
- VPN
- Mobile
- CAS
- Shibboleth
- Radius
- Operating Systems
- What features does the solution have to facilitate broad deployment throughout a university community?
- What features does the solution have to be deployed in a BYOD environment?
- Is the solution scalable and flexible enough to be adapted to potential future applications?
Usability and Accessibility
- Are users required to remember secrets? How many? What is the required complexity of those secrets?
- Does adding more accounts for the user increase the burden on the user?
- Are users required to carry an additional object (to be used as an authentication token)?
- What physical effort is required of the user?
- How easy is the solution to learn?
- How much time is required to perform an authentication? To associate a new account?
- How reliable is the authentication process (i.e., are false negatives common)?
- How easy is it to recover from a lost token or forgotten credentials?
- How accessible is the solution? Do certain disabilities preclude its use?
Security
- Is the solution resistant to physical observation?
- Is the solution resistant to targeted impersonation (e.g., by an acquaintance)?
- Is the solution resistant to throttled guessing?
- Is the solution resistant to unthrottled guessing?
- Is the solution resistant to internal observation (e.g., by intercepting traffic across a network or within an endpoint device?
- Is the solution resistant to leaks from other verifiers?
- Is the solution resistant to phishing?
- Is the solution resistant to theft?
- Does the solution require a trusted third party?
- Is explicit user consent required to complete authentication?
- Can authentication with one verifier be linked with another?
- Does the solution require secrets to be stored on a server?
- Is the technology mature? Has it been reviewed by a sufficiently large community?
- Is the solution proprietary? Can the implementation be assessed independently by users?
Product Support
- What support services are a provided by the vendor? By third parties?
- Are add-ons available from third parties? What other vendors have integrated the solution with their products?
- Is there a viable user community?
Compliance
- Does the solution satisfy the authentication criteria for well-known assurance profiles?
- Does the solution satisfy the authentication criteria for compliance with FERPA, HIPAA, PCI, and other requirements for higher education?
- Does the solution conform with applicable standards, particularly FIPS 140-2 and NIST 800-63-2? Are there plans for alignment with the Fido Alliance?
Cost
- What is the purchase cost of the solution?
- What are the support costs of the solution?
- What is the startup cost?
- What are the ongoing operational costs? What are the staffing requirements?
- What is the fixed cost per user?
- Are there variable "per user" costs?