Middleware End-To-End Diagnostics Advisory Group (MW-E2ED) Middleware End-To-End Diagnostics Advisory Group (MW-E2ED) The MW-E2ED Advisory Group is dormant for now. Please contact Steve Olshansky, MW-E2ED flywheel, with questions or comments. Minutes || Mailing List || Documents Background Since the beginning of the Internet2 effort, the number and complexity of distributed application initiatives has exploded, and each must create its own framework for providing diagnostic tools and performance metrics. These distributed applications have become increasingly dependent not only the system and network infrastructure that they are built upon, but also each other. It is therefore becoming extremely complex to diagnose problems and provide a way to provide diagnostic and performance information to the application development and enterprise operations efforts. The following documents will give an overview of the problem, vision, concept, direction, and status of the effort. NSF Middleware Initiative: NMI-EDIT Components: EDDY (End-to-end Diagnostic DiscoverY) - A Diagnostic Tool Developers Platform Latest Software Release: Version 0.5.2 Update 4 (June 9th, 2008) EDDY Infosheet (PDF) EDDY Top Network Talker Diagnostic User Interface Example Internet2 End-to-End Diagnostic Initiative: Vision and Direction (PDF) The Case for Comprehensive Diagnostics (White Paper) (PDF) User Perspectives of Diagnostics - A Multidimensional View (PDF) EDDY (End-to-End Diagnostic Discovery) Architecture, Concept and Effort Status (PDF) While the requirement for providing wide variety of diagnostic and application specific performance tools requires access to a rich set of log and audit data, there is a high possibility of inferring specific user behavior through correlation of this data. Therefore, it is paramount that limiting access to this data and using data anonymization methods be essential to insure the privacy of the end user, or an enterprise as a whole. Charter 1. Gather feature requirements from the perspective of the origin/target application developers, administrators, operations and their support staff to create a distributed diagnostic platform, which is application independent. This will feed the design effort that will include a rich tool set for problem discovery, forensic analysis and application performance baselining. While being highly conscious of privacy requirements with respect to the end user as well as enterprises and federations, document privacy requirements and make recommendations for an evolutionary deployment of the features over three years. This will ensure that an initial set of simple high impact tools can be delivered quickly without the delay of the full implementation of privacy features, which will be deferred until year two. 2. Engage with specific high impact NMI-EDIT and GRIDS initiatives to help foster their adoption, 1 of 4 Middleware End-To-End Diagnostics Advisory Group (MW-E2ED) development and deployment efforts. 3. Survey and report on market, internal and external research activities, as well as standards efforts in the distributed diagnostic space. 4. Deliver a simple but useful diagnostic and performance analysis tool set in year one, to quickly aid in the evolution of Internet2 Middleware distributed application initiatives, which uses an extensible platform for more complex and feature rich tool deployment in year two and beyond. Objectives Supporting NMI-EDIT initiatives through the development of a diagnostic tool platform that; Assists the deployment efforts in the configuration and testing phases Reduces the problem discovery and analysis time during the operation and maintenance phases Assists application, developers, administrators and operators by; Allowing the collection of a wide variety of diagnostic log formats Establishing common event records to share inter and intra-organizationally, to aid in the proof and repudiation of faults Providing access to application, system and network events that are both internal and external to the application to aid the bug fixing process Enabling rapid development of diagnostic applications for pre and post production phases Current Activities Dissemination EDDY Infosheet (PDF) Effort Timeline 2005-2006, 2006-2007 (PDF) Process (PDF) Presentations Requirements Gathering for the Tool Deliverable Scenario Compilation Process Interviews With Key Individuals and Groups Requirement Matrix Survey of the Diagnostic Space Architecture Diagnostic Backplane (PDF) Common Event Record High Level Overview (PDF) Detailed Description of Concept (PDF) Formal Specification (work in progress) Pilot for Testing the Common Event Record and Diagnostic Backplane Concept (ccBay) EDDY (End-to-end Diagnostic DiscoverY) - The CMU Diagnostic Initiative and Developers Toolkit (v0.5.2.2 Murphy Beta Release 11/5/07) Advisory Group Chair Advisory Group Flywheel Chas DiFatta, Carnegie Mellon University Steve Olshansky, Internet2 NOTE WELL: All Internet2 Activities are governed by the Internet2 Intellectual Property Framework. 2 of 4 Middleware End-To-End Diagnostics Advisory Group (MW-E2ED) Minutes of MW-E2ED Conference Calls 10-September-2008 7-February-06 29-July-04 4-June-2008 14-December-05 15-July-04 7-May-2008 16-November-05 1-July-04 5-March-2008 3-November-05 17-June-04 1-August-2007 8-September-05 3-June-04 6-June-2007 2-June-05 8-April-04 7-March-2007 21-April-05 25-March-04 4-December-2006 7-April-05 12-February-04 Fall Internet2 Member Meeting BoF 24-February-05 29-January-04 1-November-2006 12-January-05 15-January-04 4-October-2006 16-December-04 18-December-03 12-July-06 18-November-04 4-December-03 3-May-06 4-November-04 20-November-03 5-April-06 21-October-04 6-November-03 8-March-06 23-September-04 23-October-03 22-February-06 12-August-04 13-October-03 Mailing List To subscribe to the list, send email to pubsympa at internet2 dot edu with the following in the *subject line*: subscribe For example: subscribe mw-e2ed Jane Doe To unsubscribe, send email to pubsympa at internet2 dot edu with the subject: unsubscribe mw-e2ed Draft Documents These documents are works in progress. For more information on the status of these documents, see the Internet2 Document Guidelines. For reference see also the Internet2 Document Library. (Coming Soon) Development of this software was supported with funding from Internet2, Carnegie Mellon University, and the NSF Middleware Initiative (NSF 02-028, Grant No. OCI-0330626). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation (NSF). 3 of 4 Middleware End-To-End Diagnostics Advisory Group (MW-E2ED) 4 of 4