Performance WG Face-to-Face Meeting
at Internet2 Spring Member Meeting
Monday, April 27, 2009
Internet2 Update [Jeff Boote]
An update on the software releases available for Internet2 performance and measurement tools and a preview of the roadmap.
REDDnet's Use of Performance Tools [Ezra Kissel]
REDDnet has disk depots that cache data placed throughout the U.S, and they have been experiencing challenges moving data between the depots.
Highlights:
- REDDnet provides "Working storage" to help manage the logistics of sharing, moving and staging large datasets across wide areas and distributed collaborations.
- Participating Institutions: Vanderbilt, Tennessee, Stephen F. Austin, NC State, Nevoa Networks, Delaware
- Host Sites: Caltech, Florida, Michigan, ORNL, SDSC, TACC, UC Santa Barbara (Stephen F. Austin, Tennessee, Vanderbilt)
- Tools used for performance monitoring inlcude:
- OWAMP (3.1)
- BWCTL (1.3)
- NDT client (3.5)
- perfSONAR-PS perfSONAR-BUOY (regular testing framework for bwctl)
- Performance monitoring includes:
- TCP tuning on all hosts
- Picked a set of hosts to investigate from the "worst offenders"
- Divide and conquer approach (Test from depot to POP, Divide path, Narrow down where the problem "ends")
- Examples:
- REDDnet Umich and CHIC I2 POP
- REDDnet Vanderbilt to Atlanta I2 POP
Internet2 and Cisco Telepresence [Aaron Brown]
A behind-the-scenes look at the planning and setup for the Cisco Telepresence Demo shown at Wednesday's General Session.
Aaron Brown presented on "Regular Latency Monitoring Or: How I Learned to Start Worrying and Hate the Jitter"
View the slides on Latency Monitoring
Highlights
- Throughput testing is being done by LHC, ESNET and others
- But what about latency testing for latency sensitive applications such as Cisco Telepresence Demo?
- Cisco Telepresence Limits
- 10 ms jitter
- 160 ms delay
- 0.05% loss
- Polycom Limits
- 30-35 ms jitter
- 300 ms delay
- <1% loss
- Goals:
- Measure delay/jitter/loss between points
- Be able to fix any issues that come up
- Approach: Deployed measurement machines at the endpoints and set up regular tests between the machines
- Tools: : perfSONAR-BUOY and OWAMP
- Analysis software was written or modified to make it easy to view and understand the data.
- Monitoring included analysis of network health, host health, path status, highly utilized link, and cross trafic
- Several potential issues identified, and all were solved and verified through diagnostics and monitoring
Interoperability Testing with Europe - Update
Tom Throckmorton presented an update on the Multi-Vendor 10 Gigabit Testing that Matt Zekauskus and Tom discussed at the Performance WG at the Feb. 2009 Joint Techs in College Station. The goal is to determine how well 1G and higher speed circuits work between differing vendor hardware over long distance
Tom reported that interoperability testing on connecting Internet2 and Dante is ongoing. There was a prior set of tests at 1 gig reported on in February 2009. There had been limitations and problems w interruptibility .
Since the last update, in Feb 2009, Dante and Internet2 and CNC have done product evaluation on interrupt testing from ? company out of Denmark. This has been an opportunity to jointly evaluate the interrupt testing before turning the circuit over to production.
This system is a tenth of the cost of other interrupt testers. SPGA systems. Very high performance led to low cost.
Deante had received testers in Jan 2009. Internet2 got the testers towards the end of Feb 2009 and had an aggressive timeframe for completing testing. Having equiment on hand allowed us to complete the testing in timely fashion and also to complete tests with a higher degree of confidence than w using ?BCs for commodity systems. Issues around driving scars?? at sufficient rate with this test equipment in place. We drove the circuit almost to full capacity. Did suite of tests at various packet sizes. We were able to iterate through the same set of tests independently and get the same results consistently, leading to high confidence in the numbers.
One issue emerged as a result of this testing. In one direction, the frame size got below 64 bites. After a number of back-to-back tests, and repeated tests to be sure we got numbers accurately, we surmised a limitation on ? in one side of connection. Based on ... not a problem interuptablility wise.
Overall we got excellent results from this gear. More consistent than out of PCs.
Another positive thing was interaction wtih the vendor. They were eager to please and responsive to issues we raised with them. Made corrections for us based on feedback we had given them. The underlying circuit was turned over for production in mid april and it's been carved up in different ways to serve connections between Dante and a couple of points in the U.S.
Hope to provide a general interrupt test report that will be delivered at end of May and a product evaluation around end of June.
Some ideas surfaced on how we could make improvements in using commodity systems to do testing, and there are more things to look at.
Dante is interested in purchisng these testers for some use. Not sure they are attractive otherwise.
If some one wants to learn more, contact Tom Throckmorton or Matt Zekauskus.
Assembling a PERT team in the U.S.
http://www.geant2.net/server/show/conWebDoc.1602
Discussion of establishing a team of network engineers representing each of the RONs that would be available on a rotating basis to troubleshoot complex, multi-domain issues.
WG Charter
Carla presented the draft WG Charter, and invited comments. Carla would like volunteers to serve with her as a co-chair of the working group.