Performance WG Face-to-Face Meeting
at Internet2 Spring Member Meeting
Monday, April 27, 2009
Internet2 Update [Jeff Boote]
An update on the software releases available for Internet2 performance and measurement tools and a preview of the roadmap.
REDDnet's Use of Performance Tools [Ezra Kissel]
REDDnet has disk depots that cache data placed throughout the U.S, and they have been experiencing challenges moving data between the depots.
Highlights:
- REDDnet provides "Working storage" to help manage the logistics of sharing, moving and staging large datasets across wide areas and distributed collaborations.
- Participating Institutions: Vanderbilt, Tennessee, Stephen F. Austin, NC State, Nevoa Networks, Delaware
- Host Sites: Caltech, Florida, Michigan, ORNL, SDSC, TACC, UC Santa Barbara (Stephen F. Austin, Tennessee, Vanderbilt)
- Tools used for performance monitoring inlcude:
- OWAMP (3.1)
- BWCTL (1.3)
- NDT client (3.5)
- perfSONAR-PS perfSONAR-BUOY (regular testing framework for bwctl)
- Performance monitoring includes:
- TCP tuning on all hosts
- Picked a set of hosts to investigate from the "worst offenders"
- Divide and conquer approach (Test from depot to POP, Divide path, Narrow down where the problem "ends")
- Examples:
- REDDnet Umich and CHIC I2 POP
- REDDnet Vanderbilt to Atlanta I2 POP
Internet2 and Cisco Telepresence [Aaron Brown]
A behind-the-scenes look at the planning and setup for the Cisco Telepresence Demo shown at Wednesday's General Session.
Aaron Brown presented on "Regular Latency Monitoring Or: How I Learned to Start Worrying and Hate the Jitter"
View the slides on latency monitoring
Highlights
- Throughput testing is being done by LHC, ESNET and others
- But what about latency testing for latency sensitive applications such as Cisco Telepresence Demo?
- Cisco Telepresence Limits
- 10 ms jitter
- 160 ms delay
- 0.05% loss
- Polycom Limits
- 30-35 ms jitter
- 300 ms delay
- <1% loss
- Goals:
- Measure delay/jitter/loss between points
- Be able to fix any issues that come up
- Approach: Deployed measurement machines at the endpoints and set up regular tests between the machines
- Tools: : perfSONAR-BUOY and OWAMP
- Analysis software was written or modified to make it easy to view and understand the data.
- Monitoring included analysis of network health, host health, path status, highly utilized link, and cross trafic
- Several potential issues identified, and all were solved and verified through diagnostics and monitoring
Interoperability Testing with Europe - Update
Tom Throckmorton presented an update on the Multi-Vendor 10 Gigabit Testing that Matt Zekauskus and Tom discussed at the Performance WG at the Feb. 2009 Joint Techs in College Station.
Tom reported that interoperability testing on connecting Internet2 and Dante is ongoing . prior set of tests at 1 gig. Last update we reviewed some of the work using standard PCs for doing testing. Talked about some of the limitations and problems w interuptabliltiy .
Since the last update, in Feb 2009, Dante and Internet2 and CNC have done product evaluation in tester from ? company out of Denmark. This has been an opportuntiy to do joint evaluation on the interrupt testing before turning the circuit over to production.
10th of cost of other interrupt testers. SPGA systems. Very performant that led to low cost.
That's where we were a beginning of Feb.
Deante had received testers in Jan 2009. We got ours towards end of Feb and had an aggressive timeframe for completing testing. Having equiment on hand allowed us to complete that in timely fashion compared with before and also to complete w a higher degree of confidence than w using ?BCs for commodity systems. Issues around driving scars?? at sufficient rate with this test equipment in place. We drove the circuit almost to full capacity. Did suite of test at various packet sizes. Able to iterate thru same set of tests independently and get the same results consistently. Very high confidence in numbers we were seeing there.
One issue emerged as a result of this testing. In one direction we saw frame sized got below 64 bites. After a number of back-to-back tests, and repeated tests, to be sure we got numbers accurately, we surmised a limitation on ? in one side of connection. Based on ... not a problem interuptablility wise.
Overall we got excellent results from this gear. More consistent than out of PCs.
Another positive thing was interaction wtih the vendor. They were eager to please and responsive to issues we raised with them. Made corrections for us based on feedback we had given them. The underlying circuit was turned over for production in mid april and it's been carved up in different ways to serve connections between Dante and a couple of points in the U.S.
Hope to provide a general interrupt test report that will be delivered at end of May.
Space on Internet2 reserved for...
Also, product evaluation sometime around end of June.
A couple of interesting findings.
Ways we could have made improvements in how we were using commodity systems to do testing.
We have more things to look at.
Dante interested in purchisng these testers for some use. Not sure attractive otherwise.
If some one wants to learn more contact Tom or Matt
Assembling a PERT team in the U.S.
http://www.geant2.net/server/show/conWebDoc.1602
Discussion of establishing a team of network engineers representing each of the RONs that would be available on a rotating basis to troubleshoot complex, multi-domain issues.
WG Charter
Carla presented the draft WG Charter, and invited comments. Carla would like volunteers to serve with her as a co-chair of the working group.