National Laboratory for Applied Network Research

National Laboratory for Applied Network Research
Measurement and Operations Analysis Team

Quarterly Report for
April 1999 to June 1999

  1. Summary

    Significant progress had been made in planning for the next round of passive monitors, based on the New Zealand DAG3 technology. We are planning on deploying 25+ machines in close collaboration with Internet2 sites. We expect that during the third quarter we will begin to order DAG3 cards in sufficient quantities.

    Effort with the Active Measurement Program included expanding and stabilizing the system. More machines have been added, and the central data collectors have been evolving towards redundant machines.

    We continued to work on BGP analysis and visualization, and presented initial visualization prototypes.

    Work has progressed with version 2 of the Cichlid visualization engine.

    Two workshops were held at SDSC, in collaboration with Internet2 and NGI participants, one on the 29/30 June 1999 among measurement sites, and the other on the 1 July 1999 among a small set of people with significant network analysis background in the community.

    We still have mostly students working on the project, but are attempting to hire some staff.

  2. Passive measurements

    Instigated by Internet2 staff, we are planning to deploy approximately 25 more passive monitors in the foreseeable future. Those would be a mixture of ATM and PoS, as well as OC12 and OC3. We also still intend to deploy two DS3 monitors, for which we already have the monitoring cards.

  3. Active Measurement Program

    AMP summary:

    As of 7 August 1999, we have 84 AMP machines sent out. NSF has 154 awarded HPC sites, three of which are dual-awardees:

    Five AMP sites are not HPC awardees:

    That all considered, the list of 154 HPC awardees leaves 72 institution who have not committed to deploying an AMP machine. The statistics are: 

             84  sent out AMP machines
        + 3  dual-awardees
        - 5  non-HPC
        ---
         82  HPC sites served
        +72  remaining sites
        ---
        154  HPC institutions

    The 84 sites are:

    AMP machines are now deployed at about half of the HPC awardee institutions. The list of the remaining 72 HPC institutions with an AMP opportunity as of the beginning of August is:

    Todd Hansen assisted Tony McGregor with the AMP project. One of his projects was to make a grid data display based on the existing status page.

    This data report page generates data on three different aspects of the data, RTT, loss and numbers of samples missing. The biggest benefit of this grid mechanism is that you can compare the differences between the different systems. Another tool that Todd developed was an AMP raw data query page.

    This is available for users who wish to have access to the raw data to do their own analysis. Tony and Todd have also been working on setting up a mirror for the central data collection machine, amp, called volt. They want to have a redundant system in case one should crash. It will also allow them to distribute some of the data processing jobs to lighten the disk load.

    Ryan Kassel, is helping to coordinate the deployment of new AMP monitors, contacting new sites, and to help manage information supplied by those sites hosting AMP monitors. Currently, Ryan is engaged in several different projects. His first aim is to improve usability of the data obtained by AMP monitors (both for researchers and non-technical people). So far, in conjunction with Jeff Brown, he has authored a page where a researcher can download a full days worth of AMP data (about 92M compressed) and then, use a ready-made Cichlid server to view the data animated minute by minute. In the future, he plans to work with Jeff and Todd to implement a Cichlid animations page that shows daily, weekly, and monthly AMP data to viewers via the world wide web.

    This page can be viewed at: http://moat.nlanr.net/AMP/~rkassel/amp/data/amp_data.html

    Ryan is also assisting Tony and Todd to refine and debug the AMP data collection and processing. The growing amount of data brings more complexity and more opportunities for optimization and simplification. One way to manage this complexity is with a more user-friendly interface for the AMP web pages, which is where researchers go to view data gathered by the AMP monitors. Ryan is currently working with SDSC's Mike Gannis and Edna Nerona to update the AMP web pages.

    Finally, Ryan will be working on a paper for "Event Triggers", that will help network administrators and researchers identify and evaluate the significance of problems over networks.

  4. BGP analysis

    Neil Cotofana learned perl for the purpose of bgp data manipulation. For the time being, he decided to use preexisting visualization and bgp routing-monitoring tools, as they are suitable for my needs. He undertook some preliminary visualizations with CAIDA's Otter visualization tool and their gnuplot perl module. Neil most recently started using Cichlid for 3-D visualizations of time series distribution and changes of prefix lengths based on the University of Oregon's RouteViews project data, and similar time series visualizations and analysis for the vBNS, specifically, using the vBNS gated BGP session logs.

    Work will continue on the above, looking at longer time frames, but it will also focus on growth, changes, and stability in the vBNS, based on the network/prefix information contained in the vBNS gated BGP session logs. Neil will additionally investigate short term fluctuations and phenomena of the network/prefixes, also based on the aforementioned data logs.

  5. Cichlid visualization tool

    Jeff Brown gave Cichlid a new, more object-oriented architecture, and got the old bar-chart graphs updated to use it. The old architecture (all the function calls in the server API, etc.) was bound to bar charts, since it was written only with them in mind. The user had one set of calls, and they all referred to X and Y coordinates, and height. The internal code also processed everything in these terms, which would have made it a nightmare to add support for something of a different shape.

    Under the new architecture, the user asks that an object be created -- the constructor function they call determines the object type and interface. Then, each object has specific calls to manipulate that object appropriately, reading and writing data, setting labels, etc.

    This makes the user's (server writer's) job much more logical, but it also greatly simplifies the internals of Cichlid. Instead of saying, "IF this is a barchart, THEN allocate N bytes of memory and go here, ELSE IF it's a vertex set, make sure the coordinates are valid, etc. etc. etc.", most of the Cichlid code becomes abstract with regard to the data: "Send yourself to this socket, draw yourself at this location, etc. etc. etc."

    Then, to the new architecture Jeff added the now-infamous vertex/edge graphs. Barcharts just provide an animated bar chart, much like one expects from a favorite spreadsheet. Vertex/edge graphs (which CS and Math people tend to just call "graphs") consist of points (vertices) which are interconnected by edges. They can be used to model networks, freeways, BGP trees, variable lifetime in an optimizing compiler, and other things.

    Jeff also wrote a library for encoding/decoding data that addresses some alignment problems that could lead to crashes on some processors. It also greatly simplifies sending and receiving data.

    Next came the integration of the source tree with the Windows-adapted version. Using the copy of the old Cichlid that was ported to Win32, Jeff identified the "problem areas" in the code and wrote a module to isolate them. With some Windows logistical help from Todd, once that module was working the rest of the code became quite portable, and the same code that compiles under unix will compile under Win32.

    Finally, with the new version reasonably stable, Jeff wrote some demo servers -- most notably the "vbns_fake" server which is a 3-D representation of the vBNS, showing traffic on the network links -- which were displayed at the measurement workshop on June 29th and 30th, and will be shown as part of a presentation on remote data visualization at SIGGRAPH '99.

    Here is a screen shot of the "bar chart" mode, with data coming from a test server. This barchart actually has multiple bars "stacked" on top of each other, which the original code would not support. Though it may have been painful to add them to the old code, Jeff coded and tested multi-stack support under the new architecture in under two hours.

    This is an overhead screen shot of the "vbns_fake" server. A few sites and links have been clicked on, and the data server has provided informative text which is drawn in little "callout boxes".

    Here is another shot of the "vbns_fake" server, antialiased to a white background for printing.

    This is an animated gif of a look through the "vbns_fake" server in action.

  6. Published Paper

    Tony McGregor had a paper accepted for publication at the ASIA Pacific web conference ( http://www.comp.polyu.edu.hk/APWEB99/) called "Architectures to support HTTP traffic on Symetric High-bandwidth*delay circuits".

  7. Workshops

    The proceedings of the workshops held at SDSC, in collaboration with Internet2 and NGI participants, are available at http://moat.nlanr.net/Workshops/.

    The "Measurement and Analysis Collaborations Workshop" one held on the 29/30 June 1999 was attempting to foster collaborations among measurement sites.

    Workshop Agenda for Tuesday, 29 June 1999

    Introduction: Hans-Werner Braun, National Laboratory for Applied Network Research

    Goals and Measurements/Analysis Needs for High-Performance Communities

    Activities Overview for Measurement/Analysis: What is being measured? What is being analyzed? What is being made available?

    Supporting Measurement/Analysis Activities

    Workshop Agenda for Wednesday, 30 June 1999

    Presentations by Site Representatives: What do we measure? What do we analyze? What do we make available? Expectations for collaborations and concerted activities. Measurement/analysis issues.

    Discussions and Demonstrations

    The "Challenges and Opportunities for Measurement and Analysis in a High Performance Computing Environment" workshop held at the 1 July 1999 was creating a focus among a small set of people with significant network analysis background in the community.

    Workshop Agenda for Thursday, 1 July 1999

    Introductions: Hans-Werner Braun, National Laboratory for Applied Network Research

    Status and Goals

    Specific Work in Progress

    Related Work

    Discussions

    Todd Hansen did the initial writeup and organization of the proceedings for the two workshops. Mike Gannis improved it with his technical writer skills, and SDSC's Gail Bamber did the layout work. The proceedings will be finished around the end of August.

  8. Staffing

    Four students are working on MOAT related projects. In addition positions are posted for a manager, a system administrator, and a part time administrative support person.