Development and management of the network analysis infrastructure (NAI) continued this quarter, with good progress being made in each of our activities. The creation and progress of the Internet Trace User community has been very successful. Through this community building effort, we are exchanging information and gathering feedback regarding our passive measurement and analysis plans and strategies.
Full implementation of the system manager was achieved this quarter. It has been used extensively to update various components of the Active Measurement Project (AMP) mesh and is proving to be a valuable tool. The most useful option so far is "update changes" which allows us to update each system quickly and uniformly. The AMP site data collection functionality continues to mature and be an excellent resource. Data collection has continued at a very high level of reliability with just a few (minimal, limited) outages. (A few of these were attributed to the implementation of new functionalities, such as system manager and IPMP testing.)
Discussions regarding potential AMP collaborations were held with several sites' researchers and administrators (at JPL, Internet2 End-to-End Initiative, Canarie [Canada], Kordic [Korea], etc.). Work began in preparation for running regular iperf tests (and throughput tests in general) over the AMP network.
Performance measurement and analysis activities continued on the wireless network, including work with the Lucent and Tsunami Radio MIBs and the extraction of the SNMP data from them. Some interesting data has resulted, which is undergoing examination.
The development and refinement of tools to use with the large amount of data from our various activities is important to the evolution of each project. A notable achievement in this regard is the C implementation of the tsh2ta.pl Perl script, (which is used on each of the PMA monitors as an initial flows analysis tool). Additions and refinements to the current tools and databases (including the Cichlid servers) continues to improve accessibility and ease of use for researchers and system administrators using our data.
Nine new scientific papers were completed this quarter; seven have been accepted for various conferences and journals, two are awaiting review. A new PMA poster was created; promotional materials (Network Analysis Times, NLANR Packets, posters, etc.) continue to be distributed at conferences and meetings.
The community building effort to address and maximize PMA project planning that began last quarter continues to develop momentum. The Internet Trace User community is a forum for announcements (such as the availability of new traces and conferences relevant to network measurements), and is intended to spark discussion and gather feedback on measurement plans and strategies and specifically what the needs of the community are, (such as which kinds of traces need to be taken, for what durations, during what time frame, etc.). In the future, research done on the traces will also be announced, (such as papers published and new results available on the web). For more information, see the archive of mailings at http://moat.nlanr.net/PMA/Traces/archive and http://moat.nlanr.net/PMA/traces.html.
We continue to devise methods to achieve the most effective deployment of Passive Measurement and Analysis (PMA) monitors with the Dag3 interface technology. The Dag3 optical interface card has a dynamic input signal power range of -14 dbm to -26 dbm. This creates the need for more care in choosing the signal splitters used to tap into access points at network nodes. Procedures to measure the signal levels needed to calculate the split ratios prior to selection and purchase of the splitters were developed, and are being implemented. This work will continue as new locations are being selected for PMA Dag3 monitors. This planning is expected to provide flexibility, resulting in network data collection better supporting research to understand the performance of selected links as the needs of the research community dictates.
Due in part to the successful use of the San Diego Supercomputer Center's (SDSC's) HPSS storage system for our Active Measurement Project (AMP) data, an HPSS account has also been created for PMA data storage and retrieval. This account was created at the end of the quarter, and we are preparing to utilize this resource. The steps necessary to continuing to protect privacy as the PMA HPSS storage account is populated are under development and discussion. Work on these anonymization procedures has begun.
Plans are being developed for the collection of Abilene backbone measurements (ring of monitors surrounding a router); router instrumentation was begun. Extensive planning and discussions were held regarding this collaboration.
A four-hour trace was collected at Colorado State University; analysis of this data is in progress. A five-day continuous trace at NZIX (New Zealand Internet Exchange) was published, including graphs of the data. (http://moat.nlanr.net/Traces/Kiwitraces/nzix.html).
We continue to survey and publish information related to network topology at PMA sites. Preliminary information is available at http://moat.nlanr.net/PMA/Sites/index.html.
The Active Measurement Project (AMP) Site Information and Status database is continuing to mature and serve as a valuable tool in the efficient maintenance and management of the network analysis infrastructure (NAI). Data collection has continued at a very high level of reliability with minimal outages; however, limited equipment failures occurred which were easily diagnosed and corrected. In addition, some outages have occurred that can be attributed to new implementations, such as the system manager and IPMP testing.
Full implementation of the system manager was achieved this quarter. It has been used extensively to update various components of the amp mesh. We have also used it in a few situations to bring systems up to speed and into the mesh. The most useful option so far is "update changes" which allows us to update each system quickly and uniformly.
There has been little AMP infrastructure growth during the last quarter, due in part to the fact that a majority of NSF HPC sites are already participating. Another contributing factor in the deployment slowdown is the need for redesign and development of the AMP monitors. The AMP monitor configuration redesign involves two factors. First, new system boards are needed for the AMP monitors because the previous system board is no longer available. Second, a test is underway to upgrade the OS from FreeBSD 3.0 to FreeBSD 4.2. These investigations and determinations are expected to be completed shortly and will allow for site system upgrades.
Last quarter the use of the HPSS data storage and retrieval system at SDSC was being investigated for use with the oldest AMP data. This was needed to circumvent the need to continually expand the disk storage capability of the AMP data collection servers, AMP and VOLT. The archiver to send older AMP data to the SDSC HPSS has been implemented and refined; it is fully functional.
In the steps required to upgrade VOLT's RAM, significant time was spent on resolving a number of issues with the disk array. We are still investigating what to do about making the AMP/VOLT disk arrays more easily recoverable in the event of a drive failure. The anticipated end result will be a system better able to cope with system failure and easier and less time consuming recovery procedures. We are investigating software RAID in this regard. (Currently, it does not appear that RAID will be the appropriate option, as good performance and ease of recovery are necessary. Therefore, the current best option may be to use separately mounted disks with some kind of space balancing between the disks.)
Figure 1. This graph is an overview of how IPMP end-to-end measurements compare with ICMP end-to-end measurements. Each data point in the graph represents the mean difference between the IPMP data set and the ICMP data set for the 24 hour period. Both IPMP and ICMP measurements were taken 30 seconds apart. The measurements were conducted on the High Performance Connection (HPC) mesh, using the AMP infrastructure (of approximately 120 monitors). A preliminary analysis of this data shows that about 90% of the IPMP and ICMP measurements taken on this mesh are very similar, i.e., the difference between the average IPMP and the average ICMP measurement is zero. For those datapoints where there is a difference between the two measurement methods, we have found that the majority of those differences are attributable to a few sites, suggesting possible protocol based scheduling at border routers. Graphs taken on weekdays look very similar to each other; however those taken on weekends show significant shape differences. For more information, see http://moat.nlanr.net/~mjl/graphs/.
The IPMP measurement protocol is now deployed over the AMP mesh and we are currently undertaking comparative measurements. (See Figure 1 above for example.) Weaknesses in measurement techniques were discovered during result comparisons of ipmp_ping to results produced with fping-1.20. This lead to further questions, such as: What is the impact of sending warm-up packets before sending the actual measurement packets onto the network? How long before conducting the actual measurements should the warm-up packets be sent? Additional questions are anticipated and will be investigated. Guidelines and a how-to on using the ipmp_ping client have been written. Publicly available versions will follow.
The previous student work on event detection had to be discarded and the event detection algorithm redesigned and the code rewritten preceding the paper submission deadline for the PAM 2001 workshop. The windows concept was maintained. The paper will be published in the conference proceedings. Authors' version available: http://moat.nlanr.net/PubsResources/workinprog/eventdetection.
Work began in preparation for running regular iperf tests (and throughput tests in general) over the AMP network. On-demand tests are the first objective, and should be running in the first part of next quarter.
Discussions regarding the development AMP collaborations have been held with the Jet Propulsion Lab (JPL), the Canarie network in Canada, and KORDIC in Korea. If implemented, each would minimally involve the deployment of an AMP monitor. Discussions have also been held with Internet2 (I2) on how AMP can be involved in the I2 End-to-End performance initiative.
A new set of simulations were created to investigate the sensitivity of the simulator to changing aspects of its design. Also, a new graphic summarizing the routers that are taken, is in the final stages of development.
Performance measurement and analysis activities on the wireless network (HPWREN) have included work with the Lucent and Tsunami Radio MIBs and the extraction of the SNMP data from them. The MIBs show vital information such as link health, link usage, signal and noise levels. Factors affecting network performance can be revealed using this data, as well as in comparison with the corresponding weather data. The data of most interest so far are the received signal levels (which include the minimum, maximum and current levels) and the bit error rate graphs. Also of interest are the comparisons of the performance data with the weather data. The current received signal levels fluctuate with respect to weather variations. In particular, the current received signal levels dip lower as the recorded rain level increases. See Figure 2 below; data and graphs are available at: http://stat.hpwren.ucsd.edu/.
There are issues relating to bandwidth management in the Lucent radios that require investigation, which has begun. There are also some questions relating to some of the performance measurements being done in the Tsunami radios, which are also being researched.
Figure 2. Note that the received signal level (RSL) tends to decrease more than expected with normal fluctuations in response to the appearance of, and increasing, rainfall; and stabilizes to normal levels with decreasing rainfall. We believe that the decrease in RSL which can be seen slightly before rainfall is detected can be explained as follows: In Southern California rain tends to move from west to east; therefore, it is raining in the network path before the rain registers on our sensor.
Other Activities
Some refinements of the weather collection pages were made (corrections, installing new hardware, etc.). A "parts" database was begun on the all the parts for the various pieces of equipment we use.
Several Perl/CGI scripts were written that will allow sites that host our remote AMP and PMA monitors to fill out a survey with details regarding the specific environment in which their monitor is located. The scripts also allow viewing and searching (internally only) of the returned survey replies. These will be finished at the beginning of next quarter.
The development and refinement of tools to use with the large amount of data from our various activities is important to the evolution of each project. A notable achievement in this regard is the C implementation of the tsh2ta.pl Perl script, which is used on each of the PMA monitors as an initial flows analysis tool. Rewriting this tool in C has improved time and memory performance of the passive monitors substantially. This is the first step towards the possibility of real-time analyses and/or continuous traces. Also, for some of our more heavily trafficked sites, where standard analyses have not been consistent due to exhaustion of resources, this implementation will have a facilitating effect. An improved Web interface and new site information page for PMA was also created.
Additions and improvements to the Cichlid servers continues. A Cichlid server that visualizes information gathered from a trace file was written. Work continued on the seismic Cichlid server. The wireless Cichlid server was updated and improved. The throughput Web page for the wireless network was improved; information can now be displayed in raw format for any time period.
An initial prototype site summary graph for AMP was developed. This graph will allow users to see changes in groups of machines over time. All sites will be graphed based on their average measurements each week. As this develops, it will be possible to identify subsets of sites that are affected by a change, or event.
Work on the next issue of the Network Analysis Times is close to complete; the new issue will be published in April, available at: http://moat.nlanr.net/NATimes/. Printed copies of the full issue will be created for distribution at conferences and meetings (in particular, the PAM2001 workshop, later in April). A new issue of NLANR Packets was published in late January, available at: http://www.nlanr.net/NLANRPackets/v2.1/. The Table of Contents Summary is available as a single-sheet hard copy, which is used for distribution.
A new PMA poster was completed. Distribution of these new PMA posters began at the APAN/TransPAC/NLANR/Internet2 Techs Workshop in January and has continued. A new Web page was created to provide access to pdf copies of our current posters for AMP, PMA, Cichlid, and NLANR overall; please see http://moat.nlanr.net/PubsResources/posters.html.
Seven papers were accepted this quarter, an additional two are awaiting review:
Gao, J., V. P. Roychowdhury, R. Ritke, and I. Rubin. IP Packet Level vBNS Traffic Analysis and Modeling. Submitted to SPECTS 2001.
GAO, J. and R. Ritke. Long-Range-Dependence and Multifractal modeling of vBNS Traffic. Accepted by 2001 Applied Telecommunications Symposium (ATS'01), April 22-26, 2001, Seattle, Washington.
McGregor, A.J. and H-W. Braun. Automated Event Detection for Active Measurement Systems. Accepted by PAM2001, the Netherlands, April 2001.
McGregor, A.J. And M. Pearson. Sensitivity Analysis of Event Driven Simulation Results. Accepted by PAM2001, the Netherlands, April 2001.
McGregor, A.J., M.W. Pearson, and J.C. Cleary. Improving the performance of HTTP over high bandwidth-delay product circuits. Accepted by the Communication Networks and Distributed Systems modeling and Simulation Conference (CNDS) '01.
Micheel, J., H-W. Braun, and I. Graham. Storage and bandwidth requirements for passive Internet header traces. Submitted to the NRDM workshop, May 2001.
Micheel, J., I. Graham, and N. Brownlee. The Auckland data set: an access link observed. Accepted by the 14th ITC Specialists Seminar on Access Networks and Systems, Barcelona/Gerona, Catalonia, Spain, April, 25-27, 2001.
Ritke, R., X. Hong, and M. Gerla. Contradictory Relationship between Hurst parameter and Queueing Performance (extended version). Accepted by Telecommunication Systems for inclusion in a special issue devoted to the publication of extended versions of the best papers presented at SPECTS. February 2001.
Ryu, B., D. Cheney, and H-W. Braun. Internet Flow Characterization -- Adaptive Timeout and Statistical Modeling. Accepted by PAM2001, the Netherlands, April 2001.
NLANR/MOAT was represented this quarter at the following meetings:
Ronn Ritke attended a Network presentation by, and gave an NLANR presentation to, a group of Chinese Delegates, March 19 at San Diego Supercomputer Center (SDSC).
NPACI/SDSC All-Hands Meeting, 26 February, San Diego. (Student researcher Justin Fields presented a demonstration of the Cichlid wireless and seismic servers at the poster session. Please see Figure 3 below, for a sample image. AMP, PMA, Cichlid, and the general NLANR poster were presented by Ronn Ritke. Mike Gannis facilitated the NLANR poster exhibits. In addition, several members of the group attended various sessions of the meeting.)
Figure 3. This image is a Cichlid 3-D visualization system generated graph showing the analysis of a trace file. Trace files are 1.5 minutes to 2 minutes in length (time, x-axis).The analysis shows the network statistics for the trace file; each metric is shown per second/10000. From left to right on the y-axis: smtp bytes [red], ftp bytes [green], html bytes [blue], udp bytes [brown], tcp bytes [1st gray]; the number of bytes and packets form the last two gray rows, respectively. Please click on image for larger view with more detail.
APAN/TransPAC/NLANR/Internet2 Techs Workshop, East-West Center, Honolulu, Hawaii, 28-31 January 2001. (Ronn Ritke presented; Jörg Micheel attended.)
Spring Internet2 Member Meeting, 7-9 March 2001, Washington, DC. (Ronn Ritke)
The NLANR meeting (Feb. 14) and NSF-NLANR meeting (Feb. 15) were attended (teleconference) by Hans-Werner Braun and Ronn Ritke.
I2 End-to-End Planning Meeting, Ann Arbor Michigan, Jan. 9, 2001. (Ronn Ritke)
Preparatory work and networking were done with a view to creating future collaborations in a number of areas this quarter. The community building effort regarding our passive measurement activities (PMA) is establishing an ongoing dialogue with researchers who use our traces in their work. (For more information on this effort, please see the PMA section above.) Also, new contacts have been made for potential collaboration on passive measurements with John Hicks at Indiana/TransPAC and researchers at AARNet and the Pacific/Northwest GigaPOP.
We are working with Internet2 and Abilene administrators and researchers in order to establish a monitoring ring around an Abilene backbone router (PMA) and how AMP can be involved in the I2 End to End initiative. (Jörg Micheel, Ronn Ritke, and Tony McGregor)
Ronn Ritke was chosen to be a member of the Internet2 End-to-End Performance Design Team and has participated in ongoing conference calls, a meeting in Michigan, and a teleconference. He helped with the Design Team Report and reviewed other documents and worked with Matt Zekauskas regarding details for the measurement infrastructure.
Discussions have been held regarding AMP collaborations (and possible AMP deployments) with the Jet Propulsion Lab (JPL), with the Canarie network in Canada, and the Korean KORDIC site. (Tony McGregor and Ronn Ritke)
A collaborative effort with SDSC's SKIDL lab (an off-shoot of the Data Mining group) has been initiated, wherein the data mining tools and techniques of the SKIDL lab will be applied to our archived BGP data. (This collaboration was initiated by student researcher, Neil Cotofana.)
Ronn Ritke participate in the planing, coordination, and organization of the Extreme Networking WORKSHOP 2 -TECHNOLOGIES, held January 8-9, 2001, San Diego Supercomputer Center, UCSD.
We are working with John Hicks and others at the Indiana University GigaPop and TransPAC regarding a potential collaboration on passive measurements and analysis. As there are differences in data handling needs, we have referred them to WAND to obtain Dag cards directly.
We also maintain several continuing collaborations:
Continued work with Phil Dykstra and Cindy Dykstra, WareOnEarth (WCI, Inc.), regarding the AMP peer network with Defense Research and Engineering Network (DREN), as well as other activities.
Continued work with Ian Graham and others in the Waikato Applied Network Dynamics (WAND) group, regarding the Dag3 cards for the PMA machines, and other activities.
Continued work with Frank Vernon (Scripps Institution of Oceanography [SIO] geophysicist) regarding his remote earthquake sensors and methods to visualize his seismic data.
Continued work with Nevil Brownlee of the University of Auckland and CAIDA, on passive measurement and analysis (PMA) related activities.
Continued work with Hank Nussbacher (Israel InterUniversity Computation Center, Tel-Aviv University) on collecting OC3 monitoring data between Israel and the U.S. and related activities.
Continued work with other NLANR sites and their researchers and administrators, including weekly conference calls, reviewing documents for the "NLANR2" proposals, and other activities.
Continued work with Kevin Walsh and Jay Dombrowski of the network systems group at SDSC (SDSC Enterprise Network Services [ENS]) on creating a data compute engine/repository containing a replica of AMP data, for our use to conduct intensive analyses on the AMP data, and for their use to correlate with other data to determine the performance characteristics of application behavior between the NPACI partners and SDSC. This repository will be a special purpose server that contains a replica of the AMP data. The benefit of this system is that it will allow us to do heavy processing of the data without impacting the performance of our data collectors (AMP and VOLT) and Web servers.
Graduate student Pavana Yalamanchili, gave a presentation this quarter to Dr. Ramesh Rao's group (Electrical and Computer Engineering researchers and students) about the SNMP data that she is working on, the analyses that are being performed, and how it is being used in the HPWREN project. Dr. Rao is the Director of the Center for Wireless Communication at UCSD; he is Pavana's academic advisor.
Graduate student Jamie Curtis wrote a paper for the New Zealand Computer Science Research Students Conference - "Review of Bandwidth Estimation Techniques"; he will be presenting in mid-April.
Additional student activities are throughout this report. Undergraduate students Neil Cotofana's initiation of a collaboration involving our BGP data and Data Mining techniques (in collaboration section), David Cheney's work on the PMA Web interface and retrieval/analysis tools for the data (in documentation section), and Justin Fields work with Cichlid servers (in documentation and tools section).
Passive Measurement and Analysis (PMA)activities:
Active Measurement Project (AMP) activities:
Measurement and Analysis on the wireless network (HPWREN):
-30-