The major areas of progress in this quarter have been in
- creating local machine storage and computing engines in support of the analysis infrastructure system
- creation of a distributed 3D real-time visualization engine
- groundwork for the active monitor development and deployment
- continuation of the work on passive monitors and network analysis
- preparational work for the Supercomputing'98 conference
- preparational work for deployment of OC12 monitors
As an overall objective we continued working on a system to provide a network analysis infrastructure (NAI), which collects data from multiple sources, including passive and active monitors, as well as SNMP and BGP information. The system also has local resources in San Diego for data storage and computation.
The external interface to this system is principally web and ftp access, via the moat.nlanr.net server.
The central NAI servers consist now of six machines. Those are
- nai.nlanr.net
160MB (18*9.1GB) disks, 256MB memory, dual 450MHz PII
file server, initial collector of network data - moat.nlanr.net
160MB (18*9.1GB) disks, 256MB memory, dual 450MHz PII
file server for public data (web and ftp access), email - four network analysis computing engines
each 18GB (2*9.1GB) disks, 256MB memory, 450MHz PII
support for researchers doing network analysis
The machines are interconnected via an internal 100BT Ethernet, which has no other machines attached. All machines are connected via additional 100BT Ethernet interfaces to SDSC's external server network. The machines are housed in a rack in SDSC's machine room:
These machines form the principal support role for the measurement data gathering machines deployed throughout the country.
One of the principal needs for making network analysis results available and comprehensible is the availability of visualization tools. We have mostly used the xmgr tool for 2D graphs. While being an excellent tool for 2D data visualization, the lack of a third dimension proved limiting. Available 3D data visualization tools, especially based on VRML, appeared somewhat limited for creating a data importing capability, and also had a user interface more conducive to object visualization. Jeff Brown, using OpenGL functions, created Cichlid tool, which currently runs in Linux (including with 3DFX based accelerator cards), FreeBSD, and SGI environments.
For the conceptualization of Cichlid it was important to understand tradeoffs relative to the location of the visualization. One extreme point would be to do real-time visualization right on the measurement machine, such as an OC3 monitor. Since typically we are physically fairly far away from the monitors, this would largely be useful for demonstration purposes. Another extreme is to do visualization only off already collected data, i.e., data off a hard disk. Obviously this would preclude real-time visualizations. A third possibility is a distributed server/client environment, with a server collecting data on a monitoring machine and analyzing it, then sending the result data to a remote client. The client then functions as the visualization engine, consuming the data while displaying it as part of a Cichlid visualization. In other words, the data generation and analysis is remote from the actual visualization, possibly compute intensive in and of itself, while a fast data visualization with a 3D user interface consumed computing power at the user-local machine.
The following two graphics are example outputs of the visualization engine.
Examples (some of which are shown above) include bucketed packet length distributions over time, and network usage based on a network address matrix.
Deployment and use of the passive OC3 monitors continued, which more than ten active machines throughout the country, and more site anticipated in the future.
As shown in the next two images, the monitors use off the shelve hardware, both for the machines themselves, as well as for the optical splitters.
The optical splitters are small and easy to insert within seconds, and, due to their size, easy to house within communication equipment.
The primary interface we use for data collection right now is OC3, based on FORE PCA-200E cards. In addition we can use data off Ethernet and FDDI interfaces, using a modified version of tcpdump (usage explained at http://moat.nlanr.net). With help of CAIDA staff we have a prototype version of an OC12 monitor, but are experiencing hardware difficulties at the card/motherboard interface. Later in the year we expect a different type of OC12 monitor cards based on work by Ian Graham and his team at the University of Waikato in New Zealand. These cards will also be Packet-Over-Sonet capable. In addition we also expect two pairs of DS3 cards from Ian Graham, for creating DS3mon machines.
While initially collecting data every hour on the OC3mon machines, then doing some analysis of the data, and only sending results to the central servers, we have started to collect regular packet traces instead, and making sanitized versions available, as well as some analysis results. Traces are available via web and ftp access. Analysis results are available in the Datacube.
- Graphs
- Routes
- Tabular
- Graphical (using Otter)
- 3D Visualization
- The IP Measurement Protocol (IPMP)
The data collected is being made available through a prototype web interface (see http://moat.nlanr.net/ActMon/) These pages include long term, weekly and daily (including the current day) graphs of RTTs and loss. There are daily graphs for every day since measurement began. The graph for the current day is live. Example graphs are shown below.
Long term
Weekly
Daily
Path records are recorded at 5 minute intervals. These are available in both tabular and graphical form. The latter uses Otter (developed by CAIDA). Examples are shown below.
In addition to the web interface the active measurement data can be viewed through a 3D visualization. A data server for Cichlid (an NLANR/MOAT developed, OpenGL, 3D graphing tool) has been developed. This provides alternative visualizations of the data which make the relationship between RTTs and loss to different sights more visible.
Two different presentations have been found to be particularly useful. The first is an active bar chart showing the RTT to different sites against the time. Loss is shown as a negative bar. A moving time series is created by introducing new samples at one end of the display and moving the older samples towards the other end.
The second display is similar but the data is presented as a rendered `nurbs' surface instead of as bars. This display can be thought of as a network `terrain'. A smooth surface indicates a network that is behaving consistently where as a rugged or peaky surface indicates areas and times of instability. This display highlights major events while reducing the visual impact of small disturbances.
In both displays local events are seen as bands across the display while events at a single remote site cause a band down the length of the display. Events at other positions cause displays between these two extremes.
Examples are shown below.
Bar Graph
Network Terrain Map
The protocols that are currently used for active measurement (such as ICMP) have significant weaknesses when used for measurement. This has been widely noted in the measurement community. To stimulate thinking about solutions to these problems a new protocol (IPMP) is being considered.
This protocol has been designed to allow routers to participate in active measurement without compromising their core packet forwarding role. Special care has been taken to make the impact on the router minimal. The protocol supports simultaneous delay and path determination with a low impact on the network. More detail is available at: http://moat.nlanr.net/ActMon/IPMP/
We continue to receive daily vBNS SNMP data trees, based on our collaboration with the vBNS team, but no active work is currently going on to analyze the data.
Two sources of BGP information are being collected: a daily snapshot off a route server in Oregon, and a continuous BGP session with a vBNS router. The latter is to capture short term routing fluctuations with a log file, while the former gives us more systemic Internet routing information. No active work is going on in analyzing the collected BGP data either.
Significant amounts of work went into the preparation for the Supercomputing'98 (SC98) conference in November in Florida. SC98 requirements were also the initiator for the development of the 3D OpenGL based Cichlid software by Jeff Brown. The SC98 demonstrations will include work by Tony McGregor on active monitoring tools, and Brynjar Viken on distributed visualization based on serves running on OC3mon machines. Tony McGregor and Brynjar Viken will attend Supercomputing'98.
Most of the OC12mon development work is being undertaken by CAIDA staff. However, significant problems arose from interactions between the machine motherboard PCI bus and the OC12 cards. This is continued to being worked on, while at the same time independent contingency cards from the University of Waikato in New Zealand are being developed by Ian Graham and his team. If these cards work as expected (we anticipate delivery of one pair of cards in the December 1998 to January 1999 time frame), these cards may become our primary choice in the for future machines. Those cards are also supposed to be POS capable, which will be of great value for Internet2 collaborations. We also expect two pairs of DS3 cards in the same time frame. Both cards are being developed for a Linux environment.
Guys Almes and Terry Rogers of Internet2 have repeatedly expressed interest in collaboration on measurement and analysis objectives with NLANR, particularly in the area of passive monitoring. We have responded expressing interest in such collaborations.