The specific work to be performed under the NLANR
agreement includes technical and engineering support and overall
coordination of the vBNS
connections at the five supercomputing centers and selected
research and
education sites, as well as testing and measurement of the vBNS
and related Internet
performance characteristics.
In November, NLANR participating institutions met during
Supercomputing '96
(SC ' 96) and
agreed that future NLANR activities would be focused into three
topical
areas: User Services,
Engineering Services, and Research. The FY1997 Program Plan
describing planned activities in these areas is available at http://www.nlanr.
net/Reports
/progplan.html.
Where
appropriate, the site specific discussions in this report are
divided into
these three categories.
During the second quarter of FY1997, all sites provided ongoing
engineering and general
support
related to the vBNS
operations and applications and participated in the NLANR caching
project http://www.nlanr.net/Cache.
Other NLANR-wide activities during this period include:
A NLANR/vBNS Workshop was held at MCI- Reston on February
3rd. Representatives
from each of the NLANR sites attended. Minutes are available at
www.nlanr.net/VBNS/Minutes/970203.html>
Details on MCI/vBNS activities during this period are covered in
their monthly and quarterly reports for the vBNS Cooperative
Agreement, available at www.vbns.net, and in
the VTCC telecon minutes cited above. Highlights of activities
during this quarter include:
January:
vBNS Engineering completed the replacement of LightStream
2020 ATM switches at Perryman (PYM), Hayward (HAY) and Downer's
Grove (DNG).
BGP peering session between the vBNS and NASA Internet at the
MFS NAP was activated.
A new operating system was installed on the NetStar at
SDSC.
Further enhancements to OC3MON statistics access were
completed: the web page
interface now gives reports on Heavy-Hitters with address/host
name identifiers,
and AS-matrix reporting includes identification of Autonomous
system by
organization, and in some cases, country and region.
February:
The vBNS Engineering group completed the cutover to OC-12
backbone.
Further enhancements to OC3MON statistics access were
completed: the web page
interface now gives reports on Heavy-Hitters with address/host
name identifiers,
and AS-matrix reporting includes identification of Autonomous
system by
organization, and in some cases, country and region.
MCI made the official donation of 6 Lightstream ATM switches
to the University of
Virginia School of Engineering. The total value of the donation
exceeds $1 million.
Switches were put into operation at UVA almost immediately, to
establish OC-3
connectivity between the Medical and Engineering schools in
support of medical
imagery and telemedicine applications. The other switches are
being used by the
Computer Science department in support of ATM-level research
within the Multimedia
group at UVA.
March:
The Vector switches in the backbone were rebooted with FORE
software to complete a
homogeneous ATM layer in terms of the software run among the vBNS
FORE switches and
backbone switches.
The vBNS engineering team met with staff of Cary NOC to
discuss scaling up support
for the vBNS toward production status. The Cary NOC is serving
both the MCI BIPP
and vBNS networks. Two new people are joining the MCI/vBNS staff
to work with the
NOC. vBNS staff will continue to meet with NOC personnel on an
ongoing basis.
John Jamison has designed a standard vBNS node associated
with all MCI locations
that will facilitate the incorporation of new connections. The
configuration will
include a FORE switch, router, Hyperstream switches, performance
host and
measurement monitoring equipment (including the OC3mon
equipment).
The University of Pennsylvania is now connected to the
vBNS.
Bruce Johnson began discussions with CLEO researchers about
possibilities for improved networking to Cornell from their 18
host
Institutions. Most sites are on ESnet, but some have low
bandwidth
connections to ESnet and high bandwidth commodity connections so
considerable study and coordination will be needed before a
proposal can be
submitted.
http://www.nile.utexas.edu/Nile/components/networks/CLEO-sites/Co
rnell/.
Cornell Theory Center is hosting a mirror site for the NASA
Mars Pathfinder
mission, see www.tc.cornell.edu/Mars/
.
Bruce Johnson is co-chairing the Engineering Working Group
for NYSERNet 2000. The
first meeting was held 3/31/97. The intent is to create two
Gigapops in New York State. Initial
work of the group is focused on creating an RFI to be sent out
May 1, 1997. For more
information, see
www.nysernet.org/nysernet2000/index.html.
Engineering:
Cornell's staff are exploring options for routing non-CTC
Cornell traffic over
the vBNS. This effort was
postponed in August 1996 pending guidance from NSF regarding the
policy regarding
supercomputer centers' hosts connecting to the vBNS. Recovering
CTC's Class B (128.84.0.0)
and renumbering internal CTC machines which were outside that
space was initially estimated at
three person-years effort. The alternative of having Cornell
only advertise internal CTC subnets
would require the vBNS to accept about 60 separate routes and be
filtered by the other
institutions. An alternative scheme is to condense CTC machines
into two quarters of the
128.84.0.0 address space and advertise them as two 18-bit nets-
this would still require
considerable renumbering of machines, but only hundreds instead
of thousands. CTC is also
working with Cisco engineers to find a better solution. Cornell
is working with the NSF and
MCI to resolve this situation as soon as possible.
Sanjay Hiranandani and Bruce Johnson assisted MCI Engineering
by replacing fiber jumpers
which had been temporarily installed during the December
upgrade.
Sanjay Hiranandani and Bruce Johnson coordinated and
implemented routing
adds and changes to Cornell's routers recommended by vBNS
staff.
Bruce Johnson and Sanjay Hiranandani used a borrowed HP ATM
analyzer to
obtain a trace of the UNI 3.0 interactions between our IBM 8260
switch and
FORE PowerHub 7000 and sent it to FORE for diagnosis. The
PowerHub will be
used to route between the SP2 and the vBNS.
Sanjay is coordinating with Rich Gallup at SDSC regarding set
up for the Sharma application (CTC/UCSD collaboration).
In support of using IP multicast video conferencing for vTCC
meetings, CTC began
investigation of vic/vat compatible applications for Macintosh
and Windows machines. CTC has
relatively few UNIX machines which have the necessary audio and
video components, so it uses
a shared SGI Indy running mrouted and vic and vat to participate
in vTCC meetings. CTC is
evaluating alternative software, e.g. Apple's QuickTime TV
client, vic for Windows from LBL,
and Netscape's Cool Talk.
Research:
CTC received the IPv6 machine. Sanjay Hiranandani began the
software installations.
Activities Planned for the 3rd Quarter FY1997:
IPv6 project - Sanjay Hiranandani will act as System
Administrator for the IPv6 machines at all the
sites to maintain consistent configurations. CTC's machine will
be
installed on Cornell's FDDI DMZ and will be connected to the
6bone via a
tunnel over IPv4. CTC will also install IPv6 on a AIX 4.2
machine which
will connect to the IPv6 PC using native IPv6.
Sanjay Hiranandani will work with k claffy on setting up servers
for Van Jacobson's pathchar utility on all the IPv6 PCs. Sanjay
will also use pathcar to study local performance on CTC's
internal networks and to the vBNS and commodity Internet.
NCAR hosted the Westnet2 regional GigaPOP meeting in
Boulder, CO on
March 14, 1997. Participants included representatives from NCAR,
University of
Colorado, Colorado State University, University of Wyoming,
University of New
Mexico, University of Utah, Arizona State University, Colorado
School of Mines,
Utah State University, University of Arizona as well as several
vendors and
ISPs. The meeting focused on a regional gigaPOP implementation.
Discussions
were held on various gigaPOP architecture options, gigaPOP
planning issues and
technical issues relevant to implementation with and without I-2
evolution.
Vendor presentations were included to gain awareness of
connectivity options and
prospective costs. The URL for the Westnet2 meeting is:
http://www.scd.ucar.edu/nets/Projects/Westnet2/prev-mtg/index.htm
l
Distributed Mesoscale Prediction (DMP) Project - The Major
Research Instrumentation (MRI)
Program is a National Science Foundation program that "assists in
the acquisition or development of
major research instrumentation at U.S. institutions". DMP is a
project to obtain MRI Program
funding to build a Distributed Numerical Weather Prediction
Laboratory (DNWPL) that uses the
Penn State/NCAR MM5 model running on dedicated computers
initially distributed at NCAR, the
University of Arizona, the University of Utah, and the Desert
Research Institution. This project
assumes that the participating institutions will be connected to
the vBNS and that the vBNS is the data
transportation vehicle for effecting data transfer among the
distributed model instances.
Data flow would be radial, with NCAR at the center. The bulk of
data would flow from the other three
sites to NCAR, with a smaller amount of data flowing away from
NCAR. The aggregate of data flow
at NCAR is estimated to be no more than 55 Mbps. Basil Irwin
will provide application support to
Distributed Mesoscale Model Project (DMP).
Engineering Services:
Chris Fair worked with MCI Engineering to identify and
correct an ATM layer
routing anomaly between NCAR and SDSC which resulted in abnormal
ICMP round trip
times across the vBNS. During the course of initializing TCP
performance testing parameters over the
vBNS, Chris Fair encountered ICMP RTTs between NCAR and SDSC
machines that
appeared inconsistent with normal vBNS performance. Chris Fair
worked with MCI
Engineering to identify and correct this ATM layer routing
anomaly between NCAR
and SDSC.
TCP performance data indicated that the increase was not related
to the Network
layer but resided in the ATM layer. Tracks for SGI to SGI
(sugarloaf.scd.ucar.edu and
kasina.nlanr.net) and Cray to Cray (paiute1-h.ucar.edu to
C90-casa.sdsc.edu) were studied in an effort
to determine the cause of the twofold increase in
the RTTs. NCSA <-> SDSC tracks indicated 51 ms RTT. While this
entity was
higher than expected, it was still less than the lesser distance
between Boulder
and San Diego by a considerable amount. NCAR traverses the
MCI/vBNS Hayward ATM
track and NCSA the MCI/vBNS Dallas ATM track to San Diego.
The data also indicated that the RTT increases were not related
to a specific
NCAR or SDSC machine (host). While the NCAR <-> SDSC track was
not dependent on
routers, the problem pointed to the ATM layer, i.e., switch
problems either with
dedicated vBNS switches or MCI backbone switches. John Jamison
was brought into
the loop and Randy Nicklas investigated.
A SDSC C90 configuration change would be required for increased
buffer pool
to support the increased bandwidth-delay product separating NCAR
and SDSC.
Radically higher RTTs between Boulder and San Diego make a great
difference in
every aspect of the vBNS performance issue. The Bandwidth-Delay
product the
machines have been tuned for basically doubled, doubling the host
memory
resources required to re-optimize TCP.
Using the information provided above, MCI Engineering traced the
high RTT problems between
Boulder and San Diego via the vBNS to be in the ATM layer between
Denver Junction and Hayward.
An outage in the MCI commercial backbone (which the vBNS rides)
caused the General Datacom
Apex ATM switches to auto-reroute ATM traffic via Seattle. MCI
indicated that similar future occurrences could be better handled
with some sort of (automatic) notification. This problem could
have gone unnoticed for some time had TCP performance
measurements
not failed at NCAR.
Chris Fair began preliminary performance tests between
aztec1-h.ucar.edu
and C90-casa.sdsc.edu. With the RTT and high BDP problem in hand
Chris Fair began nettest/nettesd
TCP performance test suites. These tests indicated widely
varying performance
numbers between aztec1-h.ucar.edu and C90-casa.sdsc.edu, ranging
from the low teens of Mbps to
over fifty Mbps. These variations appear to be host rather than
network related. Further analysis and
collection will continue with the addition of a host at PSC.
Chris Fair worked with Jamshid Mahdavi to procure login
account on PSC C90. A login
account was set up on mario-97.psc.edu at PSC for Chris Fair to
conduct nettest/nettestd TCP
performance analysis between PSC and SDSC and PSC and NCAR
machines. Host routes have been
requested from system administrators at SDSC, NCAR, and PSC to
point at each site's vBNS
NetStars.
Chris Fair worked with Jamshid Mahdavi (PSC) to configure
host routing and
the UNICOS operating system kernel on mario-97.psc.edu for vBNS
connectivity to
aztec1-h.ucar.edu, paiute1-h.ucar.edu, and c90-casa-sdsc.edu.
Host routes were added on mario-97.psc.edu for IP connections to
both NCAR Cray
J9s and the SDSC Cray C90. These were configured to hit the PSC
NetStar
GigaRouter, vbns-netstar.psc.edu, the only option for connecting
these HIPPI
hosts to the vBNS PVC mesh.
The resulting path to NCAR and SDSC then involved the GigaRouters
at each site
connected by the vBNS PVC between them.
mario% traceroute 192.67.81.32
traceroute to 192.67.81.32 (192.67.81.32), 30 hops max, 56 byte
packets
1 vbns-netstar.psc.edu (128.182.97.23) 2 ms 2 ms 2 ms
(2.0/2.000)
2 ns-atm9-0-6.sdsc.vbns.net (204.147.129.25) 62 ms 63 ms 64
ms (63.0/63.00)
3 c90-casa.sdsc.edu (192.67.81.32) 64 ms 63 ms 63 ms
(63.0/63.333)
The maximum socket buffer space was set for 1024000 bytes which
is still not
quite enough to accommodate the vBNS Bandwidth-Delay Product
(BDP) between PSC
and SDSC as seen above (64 ms RTT). Personnel at PSC will be
reconfiguring the
kernel for 2048000 bytes to accommodate larger BDPs across the
vBNS.
Chris Fair worked with Bilal Chinoy (SDSC) to configure
host routing and
operating system kernel on c90-casa-sdsc.edu for vBNS
connectivity to
mario-97.psc.edu. A host route was added on c90-casa.sdsc.edu
for IP connections to the PSC C90,
mario-97.psc.edu. It was configured to hit the SDSC NetStar
GigaRouter, ns-h-vbns.sdsc.edu, the
only option for connecting these HIPPI hosts to the vBNS PVC
mesh.
128.182.97.3 192.67.81.41 UGH 0 200127 hi1
The maximum socket buffer space was set for 1000000 bytes which
is still not
quite enough to accommodate the vBNS Bandwidth-Delay Product
(BDP) between PSC
and SDSC as seen in the traceroute above to 192.67.81.32 (64 ms).
Personnel at
SDSC will be asked to reconfigure the C90's kernel for 2048000
bytes to
accommodate larger BDPs across the vBNS.
Chris Fair worked with George Fuentes (NCAR) to configure
host routing and
operating system kernel on aztec1-h.ucar.edu and
paiute1-h.ucar.edu for vBNS
connectivity to mario-97.psc.edu. Host routes were added on
aztec1-h.ucar.edu and
paiute1-h.ucar.edu for IP connections to the PSC Cray C90,
mario-97.psc.edu. These were configured
to hit the NCAR NetStar GigaRouter, ns-vbns.ucar.edu, the only
option for connecting these HIPPI
hosts to the vBNS PVC mesh.
The maximum socket buffer space on both NCAR Crays remains at
2048000 bytes to
accommodate vBNS Bandwidth-Delay Products.
Chris Fair continued to record and analyze vBNS TCP/IP
performance between
aztec1-h.ucar.edu, paiute1-h.ucar.edu, and c90-casa-sdsc.edu.
nettest/nettestd suites executed over the
vBNS between HIPPI-attached Crays at NCAR, PSC and SDSC have
produced varied performance
baselines. While directly ATM attached hosts have shown TCP
performance in the 70 Mbps range,
hosts connected through the vBNS GigaRouters have illustrated
performance as low as 4 Mbps across
the vBNS.
Below are two instances of TCP performance across the vBNS
between
aztec1-h.ucar.edu (NCAR) and mario-97.psc.edu (PSC). The first
was taken at
0832 MST on March 2, 1997 and the second at 0847 MST on March 3,
1997.
It has been proposed that perhaps some of the performance
degradation being seen
here is due to fragmentation by the GigaRouters, AAL5 related,
host memory
constraints or CPU loading.
The instances below are taken between the SDSC C90 and the PSC
C90 also on the
morning of March 3, 1997.
Plots of these and other performance tracks across the vBNS are
being prepared
for WWW posting in an effort to develop a histogram and possible
explanation of
these varying throughput results.
Chris Fair installed and configured new Cisco Systems
LightStream 1010 ATM
switch at the NCAR site. This switch will supplant the Fore
Systems ASX-200 ATM switch currently
used for the vBNS' PVC-based Logical IP Subnet (LIS). A patch
between this switch and the Cisco
LS1010 was implemented to extend the entire NCAR test3 and test5
PVC
fabric to the new switch while cutover occurs.
Chris Fair, Paul Hyder, Duane Wessels and Marla Meehl met to
discuss vBNS
PVC fabric changes in lieu of vBNS OC-12 cutover. It was
requested that MCI
Engineering restore partial PVC mesh for vBNS performance testing
from
sugarloaf.ucar.edu. and perhaps PSC via the test3/1d1 port on the
vBNS Fore
Systems ASX-1000 ATM switch at NCAR.
Chris Fair is working with Randy Nicklas of MCI Engineering to
begin testing
between an NCAR ATM-attached host, sugarloaf.ucar.edu and the
SVC-based LIS
running between a subset of the vBNS GigaRouters and switches.
In lieu of the
retired vBNS PVC LIS fabric, the LIS is the preferred method for
all vBNS
connectivity over the new OC-12 fabric.
Chris Fair worked with Jamshid Mahdavi (PSC) to establish
performance
testing procedures between NCAR and PSC Crays.
Chris Fair continued to record and analyze vBNS TCP/IP
performance between
aztec1-h.ucar.edu, paiute1-h.ucar.edu, c90-casa-sdsc.edu, and
mario-97.psc.edu.
nettest/nettestd suites executed over the vBNS between
HIPPI-attached Crays
at NCAR, PSC and SDSC are being collected to establish
performance baselines.
Plots of these and other performance tracks across the vBNS will
be posted on
the WWW to develop a histogram of TCP throughput.
Tests executed between NCAR and SDSC on 3 March, 1997 produced
very low TCP
performance throughput values. As with all nettest/nettestd
suites RTTs are
measured and the BDP for those values computed prior to setting
the socket
buffer sizes and the TCP window shift for each trial. RTTs
illustrated below
are consistent with those normally found between SDSC and NCAR.
The BDP for
these is well within the capabilities of each host's kernel
configuration.
However TCP performance of approximately 10 Mbps is far less than
expected. Details on these tests
are provided as Attachment 1.
In all of these tests, TCP Slow Start was not disabled, this may
have
resulted in some performance degradation as TCP ratcheted-up.
However the
radical improvement in TCP performance in just a day's time is
puzzling.
Time-of-day variations are expected as a result of machine loads
and network
burstness.
Research:
Duane Wessels continues at NCAR as UCSD NLANR-funded
engineer, working on
NLANR, caching research, and engineering.
Activities Planned for the 3rd Quarter FY1997:
NCAR NLANR personnel will continue to support existing vBNS
high
performance applications as well as the emergence of the
Distributed Mesoscale
Prediction (DMP) Project as a vBNS application.
NCAR NLANR personnel will continue to monitor, analyze, and
evaluate vBNS
performance issues between SCCs at the TCP layer.
NCAR NLANR personnel will continue to be active in the
Westnet2 and GigaPOP
planning, analysis, and implementation issues.
Chris Fair will work with MCI Engineering to research and
implement Logical
IP Subnets (LIS) over ATM (vBNS) using NCAR hosts which are not
directly
connected to the vBNS Fore Systems ASX-1000 ATM switch. This
will entail the
configuration of static NSAP routes between NCAR's premise ATM
switching
equipment and the vBNS switching fabric.
NSF Connections Review - Randy Butler participated in the
last round of High Performance
Connection reviews for NSF.
NCSA Staff participated in numerous network meetings
including:
NSF hosted vBNS/Internet2 meeting in Reston
Internet2 meeting in San Francisco
NANOG
STAR TAP Project - NCSA submitted a proposal to NSF for the
Science and Technology
Applications Research Transit Access point (STAR TAP) for
international connectivity to the vBNS.
In anticipation of an award NCSA, together with UIC and ANL
hosted a meeting at ANL
in March. The participants included 13 representatives from the
Asian Pacific Advance Network
collaboration (Japan, Singapore, Taiwan and Korea), three
participants from CANARIE as well as
representatives from MREN and AADS. The TAP architecture was
discussed in detail as were details
about the use of the vBNS, access to other countries, and how
APAN might
connect into the STAR TAP fabric. An APAN link is expected into
the STAR
TAP before the end of the year. Singapore has reported that they
have already begun the process to
initiate a connection directly to the STAR TAP.
User Services:
The CEWES Demonstration was a success, see 1st quarter 1997
report for project details. Numerous
networking obstacles were encountered. The original
interconnection between the two networks was
at MAE-East, with documented packet loss as high as 40%. This
kind of loss as expected degraded
performance so badly that a change had to be made. With the
help of IDREN staff and MCI, the
interconnection point was moved to the Sprint NAP. NCSA was
still able to document loss (less than
2%) but it was acceptable for the application. The measured
performance end-to-end, between a host
at NCSA and another one at CEWES was 1-3 Mbps (depending on time
of day/network congestion)
using ttcp. Multicast which uses an independent route continued
to present performance problems.
Rather than changing the MBONE topology to solve the problem, a
multicast tunnel was created between the two systems. This
resolved many of the problems
however oddly enough we continued to
document loss of multicast traffic. This has now lead to a
discussion on the vtcc and nsfconn mail lists
about measurement tools and techniques for multicast traffic.
NCSA is relatively confident that the multicast loss is a result
of the lack of a multicast flow-control mechanism.
Over the local LAN this was fine since their were no significant
bottleneck, (besides the application).
But once it was run over the WAN, it was able to flood the
network with multicast packets and loss
occured. This was worse in the NCSA-CEWES direction likely
because the NCSA host had more
horsepower and could push the packets out the door faster.
Lessons learned from the process are:
Earlier involvement of the networking personnel are needed
with the application. This allows the networking engineers
time to give the application staff feedback on their network
strategies and guide them in the right
direction. It also gets the networking personnel familiar with
the application and improves communication between the two
groups. Actual involvement by the networking staff started
roughly one month in advance of the demonstration.
Multicast performance tools. There may already be something
out there,but it would have been
preferrable to have had a tool that measures multicast throughput
and loss, sort of like ttcp but that
supported multicast.
Improved tools are needed for identifying and diagnosing
network bottlenecks. vBNS OC-12 upgrade
problems, interconnection problems, IDREN network problems and
problems with the CEWES
internal network were encountered. Each were difficult to track
down and required significant
coordination to resolve.
A white paper on the CEWES experience
http://www.ncsa.uiuc.edu/People/vwelch/projects/cewes was
submitted for the upcoming NGI
workshop. NCSA is now adding lessons learned from the
experiences of the applications developers.
NCSA has been working to define a vBNS User Support program
which would
consist of a core team of support engineers and trainers. It
will
include active collaborations with MCI engineering and other
NLANR
teams especially the PSC-lead vBNS engineering effort. NCSA
staff have
participated in numerous conference calls with MCI, PSC and NSF
to
define the program and the interface between it and the other
support
and research teams.
NCSA has proposed four vBNS related maillists and has
received feedback from NSF and
general approval from other MCI/NLANR representatives. Comments
are being incorporated and the
lists should be active soon, see Attachment
2.
A draft agenda for a vBNS workshop was developed and
circulated to the NLANR Exec mail
list for comment. The goal of the workshop is to familiarize
distributed applications developers with
network performance measurement and optimization techniques, see
http://www.ncsa.uiuc.edu/People/rbutler/workshop.
Engineering Services:
The vBNS Route Server mentioned in the last quarterly report
is now complete and can be
viewed at http://rs.ncsa.uiuc.edu. This
system runs traceroutes,
displays routing table and presents an AD Path summary.
Identified a Multicast routing problem - NCSA's PIM cloud is
not receiving DVMRP routes to the
vBNS engineering group's LAN in Reston. MCI assisted NCSA by
putting in
a static multicast route, but the DVMRP refuses to work. MCI and
NCSA will continue to work on resolution
of this problem.
From an engineering standpoint there were numerous routing
configuration
changes to allow vBNS paths to additional sites.
Research:
The hardware has been assembled and work continues on the
port of the OC3Mon/CORAL software from DOS to LINUX. During this
quarter, the ATM NIC software was ported to the system.
Staff reviewed the proposed API and other related documents for
OC3mon statistics modules.
IPv6 Project - The Pentium Pro based system from MCI for
IPv6 testing has arrived. Jon
Duggan is responsible for installing FreeBSD 2.2. The maching
will reside on the NCSA FDDI ring
(141.142.121/24) and will be added to the 6bone. The 6bone is a
nationwide network of tunnels,
encapsulating IPv6 in IPv4 to provide a testbed for IPv6
development. For more info on 6bone see:
http://www-6bone.lbl.gov/6bo
ne/. NCSA is also
attempting to get CISCO's IPv6 code for some of its routers to
enable additional experiments.
Activities Planned for the 3rd Quarter FY1997:
OC3mon/Coral: Jon Duggan will attend the OC3mon developers
meeting on April 4, 1997. The
alpha version of Linux OC3mon should be operational by the end of
April. Once the alpha version is
complete, NCSA staff will test and rework, fixing any overall
design flaws. Also exploring the
possible addition of a GLOBUS resource monitoring API to the
OC3Mons.
NCSA will develop presentations on User Services and STAR
TAP for the April 28th NLANR
meeting. Additionally NCSA is in contact with the FNC security
taskforce to define the appropriate
participatory role for the vBNS community.
NCSA staff are working with PSC to develop potential NLANR
engineering projects.
NCSA will hold a vBNS Users Workshop during the upcoming
quarter.
For the past two months, PSC's support for users has focused
primarily in determining if high
performance paths exist for specific applications.
Specifically, staff have requested ATM level
connectivity between the vBNS and ESnet in support of distributed
computing applications. PSC is
also working with a user in Stutgart, Germany to identify and
then request a path between Stutgart
and the vBNS via CANARIE. PSC has been working with users at
each site to make sure there
applications are capable of utilizing the high performance
network path once implemented. Staff
continue to collect performance information in order to identify
PSC users and applications which
have high bandwidth requirements.
PSC staff worked with the RUS (Stutgart) applications group
on getting their application running
over the vBNS. Routing between the vBNS and CANARIE has been a
major first step. Countinue
pursue the ATM level connection between ESnet and the vBNS in
support of a distributed
application with Oak Ridge National Laboratory. The vBNS
application form will soon be completed
in order to request permission for an ATM level
connection.
Outreach efforts included attending Internet 2 meetings,
specifically to provide technical
information on vBNS activities for I-2 sites. PSC also
participated and initiated engineering level
discussions between NLANR and the I-2 Engineering Group.
Engineering:
PSC has supported a range of engineering activities over the
past two months. They continue providing routine engineering,
operational and technical support for the vBNS, such as:
monitoring and supporting routing between PSC and the vBNS;
hardware support, in particular
support for the recent OC-12 upgrade; and organizational support
for the vTCC meetings. Special
activities over this period include: implementing a MBONE phone
bridge and expanding our ATM
environment.
PSC purchased and implemented an MBONE phone bridge which
allows staff to integrate
telephone based conference calls with MBONE conference sessions.
This technology allows
MBONE-challenged users to participate in the audio component of
MBONE conferences. While we
are still have some difficulties with voice levels between the
MBONE bridge technology and VAT, the
technology is useful for bi-weekly vTCC meetings.
A significant amount of time during this period was devoted
to ATM issues, for both LAN and
WAN environments. On the local level, we have significantly
expanded our ATM environment,
adding both switching infrastructure and hosts to the
environment. Efforts have primarily focused on
LAN and WAN configuration issues. PSC also worked with MCI and
Sprint to provide ATM level
connectivity between the vBNS and ESnet. Currently staff are
waiting for the connection in San
Diego to be completed.
In February, PSC provided Engineering and operational advice,
particularly routing, in support of
the University of Pennsylvania's new vBNS connection. It was
clear from the types of
questions asked by U Penn's network engineering staff that an FAQ
is needed describing the technical
issues associated with implementing a new connection.
Organizational issues were also addressed during the past two
months. Efforts include better
technical support for the vTCC, first by implementing the phone
bridge and second by re-establishing
the web based advance notes for the meeting (
www.psc.edu/~mahdavi/vtcc_notes.html).
PSC initiated efforts on developing an NLANR Engineering
website. This site has information on
routing, ATM configuration, Mbone, and other Frequently Asked
Questions about the vBNS. This site will be announced to the
broader community in April, but the
current version can be viewed at
www.psc.edu/networking/nlanr. The current
version includes an annotated list of all routes currently
announced to the vBNS. This
information includes: Route, Origin AS, Description. PSC is
currently working on a program to
automatically update this table on a regular basis.
Four ASX1000 at PSC were connected to the vBNS during this
quarter. Staff also started work
on attaching to SVC LIS. Once this project is complete, PSC will
develop a FAQ based on lessons
learned.
PSC staff worked with MCI folks on the performance monitors
running on the DEC
Alpha platforms at each site. Spurious route flaps were
identified which seem to occur infrequently at
4 AM.
Routing information and vBNS access lists are being
maintained at PSC.
Staff continued to provide MBONE support for NLANR related
activities at PSC, working with the existing NLANR sites to
better the quality of MBONE VTCC. PSC/NLANR personnel provided
support for internal
broadcast of PSC Training session. Insights gained through these
tasks will be applied to supporting
future distance learning activities over the vBNS.
Research:
The IPv6 PC was received and configured with both Linux and
FreeBSD OSes. Staff developed a
strategy for implementing GateD on the machine in order to
perform route monitoring for the vBNS.
Eventually, this function may be hooked into the NLANR
engineering web site with and include
automatic updates.
Under PSC's "TCP Enhancements" grant, staff continued work on
analysis of SACK/FACK
performance in the Internet, including the submission of a
SIGCOMM'97 paper on this subject. In
addition, staff started work on TCP "autotuning", a component of
the project
particularly relevant to the expanded vBNS and Internet 2.
Finally, PSC staff are preparing to begin
work on the National Internet Measurement Infrastructure (NIMI)
grant, which aims to
develop a scalable technology for performance Internet
measurement. This will be applied to the
vBNS as early as possible.
Activities Planned for the 3rd Quarter FY1997:
Over the next quarter, PSC plans to work on a number of
Engineering,
Development and User Services projects along with providing
continuing
organization and management support. Projects planned for the
next quarter include:
General
This coming quarter PSC will support and/or attend a number
of conferences focusing on high
performance network infrastructure. Activities include:
presentations at the MREN-sponsored
conference, the NLANR/I2 Meeting in San Diego, the ISMA workshop
in San Diego, and the NSCA sponsored vBNS Users Conference.
PSC is soliciting applications and projects for the SC '97
meeting in San Jose in November 1997, see Attachment 3.
User Services
PSC plans to continue to provide User Services support for
new and existing applications over the
vBNS. The current focus is on applications which also utilize
PSC infrastucture, however this will be
expanded as new sites connect to the infrastucture.
Engineering
PSC will continue development of the NLANR Engineering
Website to provide a place for new,
existing, planned vBNS and I-2 sites to find engineering
information on the vBNS. Items to be added
this quarter include:
Additional FAQ lists based on the existings sites
experience
on the vBNS. Areas include: routing, ATM configurations,
connecting to the
vBNS MBONE, etc.
A program which automatically updates the list of routes
heard
over the vBNS.
A summary of the status of ongoing vBNS engineering
development projects.
PSC will also provide more coordination for the various
engineering projects supported by the
existing vBNS sites. Project areas and appropriate participants
are now being identified. Current
projects we are focusing on: vBNS Performance, MBONE, and IPv6.
These projects fall into the area
of developing, testing or refining network for general use on the
vBNS.
PSC will continue to provide information and consulting
support for sites with High Performance
Connection grants or sites planning to submit proposals to this
program.
k claffy hosted the NAP panel at the February NANOG meeting,
www.nanog.org
Duane Wessels presented the status of the NLANR Caching
Project at the February NANOG
meeting, www.nanog.org
NLANR was represented at the February UCSD Industrial Liasion
Poster Session by k
claffy, T. Sterrett, and C. Maeda.
k claffy represented NLANR at UCSD meetings on vBNS
connectivity issues.
k claffy and Duane Wessels arranged for a cache/mirrors
for the NASA Mars Lander project. Kirk Goodall (NASA) has
defined
a scheme for replicating the data, using a "push" approach,
where basically a tar file of new/changed data is
copied to each mirror site and then unbundled.
Initial plans are that updates will be pushed daily
and perhaps more frequently after the
initial surge in traffic.
Due to concerns that the additional traffic could
adversely affect regular NCAR business,
NCAR has agreed to be a mirror site for only the first four
days.
k claffy participated in the program committee for the Next
Generation Internet meeting, see www.cra.org/Policy/NGI/.
k claffy attended the Computer Freedom and Privacy
discussion of PICS labelling with
Matt Blaze and Paul Resnick, see www.cfp.org.
D. Wessels attended a "Westnet2" meeting at NCAR where the
vBNS, GigaPops, and Internet II
were discussed. Five vendors made presentations, including
Sprint, US West, ANS, "Dakota" and
Genuity.
Engineering Services:
SDSC staff (Bilal Chinoy and Rich Gallup) worked with MCI and
other NLANR sites to install
various h/w and s/w at SDSC throughout the period.
Research:
Measurement/Viz
Eric Hoffman, John Hawkinson (BBN) and Darren Kerr (Cisco)
set up a loaned Cisco router /
serial console for use in NLANR/CAIDA.
Tracie Monk, k claffy and Eric Hoffman worked with
InterVU personnel to review and
provide input to their measurement methodology, www.intervu.com. k claffy
initiated an evaluation
of available data.
The web pages at www.nlanr.net/NA/Learn
were
expanded to include mice.html and flowdiff.html.
Duane Wessels loaded FreeBSD 2.2.1 the IPv6 PC from MCI.
Had some trouble with the Ethernet card
(the kernel wouldn't recognize it as 10 Mb/s mode)
LAN problems at NCAR developed which make the
machine all but unusable in its current location.
Packet loss on the local ethernet segment is
around 20% when the repeater is
required to connect that machine.
The machine will be moved to a better net
in April.
Caching
A new award was received from NSF to support the NLANR
caching hierarchy project.
Currently this project is jointly supported by the NLANR
cooperative agreement and a caching grant.
NLANR's financial support of the project will be phased out over
the summer. NLANR is hosting a
caching workshop planned for June 9-10, 1997, see
www.nlanr.net/Cache/Workshop97.
During this quarter, traffic through the NLANR caches
continued to grow, with 24G of traffic
served during a single day -- an average of 4G per cache. During
that busy period, the busiest cache
actually served more than 6G of data in one day. Each cache has
only 6G of disk to use for caching.
The NLANR caches have relatively low hit rates: 20-30%.
This is in large part because the cache disk
space is (6G) is about the same as the amount of traffic served
per day (4G). As the traffic load
increases, these caches become less and less useful as caches,
see www.nlanr.net/
Cache/Statistics/Graphs/ for
a graphical depiction of the growth in cache usage. Vital
statistics such as number of objects, memory
usage, and other details are available at www.nlanr.net/Cache/Vital
s/.
VRML visualizations of
web cache traffic are available at
http://www.nlanr.net/Cache/cacheviz.html.
In late February a couple of the cache machines ran out of
disk space on the /usr partition -- where
the cache logs are written. This occurred because: (1) a
'tail -f' was running on cache.log, so that
space was not reclaimed until the tail process was killed (the
next day); and (2) the very large size of
the access.log files -- on the busy boxes access.log can get to
be 200 Mbytes.
By disabling the logging of ICP requests, especially since
the caches log about 10x as many
ICP has HTTP requests, the log files were brought back to a more
managable size, but some
potentially interesting statistics are no longer being gathered.
Also, since UDP_HIT_OBJ replies were
previously included in the hit-rate counts, the NLANR cache hit
rates are now lower.
A review of "overlap" in which objects are held by the
caches (ie, do more than half of the
objects exist in multiple caches) found that more than half of
the objects do exist in multiple NLANR
caches...
46.6% exist in only 1 cache
44.6% exist in 2 caches
6.2% exist in 3 caches
1.9% exist in 4 caches
0.6% exist in 5 caches
This means we might consider configuring Squid to NOT cache
objects fetched from NLANR peers.
Given the good connectivity of the vBNS, this would be no
problem. This may not be a good
alternative for 'sv' cache traffic which goes over the commodity
network.
The need to acquire additional disk space for the cache
machines was discussed with NSF.
The load during the second quarter of FY1997 was causing the
cache contents to be replaced every
two days. With only 6GB for caching, NLANR's "top level" caching
infrastructure was severely
underconfigured. NSF/UCSD agreed that the new disks would be
purchased under the new caching
award, versus NLANR. Digital offered to sell 24 drives to UCSD
at a significant discount.
Duane Wessels installed a fake HTTP redirector on port 80 on
all NLANR caches. Now if someone assumes the
cache is running a HTTP server they will be directed to
http://www.nlanr.net/Cache/. This is mostly to allow people
to find out more about the machine
which may be sending them lots of HTTP requests, or ICMP
traffic.
k claffy visited LBL to discuss various caching/ICP
topics..
Activities Planned for the 3rd Quarter FY1997:
General
Hans-Werner Braun will rejoin UCSD full-time starting May 1,
1997.
Engineering Services
SDSC's Bilal Chinoy, Jay Dombrowski, and Rich Gallup will
continue to work with ESnet and MCI
personnel to establish the ESnet/vBNS interconnection at the
General Atomics facility. They will
support other NLANR/vBNS engineering requirements as
necessary.
UCSD/SDSC will assist in planning/hosting the NLANR/vBNS
workshop on April 28th and the
Internet-2 GigaPoP workshop on April 29th, and the Internet-2
Management meeting on April 30th.
Charlotte Smart [smartc@sdsc.edu] is responsible for logistical
coordination and Jay Dombrowski
[dombroh@sdsc.edu] is responsible for the MBONE broadcast on
April 28th.
T. Monk is working with NLANR sites to finalize subcontracts
/ invoices for 1996 and to
complete the supplement for allocation of Year 3 funds.
k claffy will spend three days at AT&T Research making a
presentation and discussing
measurement/visualization/graphing/web caching
activities.
k claffy will attend the NGI Program Committee meeting on
April 3rd and both k claffy and
Hans-Werner Braun will attend the NGI Workshop on May 14-
15.
Efforts will continue with InterVU (www.intervu.com) to
evaluate their performance data, develop their spec, and finalize
the MOU.
Traffic visualization efforts will continue with Merit
(routing), MCI (OC3mon), Univ. of
Colorado (GeoTrace), and LBL (pathchar).
Efforts will continue to expand and improve the CAIDA Tool
Taxonomy,
www.nlanr.net/Caidants/meastools.html.
The community will be notified of the Taxonomy availability in
early April. Networking simulator
tools will be added.
cabernet.nlanr.net will be installed at FIX-West (a
replacement for Silda). The new alpha is on loan from
NASA Ames.
New disks will be purchased and installed for NLANR caches.
D. Wessels will continue
monitoring and performance testing of caches -- note that this
activity supplements other forms of
performance testing on vBNS.
NLANR-ether/pinot router will be debugged [significant
problems were experienced during this
quarter].
Efforts to port OC3mon to alternative platforms, including
Free BSD, will continue.
Some research questions being considered include: (1) How
much traffic is currently flowing
through interconnects; (2) How to better measure congestion [ack
retransmission?]; (3) How to better
correlate routing instability with loss/delay; (4) How to
visualize routing tables/differences for PAIX,
Verio, Cerfnet, vBNS, BBN; (5) how to better correlate routing
instability with loss/delay [slac, mids,
unidata...all insufficient].
k claffy will attend the bay lisa meeting [vixie on DNS]
www.baylisa.org
k claffy will continue to work with NLANR sites as they set
up their IP v.6 machines to run
FreeBSD 2.2 and prepare for IP v.6 experiments.
k claffy / D. Wessels will work with Digital PAIX and AT&T
on web cache deployment -- will
also set up tcpdump for cache measurement studies
and for the ICMP/bgp correlation activities.
During April, k claffy will make presentations at the
IETF/IPPM working group and during the IETF Plenary
Session on CAIDA and the various
measurement/visualization/caching efforts underway at
UCSD/NLANR/CAIDA, at the NLANR/vBNS meeting, and at ISMA97.
During June she will present the results of the ISMA Workshop at
NANOG and attend the Caching Workshop.
In June, Tracie Monk will present on CAIDA at NANOG and at
INET'97.
Duane Wessels will chair a ICP/Caching BOF at the April IETF.
Duane has agreed to co-chair
the IETF's ICP Working Group. During the next quarter, an
Informational RFC will be
prepared. Duane is also hosting the Caching Workshop in Boulder,
CO in June and is making presentations on the NLANR Caching
Hierarchy and the Squid software.
Date: Wed, 05 Mar 1997 11:00:41 -0600
To: vbns-tcc@mci.net
From: Randy Butler
Subject: vBNS maillists
My thoughts are that we need four maillists. I don't care what
we call them. First we have to decide
what we need.
1. Connected Sites Notifiucation List - This is the list that
MCI sends notification to about changes
with the vBNS, schedules etc. This gets sent to ALL sites that
pay the MCI access fees PLUS the
SCCs (which NSF pays for). MCI will be responsible to identify
appropriate contacts for each site.
This is not to be viewed as a discussions list.
This should apply even to sites connected thru a Gigapop since my
understanding is that sites
connected in this manner will still need to pay MCI an access fee
to use the vBNS.
Site Technical Contacts will be able to respond back directly to
MCI Engineering on any issues they
have with the notifications. If required MCI Engineering can
escalate this to the VTCC or
carry on in the Technical Contacts discussion list.
In addition all notifications should be posted to a web site and
sent (read only) to the vBNS Users
maillist.
2. vBNS Users - This is a list that users can sign up for so
that they get all the mail about changes to
the vBNS. It should contain all vBNS notifications but not allow
them to respond back directly to
MCI. Instead they should be encouraged to work through their
local site reps. All reply mail should
go back to the vBNS Users maillist. Relavant information about
the vBNS from the other list (#3)
should be posted here. Site technical contacts should be on this
list as well so they can take up site
specific issues.
This maillist should also encourage general discussion about
applications development on the vBNS.
3. vBNS Technical Coordination Committee (vtcc) - This is a
list of vBNS technical coordinators
that will likely change as more sites are connected. This is the
group that deals with the technical
decissions for the vBNS. It includes NSF designated
representivitives MCI Engineering, Marketing,
etc. It functions as it does today.
4. Technical Contacts Discussion list - A discussion list for
the technical contacts at each of the
connected sites. It would include the vtcc representitives, and
MCI. We would have to be careful to
make sure it does not end up taking over for the vtcc (or do
we?).
I could see 3 and 4 being one list if the vtcc grows to include
all sites.
Randy
From: Gwendolyn Huntoon
X-Authentication-Warning: igi.psc.edu: localhost [127.0.0.1]
didn't use HELO protocol
To: vbns-tcc@mci.net
Subject: NLANR and SC '97
Date: Thu, 10 Apr 97 14:07:57 -0400
X-Mts: smtp
Sender: owner-vbns-tcc@mci.net
All -
I have been in contact with the folks working on SCinet '97.
As many of you may know, the name of the conference has changed
to
be SC 'XX High performance Communications and Computing. Bob
Borchers is this year's Information Architect with help from
Bob Bryant (LLNL) and Jim Brandt (SNL). This year, I believe
they want to highlight the various networking initiatives -
NGI, vBNS, I2, etc. I think there is a real opportunity for
the NLANR group to get involved to showcase activities we are
involved in along with promote the use of high performance
networks
(particularly the vBNS). I am not suggesting that we get
involved
in the wire pulling aspect (I did enough of that last year to
last
a few decades ...), but take on a higher level role - perhaps
supporting/promoting activities that SCinet hasn't been able to
do in
the past. I think will also give us an opportunity to focus
some of our efforts - nothing like demos, deadlines, etc
to push us to complete things. Below is a list of activities
which
I though we might be interested in. By working on this as a
group,
I think we can also be more efficient and effective.
PSC folks are going to the SCinet Organizational meeting
next week. While I am not going, I can make sure any thing we
would
like to be involved in is passed on to the coordinating group.
NLANR Booth
I think we will want to do this again for SC '97. This would
require
a vBNS connection for the conference. I believe Bob Borchers was
hoping
to get network type booths located near the SCinet NOC, so if we
want
a booth, we might begin these discussions. Since the NLANR booth
is network connection intensive, we might want to barter
engineering
support in return for free network connections.
Services
We might want to showcase (and perhaps offer) technologies
and services we are working on:
IPv6
WWW Cache
QoS (more than just a demo, but come up with a way for
sites/applications
connected to the vBNS to actually utilize QoS)
Statistics collection
Help Desk - for high performance applications and connections
Engineering Support - for high performance connections, including
host/connection configurations, performance tuning, etc.
Applications - pick a couple of good applications, show thme
either
in the NLANR booth, booths for each site, or both.
I also believe there are other areas at the conference we could
get
involved in:
* Offer a tutorial (there are a number of topics, we would have
to
select one) - depending on the topic, it could include
actually using new services and facilties over the vBNS.
* Run a BOF - less structured than and formal than a tutorial,
could
even take the form of creating a "User's Group".
* Submit technical papers on the research we have done to date.
Some of
you may have already done this, but we might want to do one
spread across the sites.
Let me know if you are interested.
Wendy
