Since its inception in May 1995, NLANR has undertaken a number of activities in line with its chartered mission and workscope, which fall into three areas:
.
ATM connectivity between all sites is shown at OC-3 line rate. NAP connections are DS-3 links. In April, we tested a new software release for the NetStar GigaRouter (release 4.6.14) in the vBNS test network and is now running in the production vBNS. This release fixes OSPF and BGP interoperability problems with the CISCO routers. In May the Gigarouters were configured into the vBNS OSPF cloud. The Netstar BGP code is being tested on the testnet. New software (IOS release 11.1-1) was also installed on the vBNS CISCO routers. This adds support for:
New ATM switch software from LightStream/CISCO was deployed in the test network. The new software is interoperably doing UNI 3.1 signaling with the CISCO routers in the network. SVCs are now being used in the test network and need to be deployed in the production vBNS to help cope with scaling problems of the current PVC mesh as more ATM-connected end-points are added.
The vBNS statistics are presented in three parts: traffic volume, traffic category and traffic latency. Traffic volume is mainly expressed in the daily and monthly totals of IP packets and bytes per location and per router interface by location. Maximum daily IP packets are also given by location. This set of data is derived from the related SNMP MIB variables. Traffic category describes the traffic by giving the percentages of IP packets and bytes according to protocols. Customized NNStat program on a DEC/Alpha at each site monitors all IP packets on the DMZ ring and captures header information to produce such traffic description. Traffic latency gives the mean round trip latency times generated by running "ping" between any two locations. The latency value is an indication of network operational status but also of service performance to a certain extent. For these statistical data, various graphs and tables are shown below. Network outage information is also provided with the cause and remedy described. The statistics collection is subject to such outages. More information and statistics reports available at http://www.vbns.net/reports/1996/apr_96.htm and http://www.vbns.net/reports/1996/may_96.htm.
Joel Apisdorf at MCI is almost ready to field the OC-3 packet/cell monitor. (http://www.nlanr.net/NA/Oc3mon/), which ports the functionality of the SDSC flow software (described later in this report) to a stand-alone passive monitor (http://www.nlanr.net/NA/Oc3mon/) that can attach directly to serial (e.g., OC-3) optical fiber. K. Claffy, Joel Apisdorf, and Rick Wilder will have a paper on this monitor to appear in Usenix LISA '96.
Chuck Song continues his measurement study in the test network to evaluate the Early Packet Discard algorithm as implemented in Fore Systems ASX-200BX ATM switch. The results show significant improvement of TCP performance. More measurements with larger numbers of simultaneous TCP flows are being done to complete the study.
Several special static routes were configured via the CTC HIPPI network for
high bandwidth applications, including
to NCAR/UCAR for the
HPCC Grand Challenge Gephysical Turbulence Project,
ATM PVCs to Syracuse University via Nynet and
Argonne National Lab for
Roman Markowski's demo at
the CRPC meeting
( http://www.nlanr.net/VBNS/Alloc/1995/markowski.html,
long-term project at
http://www.nlanr.net/VBNS/Alloc/fox.html),
and a route to PSC to enable researchers in New York
City, who were bottlenecked by the commercial Internet connection to
Pittsburgh, to obtain high performance connectivity.
The current IBM SP2 implementation
at CTC requires statically configured routes on
all 512 nodes to route through a HIPPI-attached
node, we now have a manual procedure to remove these
routes during a vBNS and are working with IBM to make this
type of routing dynamic.
Phil Pishioneri contributed information on TCP tuning for high performance
data transfers for AIX systems to
Jamshid Mahdavi's performance tuning web page
(
http://www.psc.edu/networking/perf_tune.html)
Phil also
established an experimental mbone tunnel from
logic.tc.cornell.edu to a
multicast router in the Program of Computer Graphics
to enable video conferencing between PCG and collaborators
at SDSC. Cornell also demonstrated a version of
CU-SeeMe modified for high bandwidth over Cornell's
ATM network. With a maximum of 1 Mb, it produced nearly 30
frames per second of video and good quality audio.
Other Cornell near-term future plans for NLANR work include testing of
Cisco's beta RSVP code to the vBNS and investigation of the use of IP
multicast for distribution of weather data worldwide.
In the process of evaluating TCP performance over the vBNS
between NCAR and SDSC SGI workstations,
Fair encountered IRIX 5.3 kernel limitation restricting
TCP window size to 60 kbyte (kb) default, maximum of 512 kb.
512 kb is insufficient to support window sizes
required by typical vBNS bandwidth-delay products,
restricting the ability to correctly tune
layer 4 and adversely affecting performance between SGI's.
IRIX 6.2 and later kernels remove this restriction.
Fair has run several nettest/nettestd suites
between directly (local) connected
ATM SGI Onyx and Indigo 2 workstations at NCAR.
Observed performance does not seem to respond to
bandwidth-delay product tuning (per RFC 1323) when
the latency is very low (~1 ms or less).
Using the largest socket buffers allowable for IRIX 5.3,
optimal performance observed is in the 60-66 mbps range.
A possible kernel/device driver interaction is suspected
since the CPU utilization during these tests
is near 95% on each machine.
Fair worked with Paul Hyder (NCAR)and John Jamison (MCI) to establish BGP
peering between NCAR and the other four SCC's. NCAR began this peering on
March 15, 1996 and all SCC to SCC traffic now traverses the vBNS.
BGP peering at the NAPs allowed for fail over to the
commodity Internet for PSC and CTC-bound
traffic during the MCI DNG fiber outage.
Commodity Internet transit also occurred for CTC to
PSC traffic for the CU Turbulence application,
following a CTC routing problem,
Fair worked with Reddy Raghurama (PSC) to establish connectivity between
CU hosts and PSC C90 and T3. vBNS routers distributed the IP network for the
four CU hosts to the other SCC's. Initially CU hosts could not reach the SP2
cluster at CTC. Fair worked with Bruce Johnson (CTC) and Phil Pishoneri (CTC)
to correct the peering relationship. Host static routes were chosen over
dynamic routing to keep application traffic from commodity Internet routing and
unsatisfactory performance. Fair also worked with Bruce Johnson and Jamshid
Mahdavi (PSC) to provide connectivity to the PSC C-90 (mario) and
the NCAR J-9 )paiute)
for Don Middleton's Cray FTP client used for the NCAR Large Dataset Transfer
and the CU Turbulence applications.
Fair worked with University of Colorado researchers to configure four CU hosts
for vBNS connectivity via NCAR ATM fabric. Routing was established via the NCAR
Cisco 7000 to NetStars at PSC and CTC. This results in one extra hop via the
NCAR Cisco 7000 as opposed to direct connectivity via the NCAR Lightstream 2020.
The CU researchers have expressed interest in direct Lightstream connectivity to
the vBNS.
Fair presented a seminar on the vBNS for the NCAR SCD Seminar Series. The
seminar was multicast to the vBNS and commodity Internet, and received
great interest, particularly at PSC which had approximately 50 viewers.
The presentation slides are posted on the Web at:
http://www.scd.ucar.edu/nwg/Presentations/vbns/
Cornell Theory Center (CTC)
When the experiment to route inter-SCC traffic via the vBNS was proposed,
Cornell had difficulty establishing BGP peering to the vBNS because
the Cisco routers connecting Cornell's commodity Internet path
via NYSERNET were in a different building from the vBNS node and
only connected via Cornell's FDDI backbones with non-BGP-capable routers.
To make BGP routing to the vBNS possible,
Cornell extended the FDDI LAN (132.236.77.0)
connecting to the vBNS routers from Frank H. T.
Rhodes hall to the Computer and Communications Center to connect to the two
Cisco 7000 routers which provide Cornell's "commodity" Internet path. Thus
these Cisco routers could make the decision as to whether to route traffic
via the vBNS or NYSERNET. Cornell installed new FDDI interfaces in these
routers and established BGP peering between Cornell's Cisco routers and the
vBNS Cisco router at Cornell. Cornell is currently advertising routes to
all three of their Class B addresses, 128.84.0.0, 128.253.0.0, and
132.236.0.0, since subnets of these have been intermixed on the Campus.
However, work has begun to investigate consolidation of the Theory Center's
Class B, 128.84.0.0, such that only one network or two quarters of
128.84.0.0 would be dynamically routed via the vBNS.
CTC extended the FDDI LAN that connects vBNS routers to Frank H.
T. Rhodes hall Computer and Communications Center to connect to the
two Cisco 7000 routers that provide
Cornell's commodity path to Nysernet/Internet.
Cornell then installed new FDDI interfaces in these
routers and established BGP peering with the vBNS Cisco router.
Cornell is currently advertising routes for all
three of their Class B addresses,
since subnets of these have been intermixed on the campus.
However, work has begun to investigate
consolidation of the Theory Center's Class B so that only
one network or two subnets of 128.84.0.0 would be
dynamically routed via the vBNS.
National Center for Atmospheric Research
(NCAR)
Personnel
Chris Fair
continues as NCAR NLANR-funded engineer, and
NCAR staff network technicians continue to provide
NLANR-funded technical support.
Duane Wessels continues as SDSC NLANR-funded engineer,
working on NLANR, caching research and engineering.
Accomplishments
Fair continued worked with Claffy, Wessels, Digital and
Cisco to troubleshoot and correct illegal length FDDI packet problem
plaguing Harvest cache DEC Alpha.
After replacing all Cisco AGS+ router FDDI interfaces, appliques and
Cisco Cbus controller, the
final resolution was to disable autonomous switching on
the Cisco Cbus controller. Since this change,
no illegal length FDDI packets have halted the Alpha.
Applications
Large Data Set Transfer (LDT)
Large files (25 G bytes nominal) are
being successfully transferred
between NCAR and PSC Mass Storage System (MSS) via tuned, multiple
FTP sessions. We are observing throughput in the 70 Mbps
range with the nettest/nettestd utility and
actual FTP rates in the (ncftp) 20-40 Mbps (avg) range.
http://www.scd.ucar.edu/vg/DCSL/LDT/LDT.html
The experimental remote CSM data site continues on the direct-ATM-connected NLANR machine, kasina.nlanr.net. Kasina's HTTP server was configured to support new datatypes associated with the CSM data, and a collection of CSM data was organized on the system. Sample datasets contained variables for both the atmospheric and ocean component of the coupled climate model.
Butler is also organizing a meeting in Washington on July 31st, in order to arrive at a way for US carriers (Sprint and AT&T) to interconnect with each other, other international carriers and USA ATM networks in support of the GIBN project. Additional participants will include representatives from MREN, ESnet, vBNS and CANARIE. Representatives from SC96 will also attend this meeting to explore linking GIBN applications into the show this year via the GIBN network and likely through the vBNS.
PSC engineering staff also provided ongoing routing support for the vBNS, including configuration and management of local BGP peering sessions and installation of specific host routes to high performance hosts at other sites. In addition, Matt Mathis at PSC continued to work on engineering for the internal vBNS Mbone and configuration of local PSC mrouters.
We have also worked with MCI and local access providers in planning new connections to the vBNS, and to extend the vBNS to the David Lawrence Convention Center in Pittsburgh for Supercomputing '96 in November. We worked with several sites pursuing access to the vBNS for research projects requiring high performance networking (via the NSF Connections '96 program), including discussions with MCI and NSF regarding possible architectures. We also worked with Sprint on possible solutions for vBNS access to the GMD Institute (in Germany) via the G7 experimental networking initiative.
On the administrative front, Jamshid Mahdavi continued to co-chair the vBNS Technical Coordination Committee. In addition, Mahdavi was added to the NLANR grant as a co-PI. With these new responsibilities, Mahdavi worked with the other co-PIs on the project to help better define the future goals and direction for the NLANR project. In addition, he helped organize and chair a meeting of NLANR participants at the Montreal IETF/INET meeting. At this meeting, George Strawn of NSF presented the concept of a Gigapop: an exchange point where new vBNS sites can aggregate and collectively attach to the vBNS. The NLANR team will be writing a white paper containing analysis of the idea and providing input to the NSF on how to proceed. Mahdavi has agreed to coordinate this effort.
PSC also arranged for a large audience to attend a remote seminar talk presented by Chris Fair of NCAR. This seminar, given by Fair to the local NCAR community, was broadcast via Mbone to the Internet, and in parallel at high bandwidth over the vBNS. In Pittsburgh, two meeting rooms were set up with sound systems and large screen displays attached to Mbone capable workstations. The high quality transmission of the seminar over the vBNS was viewed by approximately seventy-five people, including a workshop class consisting of twenty-two PSC users. The participants in the seminar also had the opportunity to ask questions of the speaker. This seminar was the first in what we hope will become an active vBNS seminar series, and clearly demonstrated to a large audience the capabilities of the new technologies embodied by the vBNS.
In addition, the PSC also actively worked on applications support through debugging of performance problems on the vBNS (including the discovery of a SONET layer physical problem in the vBNS infrastructure); development of tuning libraries for use by applications needing access to the highest available bandwidth on the vBNS; and the development and distribution of documentation on techniques required for obtaining the best performance on the vBNS. During the next quarter, the NLANR team at PSC plans to be even more aggressive in soliciting new applications for use on the vBNS and in working with scientists in obtaining the best possible performance for these applications.
Following the February NLANR statistics workshop, several activities have emerged.
In line with the goal of using the information gathered from network analysis to improve the performance of the architecture, NLANR continues work on its information caching prototype, and has submitted a proposal to NSF for continued evolution of the system. A key result of Internet workload analysis has been the recognition of the existence of an increasing proportion of web traffic as measured at reachable locations, as well as simulation results that indicate that caching could leverage the high degree of redundancy in web document transmissions.
The international participation is still extraordinary, and dwarfs U.S. interest in caching in general. Following is a list of sites using the NLANR caches:
The sites shown in bold have established ``mutual parent'' relationships with us. That is, we agree to be their parent cache for URLs located in the U.S., and they are our parent for URLs located within their country.
So far, we have not placed any restrictions on using the caches. We accept connections from any address and will process any URL request. This allows, for example, sites in New Zealand to retrieve URLs from the U.K. through us. We did announce in May that we would have to eventually deploy access policies in order to maintain a rational structure of the hierachy.
Our plan for access control includes the following objectives:
The list below shows how access will be configured for each site. This list was generated by ranking all NLANR clients for the month of June by number of requests. Low ranking sites from countries with higher ranking caches have been dropped.
ebone-proxy.univie.ac.at 192.174.65.0/24 it pb cardrout.card.ucl.ac.be 130.104.0.0/16 it pb gateway.gmcc.ab.ca 198.161.33.0/24 bo it news.fudan.sh.cn 202.120.224.0/24 sv sd tinderbox.netman.dk 193.88.72.0/24 it pb ababel.ecm.ub.es 161.116.0.0/16 it pb iitk.ernet.in 202.141.40.0/24 it pb light.imasy.or.jp 202.227.24.0/24 sv sd cosmos.kaist.ac.kr 143.248.0.0/16 sv sd isis.waikato.ac.nz 140.200.0.0/16 sv sd sunsite.icm.edu.pl 148.81.0.0/16 it pb sunsite.nada.kth.se 130.237.0.0/16 it pb pacific.net.sg 203.120.90.0/24 sv sd zaphod.arcom.spb.su 194.135.104.0/24 bo it alpha.icmp.lviv.ua 193.124.228.0/24 bo it norse.mcc.ac.uk 130.88.0.0/16 it pb phoenix.doc.ic.ac.uk 193.63.255.0/24 it pb sirius.tadpole.co.uk 192.52.161.0/24 it pb connect.com.au 192.189.54.0/24 sv sd manifera.itd.uts.edu.au 138.25.0.0/16 sv sd iliad.lib.mq.edu.au 137.111.0.0/16 sv sd netra.geko.net.au 203.2.239.0/24 sv sd holly.flex.com.au 203.19.214.0/24 sv sd starwon.com.au 203.15.243.0/24 sv sd geog.unsw.edu.au 149.171.0.0/16 sv sd topdown.bns.com.au 203.19.43.0/24 sv sd bbs.ausom.net.au 203.8.161.0/24 sv sd vweb.access.net.au 203.13.25.0/24 sv sd eyre.iinet.net.au 203.19.149.0/24 sv sd alpha.xavier.sa.edu.au 203.17.192.0/24 sv sd spock.rz.uni-frankfurt.de 141.2.0.0/16 it pb rz.ruhr-uni-bochum.de 134.147.0.0/16 it pb uran.informatik.uni-bonn.de 131.220.0.0/16 it pb doener.unix-ag.uni-kl.de 131.246.0.0/16 it pb aiken.zw.klautern.fh-rpl.de 143.93.0.0/16 it pb proxy.germany.eu.net 194.175.173.0/24 it pb www-proxy.iafrica.com 196.7.0.0/24 sv sd umomware.momentum.co.za 196.13.225.0/24 sv sd groa.uct.ac.za 137.158.128.0/24 sv sd iphil-communications.ph 204.70.35.0/24 sv sd crispin.i-manila.com.ph 203.172.14.0/24 sv sd vultur.vslib.cz 147.230.0.0/16 it pb adis.cesnet.cz 194.50.6.0/24 it pb digi.feld.cvut.cz 147.32.0.0/16 it pb ax0rm1.roma1.infn.it 141.108.0.0/16 it pb server1.infn.it 131.154.0.0/16 it pb www.forza-italia.it 194.166.23.0/24 it pb mail1.fw-bc.sony.com 198.83.177.0/24 it sv telford.nsa.hp.com 15.0.0.0/8 sv sd gatekeeper.hunter.com 199.217.148.0/24 uc thesparc.wmsoft.com 194.64.125.0/24 it uc ns.nttlabs.com 204.162.36.0/24 sv zmark.cts.com 204.94.75.0/24 sd ucar.edu 128.117.0.0/16 bo colorado.edu 128.138.0.0/16 bo msu-buzz.edu 206.26.77.0/24 bo syr.edu 128.230.0.0/16 it sv cornell.edu 128.84.0.0/16 it trincoll.edu 157.252.0.0/16 bo bxscience.edu 204.217.8.0/24 uc bo colostate.edu 129.82.0.0/16 bo missouri.edu 128.206.0.0/16 it sv psc.edu 147.72.0.0/16 pb sunysb.edu 129.49.0.0/16 it diogenes.sedl.org 198.213.9.0/24 sv valinor.oceana.org 205.233.219.0/24 pb uc janus.sedl.org 198.213.9.0/24 uc bo ren.ee.lbl.gov 131.243.0.0/16 sv ange.nuri.net 203.255.112.0/24 sv zuse.space.net 194.59.182.0/24 it uc isca10.fugger.net 194.77.44.0/24 uc tm3.cni.net 194.115.51.0/24 pb binabik.gatewest.net 198.163.227.0/24 uc musclewood.hq.cic.net 198.108.58.0/24 uc shell.wisp.net 198.67.33.0/24 it
A busy cache will typically handle 150,000 requests and serve 3 Gbytes of Web objects. Cache hit rates are usually in the range of 5-10% percent and a couple of percentage points higher when weighted by byte volume.
By far, the largest number of requests are in the .com domain. Often 60% of all requests are for .com URLs. Another 20% are for .edu, .net, and .org. We calculate that the caches are currently only being used to about 15% of their capacity. The cache software should be able to handle one million requests and 16 Gbytes per day. However, at this rate, we would be serving twice as much data per day than we could store on disk; additional disk space would be desirable.
We have also been exploring metrics to indicate how well client caches are using parent caches. One measure is relative number of ICP requests vs HTTP requests (queries vs fetches). We probably do not want to see only one HTTP fetch for every 10 ICP queries. However, for sibling caches we could expect a pretty low HTTP:ICP ratio because a cache only fetches from siblings on a HIT.
So another measurement is hit rate. If a cache gets goot hit rates from us, then maybe the HTTP:ICP ratio doesn't matter so much.
So what we are tentatively calling "cache utilization" is
HTTP_COUNTS HTTP_HITS utilization = ------------- + -------------- ICP_COUNTS HTTP_COUNTSHere's calculated utilization values for 'sv' on 7 May 1996:
HIT H/I REVERSE FQDN RATE RATIO UTIL ----------------------- ---- ---- ---- edu.georgetown.webcache 0.56 + 1.00 = 1.56 gov.lbl.ee.ren 0.24 + 0.99 = 1.23 org.sedl.diogenes 1.00 + 0.21 = 1.21 edu.syr.rocket 1.00 + 0.17 = 1.17 [206.106.252.226] 1.00 + 0.14 = 1.14 br.com.gns.srv1-bsb 0.33 + 0.80 = 1.13 net.nuri.ange 0.43 + 0.63 = 1.06 [193.175.197.97] 1.00 + 0.04 = 1.04 uk.ac.ic.doc.phoenix 1.00 + 0.04 = 1.04 IT 0.07 + 0.94 = 1.01 com.hp.access.paloalto 0.20 + 0.80 = 1.00 BO 0.15 + 0.85 = 1.00 com.sony.fw-sj.user1 0.00 + 1.00 = 1.00 au.com.flex.holly 0.37 + 0.62 = 1.00 SD 0.08 + 0.91 = 0.99 com.nttlabs.ns 0.03 + 0.95 = 0.98 com.vivid.greedo 0.14 + 0.80 = 0.94 PB 0.07 + 0.87 = 0.94 dk.netman.tinderbox 0.00 + 0.94 = 0.94 jp.or.imasy.light 0.27 + 0.66 = 0.93 net.mci.sanfrancisco.ip 0.16 + 0.72 = 0.88 nz.ac.waikato.isis 0.26 + 0.62 = 0.87 au.edu.uts.itd.manifera 0.22 + 0.64 = 0.85 cz.vslib.vultur 0.78 + 0.07 = 0.85 net.nlanr.oceana 0.00 + 0.80 = 0.80 jp.or.imasy.luck.yebisu 0.50 + 0.26 = 0.76 au.edu.unsw.geog.electr 0.17 + 0.55 = 0.72 localhost 0.01 + 0.69 = 0.70 edu.ucar.scd.surf 0.20 + 0.50 = 0.70 au.net.access.power4 0.64 + 0.06 = 0.70 au.com.connect.myangup 0.17 + 0.52 = 0.69 it.infn.lnf.pccarboni 0.40 + 0.28 = 0.68 uk.ac.mcc.norse 0.64 + 0.04 = 0.68 de.tu-chemnitz.hrz.hasi 0.60 + 0.03 = 0.63 au.net.geko.netra 0.08 + 0.48 = 0.57 de.uni-frankfurt.rz.spo 0.30 + 0.25 = 0.55 uk.co.tadpole.sirius 0.50 + 0.02 = 0.52 au.edu.mq.lib.iliad 0.18 + 0.34 = 0.52 au.net.access.vweb 0.33 + 0.11 = 0.45 de.uni-bonn.informatik. 0.19 + 0.10 = 0.29 de.uni-kl.unix-ag.doene 0.00 + 0.00 = 0.00 org.sedl.janus 0.00 + 0.00 = 0.00 br.com.gns.pixel-bsb 0.00 + 0.00 = 0.00 de.dkrz.www 0.00 + 0.00 = 0.00We expect that around 0.80 utilization is a reasonable threshold; we may want to ask any child cache falling below that point to use a different (or fewer) parent cache(s).
A full collection of statistics can be found at http://www.nlanr.net/Cache/Statistics/ .
Because some of the cache machines are located at supercomputer center sites, they are able to communicate with each other over the vBNS. This is advantegeous because it provides a high-speed cache-to-cache communication channel. It means that in most cases, there is very little penalty to retrieve an object through another NLANR cache. The FIX-West cache is not on the vBNS, but we have established a collaboration with DREN which enables us to leverage the DREN connectivity between NOSC in San Diego and FIX-West.
Because the supercomputer centers are not optimal locations within the Internet topology, we are still trying to locate caches at top-level interconnection points (such as the FIX'es and NAP's). Because the supercomputer sites are one or two ISP's ``underneath'' from the backbone providers, U.S. users may find little benefit in using an NLANR cache. The FIX-West cache is actually in a very good location and this makes it more popular than the others, and we continue trying to convince NAP service providers to participate in this project.
The current version of the caching code, squid has many powerful features: Support for SSL, graceful shutdown without loss of current cache obejects, store.log that logs the actions of the storage manager and swap status/ttl of a requested object. private objects, proper parsing of HTTP reply codes, support for If-Modified-Since GET, improvements to the access log, metadata reloads in the background, flexible access control scheme, using SIGHUP to reconfigure the cache, an ftpget server to avoid forking, and assigning weights to cache neighbors.
We make sanitized versions of our daily cache access log files available for anyone to use. Some organizations are using these logs to analyze trends in WWW access patterns. At least one is using them to determine which Web sites are the most popular. The log files are available via anonymous ftp from ftp://oceana.nlanr.net/Traces/Caching/.
The Harvest project included the idea of providing a registration service for caches. This would allow cache users to locate one another and set up local cache hierarchies. The initial registration service was via email, which proved to be problematic.
NLANR now operates a cache registration service based on UDP announcement messages. If configured, a cache will send an announcement at regular intervals (i.e. once per day) or every time the cache daemon starts. The announcement includes hostname and ports, software version, contact information, geographical location (latitude/longitude), and any additional comments the administrator wishes to add. The registration database can be viewed at http://www.nlanr.net/Cache/Tracker/ .
We provide daily summary reports of cache activity. These reports include total traffic through the caches, hit-rate percent for number of requests and by traffic volume, and a breakdown of the request result types (hit, miss, error, etc). The reports also give the top 20 server sites being accessed, the top clients, the top URLs, and a list of object types (loosely determined from the URL). These are available at http://www.nlanr.net/Cache/Statistics/ .
In collaboration with a small, private corporation, we are in the process of hiring a student intern to assist with analysis of our extensive cache log data. We would like to see the intern use this research opportunity for a masters thesis. We also hope to hire several student interns during the coming school year to work with statistics and visualization of the caching architecture.
Unfortunately we had a streak of bad luck with Digital's Unix operating system. Our moderately busy, FDDI-attached machines exercised a few different bugs which caused the machines to hang very often (i.e. daily). First, a problem with the FDDI driver would hang a machine if it received too many illegal packets coming from a Cisco AGS+ router. Second, every received (small) packet would be placed into a separate `mbuf' of 8192 bytes. After some time, megabytes of RAM would be consumed by the network. Third, a bug in the TCP code would cause the maximum window size to become huge. This caused megabytes of data to become stored in a socket receive queue and led to a shortage of mbufs described above.
After two months of dealing with Digital technical support we were finally put in touch with kernel developers who provided us with fixes for these problems. Since then, the machines have been operating without any unexpected hangs.
We have written status reports and given a number presentations on this project. Claffy has given presentations at NANOG, IETF the Federal Networking Council (FNC), and numerous ISP's. Wessels has presented the project at NCAR and SDSC.
NLANR operates a number of mailing lists related to this project:
Claffy, Tamara Munzner (Stanford), and Eric Hoffman (Ipsilon)
have produce 3-D VRML representations
of the NLANR cache hierarchy. Using a database which translates
IP addresses into geographical lat/long coordinates,
they are able to generate a global view
of which sites are using the NLANR root
caches. These visualizations are available at
http://www.nlanr.net/Cache/cacheviz.html
,
with a view updated daily at
http://www.nlanr.net/Cache/daily.html
.
Claffy has been working with the NAPs, encouraging them to install caches at their sites. So far, two have placed orders for machines. We are excited about getting caches at the NAPs since they will be much more logical locations for many U.S. users.
We make current usage
statistics publicly available at
http://www.nlanr.net/Cache/Statistics
in several formats
Each of the cache machines is a high-end Digital Unix server with 10G of disk, 128M of RAM, and FDDI interfaces. The cache software keeps object metadata and hot objects in virtual memory. We have found that with 128M of RAM we must limit our cache size to around 200,000 objects or 3.3G bytes. If the cache process size grows too large, the system must swap the process to disk, which degrades overall performance.
As the above graph indicates, a cache will often serve 2G bytes of data per day, more than half of its entire cache size. We typically observe cache hit rates ranging from 10-20%, not so surprising since
We currently have an `open access' policy that allows anyone to use any of the caches. However, within the next month we will begin to implement some access control to encourage a more logical architecture. We would like to establish peer relationships with only one or two large caches in each foreign. Our proposal for this policy shift has already raised some interesting issues. In some countries, competing providers do not have local peering agreements, so it is not realistic to expect these providers to share a single large cache for their country.
Ongoing activities will be in areas to:
Other research questions that require attention include:
Ongoing activities will be in areas to:
NLANR continues to support multimedia conferencing and supporting multicast communications infrastructure to use and provide a development environment for such tools on the vBNS.
Claffy is also working on developing tools for
Mbone visualization, as mentioned
earlier, with Tamara
Munzner
at Stanford, Bill Fenner at Xerox, and Eric Hoffman at Ipsilon.
Their paper, Visualizing the Global Topology of the MBONE
was accepted to
Information Visualization '96, a symposium associated
with IEEE Visualization '96, to be held in San Francisco, Oct 27 - Nov
1 1996. The tool outlined in the work enables one
to zoom in on logical subsets of the Mbone infrastructure
such as the
vBNS-related subset of tunnels, illustrated to the right.
NLANR still supports text-based collaborative environments, in particular two online collaboration communities: Oceana, a K-12 project sponsored in conjunction with Digital Equipment Corporation; and NLANR MOO, an environment for parties interested in NLANR issues to chat, explore, and build within an online community.
Claffy has hired some students to start building a web repository of links to resources for an Internet engineering curriculum to enable instructors to share instructional resources (e.g., slides, problem sets, exams, outlines, notes).
NCRI recognizes that many of the networking as
well as computational technologies will find their way
outside of the NSF-spawned R&E environment,
just as they did with the NSFNET program.
NCRI continues to take care to optimize the interaction
between the research infrastructure and the market-provided
infrastructure, while pushing the technology
envelope gracefully, such as with the recently announced
NSFNET Connections Program.
The neutrality of the constituents of NLANR also make it
a good basis for infrastructural and interagency network activities,
such as assisting NSF with this program, and continued
collaboration among U.S. federal agencies.
NLANR facilitates collaboration in conjunction
with various other organizations,
some of which are actively
supporting NLANR activities.
Goal of meeting: focus NLANR on key areas where we can make an impact.
Agenda for next quarter/year
NLANR quarterly meeting (Tuesday 27 june 1996, IETF, Montreal)
Appendix A: example statistics of NLANR caching system
FIRST ACCESS: Wed Jun 12 00:00:12 1996
LAST ACCESS: Thu Jun 13 00:00:01 1996
SUMMARY OF PROTOCOL USAGE
UDP COUNTS TCP COUNTS TCP BYTES
Protocol counts %all %hit counts %all %hit Mbytes %all %hit
----------------------- ----------------- ----------------- ------------------
http 310108 99% 20% 148251 99% 14% 1829.65 82% 19%
ftp 3106 1% 0% 1668 1% 1% 391.77 18% 0%
gopher 562 0% - 291 0% - 7.59 0% -
cache_object 0 - - 97 0% - 0.71 0% -
file 5 0% - 0 - - 0.00 - -
----------------------- ----------------- ----------------- ------------------
SUMMARY OF CLIENT USAGE
UDP COUNTS TCP COUNTS TCP BYTES
Client counts %all %hit counts %all %hit Mbytes %all %hit
----------------------- ----------------- ----------------- ------------------
IT 913 0% 4% 719 0% 4% 10.67 0% 8%
PB 876 0% 4% 689 0% 4% 11.91 1% 7%
SD 760 0% 2% 635 0% 1% 10.55 0% 2%
BO 171 0% 1% 149 0% 1% 2.67 0% 7%
UC 1 0% - 1 0% - 0.00 0% -
----------------------- ----------------- ----------------- ------------------
[203.255.114.9] 54431 17% 26% 37738 25% 14% 497.81 22% 23%
nz.ac.waikato.isis 51852 17% 18% 33426 22% 9% 484.34 22% 8%
au.edu.uts.itd.manifera 56402 18% 45% 28458 19% 22% 392.56 18% 14%
de.uni-frankfurt.rz.spo 40300 13% 4% 12105 8% 5% 199.89 9% 15%
net.mci.sanfrancisco.ip 27383 9% 3% 19940 13% 3% 344.83 15% 8%
au.edu.mq.lib.iliad 23805 8% 3% 2 0% - 0.04 0% -
au.net.geko.netra 14877 5% 26% 6706 4% 32% 95.11 4% 31%
uk.ac.mcc.norse 16401 5% 17% 858 1% 41% 15.60 1% 52%
au.com.flex.holly 4108 1% 19% 2365 2% 24% 55.11 2% 52%
au.edu.unsw.geog.electr 3230 1% 3% 1896 1% 3% 30.41 1% 4%
au.com.bns.topdown 3132 1% 20% 1190 1% 37% 16.75 1% 27%
cz.vslib.vultur 3498 1% 14% 170 0% 53% 3.99 0% 31%
au.net.iinet.eyre 2435 1% 11% 518 0% 16% 11.61 1% 5%
au.net.ausom.bbs 1946 1% 17% 312 0% 31% 3.10 0% 24%
org.sedl.diogenes 1724 1% 12% 199 0% 100% 1.60 0% 100%
au.edu.sa.schools.catho 749 0% 34% 696 0% 34% 19.73 1% 21%
edu.syr.rocket 1267 0% 14% 106 0% 100% 0.74 0% 100%
com.hp.nsa.telford 748 0% 26% 564 0% 20% 10.34 0% 5%
uk.co.tadpole.sirius 1085 0% 5% 20 0% 100% 0.81 0% 100%
gov.lbl.ee.ren 265 0% 11% 256 0% 14% 2.21 0% 10%
----------------------- ----------------- ----------------- ------------------
SUMMARY OF SERVER USAGE
UDP COUNTS TCP COUNTS TCP BYTES
Server counts %all %hit counts %all %hit Mbytes %all %hit
----------------------- ----------------- ----------------- ------------------
http://com.netscape.hom 18151 6% 68% 2989 2% 18% 36.36 2% 16%
http://com.yahoo.www 5722 2% 26% 2031 1% 13% 15.74 1% 14%
http://com.excite.www 2705 1% 45% 1129 1% 34% 7.95 0% 35%
http://com.geocities.ww 2526 1% 20% 1239 1% 18% 8.58 0% 22%
http://jp.or.bekkoame.w 2237 1% 29% 1319 1% 25% 16.55 1% 27%
http://com.microsoft.ww 2385 1% 43% 842 1% 33% 46.20 2% 47%
http://com.cnn 1739 1% 42% 1107 1% 9% 7.11 0% 10%
http://com.sportszone.e 1710 1% 29% 899 1% 21% 5.31 0% 21%
http://com.infoseek.ima 2339 1% 58% 179 0% 49% 1.20 0% 47%
http://com.digits.count 1515 0% - 687 0% - 0.37 0% -
http://com.lycos.www 1613 1% 33% 499 0% 12% 3.70 0% 14%
http://com.aol.members 1426 0% 7% 612 0% 5% 5.51 0% 7%
http://com.happypuppy 1460 0% 44% 515 0% 4% 2.48 0% 16%
http://com.finals.www 1397 0% 42% 547 0% 25% 16.74 1% 29%
http://kr.co.netalk.www 1127 0% 63% 788 1% 2% 6.82 0% 34%
http://com.prodigy.page 1247 0% 17% 558 0% 16% 13.63 1% 21%
http://com.pathfinder 1036 0% 2% 647 0% 2% 5.83 0% 3%
http://com.cnn.www 998 0% 26% 528 0% 15% 5.94 0% 9%
http://com.nba.www 1176 0% 47% 332 0% 25% 7.94 0% 5%
http://com.compuserve.o 1074 0% - 415 0% 0% 4.13 0% 0%
http://no.lek.www 857 0% 51% 527 0% 48% 9.34 0% 29%
http://com.mckinley.ima 1136 0% 73% 216 0% 41% 0.83 0% 54%
http://edu.caltech.magi 863 0% 7% 455 0% 9% 8.71 0% 20%
http://net.infi.www 911 0% 8% 351 0% 1% 0.92 0% 4%
http://com.cris.www 842 0% 6% 410 0% 6% 2.14 0% 13%
----------------------- ----------------- ----------------- ------------------
SUMMARY OF URL TYPES
UDP COUNTS TCP COUNTS TCP BYTES
Type counts %all %hit counts %all %hit Mbytes %all %hit
----------------------- ----------------- ----------------- ------------------
Image 215576 69% 26% 98816 66% 18% 1015.50 46% 22%
HTML 43829 14% 7% 22542 15% 6% 161.24 7% 8%
Other 29423 9% 8% 16266 11% 5% 545.59 24% 13%
Directory 21928 7% 6% 10970 7% 7% 57.96 3% 8%
Bundle 1149 0% 3% 685 0% 5% 284.15 13% 7%
SHTML 659 0% 0% 317 0% 1% 1.91 0% 0%
Text 554 0% 3% 296 0% 2% 10.28 0% 10%
Movie 372 0% 9% 215 0% 13% 137.61 6% 15%
Audio 292 0% 15% 200 0% 7% 15.49 1% 6%
----------------------- ----------------- ----------------- ------------------
SUMMARY OF URL TOP-LEVEL DOMAINS
UDP COUNTS TCP COUNTS TCP BYTES
Domain counts %all %hit counts %all %hit Mbytes %all %hit
----------------------- ----------------- ----------------- ------------------
com Commercial 197589 63% 23% 87301 58% 16% 1336.28 60% 17%
edu Educational 25225 8% 11% 12808 9% 11% 205.52 9% 10%
net Network 21192 7% 15% 10884 7% 14% 141.34 6% 14%
Unknown 9854 3% 16% 5632 4% 11% 116.19 5% 7%
kr Korea (South) 8091 3% 26% 4861 3% 7% 58.98 3% 39%
jp Japan 7760 2% 20% 4589 3% 17% 55.13 2% 23%
org Non-Profit Organiza 7842 2% 9% 3683 2% 9% 32.19 1% 15%
uk United Kingdom 6281 2% 10% 3663 2% 10% 36.61 2% 11%
ca Canada 4021 1% 10% 2090 1% 12% 22.66 1% 12%
au Australia 2856 1% 6% 1942 1% 5% 34.17 2% 23%
gov Government 2800 1% 11% 1421 1% 11% 25.29 1% 4%
no Norway 2012 1% 31% 1098 1% 31% 18.93 1% 20%
sg Singapore 1902 1% 11% 1111 1% 5% 10.81 0% 6%
se Sweden 1909 1% 8% 958 1% 9% 15.88 1% 6%
nl Netherlands 1740 1% 12% 1051 1% 13% 12.97 1% 11%
fr France 1413 0% 9% 735 0% 11% 5.88 0% 9%
de Germany 1157 0% 7% 689 0% 6% 9.41 0% 10%
it Italy 991 0% 11% 568 0% 12% 8.66 0% 5%
tw Taiwan 799 0% 12% 460 0% 9% 7.12 0% 5%
fi Finland 763 0% 10% 421 0% 4% 10.89 0% 12%
at Austria 572 0% 17% 392 0% 21% 7.19 0% 22%
us United States 515 0% 10% 319 0% 11% 2.55 0% 8%
my Malaysia 562 0% 12% 245 0% 8% 1.45 0% 10%
dk Denmark 490 0% 28% 246 0% 26% 2.69 0% 27%
hk Hong Kong 451 0% 7% 200 0% 6% 7.80 0% 0%
il Israel 459 0% 7% 161 0% 7% 0.97 0% 7%
ch Switzerland 389 0% 5% 225 0% 5% 3.93 0% 1%
mil US Military 343 0% 9% 225 0% 12% 11.06 0% 55%
za South Africa 357 0% 15% 198 0% 5% 0.78 0% 2%
ie Ireland 312 0% 10% 203 0% 8% 1.45 0% 16%
id Indonesia 319 0% 26% 191 0% 31% 1.30 0% 30%
nz New Zealand 240 0% 4% 200 0% 4% 3.01 0% 3%
be Belgium 224 0% 4% 136 0% 4% 0.66 0% 21%
br Brazil 207 0% 11% 128 0% 10% 1.40 0% 4%
lv Latvia 162 0% 2% 143 0% 1% 2.65 0% 0%
ph Philippines 160 0% 2% 117 0% 2% 0.61 0% 4%
mx Mexico 158 0% 1% 78 0% 5% 0.56 0% 4%
gr Greece 189 0% 3% 47 0% - 1.59 0% -
pl Poland 174 0% 5% 60 0% 3% 1.29 0% 9%
cn China 133 0% 8% 91 0% 10% 0.39 0% 28%
es Spain 138 0% 1% 71 0% 1% 0.48 0% 1%
pt Portugal 99 0% - 46 0% - 0.91 0% -
int International field 80 0% 11% 59 0% 5% 0.28 0% 1%
th Thailand 99 0% 11% 30 0% 30% 0.13 0% 38%
su Soviet Union 70 0% 1% 39 0% 3% 0.94 0% 0%
cz Czech Republic 68 0% 21% 41 0% 32% 1.61 0% 2%
bg Bulgaria 77 0% 4% 29 0% 7% 0.10 0% 6%
hu Hungary 66 0% 11% 36 0% 17% 0.20 0% 4%
lu Luxembourg 56 0% 20% 44 0% 7% 0.27 0% 8%
sv El Salvador 0 - - 97 0% - 0.71 0% -
----------------------- ----------------- ----------------- ------------------
CLIENT UTILIZATION
org.sedl.janus 1.00 + 0.71 = 1.71
[206.106.252.226] 0.53 + 0.80 = 1.33
au.edu.sa.schools.catho 0.34 + 0.93 = 1.27
dk.netman.tinderbox 0.19 + 1.00 = 1.19
org.sedl.diogenes 1.00 + 0.12 = 1.12
gov.lbl.ee.ren 0.14 + 0.97 = 1.11
edu.syr.rocket 1.00 + 0.08 = 1.08
com.wmsoft.thesparc 1.00 + 0.08 = 1.08
nz.ac.waikato.osiris 1.00 + 0.08 = 1.08
it.forza-italia.www 1.00 + 0.05 = 1.05
de.uni-kl.unix-ag.doene 1.00 + 0.03 = 1.03
uk.co.tadpole.sirius 1.00 + 0.02 = 1.02
edu.bxscience.explorer 0.00 + 1.00 = 1.00
UC 0.00 + 1.00 = 1.00
com.nttlabs.ns 0.07 + 0.91 = 0.97
com.hp.nsa.telford 0.20 + 0.75 = 0.95
jp.or.imasy.light 0.00 + 0.94 = 0.94
BO 0.01 + 0.87 = 0.88
SD 0.01 + 0.84 = 0.85
[203.255.114.9] 0.14 + 0.69 = 0.84
PB 0.04 + 0.79 = 0.83
IT 0.04 + 0.79 = 0.83
au.com.flex.holly 0.24 + 0.58 = 0.82
net.nlanr.oceana 0.00 + 0.80 = 0.80
au.net.geko.netra 0.32 + 0.45 = 0.77
net.mci.sanfrancisco.ip 0.03 + 0.73 = 0.76
jp.ad.imnet.slim.cache 0.08 + 0.68 = 0.76
au.com.bns.topdown 0.37 + 0.38 = 0.75
nz.ac.waikato.isis 0.09 + 0.64 = 0.74
au.edu.uts.itd.manifera 0.22 + 0.50 = 0.73
au.edu.unsw.geog.electr 0.03 + 0.59 = 0.61
net.nlanr.cache.ch 0.00 + 0.60 = 0.60
cz.vslib.vultur 0.53 + 0.05 = 0.58
au.net.ausom.bbs 0.31 + 0.16 = 0.47
uk.ac.mcc.norse 0.41 + 0.05 = 0.47
es.ub.ecm.ababel 0.29 + 0.13 = 0.42
au.net.iinet.eyre 0.16 + 0.21 = 0.37
de.uni-frankfurt.rz.spo 0.05 + 0.30 = 0.35
uk.co.demon.hisoft 0.18 + 0.15 = 0.33
edu.missouri.phlab.nezp 0.14 + 0.17 = 0.32
au.edu.mq.lib.iliad 0.00 + 0.00 = 0.00
TOP 25 HTTP Reqeusts
446 http://home.netscape.com/home/internet-search.html
381 http://home.netscape.com/
177 http://cnn.com/SPORTS/BASKETBALL/nba.playoffs/images/nba_playoff_banner.
168 http://cnn.com/SPORTS/BASKETBALL/nba.playoffs/images/nba_footer.jpg
145 http://cnn.com/ads/SPORTS/adv1.gif
118 http://home.netscape.com/home/internet-directory.html
103 http://cnn.com/SPORTS/BASKETBALL/pro/automation/today/score_board.html
96 http://sports.yahoo.com/nba/
75 http://home.netscape.com/escapes/search/search5.html
75 http://chat.magmacom.com/lhotel/sell.html
74 http://home.netscape.com/home/whats-new.html
72 http://home.netscape.com/escapes/search/search1.html
70 http://home.netscape.com/home/whats-cool.html
64 http://six.internet-audit.com/six/ZQ0040622.gif
63 http://six.internet-audit.com/six/ZQ0040621.gif
54 http://home.netscape.com/escapes/search/search4.html
53 http://www.bytethis.net/graphics/ie_animated.gif
46 http://www.excite.com/img/ads/smi_sponsor.gif
41 http://home.netscape.com/escapes/images/exploring_ban.gif
41 http://www.yahoo.com/
40 http://home.netscape.com/images/navigation_bar.gif
32 http://www.yahoo.com/images/new2.gif
30 http://www.yahoo.com/images/search2.gif
29 http://www.yahoo.com/images/cat2.gif
21 http://home.netscape.com/escapes/search/images/elibrary_logo.gif
TOP 25 ICP Reqeusts
1112 http://home.netscape.com/home/internet-search.html
554 http://home.netscape.com/escapes/images/exploring_ban.gif
554 http://home.netscape.com/
461 http://home.netscape.com/images/navigation_bar.gif
456 http://home.netscape.com/escapes/search/images/search_excite_logo.gif
451 http://home.netscape.com/escapes/search/images/search_netlocator_logo.gi
445 http://home.netscape.com/escapes/search/images/a2z_logo.gif
441 http://home.netscape.com/escapes/search/images/alta_vista_logo.gif
440 http://home.netscape.com/escapes/search/images/dnbannerN3.gif
438 http://home.netscape.com/escapes/search/images/swcom.gif
437 http://home.netscape.com/escapes/search/images/hotbot_logo.gif
434 http://home.netscape.com/escapes/search/images/elibrary_logo.gif
433 http://home.netscape.com/escapes/search/images/lycos_logo.gif
429 http://home.netscape.com/escapes/search/images/search_infoseek_logo.gif
418 http://home.netscape.com/escapes/search/images/point_logo.gif
408 http://home.netscape.com/escapes/search/images/search_mckinley_logo.gif
408 http://home.netscape.com/escapes/search/images/search_100hot_logo.gif
408 http://home.netscape.com/escapes/search/images/search_ibminfo_logo.gif
399 http://home.netscape.com/escapes/search/images/search_opentext_logo.gif
381 http://home.netscape.com/escapes/search/images/search_yahoo_logo.gif
273 http://home.netscape.com/home/internet-directory.html
271 http://www.yahoo.com/images/search2.gif
265 http://cnn.com/SPORTS/BASKETBALL/pro/automation/today/score_board.html
257 http://six.internet-audit.com/six/ZQ0040622.gif
226 http://www.yahoo.com/
Request Distribution, combined HTTP and ICP, per half hour
+------------------------------------------------+
13437-14928| # # # # |
11944-13436|## ## # #### #####|
10451-11943|### # ### ##############|
8958-10450|##### ##### ################|
7465- 8957|############## #### #################|
5972- 7464|####################### ###################|
4479- 5971|################################################|
2986- 4478|################################################|
1493- 2985|################################################|
1- 1492|################################################|
Counts /------------------------------------------------+
Hour 0 1 2 3 4 5 6 7 8 9 10 12 14 16 18 20 22
APPENDIX B
to: mccrebsi@koi.ina.de (Basil McCrea KOI)
from: Marc Baudoin
Date: Wed, 5 Jun 1996 07:21:52 +0200
To: Duane Wessels
Links to other networking resources
acknowledgements
and disclaimers
8 july 96, comments: info@nlanr.net.