EPSCoR workshop final report

EPSCoR Satellite/Wireless Workshop

July 20-22, 2000

Hosted by the National Laboratory for Applied Network Research (NLANR)

Measurement and Network Analysis Group and SDSC (San Diego Supercomputer Center)

Final Report

Introduction

Hosted by the National Laboratory for Applied Network Research (NLANR) and the San Diego Supercomputer Center (SDSC), the EPSCOR Satellite/Wireless Workshop was held July 20-22, 2000. The workshop served as a platform and starting point to exchange state-of-the-art information from network researchers and communications providers with the state-of-practice information and point of view of disciplinary researchers with remote networking needs, as well as network administrators and educators from rural, underserved, areas. The core purpose was to begin to define the parameters which will be needed to reduce the performance gap between connected high performance national centers, such as major universities, and the capabilities currently available in rural areas and to scientists in the field. The ultimate goals include greater high performance network ubiquity and more transparency and answering the question of how to achieve the best service through potential hybrid situations.

Participants included:

Network researchers
Disciplinary researchers with needs for remote connectivity to enable and enhance the full scope of their work (in the field, at observatories, with telemetry devices placed in remote and/or distant locations)
Network administrators from EPSCOR states
Representatives of the National Science Foundation (NSF)
Communications providers

EPSCoR, the Experimental Program to Stimulate Competitive Research--is an NSF program designed to identify, develop, and utilize a state's academic science and technology resources in a way that will benefit the state and its citizens. States that have historically received lesser amounts of Federal R&D funding and have demonstrated a commitment to develop their research bases and improve the quality of science and engineering research conducted at their universities and colleges are eligible to participate in EPSCoR. The program has been operating since 1980 and currently operates in 20 states.

Congress has mandated that EPSCoR have an outreach program. As one method of outreach, EPSCoR is sponsoring a number of conferences/meetings which will discuss how to get funding, review proposals, provide networking opportunities, etc. (please refer to http://okepscor.org). A major goal of these meetings is to get EPSCoR states to work together, focusing on "strength in numbers," and encouraging the formation of partnerships with “anyone you can,” including non-EPSCoR states. This workshop is Phase I, of two, which will examine the networking needs of researchers and educational institutions in rural America.

What are the social and economic effects of tether-free and wireless communications and Internet technology? A look at Internet history can be a valuable aid: 1992 - commercial interests in the Internet explode; 1995 - NSF pulled out of NSFNet and changed to vBNS; and currently, many companies are interested in high performance connections (HPC). Much of today's direction is focused on network research - towards increased scalability, privacy, etc. Infrastructure plays a large role and is a necessary tool for enabling research and education. We must extend the reach of high performance networking through:

Network and middleware service capabilities (end to end performance extending beyond the backbone and to the last mile; integration of research and infrastructure);
Network applications; and
Forming collaborations and ventures with new participants (EPSCoR conferences like this one, for example).

What are the remote networking needs of researchers and educators?

A variety of speakers from different academic disciplines spoke to this issue, after which breakout sessions were held to further discuss and define these needs. (Please refer to http://www.sdsc.edu/Workshops/epscor/agenda.html for a complete agenda, and to http://moat.nlanr.net/EPSCoR/ and http://okepscor.org to view slide presentations used by speakers.) Various applications that can benefit from tetherless/wireless access include: field work, observatories, remote telemetry, education in rural areas, and distance learning, among others. Examples of the different (and sometimes similar) needs in various disciplines from our presenters follow.

Arecibo Observatory

The Arecibo Observatory in Puerto Rico is a National Astronomy and Ionosphere Center. The Arecibo telescope gathers massive amounts of data for its many experiments. Some instruments produce up to 10 Megabytes per second. The output is stored on 35 Gigabyte tapes and mailed to computer centers for analysis. Generating such massive amounts of data at a location that is quite remote from most researchers presents problems; for example, it would be desirable to have a solution for remote control of the telescope. Arecibo's current connectivity to the mainland consists of a recently installed T1 line; the cost of installing a T3 is prohibitive. Researchers would like to combine data from Arecibo with other observatories around the world in real time, but current solutions are highly implausible.

Greenbank National Radio Astronomy Observatory

What will be the world’s largest steerable radio telescope is currently being constructed in Greenbank, WV. The telescope will be connected by fiber optic cables to its control center a half mile away; however, the fastest connection currently available in Greenbank is a T1 line. Therefore, the Greenbank Facility is actively seeking higher bandwidth solutions; the solution must be lower in cost than a commercial T3 connection (again, the cost of which is prohibitive).

Field Sensors

The need for the development of field sensors used in biological research was discussed. The issues involve biological field stations, their creation and their networking capabilities. Researchers would like to see nanosensors, but there are questions concerning the gathering of the collected information and the archiving of that information once it is gathered. There is interest in the activities of the National Environmental Observatory Network (NEON): http://www.sdsc.edu/NEON.

Seismology Challenge

A good example of an area requiring examination and development is the use of sensors in seismology research. Earthquake sensors require very little bandwidth on their own, but they do require a constant, uninterrupted, stream of data. This factor coupled with the possibility of having hundreds, or even thousands, of sensors, which are widely dispersed, poses network engineering problems which cannot to be taken lightly. Seismologists at the Scripps Institute of Oceanography (SIO) would like to see real-time feedback from these sensors for research, as well as for the capabilities of early detection and prediction of earthquakes in real time. They would also like to get real-time seismic data from central Asia. Priority access to the network would be necessary for forecasting and warning of immediate events. For more information: http://www-ida.ucsd.edu/ANZA/home.html.

Emergency, Public Safety applications

Crisis Management: There are several issues when dealing with a crisis that depend on good, fast, and reliable access to communication facilities, often in hostile conditions. Often in time of crisis the existing communications infrastructure is not adequate or cannot be used. Dealing with crises effectively depends upon the distribution of information. It was pointed out that the first line of response in the U.S. consists of local police, fire, and medical departments which are "undertrained, understaffed, and underequipped" to deal with most serious situations. Some examples of crises include natural disasters and biological and chemical terrorist acts. Often, the proper information and methods for dealing with toxic agents, and the ability to locate yourself, others, and objects quickly and accurately could mean the difference between life and death. Additional information on the Multi-Sector Crisis Management Consortium (MSCMC) is available at: http://moat.nlanr.net/EPSCoR/george_markowsky.ppt.

What is available (new technologies), and how much will it cost?

There are a number of network access technologies. We need to find broadband “last mile” technologies – wireless, satellite, fiber – we must not look only at hardware, we need to also investigate middleware (mobility and services). There are a number of new technologies available.

Satellite technology

Satellites can provide enough bandwidth for our networking needs and they are a solution to the reach problem with traditional networks. Satellites solve the “last mile” problem and can easily get to the end user; they can go anywhere and their cost is independent of location. They are capable of the same throughput as fiber networks. Some satellites rely on ground based fiber gateways, while others communicate directly with each other.

In the past, satellites were geosynchronous and transmitted in one direction. Today's satellites are high speed, built for two-way communication, and designed around IP. There are a number of broadband IP-based providers: Astrolink, AOL/Direct PC+, EuroSkyWay, Gilat2Home, SkyBridge, Spaceway, and Teledesic; SkyBridge and Teledesic hope to be in service by 2004. Most of these providers operate between 20GHz and 30GHz (KA Band).

The market for satellite service is strong: broadband satellites capture about 7% of the telecommunications industry spending. As fiber reaches further and further, the overall demand for broadband services increases in rural areas and developing countries (expansion creates demand). Obstacles for the satellite service industry include: educating investors about the differences between broadband and narrowband service, which can be somewhat difficult, but is crucial to obtaining capital, and overcoming bad financial press, companies like Iridium have given satellite service a somewhat negative image.

Satellite definitions and some technical considerations

Geostationary orbits (GEO): GEOs are circular orbits that are oriented in the plane of the earth's equator. In a geostationary orbit, the satellite appears stationary, i.e., in a fixed position, to an observer on earth. More technically, a geostationary orbit is a circular prograde orbit in the equatorial plane with an orbital period equal to that of the earth. This is achieved with an orbital radius of 6.6107 (equatorial) earth radii, or an orbital height of 35786 km. The footprint, or service area, of a geostationary satellite covers almost 1/3 of the earth's surface (from about 75 degrees south to about 75 degrees north latitude), so that near-global coverage can be achieved with as few as three satellites in orbit.
Low Earth Orbits (LEO): LEOs are either elliptical or (more commonly) circular orbits at a height of less than 2,000 km above the surface of the earth. The orbit period at these altitudes varies between ninety minutes and two hours. The radius of the footprint of a communications satellite in LEO varies from 3000 to 4000 km. The maximum time during which a satellite in LEO orbit is above the local horizon for an observer on the earth is up to 20 minutes. (A satellite with an orbiting altitude less than geostationary altitude travels at a speed faster than the earth's orbit.)
Highly Elliptical Orbits (HEO): HEOs typically have a perigee at about 500 km above the surface of the earth and an apogee as high as 50,000 km.
Mesh: Non-geostationary satellites need to be deployed in a mesh in order to assure that one is always on the horizon. In order to provide continuous service either a moving dish or a phase array antenna must be used; for non-phase array services, two dishes are needed in order to seamlessly track satellites and ensure constant services.
Spectrum usage: The presentation on wireless communications and the state of spectrum usage in the U.S. as regulated by the FCC, covered the problems, and proposed solutions. The chairman of the FCC has declared that we are out of spectrum, (all of the available bands have been allocated), and the FCC is currently encouraging subcontracting of larger bands of spectrum. Spread spectrum technologies have been deployed in the past to alleviate some of the problem, but the use of spread spectrum has not become widespread in the commercial sector, mainly due to regulation. Per current policy, only bands designated as 'unlicensed' by the FCC are available for consumer products. There are currently three such bands: 900 MHz, 2.4 GHz, and 5.6 GHz. In particular, the 2.4 GHz range has been adopted for short distance, digital data transmissions, specifically wireless LANs, Bluetooth, and other information appliances. This generates a problem with interference, which degrades the quality of the signals that these devices use; the degradation is proportional to the number and range of these devices. Additional information can be found at: http://moat.nlanr.net/EPSCoR/dewayne_hendricks.ppt.
Quality of service, the latency problem: Because of the high latency involved with satellite service, TCP applications suffer restrictions based on the window size of the TCP stack on the host operating system.
An example scenario regarding window sizes: The maximum window size for Microsoft Windows is 64 kilobytes. The optimum window size is considered to be the product of bandwidth and delay (Round Trip Time). Therefore, on a satellite link where delay is commonly 560 ms, the realized bandwidth will be 64/.560 = 117,027 bytes per second (936 Kb/sec). For the Windows operating system, this is the best throughput that can be expected, even though more bandwidth may be available. The default window size for Windows is in fact only 8 kilobytes, which will yield only 14KB/sec (112Kb/sec).
Solving the latency problem: One solution is to tune the operating systems of everyone using the satellite link. (Advice and instructions for tuning many popular operating systems can be found at http://www.psc.edu/networking/perf_tune.html. However, it was found that a very small percentage of end users are willing to tune their operating systems themselves or even let others do so for them. Therefore, it is easier to make network improvements, which would be transparent to the user.
Black boxes for satellite TCP performance: There is a company which makes a "black box" solution for latency problems; the company is Mentat, the product is called "SkyX." SkyX has the ability to dramatically improve TCP performance through high latency channels (see http://www.internet-2.org.il/satellite-testing.html). SkyX works by renegotiating TCP connections between the SkyX boxes. It requires that the connection be symmetrically routed and it does not improve non-TCP protocols. Substantial improvements have been achieved with the SkyX system in place, reaching 19.5 mbit/s bandwidth from user computers with default window sizes (i.e., 8 KB). For additional information on this, please refer to: http://moat.nlanr.net/EPSCoR/hank_nussbacher.ppt.

Satellite provider: Teledesic

Teledesic is a non-geosynchronous provider which will be coming on-line in 2004, using satellites that communicate directly with each other, (18 GHz downlinks, and 20 GHz uplinks). They plan to offer truly global and flexible end-to-end service, with flexible network technologies, rapid deployment, bandwidth on demand, multiple class service, network provisioning, the ability to deal with bursty behavior, universal access, a homogenous network, and guaranteed Quality of Service standards (low latency, high availability, high data integrity, good customer support). Systems for typical users will have 150 mbits/sec downlink, and 2 mbits/sec uplink with a latency of less than 250 ms; power users will have more uplink speed.

Teledesic boasts lower latency in their satellite system than geosynchronous satellites. They originally proposed 900 satellites at 7 km; then changed to 300 satellites at 14km to be more realistic. The current thinking is to have fewer satellites at a higher altitude, but that will increase the latency. (For additional information: www.teledesic.com.)

How much will it cost?

Hybrid satellite/land line: Tel Aviv University has a dual ATM network with an OC-3 primary connection and an E3 backup. The E3 (34 mbits/sec) connection to the European backbone costs $157,000 per month. Tel Aviv also has a T3 (45 mbits/sec) satellite link to Star Tap in Chicago, which costs $198,000 per month. Collectively these links are used to supply Internet service to eight (8) universities in Israel. The universities are then linked together with a 155 mbit/sec network, as well as a 2 mbit/sec backup network.
Satellites to offset bandwidth usage: There are a few companies, such as Cidera, who do broadcast Internet services via KA band satellites. Cidera in particular sells a range of services which includes Usenet news, Streaming Media, and Web-cache broadcasting. This is a unidirectional service which requires a 1.2 meter satellite dish but is reasonably priced considering the cost of land based bidirectional bandwidth. It was pointed out that if you receive all Usenet news, it consumes 12 megabits/sec of your available bandwidth. If you are paying $28,000 -$37,000 per month for a T3 (which is common for urban areas) then Usenet traffic could be costing up to $8,000/month. This example is a good case for satellite streaming, which costs between $350 and $650 per month, depending on what services you request. There are several competitors to Cidera, including I-beam, which was used by Buchnell University to offset Napster traffic.
Wireless initial investments vs. leased telephone lines: It was pointed out by many attendees that, where wireless equipment tends to cost more for the same bandwidth capabilities as wired equipment, airwaves are much lower maintenance and basically free. Virginia has begun to use LMDS (local multipoint distribution system) for its “last mile” connections, which they plan to provide to homes for under $100 per month for T1 speed connectivity. This is significantly less than the telephone company equivalent.
Satellite vs. fiber: Nussbacher points out that fiber is outpacing satellite networks in both quality and economy. Satellite is still excellent for broadcasting, however.

Success stories – Examples of the benefits to researchers and remote educators

Remote research data collection

David Hughes has had numerous successes connecting field researchers to their instruments in remote areas through wireless technologies. His applications usually do not need high bandwidth, therefore, his solutions are often inexpensive in addition to practical. For more information, please see the “Biological Science by Wireless” Projects Web pages: http://wireless.oldcolo.com.

Networking accomplishments

A Wireless Solution?: NLANR has been working closely with seismologists at SIO to set up a wireless network solution to connect earthquake sensors to the Internet. This project will hopefully demonstrate the effectiveness of wireless broadband networks in research and educational applications, including telemetry, field stations, and remote education. It is anticipated that approximately 400 sensors will be deployed across the southwestern U.S. For additional information, please see: http://moat.nlanr.net/EPSCoR/hans-werner_braun.ppt and http://moat.nlanr.net/HPWREN.

Kingdom of Tonga: The Dandin Group has been consulting with the Prince of Tonga with the intention of installing a wireless communications grid including telephone and Internet service. 11,000 homes and 6,500 telephone customers will be connected via wide band spread spectrum wireless devices. This is not only a demonstration of the versatility of wireless communications, being spread out over the 70 inhabited islands of Tonga, but also is a demonstration of the abilities of spread spectrum devices to scale appropriately. For additional information, please see: http://www.dandin.com and http://moat.nlanr.net/EPSCoR/dewayne_hendricks.ppt.

AN-MSI (Tribal Sovereignty): Based on their Kingdom of Tonga venture as a proof of concept, The Dandin Group is planning to deploy pilot projects on Tribal lands in conjunction with AIHEC (the American Indian Higher Education Council). It is expected that these pilot programs will offer high speed voice and data communications to tribal lands with a setup cost of $450 per home. Tribal Sovereignty makes this possible, as they are a “regulatory haven,” where FCC rules do not necessarily apply. It is hoped that these demonstrations of the possibilities of wide band wireless communications will inspire renegotiations of the currently limiting FCC legislation. For additional information, please see: http://www.dandin.com and http://moat.nlanr.net/EPSCoR/dewayne_hendricks.ppt.

Where do we go for funding? What do the funding source(s) want?

Money is available. NSF page with links to overviews of the various directorates and programs, including links to funding information (on the navigation bar): http://www.nsf.gov/start.htm
Go to the following NSF site, find the appropriate directorate (area of research) and program officer; contact them to discuss funding proposals: http://www.nsf.gov/home/nsforg/orglist.htm
Proposal development conference: How NSF reviews proposals; small groups will go through a mock proposal review.
In what direction is future funding going? Currently there are several NSF initiatives with grants in the areas of nanotechnology, biocomplexity, information technologies, distance learning, and so called "Living Laboratories."
There are two major EPSCoR awards totaling over $36 million for 36 months per state with congressional approval: the Infrastructure Development and Co-Funding Initiative.
EPSCoR Outreach: program officers come to various states to discuss funding and quality of proposal. Outreach conference presentation link: http://okepscor.org.
NSF EPSCoR topical conference schedules (see Nancy Dixon’s overhead)
Hints: It is a good idea to get EPSCoR certification before going to NSF for funding because NSF gives preference to EPSCOR states when awarding grants. One goal is to get EPSCoR states to work together, to form partnerships with anyone you can - “strength in numbers.”
SBIR: small business innovative research (conference should be in 2/01, or 3/01).

The next step(s)

Projected needs and probable paths to take
Phase 2 (October, 2000 in Las Vegas, NV)
Action items and flip chart summaries (follow)

Action items and flip chart summaries

Open discussion after break-out session - Friday

Financial considerations:

Question: Economies of networks?
Savings with networks (costs, time, personnel)
Take comparisons (service vs. device model)

Research applications:

Two observatories seem to have similar problems
East coast groups have many bio/environmental sensors

Technical considerations:

Repeaters between observatory and telecom station
Electrical power grid
Can we break data feeds into smaller pieces?
Separate control problem(s) from research problems
Access to and distribution of real-time data
Leading communities are “in corridors” (also included in societal section)

Collaborations:

Start teleconferencing; bring more faculty
How can we bring more researchers to our groups?
Make partnership centers
Find vendors who may benefit from collaboration

Education applications (remote education):

Rural schools (especially math/science) need high performance connectivity -- Tribal lands
Bring reserves to public (through wireless)
Stereo video cameras on boats; let kids use them

Societal issues:

Leading communities are “in corridors” (also included in technical section)
Growing rate of unemployment, lack of trained individuals
Populations becoming more urbanized; what about other parts of the state (less telecom in these areas)

Action Items (from Saturday’s open discussions)

Ongoing activities: