NCAR's Boulder PoP at CU's Telecommunications Building
In March 2004, NETS relocated gin, NCAR's Juniper M20 border router, to the
University of Colorado's Telecommunications Building. CU Telecom, as
the building is called, is situated on the ring portion of the
BRAN fiber. Placing network equipment on this ring increases network
reliability in case of a BRAN fiber cut. NCAR, The City and County
of Boulder, Colorado State University, the University of Colorado
at Boulder, and NOAA all benefit from this increased reliability.
In early 2008, the majority of gin's routing functionality was
moved to the Sup720 3BXL tcom-gs-1. In early 2009, gin was
decomissioned and the ICG OC-12 was replaced with a Level3 GigE DIA
service, using the same demarcation point at Level3/ICG Boulder.
BPoP Network Architecture
The BPoP installation at CU Telecom now consists of:
This equipment is in Rack 6, at the end of a row in the basement of CU TCOM.
Connections to BPoP participants, including NCAR, are made thru GigE and 10-GigE connections to
tcom-gs-1. The MOVAZ WDM equipment terminates a lambda from
Longmont for a Boulder County MAN connection. All BRAN fibers are routed up into fiber trays to
the BRAN rack in the row facing Rack 6, about 15 feet away.
Out-of-band and console access is provided thru
CU's Cyclades terminal server, neo-ts-tcom.colorado.edu
- an NCAR Cisco 6509 switch/router, tcom-gs-1, for GigE and 10-GigE connections
- A pair of BiSON Movaz (Adva) wave-division multiplexers for BiSON connections (east and west)
- a GigE connection to CU's tcom switch, for CU and Level3 DIA connections.
tcom-gs-1 connects to two redundant internet providers:
- Level3 via CU's tcom switch, over BRAN to the Level3/ICG Boulder hub on Walnut Street.
- FRGP via BiSON-A and B, both sides of BRAN to the Level3/ICG Boulder hub and on to the FRGP's Level3 colocation in Denver.
BPoP BRAN Connection Specifics
Portions of all BPoP connections are carried over BRAN fiber.
NCAR has two 10-GigE connections to the BPoP hardware - one to Mesa Lab
and one to Foothills Lab. All 6 pairs of NCAR's red-tube BRAN fiber
were terminated at TCOM specifically for the BPoP relocation. RD-11/12
southbound is patched to Mesa Lab at ECOT, and RD-11/12 northbound is
patched at ICG to WH-11/12 to connect Foothills lab.
A table of NCAR's BRAN fiber utilization is available.
CSU originally installed the MOVAZ multiplexer at CU TCOM to bring their
connection from Fort
Collins on ICG and BRAN fiber. This was eventually expanded into
the full BiSON ring, which provides redundant paths into Level3 Denver
Colo at 1850 Pearl. CSU no longer peers directly with the BPoP
router at TCOM. However, CSU obtains Level3 DIA out of BPoP via BiSON -> tcom-gs-1 (at layer 2) -> CU's switch.
City of Boulder
The City of Boulder uses two GigE connections Etherchanneled together
to provide redundancy. BRAN fibers BL-1/2 are used both north and
southbound to the City's switch at the Public Safety building.
From the FRGP perspective, the City and County of Boulder are
both secondaries behind the NCAR router.
NOAA has a GigE as well as a 10-GigE connection to tcom-gs-1, carried
on BRAN directly to ECOT, where its cross-patched to NOAA.
Like CSU, they use tcom-gs-1 (at layer 2) for access to Level3
DIA and the FRGP.
The 10-GigE connection to the FRGP uses the BiSON-B path to Denver as
primary, to avoid sharing a path and vulnerability with the Level3 DIA
service. The FRGP is also reachable over the GigE BiSON-A path.
The primary FRGP connection is carried on BRAN RD-9/10
NCAR's "inner ring" Red tube is patched to White at ECOT, which is
spliced back to Red at Pearl Street, so it arrives at Level3/ICG from
on RD-9/10. At Walnut, the connection is patched to ICG IRU fiber to
at 1850 Pearl in Denver.
DIA service has replaced the ICG OC-12. It is patched on BRAN
BL-1/2 along the same path as the original ICG OC-12. From
Walnut, it rides a Level3 "crossroads" GigE backhaul to 1850 Pearl,
where it lands on car1.denver1.level3.net for "HSIP" service.
Emergency console access is provided by a telephone modem and CU's
network, which doesn't rely on the BPoP installation. Console cables
are provided to the 6509, and the MOVAZ shelves. Access
neo-ts-tcom.colorado.edu remotely by dialing 303-417-0020 or by "ssh -l
ncarbpop neo-ts-tcom.colorado.edu". Access details are found in the
To gain physical access to the Telecom building, contact Robert Roybal or David Lindblad
or the ITS help desk. Normally,
contact Robert for hardware/software issues and David for BRAN fiber issues. In any case,
please notify firstname.lastname@example.org
as soon as practical.
Failover in case of fiber cut
Resistance to a BRAN fiber cut was the primary motivation for locating
the BPoP equipment at CU TCOM. BPoP connections to the Internet are diverse: The Level3 connection is
made around the North and West side of BRAN, while the FRGP connection
is made around both sides of BRAN. NCAR and the City have
redundant fiber connections to the BPoP, and NOAA has a backdoor
connection to NCAR's Mesa Lab which provides them with a backup wireless path via FL.
If a fiber cut in the BRAN ring were to occur, one of the BPoP's
Internet connections could fail. Also, one of the City's connections
would fail, and the same would hold for one of NCAR's ML or FL
connections. The County's MAN connection would either stay up
or fail entirely, depending on where the cut occured.
Spare modules and parts for the Catalyst 6509 and the Movaz shelves are available at NCAR ML.