in the Super Bowl project included (left to right, bottom
row) Tarak Gandhi, Ofer Achler, Mohan Trivedi, Ramesh
Rao, and Remy Chang; (top row) Kim Harlow, Andrew Cosand,
Junwen Wu, Leah Vining, Kohsia Huang, Giovanni Gualdi,
Shinko Cheng, Kate Tull, and Dashan Gao
- Behind the scenes and away from the glare of publicity, more than
a dozen faculty and students affiliated with Cal-(IT)˛ mounted
a coordinated effort to put cutting-edge public-safety technologies
in place for Super Bowl XXXVII. And they did so in a relatively
concentrated period of time: barely three weeks from the first planning
meeting to Game Day, Sunday, January 26. "It was a Herculean
effort on the part of all our people and all their counterparts
at local, state and national agencies," said Ramesh Rao, Cal-(IT)˛'s
division director at UCSD, who oversaw the institute’s involvement.
"Cal-(IT)˛ partners stepped up to help in crucial ways
at critical times to make this effort a success, and the entire
exercise illustrates why San Diego has earned a reputation as a
place where public and private sectors work together on solutions
to important problems."
included newly-developed software to glean security data automatically
from cameras wirelessly hooked up to the Internet. The cameras monitored
the southern perimeter of QUALCOMM Stadium, a high-traffic intersection
in downtown San Diego, and a waterfront command center staffed by
San Diego Police and other security personnel. Separately, Cal-(IT)˛
researchers developed a service that allowed police to call a toll-free
number to get voice-activated traffic updates for major roads and
highways. "While our systems played only a small part in the
event’s overall security," added Rao, an electrical and
computer engineering professor at UCSD’s Jacobs School of
Engineering, "we were able to take experimental hardware and
software code and turn them into working systems almost overnight
- and they worked flawlessly!"
San Diego spent
roughly $2 million on security for the game (with half of that coming
from the National Football League), including the cost of equipping
QUALCOMM Stadium itself with more than 50 video surveillance cameras.
But the City invited public agencies, private companies and research
organizations to contribute - at their own cost - backup systems
outside the stadium that could also be monitored by police and security
personnel. "“It is remarkable how quickly and successfully
San Diego technology institutions pulled together on this,"
said Cal-(IT)˛'s Rao. "And it would not have happened
without the leadership of Assistant Chief of Police Bill Maheu,
who was in charge of Super Bowl security, and of Bob Davis, a police
advisor who coordinated the participation of technology providers."
Bowl XXXVII sign from Seaport Village Command Center
Rao led the
institute's effort, together with Cal-(IT)˛'s Intelligent Transportation
and Telematics layer leader at UCSD, Mohan Trivedi. A Jacobs School
professor, Trivedi and a team of 13 students and researchers from
UCSD'’s Computer Vision and Robotics Research (CVRR) lab installed
a variety of cameras, software and other equipment to capture and
analyze video streams in real time over the Internet – turning
the cameras into 'smartcams.' "On a temporary basis, we deployed
roughly $150,000 worth of cameras, video server, computer and large
disk storage equipment that we only have because of the support
of agencies that fund our research on computer vision and homeland
security," said Trivedi. "And even though most of the
systems were up for just the weekend, we could only be successful
because of the nearly four years of research - and funding - that
allowed us to develop these technologies."
Among the agencies
sponsoring computer vision-related work in Trivedi's lab: the U.S.
Defense Threat Reduction Agency, as well as the Technical Support
Working Group, an inter-agency forum that coordinates R&D for
combating terrorism; Space and Naval Warfare Systems Command (SPAWAR);
Center for Commercialization of Advanced Technologies; University
of California-based Industry-University Cooperative Research Program;
Caltrans; Office of Naval Research (ONR); National Science Foundation’s
Information Technology Research program; and the William J. von
Liebig Center for Entrepreneurism and Technology Advancement.
of researchers included Ofer Achler [a Cal-(IT)˛ undergraduate
fellow in 2001], Jun Wen Wu [Cal-(IT)˛ graduate fellow in 2001-02],
and Dashan Gao [who is a Cal-(IT)˛ graduate fellow in the current
2002-03 academic year].
The CVRR researchers
deployed cameras and other equipment in differing configurations
at multiple locations, with all feeds viewable - and, in some cases,
customizable - over the Internet, from two command centers (one
at San Diego Downtown, the other at San Diego's Seaport Village.)
The research team also developed a series of software modules tailored
to the needs of law-enforcement authorities and security personnel.
Most were based on previous computer algorithms developed in the
CVRR lab but required tweaking to fit the occasion. "Our algorithm
development team was working almost 20-hour shifts all the way up
to kickoff time," said Trivedi. "They were able to develop
various applications for robust event recognition and event database
The four locations,
and some of the hardware/software capabilities installed at each,
Stadium Perimeter: The southern perimeter of the stadium'’s
property borders on the San Diego River channel. To monitor possible
intrusions and activity in the area, Trivedi deployed two cameras
on a 60-foot tall cherry-picker supplied by UCSD Physical Plant Services,
and positioned just south of the riverbed on an SDSU Foundation site
that was a staging ground for SDSU's major ShadowBowl exercise (http://www.shadowbowl.org).
"High-speed Internet connectivity and power was also provided
by SDSU," said Trivedi. "We are indebted to Dave Warner's
group that was staging the ShadowBowl activity, and to SDSU professor
Eric Frost, one of our key partners in Cal-(IT)˛, who helped
establish the technical and logistical interaction."
deploys cameras across from QUALCOMM Stadium
Triton sign on top of crane across from QUALCOMM Stadium
installed atop the crane included a high-resolution thermal imaging
system able to detect humans (and animals, including several jackrabbits!)
in unlit and visually cluttered scenes near the riverbed. CVRR researcher
Ed Childers designed a specialized housing, optical filter and mounting
assembly for the thermal imaging system. A team of technicians at
the Scripps Institution of Oceanography's Engineering Shop developed
the housing assembly within a day. Researchers also deployed a high-resolution,
pan-tilt-zoom camera on top of the boom truck. It provided a live
video feed at 30 frames per second, covering from the south side
of the riverbed all the way to the stadium gates. The system provided
temporally and spatially registered multimodal (thermal 8- 14 um
and visible .4-.7 um) video from the site. "This is a very
unique feature to allow development of surveillance systems which
can operate day and night,"said Trivedi. "And law-enforcement
officials were able to operate the camera over the secure Internet
from their multiple locations."
Road: Tapping into a video camera already monitoring traffic
activity on the road in front of QUALCOMM Stadium, CVRR researcher
Ofer Achler ran the video feed through the latest version of his
experimental vision-based traffic analysis and notification software.
The program uses algorithms to count cars and vehicle speeds, and
worked in both day and night conditions. The live traffic flow information
was also fed to a separate traffic monitoring system developed by
Cal-(IT)˛ researcher Ganz Chockalingam (see below).
District: San Diego Police and city technicians mounted a
CVRR omni-directional camera with outdoor housing, axis server, power
reset module, and wireless antenna on a traffic signal pole at the
intersection of Sixth Avenue and G Street in downtown San Diego’s
historic Gaslamp District. The real-time, 360-degree video was uploaded
to the over a high-speed 802.11b-based wireless network provided by
Sky River, a provider of broadband wireless communications for business.
Noted Trivedi: "Sky River helped generously with access, network
provisioning and installation support - and under very tight time
detection software running on Seaport Village video
were able to view the live panoramic video, as well as corrected
(unwrapped) video views, and were able to customize their views
of the video stream. That capability was available not just from
both command centers, but also using a handheld Digital Tele-viewer
(developed in Trivedi's lab) that was hooked up wirelessly to the
Internet; it allowed officers to select pan, tilt, and zoom parameters
for custom viewing.
installed a "dynamic crowd status analyzer and recorder,"
using input from the Gaslamp omnicam. The brand-new software module
used pattern recognition to estimate the size of the crowd gathered
in the secured "party" area of the district, as well as
the flow of pedestrians on both Sixth and G streets. The numbers
were updated every two minutes in a database for query and analysis.
The program was designed at CVRR to work in the daytime as well
as with night-time illumination. "A number of officials indicated
long-term interest in this program as a potentially very important
tool for emergency response teams," noted Trivedi.
Bowl XXXVII sign from Seaport Village Command Center
Village: To cover perimeter security around the command
center building set up on the waterfront in Seaport Village (overseeing
security operations in the village and in the Gaslamp District),
the CVRR team installed a high-resolution rectilinear camera with
full pan-tilt-zoom controls operated by authorities over the Internet.
The video feed, at almost 30 frames per second, was also fed through
a "virtual fencing and vehicle/person detector." Using
the high-resolution feed, the software module detected and counted
vehicles and pedestrians around the Seaport Village command center.
All the information was also recorded and logged for later analysis.
In Cal-(IT)˛ division director Ramesh Rao's lab, researcher
Ganz Chockalingam had developed a traffic alert system based on
speech recognition. It gave mobile users access by phone to Caltrans
information on traffic flows around San Diego County—updates
that were previously only available over the Web. Meanwhile, the
San Diego police were looking for an easy way to get up-to-the-minute
traffic information for major thoroughfares near QUALCOMM Stadium
(Friars Road in particular), and popular "destinations"”
for visitors. They also wanted a way to notify drivers of special
events such as accidents or road closures - features that Chockalingam,
with technical support from Gopal Sankarnaryanan and Voxeo Corporation,
incorporated in the system.
Even on the
Web, Caltrans only provides traffic flow information for major freeways
in San Diego. But Cal-(IT)˛ was already installing a camera
to monitor traffic on Friars Road in front of the stadium. Using
vision-based software developed in Mohan Trivedi's lab (see above),
a computer estimated traffic flow on the road, and fed that information
to the voice-assisted traffic service. Police could then call in
from anywhere, ask for "Friars Road East," and get an
update such as 'Traffic on Friars Road East is flowing at 20 miles
per hour.' Notes Chockalingam: "The San Diego Police Department
used the system to help manage and route traffic around QUALCOMM
"Powered by Cal-(IT)˛," went online a few days before
the Super Bowl, and has since been made available to all San Diegans,
free of charge. "We quickly realized that this service can
be a great tool for commuters, so we created a second toll-free
line for the public at large," said Cal-(IT)˛'s Ramesh
Rao. "This is a very tangible way in which innovation happens:
we were working on the technology, and the security and safety requirements
for of the Super Bowl made it important that we push this technology
into the real world sooner than we had originally planned. Now,
for the foreseeable future, we will continue to provide this service
at no cost to commuters."
number is 1-866-500-0977, and the service operates around the clock.
Network for Homeland Security
Cal-(IT)˛ activities at the Super Bowl were carried out in
connection with the San Diego Regional Network for Homeland Security,
and coordinated through the efforts of Bob Welty, SDSU Director
of Homeland Security projects. Led by UCSD and SDSU, the recently-formed
network is charged with forging a coordinated regional response
to the scientific, technological, and operational challenges associated
with homeland security, by reaching out across the community’s
diverse institutional sectors to involve participation of interested
public and private organizations. The Regional Network is co-chaired
by Dolores Wozniak, Dean of SDSU's College of Health & Human
Services; and Mark Thiemens, Dean of UCSD's Division of Physical
Cal-(IT)˛ researchers see the institute's participation in
Super Bowl security efforts as a starting-point for future collaborations.
"To give you one example," said Trivedi, "law-enforcement
officials from Jacksonville saw some of our systems working in the
Command Center and are talking to us about this capability for when
they host the Super Bowl. I am extremely proud of our team. Without
everyone's professionalism, dedication, technical competence, creativity,
teamwork, and 'can-do' attitude, we could never have deployed so
much technology so quickly."
The effort also
allowed the institute to engage new partners, including companies
such as Sky River and WFI, a San Diego-based company specializing
in network architecture and dimensioning of mobile and high speed
wireless data systems. Concluded Cal-(IT)˛'s Rao: "This
event gave us a chance to reach out not just at the agency level,
but to first responders, who were able to give us feedback on the
spot about how the technology could best serve their needs. We expect
to continue getting valuable feedback from everyone who participated
in this effort, and that feedback will allow us to develop future
enhancements of these technologies - and get them deployed faster
thanks to these new relationships that are being forged."
Vision and Robotics Research Lab http://cvrr.ucsd.edu
Diego State University http://www.sdsu.edu/
Jacobs School http://www.jacobsschool.ucsd.edu/
Diego Supercomputer Center http://www.sdsc.edu/
Diego Police Department http://www.sannet.gov/police/
of San Diego Emergency Medical Service
Threat Reduction Agency http://www.dtra.mil
Support Working Group http://www.tswg.gov
and Naval Warfare Systems Command http://www.spawar.navy.mil
for Commercialization of Advanced Technologies http://www.ccatsandiego.org
Industry-University Cooperative Research Program
of Naval Research http://www.onr.navy.mil
ITR Program http://www.itr.nsf.gov/
J. von Liebig Center http://www.vonliebig.ucsd.edu/
Physical Plant Services http://pps.ucsd.edu/
Institution of Oceanography http://sio.ucsd.edu