Participants in the Super Bowl project included (left to right, bottom row) Tarak Gandhi, Ofer Achler, Mohan Trivedi, Ramesh Rao, and Remy Chang; (top row) Kim Harlow, Andrew Cosand, Junwen Wu, Leah Vining, Kohsia Huang, Giovanni Gualdi, Shinko Cheng, Kate Tull, and Dashan Gao

2.4.03 - Behind the scenes and away from the glare of publicity, more than a dozen faculty and students affiliated with Cal-(IT)² mounted a coordinated effort to put cutting-edge public-safety technologies in place for Super Bowl XXXVII. And they did so in a relatively concentrated period of time: barely three weeks from the first planning meeting to Game Day, Sunday, January 26. "It was a Herculean effort on the part of all our people and all their counterparts at local, state and national agencies," said Ramesh Rao, Cal-(IT)²'s division director at UCSD, who oversaw the institute’s involvement. "Cal-(IT)² partners stepped up to help in crucial ways at critical times to make this effort a success, and the entire exercise illustrates why San Diego has earned a reputation as a place where public and private sectors work together on solutions to important problems."

[Watch video 6:00]

The project included newly-developed software to glean security data automatically from cameras wirelessly hooked up to the Internet. The cameras monitored the southern perimeter of QUALCOMM Stadium, a high-traffic intersection in downtown San Diego, and a waterfront command center staffed by San Diego Police and other security personnel. Separately, Cal-(IT)² researchers developed a service that allowed police to call a toll-free number to get voice-activated traffic updates for major roads and highways. "While our systems played only a small part in the event’s overall security," added Rao, an electrical and computer engineering professor at UCSD’s Jacobs School of Engineering, "we were able to take experimental hardware and software code and turn them into working systems almost overnight - and they worked flawlessly!"

San Diego spent roughly $2 million on security for the game (with half of that coming from the National Football League), including the cost of equipping QUALCOMM Stadium itself with more than 50 video surveillance cameras. But the City invited public agencies, private companies and research organizations to contribute - at their own cost - backup systems outside the stadium that could also be monitored by police and security personnel. "“It is remarkable how quickly and successfully San Diego technology institutions pulled together on this," said Cal-(IT)²'s Rao. "And it would not have happened without the leadership of Assistant Chief of Police Bill Maheu, who was in charge of Super Bowl security, and of Bob Davis, a police advisor who coordinated the participation of technology providers."

Super Bowl XXXVII sign from Seaport Village Command Center

Rao led the institute's effort, together with Cal-(IT)²'s Intelligent Transportation and Telematics layer leader at UCSD, Mohan Trivedi. A Jacobs School professor, Trivedi and a team of 13 students and researchers from UCSD'’s Computer Vision and Robotics Research (CVRR) lab installed a variety of cameras, software and other equipment to capture and analyze video streams in real time over the Internet – turning the cameras into 'smartcams.' "On a temporary basis, we deployed roughly $150,000 worth of cameras, video server, computer and large disk storage equipment that we only have because of the support of agencies that fund our research on computer vision and homeland security," said Trivedi. "And even though most of the systems were up for just the weekend, we could only be successful because of the nearly four years of research - and funding - that allowed us to develop these technologies."

Among the agencies sponsoring computer vision-related work in Trivedi's lab: the U.S. Defense Threat Reduction Agency, as well as the Technical Support Working Group, an inter-agency forum that coordinates R&D for combating terrorism; Space and Naval Warfare Systems Command (SPAWAR); Center for Commercialization of Advanced Technologies; University of California-based Industry-University Cooperative Research Program; Caltrans; Office of Naval Research (ONR); National Science Foundation’s Information Technology Research program; and the William J. von Liebig Center for Entrepreneurism and Technology Advancement.

Trivedi's team of researchers included Ofer Achler [a Cal-(IT)² undergraduate fellow in 2001], Jun Wen Wu [Cal-(IT)² graduate fellow in 2001-02], and Dashan Gao [who is a Cal-(IT)² graduate fellow in the current 2002-03 academic year].

The CVRR researchers deployed cameras and other equipment in differing configurations at multiple locations, with all feeds viewable - and, in some cases, customizable - over the Internet, from two command centers (one at San Diego Downtown, the other at San Diego's Seaport Village.) The research team also developed a series of software modules tailored to the needs of law-enforcement authorities and security personnel. Most were based on previous computer algorithms developed in the CVRR lab but required tweaking to fit the occasion. "Our algorithm development team was working almost 20-hour shifts all the way up to kickoff time," said Trivedi. "They were able to develop various applications for robust event recognition and event database generation."

The four locations, and some of the hardware/software capabilities installed at each, included:

Crane deploys cameras across from QUALCOMM Stadium UCSD Triton sign on top of crane across from QUALCOMM Stadium

1. Riverbed/QUALCOMM Stadium Perimeter: The southern perimeter of the stadium'’s property borders on the San Diego River channel. To monitor possible intrusions and activity in the area, Trivedi deployed two cameras on a 60-foot tall cherry-picker supplied by UCSD Physical Plant Services, and positioned just south of the riverbed on an SDSU Foundation site that was a staging ground for SDSU's major ShadowBowl exercise (http://www.shadowbowl.org). "High-speed Internet connectivity and power was also provided by SDSU," said Trivedi. "We are indebted to Dave Warner's group that was staging the ShadowBowl activity, and to SDSU professor Eric Frost, one of our key partners in Cal-(IT)², who helped establish the technical and logistical interaction."

The equipment installed atop the crane included a high-resolution thermal imaging system able to detect humans (and animals, including several jackrabbits!) in unlit and visually cluttered scenes near the riverbed. CVRR researcher Ed Childers designed a specialized housing, optical filter and mounting assembly for the thermal imaging system. A team of technicians at the Scripps Institution of Oceanography's Engineering Shop developed the housing assembly within a day. Researchers also deployed a high-resolution, pan-tilt-zoom camera on top of the boom truck. It provided a live video feed at 30 frames per second, covering from the south side of the riverbed all the way to the stadium gates. The system provided temporally and spatially registered multimodal (thermal 8- 14 um and visible .4-.7 um) video from the site. "This is a very unique feature to allow development of surveillance systems which can operate day and night,"said Trivedi. "And law-enforcement officials were able to operate the camera over the secure Internet from their multiple locations."

2. Friars Road: Tapping into a video camera already monitoring traffic activity on the road in front of QUALCOMM Stadium, CVRR researcher Ofer Achler ran the video feed through the latest version of his experimental vision-based traffic analysis and notification software. The program uses algorithms to count cars and vehicle speeds, and worked in both day and night conditions. The live traffic flow information was also fed to a separate traffic monitoring system developed by Cal-(IT)² researcher Ganz Chockalingam (see below).

Vehicle detection software running on Seaport Village video feed

3. Gaslamp District: San Diego Police and city technicians mounted a CVRR omni-directional camera with outdoor housing, axis server, power reset module, and wireless antenna on a traffic signal pole at the intersection of Sixth Avenue and G Street in downtown San Diego’s historic Gaslamp District. The real-time, 360-degree video was uploaded to the over a high-speed 802.11b-based wireless network provided by Sky River, a provider of broadband wireless communications for business. Noted Trivedi: "Sky River helped generously with access, network provisioning and installation support - and under very tight time constraints."

Authorities were able to view the live panoramic video, as well as corrected (unwrapped) video views, and were able to customize their views of the video stream. That capability was available not just from both command centers, but also using a handheld Digital Tele-viewer (developed in Trivedi's lab) that was hooked up wirelessly to the Internet; it allowed officers to select pan, tilt, and zoom parameters for custom viewing.

Trivedi also installed a "dynamic crowd status analyzer and recorder," using input from the Gaslamp omnicam. The brand-new software module used pattern recognition to estimate the size of the crowd gathered in the secured "party" area of the district, as well as the flow of pedestrians on both Sixth and G streets. The numbers were updated every two minutes in a database for query and analysis. The program was designed at CVRR to work in the daytime as well as with night-time illumination. "A number of officials indicated long-term interest in this program as a potentially very important tool for emergency response teams," noted Trivedi.

Super Bowl XXXVII sign from Seaport Village Command Center

4. Seaport Village: To cover perimeter security around the command center building set up on the waterfront in Seaport Village (overseeing security operations in the village and in the Gaslamp District), the CVRR team installed a high-resolution rectilinear camera with full pan-tilt-zoom controls operated by authorities over the Internet. The video feed, at almost 30 frames per second, was also fed through a "virtual fencing and vehicle/person detector." Using the high-resolution feed, the software module detected and counted vehicles and pedestrians around the Seaport Village command center. All the information was also recorded and logged for later analysis.

'Powered by Cal-(IT)²'
In Cal-(IT)² division director Ramesh Rao's lab, researcher Ganz Chockalingam had developed a traffic alert system based on speech recognition. It gave mobile users access by phone to Caltrans information on traffic flows around San Diego County—updates that were previously only available over the Web. Meanwhile, the San Diego police were looking for an easy way to get up-to-the-minute traffic information for major thoroughfares near QUALCOMM Stadium (Friars Road in particular), and popular "destinations"” for visitors. They also wanted a way to notify drivers of special events such as accidents or road closures - features that Chockalingam, with technical support from Gopal Sankarnaryanan and Voxeo Corporation, incorporated in the system.

Even on the Web, Caltrans only provides traffic flow information for major freeways in San Diego. But Cal-(IT)² was already installing a camera to monitor traffic on Friars Road in front of the stadium. Using vision-based software developed in Mohan Trivedi's lab (see above), a computer estimated traffic flow on the road, and fed that information to the voice-assisted traffic service. Police could then call in from anywhere, ask for "Friars Road East," and get an update such as 'Traffic on Friars Road East is flowing at 20 miles per hour.' Notes Chockalingam: "The San Diego Police Department used the system to help manage and route traffic around QUALCOMM Stadium."

The service, "Powered by Cal-(IT)²," went online a few days before the Super Bowl, and has since been made available to all San Diegans, free of charge. "We quickly realized that this service can be a great tool for commuters, so we created a second toll-free line for the public at large," said Cal-(IT)²'s Ramesh Rao. "This is a very tangible way in which innovation happens: we were working on the technology, and the security and safety requirements for of the Super Bowl made it important that we push this technology into the real world sooner than we had originally planned. Now, for the foreseeable future, we will continue to provide this service at no cost to commuters."

The toll-free number is 1-866-500-0977, and the service operates around the clock.

Regional Network for Homeland Security
Cal-(IT)² activities at the Super Bowl were carried out in connection with the San Diego Regional Network for Homeland Security, and coordinated through the efforts of Bob Welty, SDSU Director of Homeland Security projects. Led by UCSD and SDSU, the recently-formed network is charged with forging a coordinated regional response to the scientific, technological, and operational challenges associated with homeland security, by reaching out across the community’s diverse institutional sectors to involve participation of interested public and private organizations. The Regional Network is co-chaired by Dolores Wozniak, Dean of SDSU's College of Health & Human Services; and Mark Thiemens, Dean of UCSD's Division of Physical Sciences.

Going forward, Cal-(IT)² researchers see the institute's participation in Super Bowl security efforts as a starting-point for future collaborations. "To give you one example," said Trivedi, "law-enforcement officials from Jacksonville saw some of our systems working in the Command Center and are talking to us about this capability for when they host the Super Bowl. I am extremely proud of our team. Without everyone's professionalism, dedication, technical competence, creativity, teamwork, and 'can-do' attitude, we could never have deployed so much technology so quickly."

The effort also allowed the institute to engage new partners, including companies such as Sky River and WFI, a San Diego-based company specializing in network architecture and dimensioning of mobile and high speed wireless data systems. Concluded Cal-(IT)²'s Rao: "This event gave us a chance to reach out not just at the agency level, but to first responders, who were able to give us feedback on the spot about how the technology could best serve their needs. We expect to continue getting valuable feedback from everyone who participated in this effort, and that feedback will allow us to develop future enhancements of these technologies - and get them deployed faster thanks to these new relationships that are being forged."

Related Links

Computer Vision and Robotics Research Lab http://cvrr.ucsd.edu

San Diego State University http://www.sdsu.edu/

UCSD Jacobs School http://www.jacobsschool.ucsd.edu/

San Diego Supercomputer Center http://www.sdsc.edu/

San Diego Police Department http://www.sannet.gov/police/

ShadowBowl 2003 http://www.shadowbowl.org

County of San Diego Emergency Medical Service
http://www.co.san-diego.ca.us/cnty/cntydepts/health/services/emergmed.html

Sky River http://www.skyrivercomm.com/

WFI http://www.wfinet.com/

Defense Threat Reduction Agency http://www.dtra.mil

Technical Support Working Group http://www.tswg.gov

Space and Naval Warfare Systems Command http://www.spawar.navy.mil

Center for Commercialization of Advanced Technologies http://www.ccatsandiego.org

UC Industry-University Cooperative Research Program http://uc-industry.berkeley.edu/

Caltrans http://www.dot.ca.gov

Office of Naval Research http://www.onr.navy.mil

NSF ITR Program http://www.itr.nsf.gov/

William J. von Liebig Center http://www.vonliebig.ucsd.edu/

Qualcomm Stadium http://www.sannet.gov/qualcomm/

UCSD Physical Plant Services http://pps.ucsd.edu/

SDSU Foundation http://www.foundation.sdsu.edu

Scripps Institution of Oceanography http://sio.ucsd.edu