Architecture Overview

The NeST is a flexible and extendible architecture for coordinated extraction, dissemination, and processing of data from arbitrary sources of sensory data. These data sources include but are not limited to rectilinear, thermal, and omni-directional cameras, audio transducers, and seismic sensors. Any sensor device that can establish a TCP/IP connection can join the NeST (via wired or wireless link) and share data.

The concept of a sensor is generalized in the NeST to include arbitrary software applications operating on data. A “virtual sensor” may be as simple as a compression algorithm operating directly on the sensor output, or as complex as an inference module that requires access to multiple sources of real-time or historical sensor data and databases to achieve its goal.

The complete architecture consists of a centralized server to which multiple clients may connect and share/collect information. A multi-platform client connectivity library allows arbitrary clients to become active members of the NeST. Example client hardware includes mobile devices (ie. PDA based), remote sensor interface devices, as well as software modules such as: sensor fusion and inference modules, visualization and archival applications. Modularity and extendibility is acheived in part by multiple messaging layers that allows clients to be selective over the types of data they receive. Additional layers are easily defined as needed by new applications.

The server core is written in Java, and the client library has been ported to C++ and Java. Current functional clients include HP Ipaq PDAs, HP Tablet PCs, embedded TINIOS hardware, and laptop/desktop environments.


A core client of the NeST client is a flexible visualization and interface application that pulls data from the NeST and renders the data registered into a 3D environment. He has modeled and textured both the exterior and selected interior spaces of the SERF complex (where the CVRR lab is located). The images below demonstrate tracking data filtered through the NeST to the visualization application where 3D positions are depicted using rendered boxes. Registered video from the CVRR axis camera network can be streamed onto the geometry surfaces.


Privacy Filtering

An important component of The NeST is a privacy and security filtering shell surrounding the server core. Data entering the NeST is processed and checked against application definable privacy filters. The video below illustrates tracking data extracted from a video stream. By default, the identities of individuals in the embedded video stream are blocked. A privacy filter has been enabled that only allows the identity of running individuals to pass through the system. Similar filters can be constructed for other behaviors.

No Restrictions Running Filter Track Geometry Filter