In this study, we describe an integrated solution for video surveillance in a fortified environment. The focus of this study is on identification of real time objects on different environments. The system is composed of robust object detection module, which normally detects the presence of abandoned objects, concealed objects hidden inside the human clothing, objects in dark environment and performs image segmentation with the intention of facilitating human operator’s task of retrieving the cause of a buzzer. The abandoned objects are detected by image segmentation based on temporal rank order filtering. Image fusion technique which fuses a color visual image and a corresponding IR image for concealed objects in guarded environment and in some cases like dark environment heat signature can be used for detecting real time objects etc. In the clips of interest, the key frame is the one depicting a person leaving a dangerous object and is determined on the basis of a feature indicating the movement around the dangerous region.