Video surveillance is an active research area in Computer Vision and Machine Learning. It received a lot of attention in the last few decades. Maritime surveillance is the act of effective detection/recognition of all maritime activities that have impact on economy, security or the environment. The maritime environment is a dynamic environment. Factors such as constant moving of waves, sun reflection over the sea surface, rapid change in lightning due to the sun reflection over the water surface, movement of clouds and presence of moving objects such as airplanes or birds, makes the maritime environment very challenging. In this work, we propose a method for detecting a motion generated by a maritime vehicle and then identifying the type of this vehicle using classification methods. A new maritime video database was created and tested. Classifying the type of vehicles have been tested by comparing 13 image features, and two SVM solving algorithms. In motion detection part, multiple smoothing filters were tested in order to minimize the false positive rate generated by the water surface movement, the results have been compared to optical flow, a well known method for motion detection.