The detection of fire smoke in highway tunnels and its early warning are of great importance. Traditional fire smoke detectors have many disadvantages, such as slow response time and limited functionality. Accordingly, a fire smoke detection method based on image features associated with smoke is proposed, which can analyze the characteristics of color and the texture of fire smoke from video images. First, the area in an image associated with motion is extracted using the improved Vibe algorithm. Second, the suspected area corresponding to smoke is identified in YUV color space and segmented using a color filtering method. Finally, the color moments, uniform local binary mode, and gray level cooccurrence matrix are extracted from the image having the suspected area corresponding to smoke and are subsequently used to form the input vector of a machine learning classifier for recognizing fire smoke in highway tunnels. To meet the complex requirements of tunnel environments, the fire smoke recognition effects of three machine learning classifiers, i. e., back propagation (BP) neural network, support vector machine, and random forest algorithms, are compared and analyzed, and the optimal algorithm among them is selected as the smoke recognition classifier for highway tunnels. The classifier is tested using the simulated test videos of fire smoke in highway tunnels as well as the video of a real fire in a highway tunnel. The results show that the detection system based on BP neural network algorithm has the best recognition performance and the selected smoke features have high recognition accuracy for identifying fire smoke in complex highway tunnel environments.