Recently, the spatial modulation (SM) technique has been proposed for visible light communication (VLC). This paper investigates the average symbol error rate (SER) for the VLC using adaptive spatial modulation (ASM). In the system, the analysis of the average SER is divided into two aspects: the error probability of the spatial domain and the error probability of the signal domain when the spatial domain is correctly estimated. Based on the two aspects, the theoretical expression of the average SER is derived. To further improve the system performance, an optimization problem is proposed to optimize the modulation orders on the LEDs. The ASM based and the candidate reduction (CR)-ASM based optimization algorithms are proposed to solve the problem, respectively. Numerical results show that the derived theoretical values of the average SER are quite accurate to evaluate the system performance. Moreover, compared with the existing schemes, the proposed two algorithms are better choices for VLC.