Fouling is a major obstacle and challenge in membrane-based separation processes. Caused by the sophisticated interactions between foulant and membrane surface, fouling strongly depends on membrane surface chemistry and morphology. Current studies in the field have been largely focused on polymer membranes. Herein, we report a molecular simulation study for fouling on alumina and graphene membrane surfaces during water treatment. For two foulants (sucralose and bisphenol A), the fouling on alumina surfaces is reduced with increasing surface roughness; however, the fouling on graphene surfaces is enhanced by roughness. It is unravelled that the foulant-surface interaction becomes weaker in the ridge region of a rough alumina surface, thus allowing foulant to leave the surface and reducing fouling. Such behavior is not observed on a rough graphene surface because of the strong foulant-graphene interaction. Moreover, with increasing roughness, the hydrogen bonds formed between water and alumina surfaces are found to increase in number as well as stability. By scaling the atomic charges of alumina, fouling behavior on alumina surfaces is shifted to the one on graphene surfaces. This simulation study reveals that surface chemistry and roughness play a crucial role in membrane fouling, and the microscopic insights are useful for the design of new membranes towards high-performance water treatment.