Colonies of bacterial cells endowed with a pili-based self-propulsion machinery represent an ideal model system for studying how active adhesion forces affect structure and dynamics of many-particle systems. As a novel computational tool, we describe here a highly parallel molecular dynamics simulation package for modeling of textit{Neisseria gonorrhoeae} colonies. Simulations are employed to investigate growth of bacterial colonies and the dependence of the colony structure on cell-cell interactions. In agreement with experimental data, active pilus retraction is found to enhance local ordering. For mixed colonies consisting of different types of cell types, the simulations show a segregation of cell types depending on the pili-mediated interactions, as seen in experiments. Using a simulated experimental setup, we study the power-spectral density of colony-shape fluctuations and the associated fluctuation-response relation. The simulations predict a strong violation of the equilibrium fluctuation-response relation across the measurable frequency range. Lastly, we illustrate the essential role of active force generation for colony dynamics by showing that pilus-mediated activity drives the spreading of colonies on surfaces and the invasion of narrow channels.