The concept of Koszul differential graded algebra (Koszul DG algebra) is introduced. Koszul DG algebras exist extensively, and have nice properties similar to the classic Koszul algebras. A DG version of the Koszul duality is proved. When the Koszul DG algebra $A$ is AS-regular, the Ext-algebra $E$ of $A$ is Frobenius. In this case, similar to the classical BGG correspondence, there is an equivalence between the stable category of finitely generated left $E$-modules, and the quotient triangulated category of the full triangulated subcategory of the derived category of right DG $A$-modules consisting of all compact DG modules modulo the full triangulated subcategory consisting of all the right DG modules with finite dimensional cohomology. The classical BGG correspondence can derived from the DG version.