In an earlier paper, the first two authors defined orientations on hypergraphs. Using this definition we provide an explicit bijection between acyclic orientations in hypergraphs and faces of hypergraphic polytopes. This allows us to obtain a geometric interpretation of the coefficients of the antipode map in a Hopf algebra of hypergraphs. This interpretation differs from similar ones for a different Hopf structure on hypergraphs provided recently by Aguiar and Ardila. Furthermore, making use of the tools and definitions developed here regarding orientations of hypergraphs we provide a characterization of hypergraphs giving rise to simple hypergraphic polytopes in terms of acyclic orientations of the hypergraph. In particular, we recover this fact for the nestohedra and the hyper-permutahedra, and prove it for generalized Pitman-Stanley polytopes as defined here.