Originally, the Hubbard model has been derived for describing the behaviour of strongly-correlated electrons in solids. However, since over a decade now, variations of it are also routinely being implemented with ultracold atoms in optical lattices. We review some of the rich literature on this subject, with a focus on more recent non-standard forms of the Hubbard model. After an introduction to standard (fermionic and bosonic) Hubbard models, we discuss briefly common models for mixtures, as well as the so called extended Bose-Hubbard models, that include interactions between neighboring sites, next-neighboring sites, and so on. The main part of the review discusses the importance of additional terms appearing when refining the tight-binding approximation on the original physical Hamiltonian. Even when restricting the models to the lowest Bloch band is justified, the standard approach neglects the density-induced tunneling (which has the same origin as the usual on-site interaction). The importance of these contributions is discussed for both contact and dipolar interactions. For sufficiently strong interactions, also the effects related to higher Bloch bands become important even for deep optical lattices. Different approaches that aim at incorporating these effects, mainly via dressing the basis Wannier functions with interactions, leading to effective, density-dependent Hubbard-type models, are reviewed. We discuss also examples of Hubbard-like models that explicitly involve higher $p$-orbitals, as well as models that couple dynamically spin and orbital degrees of freedom. Finally, we review mean-field nonlinear-Schrodinger models of the Salerno type that share with the non-standard Hubbard models the nonlinear coupling between the adjacent sites. In that part, discrete solitons are the main subject of the consideration. We conclude by listing some future open problems.