Electricity prices strongly depend on seasonality on different time scales, therefore any forecasting of electricity prices has to account for it. Neural networks proved successful in forecasting, but complicated architectures like LSTM are used to integrate the seasonal behavior. This paper shows that simple neural networks architectures like DNNs with an embedding layer for seasonality information deliver not only a competitive but superior forecast. The embedding based processing of calendar information additionally opens up new applications for neural networks in electricity trading like the generation of price forward curves. Besides the theoretical foundation, this paper also provides an empirical multi-year study on the German electricity market for both applications and derives economical insights from the embedding layer. The study shows that in short-term price-forecasting the mean absolute error of the proposed neural networks with embedding layer is only about half of the mean absolute forecast error of state-of-the-art LSTM approaches. The predominance of the proposed approach is also supported by a statistical analysis using Friedman and Holms tests.