Let $f$ be a polynomial in the free algebra over a field $K$, and let $A$ be a $K$-algebra. We denote by $S_A(f)$, $A_A(f)$ and $I_A(f)$, respectively, the `verbal subspace, subalgebra, and ideal, in $A$, generated by the set of all $f$-values in $A$. We begin by studying the following problem: if $S_A(f)$ is finite-dimensional, is it true that $A_A(f)$ and $I_A(f)$ are also finite-dimensional? We then consider the dual to this problem for `marginal subspaces that are finite-codimensional in $A$. If $f$ is multilinear, the marginal subspace, $widehat{S}_A(f)$, of $f$ in $A$ is the set of all elements $z$ in $A$ such that $f$ evaluates to 0 whenever any of the indeterminates in $f$ is evaluated to $z$. We conclude by discussing the relationship between the finite-dimensionality of $S_A(f)$ and the finite-codimensionality of $widehat{S}_A(f)$.