We present an approach to market-consistent multi-period valuation of insurance liability cash flows based on a two-stage valuation procedure. First, a portfolio of traded financial instrument aimed at replicating the liability cash flow is fixed. Then the residual cash flow is managed by repeated one-period replication using only cash funds. The latter part takes capital requirements and costs into account, as well as limited liability and risk averseness of capital providers. The cost-of-capital margin is the value of the residual cash flow. We set up a general framework for the cost-of-capital margin and relate it to dynamic risk measurement. Moreover, we present explicit formulas and properties of the cost-of-capital margin under further assumptions on the model for the liability cash flow and on the conditional risk measures and utility functions. Finally, we highlight computational aspects of the cost-of-capital margin, and related quantities, in terms of an example from life insurance.