Toward quantum machine learning deployed on imperfect near-term intermediate-scale quantum (NISQ) processors, the entire physical implementation of should include as less as possible hand-designed modules with only a few ad-hoc parameters to be determined. This work presents such a hardware-friendly end-to-end quantum machine learning scheme that can be implemented with imperfect near-term intermediate-scale quantum (NISQ) processors. The proposal transforms the machine learning task to the optimization of controlled quantum dynamics, in which the learning model is parameterized by experimentally tunable control variables. Our design also enables automated feature selection by encoding the raw input to quantum states through agent control variables. Comparing with the gate-based parameterized quantum circuits, the proposed end-to-end quantum learning model is easy to implement as there are only few ad-hoc parameters to be determined. Numerical simulations on the benchmarking MNIST dataset demonstrate that the model can achieve high performance using only 3-5 qubits without downsizing the dataset, which shows great potential for accomplishing large-scale real-world learning tasks on NISQ processors.arning models. The scheme is promising for efficiently performing large-scale real-world learning tasks using intermediate-scale quantum processors.