Hybrid quantum-classical (HQC) algorithms have been successfully applied to quantum approximate optimization algorithms, variational quantum compiling and quantum machine learning models. The performances of HQC algorithms largely depend on the architecture of parameterized quantum circuits (PQCs). Quantum architecture search (QAS) aims to automate the design of PQCs with intelligence algorithms, e.g., genetic algorithms and reinforcement learning. Recently a differentiable quantum architecture search (DQAS) algorithm is proposed to speed up the circuit design. However, these QAS algorithms do not use prior experiences and search the quantum architecture from scratch for each new task, which is inefficient and time consuming. In this paper, we propose a meta quantum architecture search (MetaQAS) algorithm, which learns good initialization heuristics of the architecture (i.e., meta-architecture), along with the meta-parameters of quantum gates from a number of training tasks such that they can adapt to new tasks with a small number of gradient updates, which leads to fast learning on new tasks. Simulation results of variational quantum compiling on three- and four-qubit circuits show that the architectures of MetaQAS converge after 30 and 80 iterations respectively, which are only 1/3 and 2/3 of those needed in the DQAS algorithm. Besides, MetaQAS can achieve lower losses than DQAS after fine-tuning of gate parameters.