Large "instruction-tuned" language models (finetuned to respond to instructions) have demonstrated a remarkable ability to generalize zero-shot to new tasks. Nevertheless, they depend heavily on human-written instruction data that is limited in quant
ity, diversity, and creativity, therefore hindering the generality of the tuned model. We introduce Self-Instruct, a framework for improving the instruction-following capabilities of pretrained language models by bootstrapping off its own generations. Our pipeline generates instruction, input, and output samples from a language model, then prunes them before using them to finetune the original model. Applying our method to vanilla GPT3, we demonstrate a 33% absolute improvement over the original model on Super-NaturalInstructions, on par with the performance of InstructGPT_001, which is trained with private user data and human annotations. For further evaluation, we curate a set of expert-written instructions for novel tasks, and show through human evaluation that tuning GPT3 with Self-Instruct outperforms using existing public instruction datasets by a large margin, leaving only a 5% absolute gap behind InstructGPT_001. Self-Instruct provides an almost annotation-free method for aligning pre-trained language models with instructions, and we release our large synthetic dataset to facilitate future studies on instruction tuning.
The objective of this research is to conduct a systematic literature review, analyzing the influence of implementing the ChatGPT tool in the field of education. The data for this study was gathered through a systematic review of studies published sin
ce the launch of ChatGPT in November 2022. Three prominent educational databases (Web of Science, Taylor& Francis Online, Eric) were utilized for this purpose. The study incorporated a sample of 18 relevant studies, and a descriptive and quantitative methodology was employed to present the most noteworthy findings. The outcomes indicate that the incorporation of ChatGPT in the educational setting positively impacts the teaching and learning processes. Nevertheless, the results also shed light on topics such as factors that determine students' attitudes toward the application, positive and negative effects, and how to ensure academic integrity when applying AI in education. Despite ChatGPT's potential to enhance the educational experience, its successful integration hinges on educators being well-versed in its functionalities. These insights lay a robust foundation for future research endeavors and informed decision-making concerning the incorporation of ChatGPT in educational contexts.
