Multi-task learning with transformer encoders (MTL) has emerged as a powerful technique to improve performance on closely-related tasks for both accuracy and efficiency while a question still remains whether or not it would perform as well on tasks t
hat are distinct in nature. We first present MTL results on five NLP tasks, POS, NER, DEP, CON, and SRL, and depict its deficiency over single-task learning. We then conduct an extensive pruning analysis to show that a certain set of attention heads get claimed by most tasks during MTL, who interfere with one another to fine-tune those heads for their own objectives. Based on this finding, we propose the Stem Cell Hypothesis to reveal the existence of attention heads naturally talented for many tasks that cannot be jointly trained to create adequate embeddings for all of those tasks. Finally, we design novel parameter-free probes to justify our hypothesis and demonstrate how attention heads are transformed across the five tasks during MTL through label analysis.
Stem cells have unique capability to differentiate into many cell types that can
normally replace the loss in some cells of the body due to tissue injury. Umbilical cord blood (UCB) and
umbilical cord (UC) are the two main sources for hematopoietic
stem cells (HSCs) and mesenchymal stem
cells (MSCs), respectively, which constitutes the basis for stem cell banks that have been established
worldwide and very recently in Syria. Research in our region has mainly focused on cell storage and
freezing protocols, and only few studies were conducted to prove the ability of the stored cells to
differentiate into their destined lineages. This study aimed to test the potential of cryopreserved MSCs
isolated from an umbilical cord taken from new delivery at Maternity University Hospital in Damascus, to
differentiate into various types of cells in response to growth and induction factors specific to cell lineages.
This research aims at establishing
and testing protocols for isolation, in vitro proliferation, phenotyping, and
differentiation of MSCs embedded in umbilical cord tissues of Syrian
newborns.
MSCs were isolated from two caesarian births depending
on their
adherence characteristic on plastic surfaces, and cells were cultured in MSC
medium for their growth and proliferation. Cell phenotyping was performed by
flow cytometry using fluorescent monoclonal antibodies specific to MSCs’
surface markers. Cultured cells were passed several times and a portion of
these cells was cryopreserved in liquid nitrogen, and cell viability was assessed.
Differentiation of these MSCs into adipocytes was conducted using culture
medium Indomethacin and hydrocortisone.