Transformer Feed-Forward Layers Are Key-Value Memories

Feed-forward layers constitute two-thirds of a transformer model's parameters, yet their role in the network remains under-explored. We show that feed-forward layers in transformer-based language models operate as key-value memories, where each key correlates with textual patterns in the training examples, and each value induces a distribution over the output vocabulary. Our experiments show that the learned patterns are human-interpretable, and that lower layers tend to capture shallow patterns, while upper layers learn more semantic ones. The values complement the keys' input patterns by inducing output distributions that concentrate probability mass on tokens likely to appear immediately after each pattern, particularly in the upper layers. Finally, we demonstrate that the output of a feed-forward layer is a composition of its memories, which is subsequently refined throughout the model's layers via residual connections to produce the final output distribution.

Studying the engineering and physical properties of seeds for some crops and their effect on the properties of feeders in seeders

A proper matching of agricultural machinery with properties of materials forms the essential requirements which lead to best performance of machine. To achieve that goal, availability of data and precision analysis on engineering and physical properties of these materials is required. This forms important and necessary data base for the design, development and structure of agricultural machinery and equipments, control and analysis of machines and determination of efficiency of machines. This research was carried out to study some engineering and physical properties of seeds of common Syrian crops to be used in the design and development of some agricultural machinery and equipments. Results provided the shapes and dimensions of cells of feeding devices for planting seeds of some crops namely, wheat, maize, lentil, pea, chick-pea, kidney bean, broad bean and cotton. In addition a set of empirical equations were established to be used to predict, with reasonable accuracy, the physical properties of crops seeds based on measurement of any of three dimensions (length, width and thickness).

Effect of lysine addition in the dietary on the growth of Cyprinus carpio fingerlings

This research aimed to study the effect of adding different rates of lysine on the weight and growth of common carp fingerlings. The experiment was carried out in the laboratory for 4 months using eight glass containers (40 liters each) with four treatments (L0 no Lysine addition, L1 (2%), L2 (3%), L3(4%) of the total weight of diet. Eight common carp fingerlings (average of tall 7.5cm ± 0.05 and average of weight 12.25g ± 0.57) were cultured in each container with Oxygen pumps and filters. Results showed by growth rate, absolute growth rate, specific growth rate and food conversion ratio (FCR) that the treatment L1 (2% lysine) was significantly the best comparing to the other treatments, where FCR was (2.2) and the specific growth ratio was (0.28 %)

Feeding a newborn "premature" in the first week of life

The search was conducted (newborn feeding "premature" in the first week of life) in order to assess nutrition in newborn and learn about nutrition during the first week of life and what is the best food in preterm infant valiant hospital generously almonds-Pediatrics-premature Division.

Developing a Controller based on Feed Forward Neural Networks and Direct Control Method to improve the efficiency of Solar Photovoltaic Energy Systems using Matlab/Simulink

This research presents a new methodology for the development of a controller based on Artificial Neural Networks and Direct control method in order to obtain the maximum available energy from Solar Photovoltaic (PV) Energy systems under different atmospheric changes of the solar insolation and ambient temperature. In this context, this research presents a new model for MPPT-ANN in order to track the Maximum Power Point of PV systems in Matlab/Simulink environment. The developed controller is based on Feed Forward Neural Network FFNN trained by Back-propagation algorithm of error to determine the optimal voltage operation of the system PV system at different atmospheric changes. This research also suggests, control algorithm based on the direct control method in order to determine the duty cycle, which used to control directly the operating of DCDC Voltage Converter, depending on a comparison of the difference between the output voltage of PV system and the optimal voltage output of the neural network. The developed controller MPPT-ANN based on a network FFNN, Characterized by fast speed to track of MPP point and achieve high efficiency for the PV system under the atmospheric changes. The simulation results completed in Matlab/Simulink environment, showed the best performance of developed controller MPPT-ANN by achieving a better dynamic performance and high accuracy when tracking the MPP, compared with the use of the another PI-ANN controller based on artificial neural network and the conventional Proportional-Integral Controller, and compared with the use of the conventional MPPTP& O based on Perturb and Observe (P&O) technique under different atmospheric changes.

Effect of black bean and thyme to the broiler diets (low-energy) on some productivity indicators at broiler chickens

This research carried out in the Video Center for Research on Animal Production of the Faculty of Agriculture at Tishreen University with the beginning of the summer of 2010 in order to study the possibility of improving the performance of broiler chickens under the conditions of the coastal area and increase economic returns. Search Results showed that the addition of black bean and thyme powder by 0.5% to a low-energy diet led to increased average live weight of age (9-28) days totaled 864.9 g G3)) compared to the control (without the addition), which amounted to an average live weight has 850.7) g) of the same age FCR feed conversion ratio stood at 2.63 at the group G1, compared with 1.89 when the Control CSearch results also showed that the addition of 0.25% of the black bean and thyme powder led to the live weight increase of 848.6, compared with 670.7 members G1) g) FCR feed conversion ratio stood at 1.96 at the G4 Group added (0.25% of the black bean powder and thyme), compared with 2.63 at the G1 group.

Study of some Productive Properties on some Broiler Strains in Syria

The current investigation was conducted to study some productive properties( body weight, body weight gain, feed intake, feed conversion ratio, carcass composition, and carcass cuts ) on some broiler strains in Syria ( Ross, Hubbard, and Shefer ).

Influence of different salinity concentrations on efficiency of feed utilization of Goldfish (Carassius auratus) fingerlings

The aim of this research is to study Influence of different salinity concentrations on efficiency of feed utilization of Goldfish (Carassius auratus) fingerlings. No mortality occurred during the period of the experiment (21 days) at different salinity concentrations (2, 4, 6, 8 and 10 g/l).

Effect of magnetic water in some productive parameters of Awassi lambs

This experiment is done on (24) male awassi lamb whish are divided into four categories: the first category took normal water and is considered as control group. The other three groups took water processed with different Magnetic Field : 500, 1500, 2000 **Gauss in a series.

The Effect Nutritional of Forage Block (Remnants of Olive Trim) on Some Productivity Indicators At The Lambs

This this research was Carried out at video cattle and sheep Center of the Department of Animal Production in the College of Agriculture in October University with the beginning of the spring of 2011 in order to study the possibility of bringing cubes forage replace part of the feed position in the diets of fattening lambs and the impact on their performance, using / 20 / Twenty male strain of Awas ages ranged between 4-5 months and the average weight of 23.5 kg and distributed to four groups to evaluate (in accordance with the decisions of 1996 NRC) the experimental treatments - An increase of waste residue olive trim diets tested in dry matter ingested more than a day for the bush CONTROL but without significant differences. - The daily increase in weight 150- 151 - 144- 134 g C-G1-G2-G3 respectively / day and feed conversion C-G1-G2 9.66-9.80-10.27 respectively were close while the manufacturing efficiency of the transaction G3-11.11 lower compared with the previous three transactions mentioned above, which fell somewhat - Digestible dry matter plants and organic matter for the treatment of G1 was the highest with significant at the level of (0.05> P) compared with treatment G3 table (5), while the digestion of crude protein for the treatment of G3 coefficient was lower compared with the rest of the transactions on the abstract level (0.05> P.) - Digest raw fiber and raw fat transactions G3-G2 coefficient was higher compared with the Control C on the abstract level (0.05> P).