How Magento 2 generates javascript translations during static content deploy
Lets dive deeper into how Magento 2 generates the translations. Lets start at the very surface - the one thing we can see from the browser.
We know that in localstorage the translations are stored with key mage-translation-storage. So lets search for this key in the core code.
/** @var \Magento\Translation\Block\Js $block */
?>
<?php if ($block->dictionaryEnabled()): ?>
<script>
require.config({
deps: [
'jquery',
'mage/translate',
'jquery/jquery-storageapi'
],
callback: function ($) {
'use strict';
var dependencies = [],
versionObj;
$.initNamespaceStorage('mage-translation-storage');
$.initNamespaceStorage('mage-translation-file-version');
versionObj = $.localStorage.get('mage-translation-file-version');
<?php $version = $block->getTranslationFileVersion(); ?>
if (versionObj.version !== '<?= /* @escapeNotVerified */ $block->escapeJsQuote($version) ?>') {
dependencies.push(
'text!<?= /* @noEscape */ Magento\Translation\Model\Js\Config::DICTIONARY_FILE_NAME ?>'
);
}
require.config({
deps: dependencies,
callback: function (string) {
if (typeof string === 'string') {
$.mage.translate.add(JSON.parse(string));
$.localStorage.set('mage-translation-storage', string);
$.localStorage.set(
'mage-translation-file-version',
{
version: '<?= /* @escapeNotVerified */ $block->escapeJsQuote($version) ?>'
}
);
} else {
$.mage.translate.add($.localStorage.get('mage-translation-storage'));
}
}
});
}
});
</script>
<?php endif; ?>First the code checks if the object already exists and if the version is newer than what we currently have. If we have a newer version we load the dictionary file which is later passed to translations components which translate the strings in javascript. Some more details on this can be found in this post.
The js-translation.json is located at the theme root for example pub/static/frontend/Vendor/theme/en_US/js-translation.json
The file is automatically generated during static content deploy. Lets keep going backwards to see the flow.
php bin/magento setup:static-content:deploy
The file is deployed in module-deploy/Service/DeployTranslationsDictionary.php which is called in deploy during static content deploy.
$packages = $deployStrategy->deploy($options);
if ($options[Options::NO_JAVASCRIPT] !== true) {
$deployRjsConfig = $this->objectManager->create(DeployRequireJsConfig::class, [
'logger' => $this->logger
]);
$deployI18n = $this->objectManager->create(DeployTranslationsDictionary::class, [
'logger' => $this->logger
]);
$deployBundle = $this->objectManager->create(Bundle::class, [
'logger' => $this->logger
]);
foreach ($packages as $package) {
if (!$package->isVirtual()) {
$deployRjsConfig->deploy($package->getArea(), $package->getTheme(), $package->getLocale());
$deployI18n->deploy($package->getArea(), $package->getTheme(), $package->getLocale());
$deployBundle->deploy($package->getArea(), $package->getTheme(), $package->getLocale());
}
}
}The file itself however is generated in Preprocessor class which is responsible for providing javascript translation dictionary.
public function process(Chain $chain)
{
if ($this->isDictionaryPath($chain->getTargetAssetPath())) {
...
$this->dataProvider->getData($themePath)));
$chain->setContentType('json');
}
}As we can see in the function the actual data however is gathered in the \Magento\Translation\Model\Js\DataProvider
public function getData($themePath)
{
$areaCode = $this->appState->getAreaCode();
$files = array_merge(
$this->filesUtility->getJsFiles('base', $themePath),
$this->filesUtility->getJsFiles($areaCode, $themePath),
$this->filesUtility->getStaticHtmlFiles('base', $themePath),
$this->filesUtility->getStaticHtmlFiles($areaCode, $themePath)
);
$dictionary = [];
foreach ($files as $filePath) {
$read = $this->fileReadFactory->create($filePath[0], \Magento\Framework\Filesystem\DriverPool::FILE);
$content = $read->readAll();
foreach ($this->getPhrases($content) as $phrase) {
try {
$translatedPhrase = $this->translate->render([$phrase], []);
if ($phrase != $translatedPhrase) {
$dictionary[$phrase] = $translatedPhrase;
}
} catch (\Exception $e) {
throw new LocalizedException(
__('Error while translating phrase "%s" in file %s.', $phrase, $filePath[0]),
$e
);
}
}
}
return $dictionary;
}A couple things happening here:
- First we are getting all the js files using \Magento\Translation\Model\Js\DataProvider::$filesUtility.
- Then we are going through each file path, reading the file content and parsing translation phrases using \Magento\Translation\Model\Js\DataProvider::getPhrases.
- Once we have found a phrase we are attempting to render it using \Magento\Framework\Phrase\Renderer\Composite::render
- The composite renderer is defined in translation modules di.xml:
<preference for="Magento\Framework\Phrase\RendererInterface" type="Magento\Framework\Phrase\Renderer\Composite" />- Once we have rendered the translation we compare it with the original phrase. If the translation differs then we add it to the dictionary.
Ok, now we know how the file content is generated. Last step is to understand how the file genration is triggered. If we take a look at the usages of the class we can see that the processor is defined here:
<virtualType name="AssetPreProcessorPool">
<arguments>
<argument name="preprocessors" xsi:type="array">
<item name="js" xsi:type="array">
<item name="js_translation" xsi:type="array">
<item name="class" xsi:type="string">Magento\Translation\Model\Js\PreProcessor</item>
</item>
</item>
<item name="json" xsi:type="array">
<item name="json_generation" xsi:type="array">
<item name="class" xsi:type="string">Magento\Translation\Model\Json\PreProcessor</item>
</item>
</item>
</argument>
</arguments>
</virtualType>The JS\PreProcessor is responsible for replacing translation calls in js files to translated strings. The PreProcessor pool is implemented in framework \Magento\Framework\View\Asset\PreProcessor\Pool.
The pool process is calling getPreProcessors which picks up our class:
foreach ($preprocessors as $preprocessor) {
$instance = $this->objectManager->get($preprocessor[self::PREPROCESSOR_CLASS]);
if (!$instance instanceof PreProcessorInterface) {
throw new \UnexpectedValueException( '"' . $preprocessor[self::PREPROCESSOR_CLASS] . '" has to implement the PreProcessorInterface.');
}
$this->instances[$type][] = $instance;
}The Pool itself is trigger by a call in \Magento\Framework\App\View\Asset\Publisher::publishAsset which occurs during \Magento\Deploy\Service\DeployStaticFile::deployFile.
Since we have come back to the static content generation we have travelled through the lifecycle of the javascript translations generation. That is as far as we go.
Normally you will never have to go as deep in core code but it is quite interesting to browse through and can be useful in case any bugs appear.
The functionality itself can be simplified to the following: during static content generation we are gathering all javascript files and parsing them for any translatable strings using regex. All of the translatable strings are attempted to be translated using the available renderer. If the translation differs from the original string the value the key -> value pair is added to the dictionary. The dictionary is saved in a file. Later when user visits the page the file is used by translations initialization and it updates the localstorage dictionary with the newest version of the strings. Finally the translations component is looking for strings in the dictionary and if it can find a match will translate.
This is fairly optimal since we can process everything during static content deployment, then load the translations once and use browser storage to work with javascript components.
Hopefully this does give you a bit of insight on how the process works. It is certainly fun to go through and see various methods and strategies used in Magento 2 core.