You are not logged in.

#1 2021-10-22 13:03:59

Ashark
Member
Registered: 2017-06-01
Posts: 21

Translations in kde-git packages

I have created a zanshin-git yesterday. I noticed that it misses translation files. Such as for example /usr/share/locale/zh_CN/LC_MESSAGES/zanshin.mo.
And the zanshin from official packages has such translations.

I used build function as in non-git version, but I thought that I had done something wrong.
I compared the src folder of the archive downloaded by zanshin, and it has "po" folder in it. While the src directory of zanshin-git does not have it.
Then I checked the other packages. For example, let's take kate-git. It was not done by me, but it has the same thing with missing translations.

I guess that those archives prepared by kde (that are downloaded for building non-git versions) brings translations from some other source. And this should be done manually for git versions.

I tried to find this info in https://wiki.archlinux.org/title/KDE_package_guidelines, but there is no one. Can you please suggest how to do it?
Then I think the guideline should be edited, so other packagers update their pkgbuilds to build with translations.

Offline

#2 2021-10-25 00:46:38

Ashark
Member
Registered: 2017-06-01
Posts: 21

Re: Translations in kde-git packages

Trying to build zanshin from git and with translations.

Cloned to "zanshin" folder. Created "build" folder near it. Getting the following while trying to configure:

$ LC_ALL=C cmake -B build -S zanshin -DBUILD_TESTING=OFF -DKDE_L10N_SYNC_TRANSLATIONS=ON -DKDE_L10N_BRANCH=trunk --trace-source=CMakeLists.txt
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(1):  project(zanshin )
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(2):  cmake_minimum_required(VERSION 3.2 )
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(4):  find_package(ECM REQUIRED CONFIG )
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(5):  set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/cmake/ ${CMAKE_MODULE_PATH} ${ECM_MODULE_PATH} )
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(7):  include(GenerateExportHeader )
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(8):  include(KDEInstallDirs )
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(9):  include(KDECompilerSettings NO_POLICY_SCOPE )
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(10):  include(KDECMakeSettings )
fatal: no upstream configured for branch 'my_another_branch'
fatal: destination path 'releaseme' already exists and is not an empty directory.
ruby: No such file or directory -- /home/andrew/Development/Lab/build/releaseme/fetchpo.rb (LoadError)
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(11):  include(FeatureSummary )
...

Looks like the errors happens in KDECMakeSettings.

Why ruby searches releaseme in build direcrory? Looks like it is assumed that KDECMakeSettings assumes you are already in build direcrory, despite the pathes you specified in cmake command, and clones releaseme "right there". I consider it as bug, am I right?
I think /usr/share/ECM/kde-modules/KDECMakeSettings.cmake at line 341 should be
    COMMAND git clone --depth 1 https://invent.kde.org/sdk/releaseme.git ${CMAKE_BINARY_DIR}/releaseme
instead of just
    COMMAND git clone --depth 1 https://invent.kde.org/sdk/releaseme.git
   
Ok, for now as a workaround, I will cd to build direcrory and edit path in cmake comand:

$ LC_ALL=C cmake -B . -S ../zanshin -DBUILD_TESTING=OFF -DKDE_L10N_SYNC_TRANSLATIONS=ON -DKDE_L10N_BRANCH=trunk --trace-source=CMakeLists.txt
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(1):  project(zanshin )
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(2):  cmake_minimum_required(VERSION 3.2 )
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(4):  find_package(ECM REQUIRED CONFIG )
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(5):  set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/cmake/ ${CMAKE_MODULE_PATH} ${ECM_MODULE_PATH} )
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(7):  include(GenerateExportHeader )
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(8):  include(KDEInstallDirs )
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(9):  include(KDECompilerSettings NO_POLICY_SCOPE )
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(10):  include(KDECMakeSettings )
fatal: no upstream configured for branch 'my_another_branch'
Cloning into 'releaseme'...
Found 0 elements for zanshin
/home/andrew/Development/Lab/zanshin/CMakeLists.txt(11):  include(FeatureSummary )

Now ruby script sees the releaseme cloned repo. But it finds 0 elements for zanshin.

I have checked that running
ruby releaseme/fetchpo.rb --origin trunk --project zanshin --output-dir /tmp/po --output-poqm-dir /tmp/poqm /home/andrew/Development/Lab/zanshin
did not found any files.
But if I add edit project as --project pim/zanshin in the command above, then it works (downloads translations)!

I opened https://invent.kde.org/frameworks/extra … ings.cmake and tried to understand where it came from.
At line 358 there is `--project "${_reponame}"` parameter for the ruby script. And it resolves to "zanshin", but not "pim/zanshin". Trying to understand why.
At line 338 there is
_repository_name(_reponame "${CMAKE_SOURCE_DIR}")
Hm, looks like if I change source dir to pim/zanshin, it will work?
Moved source to pim/zanshin, deleted build dir contents, adjusted path in -S to ../pim/zanshin.
This time there are move warnings before I reach line 10, but no luck:
  /home/andrew/Development/Lab/pim/zanshin/CMakeLists.txt(10):  include(KDECMakeSettings )
  fatal: no upstream configured for branch 'my_another_branch'
  Cloning into 'releaseme'...
  Found 0 elements for zanshin
  /home/andrew/Development/Lab/pim/zanshin/CMakeLists.txt(11):  include(FeatureSummary )
it still resolves to just "zanshin" without "pim/" prefix.

Now will use this to trace the KDECMakeSettings.cmake:

LC_ALL=C cmake -B . -S ../pim/zanshin -DBUILD_TESTING=OFF -DKDE_L10N_SYNC_TRANSLATIONS=ON -DKDE_L10N_BRANCH=trunk --trace-source=/usr/share/ECM/kde-modules/KDECMakeSettings.cmake --trace-expand
In the output I see:
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(337):  set(_reponame  )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(338):  _repository_name(_reponame /home/andrew/Development/Lab/pim/zanshin )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(300):  execute_process(COMMAND git rev-parse --symbolic-full-name @{u} OUTPUT_VARIABLE upstream_ref RESULT_VARIABLE exitCode WORKING_DIRECTORY /home/andrew/Development/Lab/pim/zanshin )
  fatal: no upstream configured for branch 'my_another_branch'
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(304):  string(REGEX REPLACE refs/remotes/([^/]+)/.* \1 gitorigin  )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(305):  if(exitCode EQUAL 0 )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(313):  if(exitCode EQUAL 0 )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(320):  if(NOT _reponame )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(321):  set(_reponame zanshin )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(323):  set(_reponame zanshin PARENT_SCOPE )

Looking at KDECMakeSettings.cmake from line 299, I realize that it really wants to be on branch with upstream.
I switched to master and run `git remote get-url --all origin` and got
  git@invent.kde.org:pim/zanshin.git
There is where it must take that "pim" prefix.

Now I repeat the previous (remind: now at master branch):

  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(337):  set(_reponame  )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(338):  _repository_name(_reponame /home/andrew/Development/Lab/pim/zanshin )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(300):  execute_process(COMMAND git rev-parse --symbolic-full-name @{u} OUTPUT_VARIABLE upstream_ref RESULT_VARIABLE exitCode WORKING_DIRECTORY /home/andrew/Development/Lab/pim/zanshin )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(304):  string(REGEX REPLACE refs/remotes/([^/]+)/.* \1 gitorigin refs/remotes/origin/master
   )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(305):  if(exitCode EQUAL 0 )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(306):  message(DEBUG Git upstream inferred as origin, upstream ref was refs/remotes/origin/master
   )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(307):  execute_process(COMMAND git remote get-url --all origin OUTPUT_VARIABLE giturl RESULT_VARIABLE exitCode WORKING_DIRECTORY /home/andrew/Development/Lab/pim/zanshin )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(313):  if(exitCode EQUAL 0 )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(314):  message(DEBUG Git URL inferred as git@invent.kde.org:pim/zanshin.git
   )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(315):  string(REGEX MATCHALL .+kde\.org[:\/]([-A-Za-z0-9\/]+)(.git)?\s*  git@invent.kde.org:pim/zanshin.git
   )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(316):  set(_reponame pim/zanshin )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(317):  message(DEBUG Repository inferred as pim/zanshin )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(320):  if(NOT _reponame )
  /usr/share/ECM/kde-modules/KDECMakeSettings.cmake(323):  set(_reponame pim/zanshin PARENT_SCOPE )

It worked!

But it is too eary to be happy. Reading the output further:

/usr/share/ECM/kde-modules/KDECMakeSettings.cmake(340):  set(releaseme_clone_commands COMMAND git clone --depth 1 https://invent.kde.org/sdk/releaseme.git )
/usr/share/ECM/kde-modules/KDECMakeSettings.cmake(343):  add_custom_command(OUTPUT /home/andrew/Development/Lab/build/releaseme COMMAND;git;clone;--depth;1;https://invent.kde.org/sdk/releaseme.git WORKING_DIRECTORY /home/andrew/Development/Lab/build COMMENT Fetching releaseme scripts to download translations... )
/usr/share/ECM/kde-modules/KDECMakeSettings.cmake(350):  set(_l10n_po_dir /home/andrew/Development/Lab/build/po )
/usr/share/ECM/kde-modules/KDECMakeSettings.cmake(351):  set(_l10n_poqm_dir /home/andrew/Development/Lab/build/poqm )
/usr/share/ECM/kde-modules/KDECMakeSettings.cmake(353):  set(extra BYPRODUCTS /home/andrew/Development/Lab/build/po /home/andrew/Development/Lab/build/poqm )
/usr/share/ECM/kde-modules/KDECMakeSettings.cmake(355):  set(fetch_commands COMMAND ruby /home/andrew/Development/Lab/build/releaseme/fetchpo.rb --origin trunk --project pim/zanshin --output-dir /home/andrew/Development/Lab/build/po --output-poqm-dir /home/andrew/Development/Lab/build/poqm /home/andrew/Development/Lab/pim/zanshin )
/usr/share/ECM/kde-modules/KDECMakeSettings.cmake(364):  add_custom_target(fetch-translations  COMMENT Downloading translations for pim/zanshin branch trunk... COMMAND git -C /home/andrew/Development/Lab/build/releaseme pull COMMAND cmake -E remove_directory /home/andrew/Development/Lab/build/po COMMAND cmake -E remove_directory /home/andrew/Development/Lab/build/poqm COMMAND;ruby;/home/andrew/Development/Lab/build/releaseme/fetchpo.rb;--origin;trunk;--project;pim/zanshin;--output-dir;/home/andrew/Development/Lab/build/po;--output-poqm-dir;/home/andrew/Development/Lab/build/poqm;/home/andrew/Development/Lab/pim/zanshin BYPRODUCTS;/home/andrew/Development/Lab/build/po;/home/andrew/Development/Lab/build/poqm DEPENDS /home/andrew/Development/Lab/build/releaseme )
/usr/share/ECM/kde-modules/KDECMakeSettings.cmake(374):  if(KDE_L10N_SYNC_TRANSLATIONS AND ( NOT EXISTS /home/andrew/Development/Lab/build/po OR NOT EXISTS /home/andrew/Development/Lab/build/poqm ) )
/usr/share/ECM/kde-modules/KDECMakeSettings.cmake(375):  execute_process(COMMAND;git;clone;--depth;1;https://invent.kde.org/sdk/releaseme.git )
fatal: destination path 'releaseme' already exists and is not an empty directory.
/usr/share/ECM/kde-modules/KDECMakeSettings.cmake(376):  execute_process(COMMAND;ruby;/home/andrew/Development/Lab/build/releaseme/fetchpo.rb;--origin;trunk;--project;pim/zanshin;--output-dir;/home/andrew/Development/Lab/build/po;--output-poqm-dir;/home/andrew/Development/Lab/build/poqm;/home/andrew/Development/Lab/pim/zanshin )
INFO -- ReleaseMe::L10n: Downloading translations for /home/andrew/Development/Lab/pim/zanshin
#<Thread:0x000055c68957c088 /home/andrew/Development/Lab/build/releaseme/lib/releaseme/translationunit.rb:109 run> terminated with exception (report_on_exception is true):
/home/andrew/Development/Lab/build/releaseme/lib/releaseme/l10n/asset.rb:42:in `gsub!': invalid byte sequence in US-ASCII (ArgumentError)
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/l10n/asset.rb:42:in `strip_comments!'
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/l10n/asset.rb:16:in `strip!'
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/l10n.rb:78:in `block (2 levels) in download'
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/l10n.rb:76:in `each'
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/l10n.rb:76:in `block in download'
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/translationunit.rb:128:in `block (2 levels) in each_language_with_tmpdir'
        from /usr/lib/ruby/3.0.0/tmpdir.rb:96:in `mktmpdir'
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/tmpdir.rb:22:in `mktmpdir'
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/translationunit.rb:128:in `block in each_language_with_tmpdir'
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/translationunit.rb:111:in `block (2 levels) in blocking_thread_pool'
/home/andrew/Development/Lab/build/releaseme/lib/releaseme/l10n/asset.rb:42:in `gsub!': invalid byte sequence in US-ASCII (ArgumentError)
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/l10n/asset.rb:42:in `strip_comments!'
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/l10n/asset.rb:16:in `strip!'
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/l10n.rb:78:in `block (2 levels) in download'
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/l10n.rb:76:in `each'
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/l10n.rb:76:in `block in download'
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/translationunit.rb:128:in `block (2 levels) in each_language_with_tmpdir'
        from /usr/lib/ruby/3.0.0/tmpdir.rb:96:in `mktmpdir'
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/tmpdir.rb:22:in `mktmpdir'
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/translationunit.rb:128:in `block in each_language_with_tmpdir'
        from /home/andrew/Development/Lab/build/releaseme/lib/releaseme/translationunit.rb:111:in `block (2 levels) in blocking_thread_pool'
-- Found Boost: /usr/lib64/cmake/Boost-1.76.0/BoostConfig.cmake (found version "1.76.0")

I did the following.

rmdir /home/andrew/Development/Lab/build/po
rmdir /home/andrew/Development/Lab/build/poqm
ruby /home/andrew/Development/Lab/build/releaseme/fetchpo.rb --origin trunk --project pim/zanshin --output-dir /home/andrew/Development/Lab/build/po --output-poqm-dir /home/andrew/Development/Lab/build/poqm /home/andrew/Development/Lab/pim/zanshin
INFO -- ReleaseMe::L10n: Downloading translations for /home/andrew/Development/Lab/pim/zanshin
INFO -- ReleaseMe::L10n: The following languages had translations files but they amount to no useful strings so they were removed: ja, zh_TW
INFO -- ReleaseMe::L10n: No translations for: as, ast, az, ar, af, bg, be@latin, bn, be, bn_IN, crh, br, csb, cy, el, en, eo, fa, fy, gd, gu, ha, hi, he, hr, hne, hsb, hy, id, is, ka, kab, kk, kn, km, ku, lb, lv, mai, ml, mk, ms, mt, ne, nso, oc, or, pa, ps, ro, rw, se, si, sq, sr@ijekavian, sr, sr@ijekavianlatin, sr@latin, ta, te, tg, th, tn, tt, uz, uz@cyrillic, vi, wa, xh, zh_HK

So when called directly, it works perfectly.

Wip version of pkgbuild here.

There are three questions remaining.
1) Why releaseme was expected to be in build dir, but was cloned into working dir. Most likely, upstream bug (kde extra cmake module).
2) Need to understand why in the command above it failes when run from cmake, saying there are some invalid byte sequence.
3) Why makepkg switches to makepkg branch? If there is no special reason, I can just checkout to master to avoid problem of non having origin.

Last edited by Ashark (2021-10-27 23:04:19)

Offline

Board footer

Powered by FluxBB