https://pantsbuild.org/ logo
#general
Title
# general
h

high-egg-2153

02/02/2021, 2:24 PM
I am trying to get flask babel translations to work. Not having any luck when using pants run. If I package the pex, unzip it, install it in a virtual directory, and then run it, everything works as expected. The BUILD file looks like this:
*python_library*(
name = "lib",
)
*resources*(
name = "translations",
sources = ["translations/**"],
)
*pex_binary*(
name = "bin",
entry_point = "<http://dash_campaign_status_report.app|dash_campaign_status_report.app>",
dependencies = [
":lib",
":translations",
"lib_dash:lib",
"lib_dash_security:lib",
]
)
If I run dependencies on the bin target I get:
dash_campaign_status_report/app.py:lib
dash_campaign_status_report/translations/en/LC_MESSAGES/messages.mo:../../../translations
dash_campaign_status_report/translations/en/LC_MESSAGES/messages.po:../../../translations
dash_campaign_status_report/translations/fr/LC_MESSAGES/messages.mo:../../../translations
dash_campaign_status_report/translations/fr/LC_MESSAGES/messages.po:../../../translations
dash_campaign_status_report/translations/messages.pot:../translation
h

hundreds-father-404

02/02/2021, 2:56 PM
How are you loading the resources in your app?
h

high-egg-2153

02/02/2021, 3:00 PM
Not really sure what is going on under the covers. In the dash app it looks like this: from flask_babel import Babel, gettext ... app = dash.Dash(name) babel = Babel(app.server) ... @babel.localeselector def *get_locale*(): return request.accept_languages.best_match(['fr', 'en']) ... app.logger.debug(gettext('Hello World!')) ...
h

hundreds-father-404

02/02/2021, 3:02 PM
Hm so the localeselector does this all? You're right that it's weird it works for package, but not run. Those are intended to behave almost the same
h

high-egg-2153

02/02/2021, 3:04 PM
So after things are initialized in the babel = ... line, whenever gettext is called it calls get_locale internally and then returns the localized string.
Not sure what the ../../../translations is when I ran dependencies. The ../../../ seems strange.
h

hundreds-father-404

02/02/2021, 3:11 PM
That second part is telling you what the original BUILD file target was. Pants makes more precise "file targets", one per each file. It copies all the metadata from the original BUILD target, and that's the relpath from the file to that BUILD target
e

enough-analyst-54434

02/02/2021, 3:41 PM
Two points of clarificaion: 1. "If I package the pex, unzip it, install it in a virtual directory,": What do you mean by the last clause - how do you "install it in a virtual directory". 2. "Not having any luck when using pants run." ... do you have a backtrace we could see?
h

high-egg-2153

02/02/2021, 4:29 PM
1. I think I mixed up the pex with my custom distribution process. Sorry for the confusion. Basically if I take the app.py and its dependencies and run them without pants (python app.py) then the translations work.
šŸ‘ 1
2. Is there a log file or something I could provide? There are no errors or exceptions when I do pants run. It just appears that the translation files cannot be found.
e

enough-analyst-54434

02/02/2021, 4:45 PM
OK. Can you try either marking your pex_binary target as
zip_safe=False
(https://www.pantsbuild.org/docs/python-target-types#pex_binary) or else
unzip=True
if you're on Pants 2.3.0.dev1 or newer?
h

hundreds-father-404

02/02/2021, 5:43 PM
Oh I didn't realize that page still exists...it's superseded by auto-generated docs at https://www.pantsbuild.org/docs/reference-pex_binary
h

high-egg-2153

02/02/2021, 5:50 PM
zip_safe = False doesn't seem to make a difference
e

enough-analyst-54434

02/02/2021, 6:17 PM
OK. Well, without much to go on in turns of the actual error, I can say that using this week's dev release you should be able to work around this by using a new
venv
mode that can be applied to
pex_binary
targets: https://github.com/pantsbuild/pants/pull/11510
That forces a PEX file to install itself in a venv and run from there. So that should work exactly like your hand rolled venv.
šŸ‘ 1
h

high-egg-2153

02/02/2021, 6:21 PM
I'll try updating to a newer version tonight and let you know how it goes.
A little more information. It looks like when I put zip_safe=False, nothing changes in the tmp folder where the pex is running. Babel is looking in a path like the following for the translations: .../.pants.d/tmpmedppps_/bin.pex/translations Should bin.pex be a directory when running zip_safe=False?
e

enough-analyst-54434

02/03/2021, 4:39 PM
No. The
zip_safe=false
extracts the code portion of the pex under
~/.pex/code/<sha1>
The
__main__,py
and the PEX `.bootstrap/`code still run from inside the zip. To get full extraction you need
unzip=True
available in
2.3.0.dev2
. Even that may nt do the trick though dependening on how the translation code works. You may need to wait for
execution_mode=venv
.
h

high-egg-2153

02/03/2021, 6:18 PM
Using 2.3.0.dev2 the path that babel is searching is now: .
cache/pants/named_caches/pex_root/unzipped_pexes/99f66b4789e43b1deae1ac4bbb0348a2be26a3c9/translations
However, the unzipped pex there does not have any of my code or the translations folder. I guess I'm missing something when it comes to pex. I thought it was self contained meaning all my code and all dependencies would in the pex.
h

hundreds-father-404

02/03/2021, 6:22 PM
I thought it was self contained meaning all my code and all dependencies would in the pex.
Hm, well, so, there is an interesting divergence between
./pants run
and
./pants package
. With
./pants run
, we do not put source files in the PEX file and they simply are in the chroot. This avoids an overhead where we need to keep rebuilding a PEX file every time you make a change to source files But with
./pants package
, source files are included because you want a single archive file For debugging, I recommend using the workflow of
./pants package
, followed by running via
dist/path.to.module/app.pex
. There are fewer moving parts with that than
./pants run
. Once that's working, we can translate to
./pants run
h

high-egg-2153

02/03/2021, 6:28 PM
I just gave this a try but unfortunately when I try to run the pex I get:
from pkg_resources import get_distribution, parse_version
ModuleNotFoundError: No module named 'pkg_resources'
I do have:
*python_requirements*(
module_mapping = {
"setuptools": ["pkg_resources"],
},
)
in my root BUILD file.
e

enough-analyst-54434

02/03/2021, 6:33 PM
All your deps are in .deps -
ls -a
?
h

hundreds-father-404

02/03/2021, 6:34 PM
And showing up via
./pants dependencies --transitive path/to:pex_binary
? if not, you could force it to be included by adding something like
//:setuptools
to the
dependencies
of your
pex_binary
target
e

enough-analyst-54434

02/03/2021, 6:34 PM
@high-egg-2153 the mapping looks good, but do you actually depend on setuptools somewhere? You need that too.
h

high-egg-2153

02/03/2021, 6:37 PM
It is in my requirements.txt file setuptools==51.1.1
e

enough-analyst-54434

02/03/2021, 6:38 PM
OK. Can you try @hundreds-father-404ā€™s suggestion of
./pants dependencies --transitive path/to:pex_binary
as a sanity check?
h

high-egg-2153

02/03/2021, 6:42 PM
I get:
./pants dependencies --transitive dash_campaign_status_report:bin
//:Flask-Babel
//:SQLAlchemy
//:boto3
//:dash
//:dash-core-components
//:dash-embedded
//:dash-html-components
//:dash-table
//:pandas
//:requirements.txt
dash_campaign_status_report/app.py:lib
dash_campaign_status_report/translations/en/LC_MESSAGES/messages.mo:../../../translations
dash_campaign_status_report/translations/en/LC_MESSAGES/messages.po:../../../translations
dash_campaign_status_report/translations/fr/LC_MESSAGES/messages.mo:../../../translations
dash_campaign_status_report/translations/fr/LC_MESSAGES/messages.po:../../../translations
dash_campaign_status_report/translations/messages.pot:../translations
lib_dash/dash_app.py:lib
lib_dash_security/security.py:lib
h

hundreds-father-404

02/03/2021, 6:43 PM
K, so for now, manually add
//:setuptools
to the
dependencies
field of
dash_campaign_status_report:bin
. We'll want to figure out why dep inference isn't working, but in a separate thread
h

high-egg-2153

02/03/2021, 6:55 PM
I had to add both setuptools and pymysql to the dependencies and then it worked. FYI - both of these are in requirements.txt and constraints.txt at the root.
h

hundreds-father-404

02/03/2021, 7:05 PM
Okay. I think we had found pymsql was needed because you don't import it in your Python code so it's not inferrable, iirc. setuptools is still a mystery, but best for another thread What's going on now with running
./pants package
, then running via
dist/dash_campagin_status_report/bin.pex
? (Where
unzip=True
)
h

high-egg-2153

02/03/2021, 7:11 PM
Its working. I also got the translation to work but it requires some code changes. By default babel is looking for translation right off the root:
~/.pex/unzipped_pexes/56cdee2343fcaf3ef97c9aba8c74d6c1167141e9/translations
With the following change it works:
translations_path = os.path.join(app.root_path, 'dash_campaign_status_report/translations')
app.config['BABEL_TRANSLATION_DIRECTORIES'] = translations_path
babel = Babel(app)
However, this probably won't work in my case once deployed to production.
h

hundreds-father-404

02/03/2021, 7:14 PM
Okay, yay that there's a workaround! The upcoming venv mode allows you to execute Pex apps much more similarly to how you were describing your non-Pants workflow I'd recommend not spending much more time on this until the venv support lands, given that it Might Just Work. Pex is all done, it's only finishing wiring it up to Pants
h

high-egg-2153

02/03/2021, 7:15 PM
Perfect. I think this will work for me for now. Once again thanks for the help.
ā¤ļø 1
4 Views