[SOLVED] Help installing AWS SAM CLI on linux; worried about Homebrew

geo909 · 2020-04-28 11:42:16

Hi all,

I need to write some python and use it with aws lambda. Amazon provides some basic IDE where you can write code online, but it's very basic and has some disadvantages for me that It's not relevant to get into here. The alternative and recommended way of developing python applications for aws lambda, seems to be by using Amazon's serverless framework:

https://docs.aws.amazon.com/serverless- … linux.html

I was about to follow the instructions, but I saw the following that freaked me out a little bit:

Note
These instructions cause your environment's default Python version to be the one installed by Homebrew. The change in default Python version occurs in Step 4: Install Homebrew.

I went through the instructions but I'm not 100% sure what is going on. Could somebody more experienced please check these instructions and confirm whether step 4 will mess my python (installed by pacman)? I assume if this is the case, this is very dangerous for my system, right? In any case, I would be grateful if you could advice on the safest approach to install AWS SAM CLI.

Last edited by geo909 (2020-04-29 09:39:01)

jasonwryan · 2020-04-28 17:59:55

https://www.archlinux.org/packages/comm … ranslator/

geo909 · 2020-04-28 19:05:47

jasonwryan wrote:

https://www.archlinux.org/packages/comm … ranslator/

I have already installed the package and tried it, but it doesn't seem to be the same thing as aws sam cli. This, or I am missing something. For example, it does not provide the 'sam' command which is what I need to use.

EDIT: Indeed, this is from the PyPi repository of this tool: "AWS SAM Translator is a library that transform SAM templates into AWS CloudFormation templates". It's not sam cli..

Last edited by geo909 (2020-04-28 19:12:06)

loqs · 2020-04-28 19:35:19

Lookins at https://github.com/awslabs/aws-sam-cli/ … s/base.txt chevron, serverlessrepo and aws_lambda_builders would need to be packaged first.
Setup a container or virtual machine to isolate brew from the host system?

Trilby · 2020-04-28 23:32:40

It only says the recommended way to install it is homebrew. All homebrew does is builds packages from source following a recipe - it's effectively makepkg for MacOS. So just grab the homebrew recipe for aws-sam-cli and translate it into a PKGBUILD.

EDIT: step 1, here it is

EDIT 2: I suck at ruby, but as I read through that, it looked like the recipe might just be offloading to pip. I then checked the cheese shop, and sure enough there it was. There are ample example PKGBUILD building from PyPi sources. Grab one of those and get cracking.

EDIT 3: boy am I slow. Apparently you found all this out a few hours ago ... sorry. EDIT 4: wait, no, you found some other AWS package on pypi, see my link above for the aws-sam-cli on pypi.

Last edited by Trilby (2020-04-28 23:45:55)

loqs · 2020-04-29 00:16:45

PKGBUILD for chevron

pkgname=python-chevron
_pkgname=chevron
pkgver=0.13.1
pkgrel=1
pkgdesc="A Python implementation of mustache "
arch=('any')
url="https://github.com/noahmorrison/chevron/"
license=('MIT')
depends=('python')
makedepends=('python-setuptools')
source=("$pkgname-$pkgver.tar.gz::https://github.com/noahmorrison/$_pkgname/archive/$pkgver.tar.gz")
sha256sums=('9abfa10ea527a0b62f6d65a528f26e0a4930be3107b61a146598f65e14c13118')

build() {
  cd "$_pkgname-$pkgver"
  python setup.py build
}

package() {
  cd "$_pkgname-$pkgver"
  python setup.py install --root="$pkgdir/" --optimize=1 --skip-build
  install -dm 755 "$pkgdir/"usr/share/licenses/$pkgname
  install -Dm 644 LICENSE "$pkgdir/"usr/share/licenses/$pkgname/LICENSE
}

Edit:
PKGBUILD for serverlessrepo. Using pypi as the github repo has no tags.

pkgname=python-serverlessrepo
_pkgname=serverlessrepo
pkgver=0.1.10
pkgrel=1
pkgdesc="A Python library with convenience helpers for working with the AWS Serverless Application Repository"
arch=('any')
url="https://pypi.org/project/serverlessrepo/"
license=('Apache')
depends=('python' 'python-pyaml' 'python-boto3' 'python-six')
makedepends=('python-setuptools')
source=("https://files.pythonhosted.org/packages/source/${_pkgname::1}/$_pkgname/$_pkgname-$pkgver.tar.gz")
sha256sums=('671f48038123f121437b717ed51f253a55775590f00fbab6fbc6a01f8d05c017')

build() {
  cd "$_pkgname-$pkgver"
  python setup.py build
}

package() {
  cd "$_pkgname-$pkgver"
  python setup.py install --root="$pkgdir/" --optimize=1 --skip-build
}

PKGBUILD for aws-lambda-builders

pkgname=python-aws-lambda-builders
_pkgname=aws-lambda-builders
pkgver=0.8.0
pkgrel=1
pkgdesc="Python library to compile, build & package AWS Lambda functions for several runtimes & framework "
arch=('any')
url="https://github.com/awslabs/aws-lambda-builders"
license=('Apache')
depends=('python' 'python-six')
makedepends=('python-setuptools')
source=("$pkgname-$pkgver.tar.gz::https://github.com/awslabs/$_pkgname/archive/v$pkgver.tar.gz")
sha256sums=('e345102768af7935a85bb2aa9bf8d4e3552a81ec7eca4c56d481877337933af9')

build() {
  cd "$_pkgname-$pkgver"
  python setup.py build
}

package() {
  cd "$_pkgname-$pkgver"
  python setup.py install --root="$pkgdir/" --optimize=1 --skip-build
}

PKGBUILD for aws-sam-cli

pkgname=python-aws-sam-cli
_pkgname=aws-sam-cli
pkgver=0.47.0
pkgrel=1
pkgdesc="CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM"
arch=('any')
url="https://aws.amazon.com/serverless/sam/"
license=('Apache')
depends=('python' 'python-chevron' 'python-click' 'python-flask' 'python-boto3'
         'python-jmespath' 'python-yaml' 'python-cookiecutter'
         'python-aws-sam-translator' 'python-docker' 'python-dateparser'
         'python-dateutil' 'python-requests' 'python-serverlessrepo'
         'python-aws-lambda-builders' 'python-tomlkit')
makedepends=('python-setuptools')
source=("$pkgname-$pkgver.tar.gz::https://github.com/awslabs/$_pkgname/archive/v$pkgver.tar.gz")
sha256sums=('e872412c9ed25d4632b4746da1415a9308c6540abe4fc657cb0135883abd7cd7')

build() {
  cd "$_pkgname-$pkgver"
  python setup.py build
}

package() {
  cd "$_pkgname-$pkgver"
  python setup.py install --root="$pkgdir/" --optimize=1 --skip-build
}

Last edited by loqs (2020-04-29 04:39:18)

geo909 · 2020-04-29 07:39:36

Thank you guys very much for your replies.

Trilby wrote:

wait, no, you found some other AWS package on pypi, see my link above for the aws-sam-cli on pypi.

I had seen that before, maybe I should have mentioned.. To make things a bit confusing, the package aws-sam-cli on PyPi is version 1 of the aws sam cli; the one that uses homebrew is version 2 (sigh).

I had previously installed aws-sam-cli via pip with the --user flag and tried it, but then I couldn't use it with certain sam commands that involved docker and required sudo. I was afraid to install it system-wide for fear again that it will mess up my python dependencies.

The folks at amazon recommend using version 2, but I may as well go on with that one now that you gave me the PKGBUILDs so I guess I can install them system-wide. I'll try them and let you know how it went. Thanks again.

EDIT: Argh!! Just looked at the homebrew formula.. Now I finally understand: it does use pip underneath with that aws-sam-cli package. I now finally understand that the word "version" in "Installing the AWS CLI version 1" and "version 2" in the aws docs refers to the actual method, not the tool itself :-/ Gee, that was confusing.

Last edited by geo909 (2020-04-29 07:47:09)

geo909 · 2020-04-29 08:20:34

So, I installed the packages. It feels I'm close, but there started to be a dependency version hell:

sam
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 584, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 901, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 792, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (docutils 0.16 (/usr/lib/python3.8/site-packages), Requirement.parse('docutils<0.16,>=0.10'), {'botocore'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/sam", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3259, in <module>
    def _initialize_master_working_set():
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3242, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3271, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 586, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 599, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 787, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'docutils<0.16,>=0.10' distribution was not found and is required by botocore

So, apparently the versions that of python-docutils and python-tomlkit are too high for aws sam cli. I'll play around, either downgrading those pacages, or changing the requirements inside the archive. If you have any recommendations, let me know. But in any case, thanks again for your advice and providing the PKGBUILDs, that really helped.

EDIT: it seems that there are very strict requirements inside the aws-sam-cli package about the versions of the dependencies. These include some importang python packages such as requests. I downgraded already a few of them and the errors keep coming..

Last edited by geo909 (2020-04-29 08:47:57)

progandy · 2020-04-29 09:08:46

I suggest you try to create a python virtual environemnt that does not include the system packages. Then running pip inside that venv should install all packages you need.
https://wiki.archlinux.org/index.php/Py … nment#venv

Last edited by progandy (2020-04-29 09:15:38)

geo909 · 2020-04-29 09:37:42

progandy wrote:

I suggest you try to create a python virtual environemnt that does not include the systemd packages. Then running pip inside that venv should install all packages you need.

Yes, finally.. It works that way. It turns out it was way simpler than I had made it be:

>python -m venv env
>source env/bin/activate
>pip install aws-sam-cli

Thank you for your help everybody.. While the solution was under my nose, you really helped me understand a lot of things involving aws-sam-cli and packaging in general.

Last edited by geo909 (2020-04-29 09:38:17)

progandy · 2020-04-29 09:45:46

Its not a great solution, but if a package requires outdated dependencies...

geo909 · 2020-04-29 11:09:56

progandy wrote:

Its not a great solution, but if a package requires outdated dependencies...

It works and is simple, so right now it's great for me as I can finally go on doing my job

Arch Linux

#1 2020-04-28 11:42:16

[SOLVED] Help installing AWS SAM CLI on linux; worried about Homebrew

#2 2020-04-28 17:59:55

Re: [SOLVED] Help installing AWS SAM CLI on linux; worried about Homebrew

#3 2020-04-28 19:05:47

Re: [SOLVED] Help installing AWS SAM CLI on linux; worried about Homebrew

#4 2020-04-28 19:35:19

Re: [SOLVED] Help installing AWS SAM CLI on linux; worried about Homebrew

#5 2020-04-28 23:32:40

Re: [SOLVED] Help installing AWS SAM CLI on linux; worried about Homebrew

#6 2020-04-29 00:16:45

Re: [SOLVED] Help installing AWS SAM CLI on linux; worried about Homebrew

#7 2020-04-29 07:39:36

Re: [SOLVED] Help installing AWS SAM CLI on linux; worried about Homebrew

#8 2020-04-29 08:20:34

Re: [SOLVED] Help installing AWS SAM CLI on linux; worried about Homebrew

#9 2020-04-29 09:08:46

Re: [SOLVED] Help installing AWS SAM CLI on linux; worried about Homebrew

#10 2020-04-29 09:37:42

Re: [SOLVED] Help installing AWS SAM CLI on linux; worried about Homebrew

#11 2020-04-29 09:45:46

Re: [SOLVED] Help installing AWS SAM CLI on linux; worried about Homebrew

#12 2020-04-29 11:09:56

Re: [SOLVED] Help installing AWS SAM CLI on linux; worried about Homebrew

Board footer