faas-cli publish -f my-function.yml --platforms linux/arm/v7 fails on pytest

[kylos101]

My actions before raising this issue

• Followed the troubleshooting guide
• Read/searched the docs
• Searched past issues faas build fails with tox error #44 seems like it may be related, but, I could be wrong.

I'm sorry that my faas friday contribution is an issue, but, at least it has a workaround! 🤙

The error indicates pytest is failing. I am sharing detailed logs below after doing faas-cli template pull and creating a new function to do this test.

Expected Behaviour

This should should pass for a newly generated function:

faas-cli publish -f my-function.yml --platforms linux/arm/v7

Current Behaviour

The publish command fails with the following errors from Docker:

Errors received during build:
- [testbot] received non-zero exit code from build, error: #1 [internal] load build definition from Dockerfile
#1 sha256:075a3d35e0cfd6f8ab29191ba59f72275e0de836100fbf47a4479d478589a10f
#1 transferring dockerfile: 1.15kB done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:c5c02775cf56ecbb91a4b522c245b084c179fca7d60edf301d56a840c8fd79e6
#2 transferring context: 2B done
#2 DONE 0.0s

#5 [auth] armhf/python:pull token for registry-1.docker.io
#5 sha256:3dc59a684d8d4154fb7662095cb4fb49b3254796e28ce4af271db93499b4fa7f
#5 DONE 0.0s

#6 [auth] openfaas/of-watchdog:pull token for registry-1.docker.io
#6 sha256:fc965b916b550c1c70d5127a16730dc7797be92605d259eb4d775900cb5b0dbe
#6 DONE 0.0s

#4 [internal] load metadata for docker.io/openfaas/of-watchdog:0.7.7
#4 sha256:bc34e2a38b7becdfdbb099e33e528b60e49b127082ee2115ee2bb3cf8404e0c6
#4 DONE 0.7s

#3 [internal] load metadata for docker.io/armhf/python:3.6-alpine
#3 sha256:e6d48735957efd2aca6bd015db24f7a4e25ff42e9137a2eedffa2864e2d879de
#3 DONE 0.7s

#14 [internal] load build context
#14 sha256:75d750aed9a9fecde279564471856cbc9904beaa7a60a6192034745d46cbad2c
#14 DONE 0.0s

#7 [stage-1  1/18] FROM docker.io/armhf/python:3.6-alpine@sha256:3f2dcb7c293c8fb2c7d58733bf323da02054977b07c7f50f1e274daf54c2971b
#7 sha256:d3e6d426630b31c375c08b79bc045333eb432358559e15b18198923007c7955b
#7 resolve docker.io/armhf/python:3.6-alpine@sha256:3f2dcb7c293c8fb2c7d58733bf323da02054977b07c7f50f1e274daf54c2971b 0.1s done
#7 DONE 0.0s

#8 [watchdog 1/1] FROM docker.io/openfaas/of-watchdog:0.7.7@sha256:f988f45b65b0282f457bed763525ec92ca493487cc033c2db0399eac17732ac4
#8 sha256:0966a2a0eb6bbfcb4864f9327ee06080148d634d81e99878e4dcd5c6b51c5e14
#8 resolve docker.io/openfaas/of-watchdog:0.7.7@sha256:f988f45b65b0282f457bed763525ec92ca493487cc033c2db0399eac17732ac4 0.1s done
#8 DONE 0.1s

#14 [internal] load build context
#14 sha256:75d750aed9a9fecde279564471856cbc9904beaa7a60a6192034745d46cbad2c
#14 transferring context: 63.18MB 1.3s done
#14 DONE 1.3s

#20 [stage-1 12/18] WORKDIR /home/app/function/
#20 sha256:5d4886239545062f7c5987311dc564c87064597224ed6a9d6dab7069f259e213
#20 CACHED

#10 [stage-1  3/18] RUN chmod +x /usr/bin/fwatchdog
#10 sha256:a917f761637ebfa961d596caee35ecf8f962252ed0a6ff413b539274b4647cf3
#10 CACHED

#9 [stage-1  2/18] COPY --from=watchdog /fwatchdog /usr/bin/fwatchdog
#9 sha256:9529337b26faa6c3fa0b2bd61c954359542dfe2f3a688e9bc930419290d56d37
#9 CACHED

#13 [stage-1  6/18] WORKDIR /home/app/
#13 sha256:2d34903acad8c20a4d2657108e671a46f6cfa708c5ee0e6824561208ea0e361e
#13 CACHED

#15 [stage-1  7/18] COPY index.py           .
#15 sha256:640f776c9056fbe53b26d009b2ac1ee1c613b9f01f11838368c7bdf2ef8c9ade
#15 CACHED

#16 [stage-1  8/18] COPY requirements.txt   .
#16 sha256:63a815bbf06530f21163b4718f5d47f55a96304e21858cf17efd9516927fea05
#16 CACHED

#19 [stage-1 11/18] RUN touch ./function/__init__.py
#19 sha256:d665af0632794f7e0fe82d0cade434b2fe0c73bc2fcf758aec1abf9f283513a3
#19 CACHED

#18 [stage-1 10/18] RUN mkdir -p function
#18 sha256:ee5110d7edcc7f389bc349a190a7fb599b44766aa68af9c14368909667e4568b
#18 CACHED

#21 [stage-1 13/18] COPY function/requirements.txt      .
#21 sha256:55d47fb5e2e878953ffe27f2d71f50efc9fa0f90beadd9bb00fe8c4e2cc6baca
#21 CACHED

#12 [stage-1  5/18] RUN chown app /home/app
#12 sha256:a4e2b41a9152ca3bcc83684ed7a2386f34d25511763a67fb387f5d6aab493b95
#12 CACHED

#17 [stage-1  9/18] RUN pip install -r requirements.txt
#17 sha256:48e4b20bc86026ac1165f7d9aaffdbda8c284d44fb8396647e8d797fb9b8b516
#17 CACHED

#11 [stage-1  4/18] RUN addgroup -S app && adduser app -S -G app
#11 sha256:0c5c95592af6724c8127d43533c7ea5905a0c80ee373b90b00e7a9c56df84bd4
#11 CACHED

#22 [stage-1 14/18] RUN pip install --user -r requirements.txt
#22 sha256:19170bae26414d676be587e449c9fdabf4836c2c944ef34014e97725c2a3e24c
#22 CACHED

#23 [stage-1 15/18] COPY function/   .
#23 sha256:a202f6f412b7f6421e99ef8e7be4bfccee2a2002e3100e9033efd64ce5094807
#23 DONE 2.8s

#24 [stage-1 16/18] RUN chown -R app:app ../
#24 sha256:44c0bdeccf4f45533be7ba76ad19b5b9dedf25036ebce41faf947c65f5245344
#24 DONE 7.6s

#25 [stage-1 17/18] RUN if [ "true" == "false" ]; then     echo "skipping tests";    else     eval "tox";     fi
#25 sha256:4b8af72c588d9de7175d86920a2242ca71cb2d8c6f246a03bac65694aa11bd7f
#25 4.573 lint recreate: /home/app/function/.tox/lint
#25 12.32 lint installdeps: flake8
#25 46.50 lint installed: flake8==3.9.2,importlib-metadata==4.6.0,mccabe==0.6.1,pycodestyle==2.7.0,pyflakes==2.3.1,typing-extensions==3.10.0.0,zipp==3.5.0
#25 46.50 lint run-test-pre: PYTHONHASHSEED='1279681707'
#25 46.50 lint run-test: commands[0] | flake8 .
#25 49.84 0
#25 49.99 test recreate: /home/app/function/.tox/test
#25 54.88 test installdeps: flask, pytest, -rrequirements.txt
#25 119.0 test installed: attrs==21.2.0,click==8.0.1,dataclasses==0.8,Flask==2.0.1,importlib-metadata==4.6.0,iniconfig==1.1.1,itsdangerous==2.0.1,Jinja2==3.0.1,MarkupSafe==2.0.1,packaging==20.9,pluggy==0.13.1,py==1.10.0,pyparsing==2.4.7,pytest==6.2.4,toml==0.10.2,typing-extensions==3.10.0.0,Werkzeug==2.0.1,zipp==3.5.0
#25 119.0 test run-test-pre: PYTHONHASHSEED='1279681707'
#25 119.0 test run-test: commands[0] | pytest
#25 121.6 Traceback (most recent call last):
#25 121.6   File "/home/app/function/.tox/test/bin/pytest", line 5, in <module>
#25 121.6     from pytest import console_main
#25 121.6   File "/home/app/function/.tox/test/lib/python3.6/site-packages/pytest/__init__.py", line 7, in <module>
#25 121.6     from _pytest.capture import CaptureFixture
#25 121.6   File "/home/app/function/.tox/test/lib/python3.6/site-packages/_pytest/capture.py", line 548, in <module>
#25 121.6     class MultiCapture(Generic[AnyStr]):
#25 121.6   File "/home/app/function/.tox/test/lib/python3.6/site-packages/_pytest/capture.py", line 616, in MultiCapture
#25 121.6     def readouterr(self) -> CaptureResult[AnyStr]:
#25 121.6   File "/usr/local/lib/python3.6/typing.py", line 510, in inner
#25 121.6     return cached(*args, **kwds)
#25 121.6   File "/usr/local/lib/python3.6/typing.py", line 1079, in __getitem__
#25 121.6     extra=self.__extra__)
#25 121.6   File "/usr/local/lib/python3.6/typing.py", line 944, in __new__
#25 121.6     self = super().__new__(cls, name, bases, namespace, _root=True)
#25 121.6   File "/usr/local/lib/python3.6/typing.py", line 118, in __new__
#25 121.6     return super().__new__(cls, name, bases, namespace)
#25 121.6   File "/usr/local/lib/python3.6/abc.py", line 133, in __new__
#25 121.6     cls = super().__new__(mcls, name, bases, namespace)
#25 121.6 ValueError: 'out' in __slots__ conflicts with class variable
#25 121.8 ERROR: InvocationError for command /home/app/function/.tox/test/bin/pytest (exited with code 1)
#25 121.8 ___________________________________ summary ____________________________________
#25 121.8   lint: commands succeeded
#25 121.8 ERROR:   test: commands failed
#25 ERROR: executor failed running [/bin/sh -c if [ "$TEST_ENABLED" == "false" ]; then     echo "skipping tests";    else     eval "$TEST_COMMAND";     fi]: exit code: 1
------
 > [stage-1 17/18] RUN if [ "true" == "false" ]; then     echo "skipping tests";    else     eval "tox";     fi:
------
Dockerfile:38
--------------------
  37 |     ARG TEST_ENABLED=true
  38 | >>> RUN if [ "$TEST_ENABLED" == "false" ]; then \
  39 | >>>     echo "skipping tests";\
  40 | >>>     else \
  41 | >>>     eval "$TEST_COMMAND"; \
  42 | >>>     fi
  43 |     
--------------------
error: failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c if [ "$TEST_ENABLED" == "false" ]; then     echo "skipping tests";    else     eval "$TEST_COMMAND";     fi]: exit code: 1

Possible Solution

I made this change to the template on my local machine, as a test, because I noticed armhf/python:3.6 is deprecated - and was pleasantly surprised arm32v7/python:3.6-alpine worked.

# I commented this out
# FROM armhf/python:3.6-alpine

FROM arm32v7/python:3.6-alpine

Steps to Reproduce (for bugs)

1. faas-cli template store pull python3-http get the python3-http templates, if you haven't
2. faas-cli new testbot --lang python3-http-armhf to build a new testbot function
3. faas-cli publish -f testbot.yml --platforms linux/arm/v7 to try building it

Context

I am doing development on my Raspberry Pi k3s netbooting cluster running at home, switched my images to armhf, and bumped into this issue.

Changing the local template for python3-http-armhf is a good short term solution. Long term, I'm not sure if this impacts more people, and if there's something I can do to help.

I'll share on Slack to see what folks are seeing. 💯

Your Environment

• FaaS-CLI version ( Full output from: faas-cli version ): CLI:

 commit:  72816d486cf76c3089b915dfb0b66b85cf096634
 version: 0.13.13

Gateway
 uri:     http://127.0.0.1:8080
 version: 0.20.12
 sha:     a6dbb4cd0285f6dbc0bc3f43f72ceacdbdf6f227


Provider
 name:          faas-netes
 orchestration: kubernetes
 version:       0.13.4 
 sha:           6f34f27a2405798b5ee2846f1654bc7754991920

• Docker version docker version (e.g. Docker 17.0.05 ):

Client: Docker Engine - Community
 Version:           20.10.7
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        f0df350
 Built:             Wed Jun  2 11:56:38 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.7
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       b0f5bc3
  Built:            Wed Jun  2 11:54:50 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.6
  GitCommit:        d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc:
  Version:          1.0.0-rc95
  GitCommit:        b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

• Are you using Docker Swarm or Kubernetes (FaaS-netes)? Kubernetes

• Operating System and version (e.g. Linux, Windows, MacOS):

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.2 LTS
Release:        20.04
Codename:       focal

• Code example or link to GitHub repo or gist to reproduce problem: shared above, let me know if more is needed?

• Other diagnostic information / logs from troubleshooting guide the function builds with my workaround, and runs in OpenFaas.

Next steps

Let's talk. I hope I am doing something wrong.

You may join Slack for community support.

[LucasRoesler]

@kylos101 if there is a better supported / official image that we can switch to, let's do it. Keeping the old image doesn't really help if it is

1. deprecated, and
2. broken.

[alexellis]

Python 3.6 seems like a rather old release now. My Ubuntu machine has 3.8 installed.

What do you recommend @LucasRoesler?

Keeping a Debian base is better for native packages than using Alpine.

Alex

[kylos101]

Hi @alexellis @LucasRoesler,

What do you think of this as a proposed change:

# new base image, replacing FROM armhf/python:3.6-alpine
FROM python:3.8.11-slim-buster

# Add non root user
RUN addgroup --system app 
RUN APPGROUPID=$(getent group app | cut -d: -f3)
RUN adduser --system --group $APPGROUPID app

I had to add the user to the group over a series of 3 lines, instead of 1, because I needed to capture the GID.

It works on on Raspberry Pi k3s cluster. Let me know if you'd like me to submit a PR, any changes, and test on x86_64 too?

[LucasRoesler]

moving from apline to debian could be a breaking change for people. It is probably a good idea to stay on an alpine image