AWS Lambda REST throws an exception in native mode

[iabughosh]

Describe the bug

I have created a Lambda Service and deployed successfully to AWS after following this guide, in JVM mode everything works fine but in native mode all REST APIs are failing when I run it through sam-local and it is also failing with same exception when I deployed it to AWS, logs from AWS is :
Mounting /tmp/tmpq2pz2roc as /var/task:ro,delegated inside runtime container START RequestId: 77fbefd3-535a-4022-a84e-7753a4e4c7a3 Version: $LATEST /var/task/bootstrap: /lib64/libc.so.6: versionGLIBC_2.32' not found (required by /var/task/bootstrap)
/var/task/bootstrap: /lib64/libc.so.6: version GLIBC_2.34' not found (required by /var/task/bootstrap) 31 May 2022 20:13:25,459 [ERROR] (rapid) Init failed InvokeID= error=Runtime exited with error: exit status 1 /var/task/bootstrap: /lib64/libc.so.6: version GLIBC_2.32' not found (required by /var/task/bootstrap)
/var/task/bootstrap: /lib64/libc.so.6: version GLIBC_2.34' not found (required by /var/task/bootstrap) END RequestId: a0170ba7-b7d0-405b-ad87-77b8b3aef629 REPORT RequestId: a0170ba7-b7d0-405b-ad87-77b8b3aef629 Init Duration: 0.19 ms Duration: 3.89 ms Billed Duration: 4 msMemory Size: 128 MB Max Memory Used: 128 MB Invalid lambda response received: Lambda response must be valid json 2022-05-31 23:13:25 127.0.0.1 - - [31/May/2022 23:13:25] "GET /hello/rest HTTP/1.1" 502 -

Kindly note that the API that I am hitting returns application/json response.

Expected behavior

API should return a JSON and not giving 5xx error.

Actual behavior

{
"message": "Internal server error"
}

How to Reproduce?

1. Clone producer project from Github under URL : https://github.com/iabughosh/lambda-producer
2. do " mvn clean package -Dnative"
3. sam local start-api --template target/sam.native.yaml --port 3100
4. curl -XGET localhost:3100/hello/rest | jq

Output of uname -a or ver

Linux pop-os 5.17.5-76051705-generic #202204271406165344057622.04~6277a18 SMP PREEMPT Wed May 25 01 x86_64 x86_64 x86_64 GNU/Linux

Output of java -version

OpenJDK 64-Bit Server VM Corretto-17.0.1.12.1 (build 17.0.1+12-LTS, mixed mode, sharing)

GraalVM version (if different from Java)

Version info: 'GraalVM 22.1.0.0-Final Java 17 Mandrel Distribution'

Quarkus version or git rev

2.9.2.Final

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.6.3

Additional information

Extra observation : even in JVM mode, if I add smallrye-health & smallrye-openapi extension, lambda deployed successfully but return out of memory exception for each request.

[quarkus-bot]

/cc @Karm, @galderz, @matejvasek, @patriot1burke, @zakkak

[galderz]

Maybe you need a newer Linux version for running native on lambda? According to this doc Amazon Linux 2022 comes with glibc 2.34.

A similar issue (for a slightly different glibc version) can be found in the aws-lambda-rust-runtime project.

Also, no idea why those log messages mention 2 different glibc versions: GLIBC_2.34' not found and GLIBC_2.32' not found

[iabughosh]

I am facing this error also on my local SAM api-server. How can I control AWS SAM Lambda version ? I installed latest SAM and facing the above error.

[Karm]

Hello, this is the same issue as #23998

Basically trying to compile (and dynamically link) and older GLIBC and running on a newer one is O.K., while to opposite is not.

e.g. you cannot build on the latest Ubuntu 20+ GH action and expect the binary to run on a Centos 7/8-like system in AWS Linux env.

If you describe your build env, we can help you adjust the system.

For example, I need a GitHub action to produce a binary that would automatically deploy to my CentOS 8 server. I use Ubuntu 18 (i.e. runs-on: ubuntu-18.04) as the glibc is compatible then. If I let it run with Ubuntu 20, it won't work once deployed to my CentOS 8 server.

[Karm]

Oh, @galderz beat me to the response. I am glad we don't contradict each other :-D It seems we replied basically the same thing.

[iabughosh]

I see. Let me dig deeper into this and come back to you, thanks!

[iabughosh]

Thanks for the hints team! I managed to overcome this by using Mandrel image builder instead of depending on my OS and my Installed Mandrel GraalVM, I used the below command :

./mvnw -Dnative clean package -Dquarkus.native.container-build=true -Dquarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-mandrel:22.0.0.2-Final-java11