Maintenance: Change the project.toml to restrict Python 3.10 version #1404
Comments
leandrodamascena
added
internal
|
Thanks for opening your first issue here! We'll come back to you as soon as we can. |
|
Locally I’m running Python 3.10 and I install the powertools via pip. When I deploy through SAM I’m using the layer. I don’t think we should limit being able to install this package when running 3.10. |
|
Hi @jplock, this is really an excellent point for us to debate here. It really works to use Python 3.10 to install via pip or even use the source code of this project, I've done some tests here and it's working "fine". But I think we should pay attention to a few points: 1 - Ok we have Python 3.10 installed and the project working, but we must remember that the highest version allowed by the Lambda Runtime is 3.9. So using local Python 3.10 (even via SAM) to run on the Lambda 3.9 runtime can cause some issues. 2 - Many good things were added in Python 3.10 and of course they are not supported by Python 3.9. Like PEP-0634 and PEP-0604 3 - Now imagine a scenario where a contributor wants to submit a PR for the project, but the contributor has Python 3.10 installed in the local environment and writes code supported only by Python 3.10. When the contributor submits the PR, all pipelines will fail because this project uses matrix to build a version and Python 3.10 isn't supported. 4 - Some examples that work in Python 3.10 and doesn't work in Python 3.9 So based on these facts I think it's protection/better governance to add this restriction to the Python version. I'd like to hear your point of view to get things I'm missing. |
|
I understand the points but who is our target audience: people who will be submitting changes to the library or people who want to use the library? I think people using the library will be more common and we shouldn’t limit how it can be installed if they happen to use the latest available Python release on their local machines. Maybe we could check with the Lambda runtime team on when 3.10 support is coming? If it’s soon, maybe we skip this. |
|
I agree with you that people using the library is the most common situation. But in my opinion even those people shouldn't be affected because the purpose of this project is to help projects using AWS Lambda and those projects must use a Python version below 3.10 because Lambda Runtime doesn't support 3.10+. Yessss!! Talking to the Lambda runtime team can be awesome as they certainly have a roadmap and can provide more information. Btw, there is an issue aws/aws-lambda-base-images#31 where people are asking for this feature/support. |
|
We'll look into this for next week's release, as we had some funky expressions with Poetry before. I appreciate everyone's point of view. For correctness, and specially given the number of breaking changes in 3.10+, we should update our support to 3.9 only. Lambda will take some time to provide support for 3.10. When they do, it'll take us some time to address some difficult typing breakage too. |
|
What about those of us running Due to the container runtime I could be running |
|
I didn't see that coming. Sometimes I forget about OCI because our tenets track managed runtimes, and that's a blind spot. This argument is valid and we shouldn't do it since we don't know how many would be impacted; classic Hyrum's Law. Here's where I'd like help from everyone - how do we best communicate that we don't make guarantees 3.10+ will work? This applies for any future Python version not yet supported as a Lambda managed runtime. |
If I didn't come across this issue today, I wouldn't have known 3.10 was unsupported. I've looked at
Whereas poetry seems to assume that the lambdaric:
you:
I think Similarly, I think it would be a shame for this library to actually put an upper limit on the python version. Some suggestions:
You mentioned earlier:
Are these breaks tracked anywhere? Searching the issues for "3.10" doesn't come up with much and I've been humming along on 3.10 without any problems until I came across this issue today. Most likely I'm using areas of the lib that aren't affected so it would be good to get a better understanding if you can point me at anything to read. Cheers:) |
|
That’s super helpful - I’ll read it more carefully this week. For the
breaking changes, I meant Python 3.10 and 3.11 breaking typing and some
other minor I can’t remember by heart now (e.g., Callable typing).
For our breaking changes (v2), the plan is to create an issue and discuss
with the community what cuts should we made as part of dropping 3.6 support
- I want it to be minimal, and strive for anything major for 3.0 as we aim
for modularisation.
…On Sun, 7 Aug 2022 at 13:26, Peter Schutt ***@***.***> wrote:
classic Hyrum's Law
one of the best <https://xkcd.com/1172/>
If I didn't come across this issue today, I wouldn't have known 3.10 was
unsupported.
I've looked at awslambdaric as an example, they have pretty much the same
classifiers as you, and the same python version specified, but because they
use setuptools their classifiers are packaged verbatim:
[image: image]
<https://user-images.githubusercontent.com/20659309/183286468-5ddbcdda-d066-4218-9d5c-fdc269330df6.png>
Whereas poetry seems to assume that the 3.10 classifier is an omission if
the python dependency specification includes 3.10. Skimming over the
docs, I couldn't see a way to override it either and I'm not sure how I
feel about that. Still though, even if the classifier could be rectified,
the Requires: Python ... still includes 3.10 in either case:
lambdaric:
[image: image]
<https://user-images.githubusercontent.com/20659309/183287282-680125af-83dd-432b-b8ca-4514538ab68d.png>
you:
[image: image]
<https://user-images.githubusercontent.com/20659309/183287297-d9d7f6ef-29ed-44ea-9f8e-da48e08da8be.png>
I think awslambdaric is a good example though, they pretty prominently
state they only support up to 3.9 in their readme, but they don't go so far
as to actually restrict 3.10+ and a github search including "awslambdaric"
and "3.10" brings up plenty of dockerfiles combining the two.
Similarly, I think it would be a shame for this library to actually put an
upper limit on the python version.
Some suggestions:
1. State the version support policy prominently - those tenets are
right at the bottom (I'd never actually made it that far down the homepage
😬).
2. Strengthen the wording of the first tenet. IMO it would have been a
pretty narrow interpretation of the words "optimise for" to go ahead and
exclude python 3.10 from support altogether.
3. You could include a compatibility section of the docs that
summarises known version incompatibilities and links to tracking issues.
4. Perhaps a runtime warning on first import if not a supported
version - maybe a bit extreme though.
5. If you decide you do want to cap the upper limit in line with the
supported runtimes, you could accept that the 3.10 ship has sailed and make
the python dependency >=3.6,<=3.10 - that way you wouldn't break
anyone now, and wouldn't be forced into 3.11 support when it becomes GA (or
whenever poetry decides to add the classifier for you).
You mentioned earlier:
... and specially given the number of breaking changes in 3.10+ ...
Are these breaks tracked anywhere? Searching the issues for "3.10"
<https://github.com/awslabs/aws-lambda-powertools-python/issues?q=is%3Aissue+3.10>
doesn't come up with much and I've been humming along on 3.10 without any
problems until I came across this issue today. Most likely I'm using areas
of the lib that aren't affected so it would be good to get a better
understanding if you can point me at anything to read.
Cheers:)
—
Reply to this email directly, view it on GitHub
<#1404 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAZPQBE4SVTR2LARPAOOBF3VX6MOTANCNFSM55LMNJGQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Ohhh 😮.. I think that's a great argument for not restricting the Python version and I didn't think about this point either. Right now this project could be running on thousands of containers using Python 3.10+ and this could have a big impact. 👏 |
|
I couldn't carve bandwidth to look into suggestions by @peterschutt but I've just closed the PR to signal our intent to not go ahead with the change given the customer impact. I'm inclined to close on the basis that any customer using OCI could use a newer Python version that might work, and sometimes it could be a minor change to allow customers benefit from faster versions. Our first tenet covers the intent well regardless. The only minor change I'd think we should make is to update our top note pointing customers to our Tenets (🤦 noticed we haven't updated to add .NET yet!)
|
|
Many thanks from those of us living on the edge:) One month till 3.11! |
|
Summary
The latest version of the Lambda runtime for Python is 3.9 and this project must have the same restriction.
The way project.toml is configured, the package on PyPi adds Python 3.10 as a supported version.
Why is this needed?
Keep the same python versions supported by Lambda Runtime
Which area does this relate to?
Governance, Other
Solution
No response
Acknowledgment
The text was updated successfully, but these errors were encountered: