Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Pyspark reduction does not preserve names root names with nw.all() #1780

Open
FBruzzesi opened this issue Jan 10, 2025 · 0 comments · May be fixed by #1787
Open

[Bug]: Pyspark reduction does not preserve names root names with nw.all() #1780

FBruzzesi opened this issue Jan 10, 2025 · 0 comments · May be fixed by #1787

Comments

@FBruzzesi
Copy link
Member

Describe the bug

Pyspark reductions do not preserve original root name when applied to nw.all().

I am tempted to say that the issue is in this line. However I could not manage to fix it without breaking the code elsewhere so far 🙈

Steps or code to reproduce the bug

Running:

df.select(nw.all().n_unique())

with a lazyframe backed by pyspark DataFrame, will result in a column named (count(DISTINCT a) + max(CAST((a IS NULL) AS INT))).

Expected results

Original names to be preserved

Actual results

Pyspark expression names

Please run narwhals.show_version() and enter the output below.

Latest

Relevant log output

No response

@camriddell camriddell linked a pull request Jan 10, 2025 that will close this issue
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant