Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot load dates outside of pandas nanosecond timestamp range with load_parquet #441

Closed
tswast opened this issue Dec 3, 2021 · 1 comment · Fixed by #442
Closed

Cannot load dates outside of pandas nanosecond timestamp range with load_parquet #441

tswast opened this issue Dec 3, 2021 · 1 comment · Fixed by #442
Assignees
@tswast
Labels
api: bigquery Issues related to the googleapis/python-bigquery-pandas API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@tswast
Copy link
Collaborator

tswast commented Dec 3, 2021

googleapis/python-db-dtypes-pandas#45

Possibly we need a fallback for out-of-range values that converts to object dtype of datetime.date instead? Or a pandas series containing pyarrow date scalars?
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery-pandas API. label Dec 3, 2021
@tswast tswast added type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. and removed api: bigquery Issues related to the googleapis/python-bigquery-pandas API. labels Dec 3, 2021
@tswast tswast mentioned this issue Dec 3, 2021
Merged
4 tasks
@tswast
Copy link
Collaborator Author

tswast commented Dec 3, 2021

Catching the out-of-bounds and fallback to object seems to make the most sense. Possibly we don't even need to manually convert all the rows and just leave the data as-is for these? It'd mean folks couldn't write strings to DATE when in that far-out range, but maybe that's OK?

@tswast tswast self-assigned this Dec 3, 2021
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery-pandas API. label Dec 4, 2021
@tswast tswast mentioned this issue Dec 6, 2021
Merged
4 tasks
gcf-merge-on-green bot pushed a commit that referenced this issue Dec 7, 2021
@tswast
fix: allow extreme DATE values such as datetime.date(1, 1, 1) in `l…
e13abaf 
…oad_gbq` (#442)

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
- [ ] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-bigquery-pandas/issues/new/choose) before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)

Fixes #441 
Towards #365 
🦕
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tswast tswast

Labels
api: bigquery Issues related to the googleapis/python-bigquery-pandas API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Milestone
No milestone
1 participant
@tswast