Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH/DOC: update pandas-gbq signature and docstring #20564

Merged
merged 13 commits into from
Apr 9, 2018

Conversation

tswast
Copy link
Contributor

@tswast tswast commented Mar 30, 2018

Delegates more of the behavior and documentation for to_gbq and
read_gbq methods to the pandas-gbq library. This duplicate
documentation was getting out of sync.

Please include the output of the validation script below between the "```" ticks:

$ scripts/validate_docstrings.py pandas.DataFrame.to_gbq

################################################################################
##################### Docstring (pandas.DataFrame.to_gbq)  #####################
################################################################################

Write a DataFrame to a Google BigQuery table.

This function requires the `pandas-gbq package
<https://pandas-gbq.readthedocs.io>`__.

See: :meth:`pandas_gbq.to_gbq`

################################################################################
################################## Validation ##################################
################################################################################

Errors found:
	Errors in parameters section
		Parameters {'kwargs', 'args'} not documented
	No returns section found
	See Also section not found
	No examples section found
(3.6.2/envs/pandas-dev)
# swast @ swast-macbookpro2 in ~/src/pandas/pandas on git:master o [16:56:32] C:5
$ scripts/validate_docstrings.py pandas.io.gbq.read_gbq

################################################################################
###################### Docstring (pandas.io.gbq.read_gbq) ######################
################################################################################

Load data from Google BigQuery.

This function requires the `pandas-gbq package
<https://pandas-gbq.readthedocs.io>`__.

See :meth:`pandas_gbq.read_gbq`.

Returns
-------
df: DataFrame
    DataFrame representing results of query

################################################################################
################################## Validation ##################################
################################################################################

Errors found:
	Errors in parameters section
		Parameters {'kwargs', 'args'} not documented
	See Also section not found
	No examples section found
(3.6.2/envs/pandas-dev)
# swast @ swast-macbookpro2 in ~/src/pandas/pandas on git:master o [16:56:39] C:4
$ scripts/validate_docstrings.py pandas.DataFrame.to_gbq

################################################################################
##################### Docstring (pandas.DataFrame.to_gbq)  #####################
################################################################################

Write a DataFrame to a Google BigQuery table.

This function requires the `pandas-gbq package
<https://pandas-gbq.readthedocs.io>`__.

See: :meth:`pandas_gbq.to_gbq`

################################################################################
################################## Validation ##################################
################################################################################

Errors found:
	Errors in parameters section
		Parameters {'args', 'kwargs'} not documented
	No returns section found
	See Also section not found
	No examples section found

If the validation script still gives errors, but you think there is a good reason
to deviate in this case (and there are certainly such cases), please state this
explicitly.

Validation script gives errors This PR removes documentation which is duplicated in the pandas-gbq docs and add intersphinx links to the relevant methods.

Checklist for other PRs (remove this part if you are doing a PR for the pandas documentation sprint):

  • closes #xxxx
  • tests added / passed
  • passes git diff upstream/master -u -- "*.py" | flake8 --diff
  • whatsnew entry

Delegates more of the behavior and documentation for `to_gbq` and
`read_gbq` methods to the `pandas-gbq` library. This duplicate
documentation was getting out of sync.
@codecov
Copy link

codecov bot commented Mar 31, 2018

Codecov Report

Merging #20564 into master will decrease coverage by <.01%.
The diff coverage is 50%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #20564      +/-   ##
==========================================
- Coverage   91.82%   91.81%   -0.01%     
==========================================
  Files         153      153              
  Lines       49256    49265       +9     
==========================================
+ Hits        45227    45235       +8     
- Misses       4029     4030       +1
Flag Coverage Δ
#multiple 90.2% <50%> (-0.01%) ⬇️
#single 41.9% <50%> (-0.01%) ⬇️
Impacted Files Coverage Δ
pandas/io/gbq.py 25% <50%> (ø) ⬆️
pandas/core/frame.py 97.15% <50%> (ø) ⬆️
pandas/plotting/_core.py 82.39% <0%> (-0.12%) ⬇️
pandas/util/_test_decorators.py 92% <0%> (+0.1%) ⬆️
pandas/util/testing.py 84.73% <0%> (+0.2%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6d610a4...1ab0934. Read the comment docs.

@jreback
Copy link
Contributor

jreback commented Apr 1, 2018

can we just update the signature (of course can have the link as well). It just provides nice tab completeion and doc-strings if its in-line.

@pep8speaks
Copy link

pep8speaks commented Apr 2, 2018

Hello @tswast! Thanks for updating the PR.

Cheers ! There are no PEP8 issues in this Pull Request. 🍻

Comment last updated on April 09, 2018 at 08:39 Hours UTC

@tswast
Copy link
Contributor Author

tswast commented Apr 2, 2018

@jreback Good point. I've updated the PR to just add the links to the pandas-gbq docs and do some light clean-ups based on scripts/validate_docstrings.py warnings.

I've built the docs locally and verified that the intersphinx links in the See Also section do render correctly.

@tswast
Copy link
Contributor Author

tswast commented Apr 3, 2018

@jreback Was this what you meant? I've reverted the function signatures back to the original (but added **kwargs for some new args added in newer versions of pandas-gbq).

Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

general philosophy here is simply to update the pandas signature for gbq when they change in impl

it’s not going to be that often

"""
authentication (eg. Jupyter/IPython notebook on remote host).
kwargs : dict
Arbitrary keyword arguments.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if these are now official kwargs then add them

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in cb178d9

@@ -1143,33 +1142,57 @@ def to_gbq(self, destination_table, project_id, chunksize=10000,

Parameters
----------
dataframe : DataFrame
DataFrame to be written
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this needs to stay

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The validation script was complaining about this one. I think because this is a method of DataFrame, so there is no dataframe argument. (I also tried self and it complained about that, too.)

################################################################################
################################## Validation ##################################
################################################################################

Errors found:
	Errors in parameters section
		Unknown parameters {'dataframe'}
		Parameter "kwargs" description should finish with "."
	No returns section found
	Missing description for See Also "pandas_gbq.to_gbq" reference
	No examples section found

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh don’t worry about that too much

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @tswast is correct that this should be removed. The method does not take a dataframe as input (it is writing self to gbq)

The main method a user calls to export pandas DataFrame contents to
Google BigQuery table.
def to_gbq(
self, destination_table, project_id, chunksize=10000,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you keep this on the previous line as it was? (to keep consistency, we don't use this style anywhere else in the file for def lines)

@@ -1144,32 +1143,58 @@ def to_gbq(self, destination_table, project_id, chunksize=10000,
Parameters
----------
dataframe : DataFrame
DataFrame to be written
DataFrame to be written to Google BigQuery.
destination_table : string
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

string -> str

destination_table : string
Name of table to be written, in the form 'dataset.tablename'
Name of table to be written, in the form 'dataset.tablename'.
project_id : str
Google BigQuery Account project ID.
chunksize : int (default 10000)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"int (default 10000)" -> "int, default 10000"

(and same comment for other similar cases below)

"""
authentication (eg. Jupyter/IPython notebook on remote host).
auth_local_webserver : boolean (default False)
Use the [local webserver flow] instead of the [console flow]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think those [..] need a trailing underscore to make them into actual links?

(you can test with python doc/make.py --single DataFrame.to_gbq)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 37b2a08

Also, thanks for the test command. Much faster than trying to build the whole docs set!

Also, adjust line breaks per review comments.
@tswast
Copy link
Contributor Author

tswast commented Apr 5, 2018

I've addressed the comments and CI is now passing again. Is there anything left to do on this PR?

@jreback
Copy link
Contributor

jreback commented Apr 5, 2018

can u add a note in whatsnew (documentation section is ok)
otherwise lgtm

@tswast
Copy link
Contributor Author

tswast commented Apr 6, 2018

Okay. Added to whatsnew. (And made a couple tweaks for recently released of Pandas-GBQ)

@jorisvandenbossche jorisvandenbossche changed the title DOC: intersphinx to pandas-gbq ENH/DOC: update pandas-gbq signature and docstring Apr 9, 2018
Copy link
Member

@jorisvandenbossche jorisvandenbossche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comment

@@ -924,6 +924,9 @@ read the `NumFOCUS blogpost`_ recapping the sprint.
to functions, methods and classes.
(:issue:`18941`, :issue:`18948`, :issue:`18973`, :issue:`19017`)
- Added a reference to :func:`DataFrame.assign` in the concatenate section of the merging documentation (:issue:`18665`)
- Updated ``to_gbq`` and ``read_gbq`` documentation to reflect changes from
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe "documentation" -> "signature and documentation" (as it is not only a doc change ?)
and move it to "other enhancements" section

Copy link
Member

@jorisvandenbossche jorisvandenbossche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quickly addressed it myself

@jorisvandenbossche jorisvandenbossche added this to the 0.23.0 milestone Apr 9, 2018
@jorisvandenbossche jorisvandenbossche merged commit 2431641 into pandas-dev:master Apr 9, 2018
@jorisvandenbossche
Copy link
Member

@tswast Thanks!

@tswast
Copy link
Contributor Author

tswast commented Apr 9, 2018

Thanks @jorisvandenbossche 🎉

@tswast tswast deleted the gbq-docs branch April 9, 2018 15:51
@jbrockmendel
Copy link
Member

can/should we remove "verbose"?

@tswast
Copy link
Contributor Author

tswast commented Dec 10, 2019

can/should we remove "verbose"?

It hasn't had any effect in pandas-gbq since 2018-04-03. Makes sense to remove in 1.0.

@jbrockmendel
Copy link
Member

@tswast great, thanks. how about private_key?

@tswast
Copy link
Contributor Author

tswast commented Dec 11, 2019

I'm a little more hesitant on private_key, as I never actually removed the code for it in pandas-gbq. Filed googleapis/python-bigquery-pandas#301 to start throwing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants