Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new: usr: JUPY-567, SDK-394: Add SDK bindings for JupyterNotebookCommand #304

Merged
merged 30 commits into from
Mar 12, 2020
Merged

Conversation

tgvr
Copy link
Contributor

@tgvr tgvr commented Feb 18, 2020

Ready for review:

Usage:

qds.py --token <> --url <> jupyternotebookcmd run|submit --path <> --uuid <> --cluster-label <> --arguments ‘{“key1”:”val1”}’ --macros <> --name <> --tags <> --notify --timeout <> --retry <> --retry-delay <> --pool <>

Test Cases:
These cases have been verified using my QiB.

  • Command Returns error if notebook with given path or given uuid does not exist.
  • Command argument validation throws error if path is not given. [Atleast one of path or UUID is mandatory]
  • Command returns error if cluster with given label does not exist.
  • Verified that arguments passed are passed to the notebook.
  • Submitted commands with the other options and verified in QiB that they are passed properly.

Also, currently creating schedules from commandline is broken. I've made a small change in scheduler.py that fixes this.

  • Verified this by running $qds.py --token <> --url <> scheduler create --data ./temp.json
    temp.json contains
    { "command_type": "JupyterNotebookCommand", "command": { "path": "Users/[email protected]/m.ipynb" }, "start_time": "2019-12-26T02:00Z", "end_time": "2020-07-01T02:00Z", "frequency": 1, "time_unit": "days", "label": "spark" }

A schedule is created, verified in UI.

@tgvr tgvr marked this pull request as ready for review February 20, 2020 10:14
qds_sdk/scheduler.py Show resolved Hide resolved
qds_sdk/commands.py Outdated Show resolved Hide resolved
qds_sdk/commands.py Outdated Show resolved Hide resolved
qds_sdk/commands.py Outdated Show resolved Hide resolved
qds_sdk/commands.py Outdated Show resolved Hide resolved
@chattarajoy
Copy link
Member

@tarungavara, once you are done with the changes please rebase your branch once with the latest unreleased branch.

qds_sdk/commands.py Outdated Show resolved Hide resolved
qds_sdk/commands.py Outdated Show resolved Hide resolved
tgvr and others added 2 commits March 2, 2020 16:40
Co-Authored-By: Joy Lal Chattaraj <[email protected]>
Co-Authored-By: Joy Lal Chattaraj <[email protected]>
@chattarajoy chattarajoy force-pushed the unreleased branch 2 times, most recently from 8a95947 to caec9b1 Compare March 2, 2020 13:09
chattarajoy
chattarajoy previously approved these changes Mar 4, 2020
@tgvr tgvr changed the title new: usr: JUPY-567: Add SDK bindings for JupyterNotebookCommand new: usr: JUPY-567, SDK-394: Add SDK bindings for JupyterNotebookCommand Mar 4, 2020
chattarajoy
chattarajoy previously approved these changes Mar 11, 2020
qds_sdk/commands.py Outdated Show resolved Hide resolved
qds_sdk/commands.py Outdated Show resolved Hide resolved
chattarajoy
chattarajoy previously approved these changes Mar 12, 2020
@chattarajoy
Copy link
Member

changes LGTM, @tarungavara please let me know when the changes are tested and ready to be merged.

@chattarajoy
Copy link
Member

chattarajoy commented Mar 12, 2020

Also, please check and fix any relevant styling issues here https://deepsource.io/gh/qubole/qds-sdk-py/run/548529df-5275-42be-9a57-15a7830b5c02/python/

@tgvr
Copy link
Contributor Author

tgvr commented Mar 12, 2020

@chattarajoy This is now ready to merge.

@chattarajoy chattarajoy merged commit bb8e5ca into qubole:unreleased Mar 12, 2020
chattarajoy added a commit that referenced this pull request Mar 18, 2020
…and (#304)

* changes

* changes

* changes

* add tests

* fix tests

* changes to make scheduler create from command line work

* Remove non ASCII characters

Co-Authored-By: Joy Lal Chattaraj <[email protected]>

* Remove redundant parenthesis

Co-Authored-By: Joy Lal Chattaraj <[email protected]>

* Update qds_sdk/commands.py

Co-Authored-By: Joy Lal Chattaraj <[email protected]>

* changes

* changes

* add support for macros

* add name option

* add support for tags

* add support for notify and timeout

* update tests

* add support for pool

* update tests

* add support for retry and retry_delay

* update tests

* add support for jupyter_notebook_id

* update tests

* replace notebook id with uuid

* remove support for uuid

* seperate validate json input method

* fix style

* some more style fixes

* changes

Co-authored-by: Gavara Tarun <[email protected]>
Co-authored-by: Joy Lal Chattaraj <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants