Enabling versioning for jupyter notebooks

Hopsworks.ai
#1

Hello,

I see there is a new git versioning feature available on the new release. But I am not sure how I can work on the cloned repo while the actual Jupiter notebook file is located in HDFS. Do I have to use HDFS copy to local and push my the file?

If you also direct me to a document that shows the steps it will be super helpful. Thank you.

#2

Hi.

When you select Git support for Jupyter, notebooks will not be stored in HopsFS, thus you won’t be able to see them in Datasets browser.

You can follow this guide on how to use it https://hopsworks.readthedocs.io/en/latest/user_guide/hopsworks/jupyter.html#jupyter-notebooks-with-version-control

#3

Thanks @antonios. I am able to setup git versioning for Jupyter notebooks. Everything works fine using CLI but git push from UI results in the following error. From the CLI I have to provide my username and token on prompt every time I push to the remote repo. Using SSH could solve the problem but I am currently using the personal access token option.

Git Push
Git Push failed with error:
fatal: could not read Username for 'https://github.com': terminal prompts disabled
#4

I followed the instructions I sent you and I didn’t encounter the problem you mentioned.

In case you missed it, you should use the HTTP web url of GitHub.
Also, watch the GIF in the instructions to get the feeling of how it works :wink:

Let me know if you’ve figured out what’s wrong or if you still have a problem.

#5

I followed the following instructions on stack overflow and it worked after that. The downside is I have to store the api key in disk for the password prompt.

Permanently authenticating with Git repositories,

Run following command to enable credential caching.

$ git config credential.helper store
$ git push https://github.com/repo.git

Username for 'https://github.com': <USERNAME>
Password for 'https://USERNAME@github.com': <PASSWORD> 
Use should also specify caching expire,

git config --global credential.helper 'cache --timeout 7200'
After enabling credential caching, it will be cached for 7200 seconds (2 hour).

Note: Credential helper storing unencrypted password on local disk.

Stack Overflow link

#6

Hi @atlantis.

To be honest I don’t really understand how you ended up there. Could you enumerate the steps you took to try reproduce it?

Cheers