Job creation in Hopsworks

Hopsworks
#1

Hi,
I am using the demo version of Hopsworks. I want to create a job to perform transformations on data before creating feature group. I am facing two below issues:

  1. I created a python script (.ipynb) and fed to the job, I wasn’t asked to add argument while creating job and when I try to run it, it asks for argument not sure why and what should I pass in.

  2. There is a job already created in Hopsworks with a .jar file and running that fails as well.

#2

Hi,
The job asks for argument when you run it so that you can run the same job with different arguments. You only need to pass an argument to a job if the job needs an argument.
For the failed job I need more info. What did it fail on? Did you get any logs?

/Ermias

#3

So, I am still trying to understand on the first issue that I shared: Will a python script for jobs not work in a demo version?

And for the logs, I haven’t created any job separately, I am going to the featurestore_tour_job and running that is failing. Is there something that am missing to run it or should I be passing any specific argument?

#4

Python script for jobs should work. As for the featurestore_tour_job it creates feature groups and training datasets, so it will fail if you try to run the same job more than once because the feature groups and training datasets already exists.

#5

I ran two python jobs and both failed. Please find the steps I followed and the error message below:

  1. Python job for data transformation -
  • I uploaded two csv files in resources folder and test.ipynb file in test_jobs folder under datasets section on the older version of java (not 2.5). The ipynb file referred to the csv file with path - “Resources/filename.csv”
  • Created the job by selecting the .ipynb file uploaded in test_job folder
  • I ran the job, it asked for argument but there was no argument required so, I left it blank and clicked “OK”. The job failed with error log: io.hops.hopsworks.exceptions.ServiceException
    io.hops.hopsworks.exceptions.ServiceException
  1. Python job to print “Hello World” -
  • I uploaded the .ipynb file in test_jobs folder. The file had code: print(“Hello World”)
  • Created the job by selecting the .ipynb file uploaded in test_job folder
  • I ran the job, it asked for argument but there was no argument required so, I left it blank and clicked “OK”. The job failed with error log:
    org.json.JSONException: No value for kernelspec
    org.json.JSONException: No value for kernelspec

I am not able to understand where is this going wrong.

#6

Can someone please respond to this?