How do I create a job for a notebook not in the /Users/ directory?

[kpendergast]

I am setting up a job to to load data from S3 into Delta using Auto loader. I can do this fine in interactive mode. When trying to create a job in the UI. I can select the notebook in the root directory I created for the project within the create job UI however instead of it being "/abc/notebook" it uses "/Users/my_username_dir/notebook" and that doesn't exist. I have full admin privileges.

[kpendergast]

Thanks @Kaniz Fatma​  if there is a best practice for how to structure projects that would also be helpful. It just seems strange to need to have files that run streaming jobs and other parts of a pipeline live inside user directories.

[User16844513407 · Accepted solution]

Hi @Ken Pendergast​, you are supposed to be able to reference any Notebook you have the right permissions on so it looks like you are running into a bug, can you please reach out to support or email me directly with your workspace ID? My email is jan@databricks.com. Sorry for the inconvenience.

[kpendergast]

Thanks for the feedback. I had reported this to my SA Kristin and seems like it has been resolved. I just tested creating a new job on a file not in a user folder and it didnt show me an error when trying to save the task. I was not even able to save a task pointing to a file not in a user's directory.

Thanks again for responding I can now move all the job files out of my user directory 😀