cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

mjbobak
by New Contributor III
  • 12536 Views
  • 5 replies
  • 9 kudos

Resolved! How to import a helper module that uses databricks specific modules (dbutils)

I have a main databricks notebook that runs a handful of functions. In this notebook, I import a helper.py file that is in my same repo and when I execute the import everything looks fine. Inside my helper.py there's a function that leverages built-i...

  • 12536 Views
  • 5 replies
  • 9 kudos
Latest Reply
amitca71
Contributor II
  • 9 kudos

Hi,i 'm facing similiar issue, when deploying via dbx.I have an helper notebook, that when executing it via jobs works fine (without any includes)while i deploy it via dbx (to same cluster), the helper notebook results withdbutils.fs.ls(path)NameEr...

  • 9 kudos
4 More Replies
Christine
by Contributor
  • 19531 Views
  • 4 replies
  • 1 kudos

Resolved! Is it possible to import functions from a module in Workspace/Shared instead of Repos?

Hi,I am considering creating libraries for my databricks notebooks, and found that it is possible to import functions from modules saved in repos. Is it possible to move the .py files with the functions to Workspace/Shared and still import functions ...

  • 19531 Views
  • 4 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Christine Pedersen​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell ...

  • 1 kudos
3 More Replies
kll
by New Contributor III
  • 2157 Views
  • 2 replies
  • 0 kudos

AttributeError: module 'pyspark.dbutils' has no attribute 'fs'

`AttributeError` when attempting to transfer files from `dbfs` filestore in DataBricks to a local directory.  import pyspark.dbutils as pdbutils   pdbutils.fs.cp("/dbfs/Data/file1.csv", "/Users/Downloads/")   Traceback (most recent call last):   ...

  • 2157 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Keval Shah​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers yo...

  • 0 kudos
1 More Replies
absolutelyRice
by New Contributor III
  • 4135 Views
  • 5 replies
  • 2 kudos

Resolved! Databricks Terraform Provider Issues Passing Providers to Child Modules

I have been following the documentation on the terraform databricks documentation in order to provision account level resources on AWS. I can create the workspace fine, add users, etc... However, when I go to use the provider in non-mws mode, I am re...

  • 4135 Views
  • 5 replies
  • 2 kudos
Latest Reply
absolutelyRice
New Contributor III
  • 2 kudos

So the answer to this was that you need to explicitly pass the provider argument to each of the data resources blocks. The docs should be updated to accommodate that. ​i.e. data "databricks_spark_version" "latest" { provider = databricks.workspace ...

  • 2 kudos
4 More Replies
tariq
by New Contributor III
  • 4188 Views
  • 4 replies
  • 0 kudos

Importing python module

I'm not sure how a simple thing like importing a module in python can be so broken in such a product. First, I was able to make it work using the following:import sys sys.path.append("/Workspace/Repos/Github Repo/sparkling-to-databricks/src") from ut...

  • 4188 Views
  • 4 replies
  • 0 kudos
Latest Reply
KrishZ
Contributor
  • 0 kudos

I too wonder the same thing. How can importing a python module be so difficult and not even documented lol.No need for libraries..Here's what worked for me..Step1: Upload the module by first opening a notebook >> File >> Upload Data >> drag and drop ...

  • 0 kudos
3 More Replies
bluetail
by Contributor
  • 11755 Views
  • 6 replies
  • 5 kudos

Resolved! ModuleNotFoundError: No module named 'mlflow' when running a notebook

I am running a notebook on the Coursera platform.my configuration file, Classroom-Setup, looks like this:%python   spark.conf.set("com.databricks.training.module-name", "deep-learning") spark.conf.set("com.databricks.training.expected-dbr", "6.4")   ...

  • 11755 Views
  • 6 replies
  • 5 kudos
Latest Reply
User16753724663
Valued Contributor
  • 5 kudos

Hi @Maria Bruevich​ ,From the error description, it looks like the mlflow library is not present. You can use ML cluster as these type of cluster already have mlflow library. Please check the below document:https://docs.databricks.com/release-notes/r...

  • 5 kudos
5 More Replies
Labels