Databricks Community

accessing power bi dataset using MDX query using windows is working but the same not working using python Linux server.

trying to access the SSAS POIWER BI dataset using MDX query from python LInux server. We are hitting roadblock. The existing setup works as expected in windows system due to adodb.dll but unable to connect in Linux. Any help would be much appreciated...

Hello Team, I am not able to find the bookstore dataset in Databricks. Please guide me to how to download this dataset

PySpark job with joins & write parquet operation fails with FetchFailedException

I'm working on a task where I transform a dataset and re-save it to an S3 bucket. This involves joining the dataset to two others, dropping fields from the initial dataset which overlapped with fields from the other two, hashing certain fields with p...

Not able to install the AIRBNB dataset when trying to run in the notebook-"Scalable ML". I am getting the error as below-:AnalysisException: Path does not exist:

file_path = f"{datasets_dir}/airbnb/sf-listings/sf-listings-2019-03-06-clean.parquet/"2airbnb_df = spark.read.format("parquet").load(file_path)3​4display(airbnb_df)AnalysisException: Path does not exist: dbfs:/user/nniha9188@gmail.com/dbacademy/machi...

Converting from RDD to Dataset, and unit test takes 3x slower. (but prod is faster)

I converted a data job fro RDD to Dataset, and I've found that, in prod, the data job runs faster, which is nice.But unit test runs 3x slower than before.My best guess is that Dataset spends time doing a lot of stuffs like encoding, optimizing, query...

After several iteration of filter and union, the data is bigger than spark.driver.maxResultSize

The process for me to build model is:filter dataset and split into two datasetsfit model based on two datasets union two datasetsrepeat 1-3 stepsThe problem is that after several iterations, the model fitting time becomes longer dramatically, and the...

How large should a dataset be so that it’s worth using Spark?

How large is considered a “large” dataset to put on the driver node?

java.io.IOException: No FileSystem for scheme: null

Getting the error when try to load the uploaded file in py notebook.# File location and type file_location = "//FileStore/tables/data/d1.csv" file_type = "csv" # CSV options infer_schema = "true" first_row_is_header = "false" delimiter = ","# The app...