- 1051 Views
- 0 replies
- 0 kudos
MLflow CVE-2023-6015 (vulnerabilities)
Hi,what mlflow version is used for the "Databricks Managed MLflow"Is there a workarround for MLflow CVE-2023-6015 (vulnerabilities) ?RegardsTimo
- 1051 Views
- 0 replies
- 0 kudos
Forum Posts
- 1318 Views
- 0 replies
- 0 kudos
run spark-sql-perf from a notebook
I am looking to run spark-sql-perf on a databricks cluster I am not sure how to create a session that add spark-sql-perf jar dependencies
- 1318 Views
- 0 replies
- 0 kudos
- 1701 Views
- 1 replies
- 0 kudos
Internal error: Attach your notebook to a different compute or restart the current compute.
I am currently using a personal computer cluster [13.3 LTS (includes Apache Spark 3.4.1, Scala 2.12)] on GCP attached to a notebook. After running a few command lines without an issue, I end up getting this errorInternal error. Attach your notebook t...
- 1701 Views
- 1 replies
- 0 kudos
- 1493 Views
- 1 replies
- 1 kudos
Resubscribing / Deleting and recreating an account
Hello,at some point I tested Databricks for a potential customer and, after the test, I cancelled the subscription.I read that it is not possible to resubscribe with the same e-mail address. Therefore, my idea would be to delete the account I created...
- 1493 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
I have a similar issue. I subscribed to Databricks using AWS account email. I cancelled it later. Now I want to start using Databricks on AWS again using the same email id and with pay as you go plan. But there is no way to re-subscribe. If this can...
- 1 kudos
- 1276 Views
- 0 replies
- 0 kudos
benchmark tpc-ds from external parquet hive structure in S#
Hi I am just getting started in databricks would appreciate some help here.I have 10TB TPCDS in S3 i a hive partition structure.My goal is to benchmark a data bricks cluster on this data.after setting all IAM credentials according to this https://doc...
- 1276 Views
- 0 replies
- 0 kudos
- 1130 Views
- 0 replies
- 0 kudos
transform a dataframe column as concatenated string
Hello,I have a single column dataframe and I want to transform the content into a stringEG df=abcdefxyzToabc, def, xyz Thanks
- 1130 Views
- 0 replies
- 0 kudos
- 1272 Views
- 1 replies
- 0 kudos
Azure DevOps load sequence
Hi Expert,How we can setup multiple notebook in a sequence order in flow for an example 1 pipeline have notebook1 - sequence 1,Notebook2- Sequence 2(in 1pipeline only)
- 1272 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Not sure how to approach your challenge but something you can is to use the Databricks Job Scheduler or if you want an external solution in Azure you can call several notebooks from DataFactory.
- 0 kudos
- 923 Views
- 0 replies
- 0 kudos
DBR 14.1 Pyspark Join on df1["col1"] = df2["col1"] syntax fails
HelloAfter upgrading my cluster from DBR 12 to 14.1 I got a MISSING_ATTRIBUTES.RESOLVED_ATTRIBUTE_APPEAR_IN_OPERATION on some of my Joinsdf1.join( df2, [df1["name"] == df2["name"], df1["age"] == df2["age"]], 'left_outer' )I resolved it by...
- 923 Views
- 0 replies
- 0 kudos
- 733 Views
- 0 replies
- 0 kudos
Seeking Advice on Optimizing Spark Scripts for Efficient Data Processing
Hello community!I'm currently working on Spark scripts for data processing and facing some performance challenges. Any tips or suggestions on optimizing code for better efficiency? Your expertise is highly appreciated! paybyplatema Thanks.
- 733 Views
- 0 replies
- 0 kudos
- 781 Views
- 0 replies
- 0 kudos
pushing .ipynb from azure DB to github converts file to .py
I was trying to push .ipynb file to github from Azure DB to github and appears that original file is converted to source code as .py.Why does databricks do this and how can I control which ones to do or not ?I need to keep some files as .ipynb.Thanks...
- 781 Views
- 0 replies
- 0 kudos
- 1408 Views
- 0 replies
- 0 kudos
Getting the Databricks login screen as response to API call from Azure Data Factory
Hi,I'm trying to call the DLT api to kickoff my delta live table flow with a web API call block from Azure Data Factory. I have two environments: one DEV and one PROD.The DEV environment works fine, the response is giving me the update_id, but the PR...
- 1408 Views
- 0 replies
- 0 kudos
- 1545 Views
- 0 replies
- 0 kudos
Upload file from REST API Response directly to ADLS
i have usecase to call rest API and then return response file with base64Is it possible save the response directly to ADLS without convert it to file first ?
- 1545 Views
- 0 replies
- 0 kudos
- 14681 Views
- 2 replies
- 2 kudos
Resolved! Chrome browser is very slow
Hi. Recently, I've found that Databricks is really slow when editing notebooks, such as adding cells, copying and pasting text or cells, etc. It's just the past few weeks actually. I'm using Chrome version 118.0.5993.118. Everything else with Chrome ...
- 14681 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
It seems related to the notebook length (number of cells). The notebook that was really slow had about 40-50 cells, which I've done before without issue. Anyway after starting a new notebook using Chrome, it seems useable again. So without a specific...
- 2 kudos
- 1548 Views
- 0 replies
- 1 kudos
Not able to use graphframes library in databricks.
I have installed graphframes library from maven repository in the cluster (13.3 LTS (includes Apache Spark 3.4.1, Scala 2.12), Standard DS4_v2). The library that I have installed is graphframes:graphframes:0.8.3-spark3.5-s_2.13.I can import the graph...
- 1548 Views
- 0 replies
- 1 kudos
- 1126 Views
- 0 replies
- 0 kudos
Unable to run query- Insert into with 'Nan' value in SQL Editor. Getting the error.
Unable to run query- Insert into with 'Nan' value in SQL Editor.Query :-Insert into ABC with values('xyz',123,Nan);Error :-org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation cannot be cast to org.apache.spark.sql.execution.datasources...
- 1126 Views
- 0 replies
- 0 kudos
