cancel
Showing results for 
Search instead for 
Did you mean: 
missing-QuestionPost
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Oliver_Angelil
by Valued Contributor II
  • 26152 Views
  • 17 replies
  • 17 kudos

Resolved! Databricks slack channel

I would like to know if there is a Databricks slack channel? And if not, would anyone be interested in joining one?

  • 26152 Views
  • 17 replies
  • 17 kudos
Latest Reply
YashP
New Contributor II
  • 17 kudos

I joined with same email and it says try different emails. How do I join?

  • 17 kudos
16 More Replies
396827
by New Contributor II
  • 5156 Views
  • 1 replies
  • 2 kudos

Discrepancies between official spark 3.3.2 and what's provided in Databricks Runtime 12.2 LTS leads to NoSuchMethodError when creating ParquetToSparkSchemaConverter

I am trying to run my spark-dependent application on a Databricks cluster:spark-3.3.2on a cluster with Databricks Runtime 12.2 LTS (spark-3.3.2, scala-2.12)I end up with:NoSuchMethodError: org.apache.spark.sql.execution.datasources.parquet.ParquetToS...

  • 5156 Views
  • 1 replies
  • 2 kudos
Latest Reply
OsamaNabih
New Contributor II
  • 2 kudos

I ran into the same issueDid you happen to find any fix?

  • 2 kudos
MattJohn_35453
by New Contributor III
  • 7623 Views
  • 3 replies
  • 1 kudos

Resolved! When creating a cluster, ERROR: "Missing required field 'spark_version'" even though it was specified

Every time I try to create a cluster over the API I get this. Even when I am literally using the example JSON from the API documentation:```{ "cluster_name": "single-node-cluster", "node_type_id": "i3.xlarge", "spark_version": "7.6.x-scala2...

  • 7623 Views
  • 3 replies
  • 1 kudos
Latest Reply
Norway
New Contributor II
  • 1 kudos

Hi, Matt. Would be super useful if you could provide a more complete code example on how it worked.

  • 1 kudos
2 More Replies
alexiswl
by Contributor
  • 4195 Views
  • 2 replies
  • 0 kudos

Init Script not 'Absolute'

Hello,I am trying to use an init script from my repo workspace. The cluster is a 'personal compute' cluster with single user access mode with the following runtime version: '13.1 (includes Apache Spark 3.4.0, Scala 2.12)' I have 'Files in Repos' set ...

  • 4195 Views
  • 2 replies
  • 0 kudos
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi, could you please enable init script logging and provide the elaborated logs here? Looks like it is not able to get the package from the repo. https://docs.databricks.com/clusters/init-scripts.html#init-script-loggingAlso, please tag @Debayan Mukh...

  • 0 kudos
1 More Replies
berdoni
by New Contributor III
  • 6142 Views
  • 6 replies
  • 6 kudos

Accreditation badge for AZURE Databricks Platform Architect

Dear support,after 48h I didn't receive the Accreditation Badge for AZURE Databricks Platform Architect. The id is E-02KZJV. Could you check please? I completed the "Azure Databricks Platform Architect Accreditation" learing plan and passed with >80%...

  • 6142 Views
  • 6 replies
  • 6 kudos
Latest Reply
Anonymous
Not applicable
  • 6 kudos

Hi @NIKOLAOS BERDOS​ I assume you will receive the ticket number since you have initiated the ticket. That means your ticket has been raised.Thanks and Regards

  • 6 kudos
5 More Replies
vijaykumar99535
by New Contributor III
  • 3786 Views
  • 2 replies
  • 1 kudos

Purge utility for databricks

I have one requirement where I need to delete older data from a folder, and folder data can be partitioned or without partition. What approach to follow to build this utility?

  • 3786 Views
  • 2 replies
  • 1 kudos
Latest Reply
vijaykumar99535
New Contributor III
  • 1 kudos

Hi @Vidula Khanna​ @Kaniz Fatma​ It does not answer my real question. We mount adls path in databricks where we read and write the data. Suppose I have one partitioned (partitioned on audit) folder like below. I want to build an utility to delete the...

  • 1 kudos
1 More Replies
yatharth
by New Contributor III
  • 4086 Views
  • 2 replies
  • 2 kudos

Resolved! Delta table transaction log

What if i delete the delta table in the 10th transaction(Checkpoint) will i be able to recover the table through time travel

  • 4086 Views
  • 2 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Yatharth Malhotra​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best ans...

  • 2 kudos
1 More Replies
chico
by New Contributor II
  • 4179 Views
  • 3 replies
  • 0 kudos
  • 4179 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @stephane cochet​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answe...

  • 0 kudos
2 More Replies
804925
by New Contributor II
  • 1950 Views
  • 1 replies
  • 1 kudos

Why doesn't my job-cluster scale down?

Hi,Running on Databricks-AWS, I have a job running on a cluster with 3 workers, 2-cores each (r6i.large), with autoscaling enabled.The Spark job has two stages: (1) highly parallelizable, cpu-intensive stage. This stage takes 15 minutes.(2) a non-par...

  • 1950 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Yoav Ben​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.

  • 1 kudos
Duy
by New Contributor II
  • 1901 Views
  • 1 replies
  • 0 kudos

Why is the current path different when executing a Notebook from Repos vs Workspace?

When executing the same Notebook from Repos and Workspace, the current path (`os.getcwd()`) is different:Repos=/Workspace/Repos/USERID/REPONAME.ide/pricingWorkspace=/home/spark-1636b562-0ce7-473b-bb17-56The issue is the Databricks VScode extension re...

  • 1901 Views
  • 1 replies
  • 0 kudos
Latest Reply
Duy
New Contributor II
  • 0 kudos

Found a work around:``` Pythonimport osnotebook_dir = os.path.dirname(dbutils.notebook.entry_point.getDbutils().notebook().getContext().notebookPath().get())notebook_dir = "/Workspace" + notebook_dircurrent_dir = os.getcwd()if current_dir != notebook...

  • 0 kudos
ccbeloy
by New Contributor III
  • 4518 Views
  • 4 replies
  • 4 kudos

How to use Databricks Spark with Ibis

Ibis now includes pyspark backend. However using the databricks-connect doesn't seem to work with Ibis. Here's the sample code the throws an error.from databricks.connect import DatabricksSessionfrom databricks.sdk.core import Configconfig = Config(p...

  • 4518 Views
  • 4 replies
  • 4 kudos
Latest Reply
ccbeloy
New Contributor III
  • 4 kudos

Hi Khanna, No solution provided yet. Can escalate with your technical team for a suitable solution please?Thanks,Calvin

  • 4 kudos
3 More Replies
Karthe
by New Contributor III
  • 3908 Views
  • 4 replies
  • 3 kudos

I could not access the deltatable from databricks. Error : org.apache.spark.sql.AnalysisException:

Hi all,​I am not able to access the delta table from the datbase.While I try to read the table via spark.read.table command, I get the below error:org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.Run...

  • 3908 Views
  • 4 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Karthikeyan Palanisamy​ I hope you are doing great!Sorry for the inconvenience. Please specify that the issue is related to Databricks Community or Community -Edition or share the screenshot. Kindly, do let me know on a priority basis.Have a grea...

  • 3 kudos
3 More Replies
Enzo_Bahrami
by New Contributor III
  • 3698 Views
  • 3 replies
  • 4 kudos

Resolved! Checklist or Document for Migrating from Dev to Prod

Hello Everyone!So we have a dev environment in Databricks and want to migrate it to prod. I need it o go over every single table, schema, notebooks, and artifacts in the databricks and make sure nothing is hard-coded for example or that there is noth...

  • 3698 Views
  • 3 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @Parsa Bahraminejad​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best an...

  • 4 kudos
2 More Replies
Mehala
by New Contributor III
  • 8994 Views
  • 12 replies
  • 13 kudos

Resolved! Connect Databricks with SSAS multidimensional Project

Hi,I just wanted to connect Databricks with SSAS multidimensional Project/want to use Databricks connection string and provider. Is that possible to do in SSAS multidimensional Project?Which Provider I need to use?Or Is there any other workaround to ...

  • 8994 Views
  • 12 replies
  • 13 kudos
Latest Reply
Anonymous
Not applicable
  • 13 kudos

Hi @Mehala M​ We haven't heard from you since the last response from @Werner Stinckens​ ​, and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to oth...

  • 13 kudos
11 More Replies