cancel
Showing results for 
Search instead for 
Did you mean: 
Community Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

HowardLJ
by New Contributor
  • 764 Views
  • 1 replies
  • 0 kudos

Delta Live Tables Slowly Changing Dimensions Type 2 with Joins

Hi,I may be missing something really obvious here. The organisation I work for has started using Delta Live Tables in Databricks for data modelling, recently. One of the dimensions I am trying to model takes data from 3 existing tables in our data la...

Community Discussions
Delta Live Tables
SCD
  • 764 Views
  • 1 replies
  • 0 kudos
Latest Reply
the_real_merca
New Contributor II
  • 0 kudos

Can it be because the default join is `inner` and that means the row must exists in both tables

  • 0 kudos
DavidKxx
by Contributor
  • 2073 Views
  • 2 replies
  • 1 kudos

Resolved! Rendering markdown images hard coded as data image png base64 in notebook

Hi all,For training purposes, I have cloned a repo from John Snow Labs into my Databricks account and am working in the notebook that you can review at https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/open-source-nlp/03.0.SparkNLP_Pretr...

DavidKxx_0-1700585428705.png
Community Discussions
image
rendering
  • 2073 Views
  • 2 replies
  • 1 kudos
Latest Reply
NateAnth
Valued Contributor
  • 1 kudos

Try changing the magic command for that cell from %md to %md-sandbox to see if that helps the image to render appropriately.

  • 1 kudos
1 More Replies
prasad_vaze
by New Contributor III
  • 4778 Views
  • 2 replies
  • 0 kudos

Resolved! Unable to add column comment on a View. Any way to update comments on multiple columns in bulk?

I noticed that unlike "Alter Table" there is no "Alter View"  command to add comment on a column in the existing view. This is a regular view created on Tables (and not Materialized view).  If the underlying table column has comment then the View inh...

Community Discussions
Data Governance
Unity Catalog
  • 4778 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @prasad_vaze, Let’s address your questions regarding altering views and mass-applying column comments:   Adding Comments to View Columns: While there is no direct “ALTER VIEW” command to add comments to specific columns in an existing view, you ca...

  • 0 kudos
1 More Replies
332588
by New Contributor II
  • 493 Views
  • 0 replies
  • 0 kudos

MLflow CVE-2023-6015 (vulnerabilities)

Hi,what mlflow version is used for the "Databricks Managed MLflow"Is there a workarround for MLflow CVE-2023-6015 (vulnerabilities) ?RegardsTimo

Community Discussions
CVE
vulnerabilities
  • 493 Views
  • 0 replies
  • 0 kudos
JCamiloCS
by New Contributor
  • 3527 Views
  • 1 replies
  • 0 kudos

Resolved! how to use R in databricks

Hello everyone.I am a new user of databricks, they implemented it in the company where I work. I am a business analyst and I know something about R, not much either, when I saw that databricks could use R I was very excited because I thought that the...

  • 3527 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @JCamiloCS , Certainly! I apologize for the oversight. Let’s dive into using R in Databricks. Databricks is a powerful platform for big data analytics and machine learning, and it integrates seamlessly with Apache Spark™.  As a business analyst, l...

  • 0 kudos
PiotrU
by Contributor
  • 12126 Views
  • 2 replies
  • 1 kudos

Resolved! Some streams terminated before this command could finish! -> java.lang.NoClassDefFoundError: scala/c

HelloI do face:Some streams terminated before this command could finish!java.lang.NoClassDefFoundError: scala/compat/java8/FutureConverters$Running some very simple query on eventhub :df = spark \.readStream \.format("eventhubs") \.options(**ehConf) ...

Community Discussions
azure
eventhub
  • 12126 Views
  • 2 replies
  • 1 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 1 kudos

Hi @PiotrU , I want to express my gratitude for your effort in selecting the most suitable solution. It's great to hear that your query has been successfully resolved. Thank you for your contribution. 

  • 1 kudos
1 More Replies
Madhawa
by New Contributor II
  • 1266 Views
  • 1 replies
  • 0 kudos

org.apache.spark.SparkException - FileReadException

Sometimes getting this kind of error "org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 12224.0 failed 4 times, most recent failure: Lost task 1.5 in stage 12224.0 (TID           ) (12.xxx.x.xxx executor 1): com.datab...

  • 1266 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @Madhawa,  One possible reason for this error is that the file you are trying to read is corrupted or incomplete. You can try to verify the integrity of the file by checking its hash value or by comparing it with a known good copy of the file. Ano...

  • 0 kudos
oleprince
by New Contributor
  • 3020 Views
  • 2 replies
  • 0 kudos

Delta table definition - Identity column

Hello,Would anyone know if it is possible to create a delta table using Python that includes a column that is generated by default as identity (identity column for which the value inserted can be manually overriden)?There seems to be a way to create ...

  • 3020 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @oleprince, Yes, it is possible to create a Delta table using Python that includes a column that is generated by default as identity. However, the generatedByDefaultAs option is not available in the DeltaTableBuilder API.   As a workaround, you ca...

  • 0 kudos
1 More Replies
Shree23
by New Contributor III
  • 607 Views
  • 1 replies
  • 0 kudos

Azure DevOps load sequence

Hi Expert,How we can setup multiple notebook in  a sequence order in flow for an example 1 pipeline  have notebook1 - sequence 1,Notebook2- Sequence 2(in 1pipeline only)   

  • 607 Views
  • 1 replies
  • 0 kudos
Latest Reply
LeTurtleboy
New Contributor II
  • 0 kudos

Not sure how to approach your challenge but something you can is to use the Databricks Job Scheduler or if you want an external solution in Azure you can call several notebooks from DataFactory.

  • 0 kudos
397973
by New Contributor III
  • 8104 Views
  • 3 replies
  • 2 kudos

Resolved! Chrome browser is very slow

Hi. Recently, I've found that Databricks is really slow when editing notebooks, such as adding cells, copying and pasting text or cells, etc. It's just the past few weeks actually. I'm using Chrome version 118.0.5993.118. Everything else with Chrome ...

  • 8104 Views
  • 3 replies
  • 2 kudos
Latest Reply
397973
New Contributor III
  • 2 kudos

It seems related to the notebook length (number of cells). The notebook that was really slow had about 40-50 cells, which I've done before without issue. Anyway after starting a new notebook using Chrome, it seems useable again. So without a specific...

  • 2 kudos
2 More Replies
data9
by New Contributor II
  • 658 Views
  • 0 replies
  • 1 kudos

Not able to use graphframes library in databricks.

I have installed graphframes library from maven repository in the cluster (13.3 LTS (includes Apache Spark 3.4.1, Scala 2.12), Standard DS4_v2). The library that I have installed is graphframes:graphframes:0.8.3-spark3.5-s_2.13.I can import the graph...

  • 658 Views
  • 0 replies
  • 1 kudos
tanmoy-km
by New Contributor
  • 6919 Views
  • 3 replies
  • 0 kudos

Duplicate queryId in the query history api response

Hello Experts,I was using the `/api/2.0/sql/history/queries` end-point to find the historical stats for queries on the warehouse endpoint.I am surprised to see exact rows duplicated in the response, even though they have different `next_page_token` H...

Community Discussions
databricks-api
sql-endpoint
  • 6919 Views
  • 3 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @tanmoy-km, Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers you...

  • 0 kudos
2 More Replies
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!