cancel
Showing results for 
Search instead for 
Did you mean: 
Community Discussions
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Nithya_r
by New Contributor II
  • 63 Views
  • 0 replies
  • 0 kudos

Access Delta sharing from Azure Data Factory

I recently got access to delta sharing and I am looking to access the data from the tables in share through ADF. I used linked services such as REST API and HTTP and successfully established connection using the credential file token and http path, h...

  • 63 Views
  • 0 replies
  • 0 kudos
egndz
by New Contributor
  • 282 Views
  • 0 replies
  • 0 kudos

Cluster Memory Issue (Termination)

Hi,I have a single-node personal cluster with 56GB memory(Node type: Standard_DS5_v2, runtime: 14.3 LTS ML). The same configuration is done for the job cluster as well and the following problem applies to both clusters:To start with: once I start my ...

egndz_2-1712845742934.png egndz_1-1712845616736.png
  • 282 Views
  • 0 replies
  • 0 kudos
EDDatabricks
by Contributor
  • 79 Views
  • 0 replies
  • 0 kudos

Lakeview dashboard dynamically change filter values

Greetings everyone,We are trying to implement a series of visualizations. All these visualizations have queries assigned to them in the form of “ Select * from test table where timestamp between :Days start and :Days end”. The is also a filter applie...

EDDatabricks_0-1712826834624.png EDDatabricks_1-1712826864074.png
Community Discussions
dashboard
filters
Lakeview
parameters
  • 79 Views
  • 0 replies
  • 0 kudos
RahulChaubey
by New Contributor II
  • 353 Views
  • 2 replies
  • 0 kudos

Do we pay just for qurery run duration while using databricks serverless sql ?

While using databricks serverless sql to run queries does we only pay for the compute resources during the run duration of the query ?

  • 353 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @RahulChaubey ,    When using Databricks Serverless SQL, the pricing model is designed to be pay-as-you-go and is based on Databricks Units (DBUs).   Let me break it down for you: Serverless SQL allows you to run SQL queries for BI reporting,...

  • 0 kudos
1 More Replies
chemajar
by New Contributor III
  • 369 Views
  • 2 replies
  • 1 kudos

Resolved! Rearrange tasks in databricks workflow

Hello,There is anyway to rearrange tasks in databricks workflow?.I would like that line that join the two marked tasks doesn't pass behind the other tasks.  It is posible that this line by one side?Thanks. 

image.png
  • 369 Views
  • 2 replies
  • 1 kudos
Latest Reply
chemajar
New Contributor III
  • 1 kudos

Thank you very much, the Visual Clarity part was what I thoght

  • 1 kudos
1 More Replies
jx1226
by New Contributor
  • 1549 Views
  • 2 replies
  • 0 kudos

How to access storage with private endpoint

We know that Databricks with VNET injection (our own VNET) allows is to connect to blob storage/ ADLS Gen2 over private endpoints and peering. This is what we typically do.We have a client who created Databricks with EnableNoPublicIP=No (secure clust...

  • 1549 Views
  • 2 replies
  • 0 kudos
Latest Reply
IvoDB
New Contributor II
  • 0 kudos

Hey @jx1226 , were you able to solve this at the customer? I am currently struggling with same issues here.

  • 0 kudos
1 More Replies
tomos_phillips1
by New Contributor II
  • 1195 Views
  • 3 replies
  • 0 kudos

GeoPandas Insall

hi,I cannot install geopandas in my notebook, ive tried all different forms of generic fix, pip installs etc but always get this error:CalledProcessError: Command 'pip --disable-pip-version-check install geopandas' returned non-zero exit status 1.---...

  • 1195 Views
  • 3 replies
  • 0 kudos
Latest Reply
shan_chandra
Honored Contributor III
  • 0 kudos

@tomos_phillips1 - can you please try install geopandas on a single node cluster. (driver only) ? %pip install geopandas  

  • 0 kudos
2 More Replies
dm7
by New Contributor
  • 210 Views
  • 0 replies
  • 0 kudos

DLT CDC/SCD - Taking the latest ID per day

Hi I'm creating a DLT pipeline which uses DLT CDC to implement SCD Type 1 to take the latest record using a datetime column which works with no issues:@dlt.view def users(): return spark.readStream.table("source_table") dlt.create_streaming_table(...

  • 210 Views
  • 0 replies
  • 0 kudos
jenshumrich
by New Contributor III
  • 167 Views
  • 0 replies
  • 0 kudos

Long running jobs get lost

Hello,I tried to schedule a long running job and surprisingly it does seem to neither terminate (and thus does not let the cluster shut down), nor continue running, even though the state is still "Running":But the truth is that the job has miserably ...

jenshumrich_0-1712742957610.png jenshumrich_2-1712743008070.png jenshumrich_3-1712743098546.png
  • 167 Views
  • 0 replies
  • 0 kudos
nikhilmb
by New Contributor II
  • 368 Views
  • 4 replies
  • 0 kudos

Error ingesting zip files: ExecutorLostFailure Reason: Command exited with code 50

Hi,We are trying to ingest zip files into Azure Databricks delta lake using COPY INTO command. There are 100+ zip files with average size of ~300MB each.Cluster configuration:1 driver: 56GB, 16 cores2-8 workers: 32GB, 8 cores (each). Autoscaling enab...

  • 368 Views
  • 4 replies
  • 0 kudos
Latest Reply
nikhilmb
New Contributor II
  • 0 kudos

Although we were able to copy the zip files onto the DB volume, we were not able to share them with any system outside of the Databricks environment. Guess delta sharing does not support sharing files that are on UC volumes.

  • 0 kudos
3 More Replies