cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Anonymous
by Not applicable
  • 10202 Views
  • 8 replies
  • 2 kudos
  • 10202 Views
  • 8 replies
  • 2 kudos
Latest Reply
djhs
New Contributor III
  • 2 kudos

I also tried to leverage this endpoint (inferred from devtools): https://<workspace_id>.cloud.databricks.com/sql/api/dashboards/import with the exported dashboard (the dbdash file) in the request payload. It returns a 200 but nothing happens. Maybe s...

  • 2 kudos
7 More Replies
KiranKondamadug
by New Contributor II
  • 1896 Views
  • 0 replies
  • 0 kudos

Databricks Mosaic's grid_polyfill() is taking longer to explode the index when run using PySpark

Pyspark Configuration: pyspark --packages io.delta:delta-core_2.12:2.4.0,org.apache.hadoop:hadoop-aws:3.3.4,io.delta:delta-storage-s3-dynamodb:2.4.0 --conf "spark.sql.extensions=io.delta.sql.DeltaSparkSessionExtension" --conf "spark.sql.catalog.spark...

Data Engineering
Delta Lake
Explode
mosaic
spark
  • 1896 Views
  • 0 replies
  • 0 kudos
burusam
by New Contributor
  • 8099 Views
  • 1 replies
  • 0 kudos

Row INSERT into table does not persist

I have this script:  from databricks import sql import os import pandas as pd databricksToken = os.environ.get('DATABRICKS_TOKEN') connection = sql.connect(server_hostname = "", http_path = "", access_token ...

  • 8099 Views
  • 1 replies
  • 0 kudos
Latest Reply
Emil_Kaminski
Contributor II
  • 0 kudos

HI, are you closing connection at the end?cursor.close() connection.close()Also, can you elaborate what you mean by saying "when I reload the table"Cheers

  • 0 kudos
giuseppegrieco
by New Contributor III
  • 23514 Views
  • 5 replies
  • 6 kudos

Workflow service principle owned can't checkout git repository

I am trying to deploy a workflow where the owner is a service principal and I am using git integration (backend with azure devops), when I run the workflow it says that it doesn't have permissions to checkout the repo.run failed with error message F...

  • 23514 Views
  • 5 replies
  • 6 kudos
Latest Reply
Anonymous
Not applicable
  • 6 kudos

Hi @Giuseppe Grieco​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us ...

  • 6 kudos
4 More Replies
js54123875
by New Contributor III
  • 13479 Views
  • 2 replies
  • 0 kudos

Power BI - Databricks Connection using Service Principal PAT Refresh

What is best practice for automatically refreshing service princpal PAT in Power BI for a connection to a Databricks dataset? Ideally when the PAT is updated it will automatically be stored in Azure Key Vault, is there a way that Power BI can pick it...

Data Engineering
Azure Key Vault
Personal Access Token
Power BI
Service Principal
  • 13479 Views
  • 2 replies
  • 0 kudos
Latest Reply
SSundaram
Databricks Partner
  • 0 kudos

May be not the recommended way, but try to create a token that do not expire for such use cases. Ideally you will need like a custom powershell type of solution to get it automated completely. 

  • 0 kudos
1 More Replies
Alexc01
by New Contributor
  • 1676 Views
  • 0 replies
  • 0 kudos

Buy Trustpilot Account

Buy Trustpilot AccountDo you want to buy Trustpilot account? Our store is the best place where you can buy Trustpilot accounts. Only fully verified Trustpilot accounts on our storeBuy Trustpilot Account

trustpilot.jpg
  • 1676 Views
  • 0 replies
  • 0 kudos
Alexrc
by New Contributor
  • 653 Views
  • 0 replies
  • 0 kudos

Buy Amazon Account

Buy Amazon AccountDo you want to buy Amazon account? Our store is the best place where you can buy Amazon accounts. Only fully verified Amazon accounts on our storeBuy Amazon Account

amazon.jpg
  • 653 Views
  • 0 replies
  • 0 kudos
AlexR
by New Contributor
  • 1802 Views
  • 0 replies
  • 0 kudos

Buy Tripadvisor Account

Buy Tripadvisor AccountDo you want to buy Tripadvisor account? Our store is the best place where you can buy Tripadvisor accounts. Only fully verified Tripadvisor accounts on our storeBuy Tripadvisor Account

tripadvisor.jpg
  • 1802 Views
  • 0 replies
  • 0 kudos
alex97
by New Contributor
  • 914 Views
  • 0 replies
  • 0 kudos

Buy Facebook account

Buy Facebook accountDo you want to buy Upwork account? Our store is the best place where you can buy Upwork accounts. Only fully verified Upwork accounts on our storeBuy Facebook account

facebook.jpg
  • 914 Views
  • 0 replies
  • 0 kudos
alexc95
by New Contributor
  • 3026 Views
  • 0 replies
  • 0 kudos

Buy Badoo account

Buy Badoo accountDo you want to buy Badoo account? Our store is the best place where you can buy Badoo accounts. Only fully verified Badoo accounts on our storeBuy Badoo account

Badoo.jpg
  • 3026 Views
  • 0 replies
  • 0 kudos
BhargaviKapu13
by New Contributor
  • 1061 Views
  • 0 replies
  • 0 kudos

Error 403

Unable to access the page while attempting the quiz for basics of the Databricks Lakehouse Platform

  • 1061 Views
  • 0 replies
  • 0 kudos
alj_a
by New Contributor III
  • 4213 Views
  • 2 replies
  • 2 kudos

Resolved! Delta Live Table - not reading the changed record from cloud file

Hi,I am trying to ingest the data from cloudfile to bronze table. DLT is working fist time and loading the data into Bronze table. but when i add new record and change a filed in existing record the DLT pipeline goes success but it should be inserted...

Data Engineering
Databricks Delta Live Table
  • 4213 Views
  • 2 replies
  • 2 kudos
Latest Reply
alj_a
New Contributor III
  • 2 kudos

Thank you Emil. I tried all the suggestions. .read works fine it picks up the new data or changed data. but my problem is it is bronze table  as target. in this case my bronze table has duplicate records. However, let me look at the other options to ...

  • 2 kudos
1 More Replies
chevichenk
by New Contributor III
  • 2791 Views
  • 1 replies
  • 1 kudos

Resolved! Why I still see delta history if I made a vacuum with just 5 hours retention

Hi, everyone!I execute a vacuum with 5 hours retention but I can see all the history of versions, even I can query those older version of the table.Plus, when I see the history version, it doesn't start with zero (supposed to be the creation of the t...

chevichenk_1-1699661687787.png
  • 2791 Views
  • 1 replies
  • 1 kudos
Latest Reply
Rom
New Contributor III
  • 1 kudos

Hi,When disk caching is enabled, a cluster might contain data from Parquet files that have been deleted with VACUUM. Therefore, it may be possible to query the data of previous table versions whose files have been deleted. Restarting the cluster will...

  • 1 kudos
MikeK_
by New Contributor II
  • 52454 Views
  • 6 replies
  • 0 kudos

Resolved! SQL Update Join

  Hi, I'm importing some data and stored procedures from SQL Server into databricks, I noticed that updates with joins are not supported in Spark SQL, what's the alternative I can use? Here's what I'm trying to do: update t1 set t1.colB=CASE WHEN t2....

  • 52454 Views
  • 6 replies
  • 0 kudos
Latest Reply
LyderIversen
New Contributor II
  • 0 kudos

Hi! This is way late, but did you ever find a solution to the CROSS APPLY-part of your question? Is it possible to do CROSS APPLY in Spark SQL, or is there something you can use instead?

  • 0 kudos
5 More Replies
boyarkaandre
by New Contributor
  • 1576 Views
  • 0 replies
  • 0 kudos

Buy ByBit Account

Buy ByBit Accounthello! Are you looking for a bybit account? You can go to our store and buy bybit account here. You can see crypto account in our store!Buy ByBit Account hereBuy ByBit Account here

bybit.jpg
  • 1576 Views
  • 0 replies
  • 0 kudos
Labels