- 697 Views
- 1 replies
- 1 kudos
Resolved! Unable to login to partner account
When I try to login with my office email to the partner acccount. It says, The service is currently unavailable. Please try again later. It says "You are not authorized to access https://partner-academy.databricks.com. Please select a platform you ca...
- 697 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello @AbhishekNakka15! Please raise a ticket with the Databricks Support Team, and include your email address so they can review your account and provide further assistance.
- 1 kudos
- 1214 Views
- 2 replies
- 1 kudos
Lakebridge Synapse Conversion to DBX and Custom transpiler
I have 2 questions about Lakebridge solution,Synapse with dedicated pool ConversionWe were conducting a PoC for Synapse to DBX migration using Lakebridge. What we have observed is that the conversions are not correct. I was anticipating all tables wi...
- 1214 Views
- 2 replies
- 1 kudos
- 1 kudos
We are also checking on this use case but got it confirmed from Databricks that it does not work for this use case yet, not sure whether it is in their roadmap
- 1 kudos
- 609 Views
- 1 replies
- 0 kudos
Databricks job runs failures Py4JJavaError: An error occurred while calling o404.sql. : java.util.No
Hi ,We had a successful running pipeline but it started failing since 20th august , no change were published. Can you please guide me resolve this issue.I've tried increasing delta.deletedFileRetentionDuration' = 'interval 365 days' but it didn't hel...
- 609 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @vishalv4476 ,The error is likely due to a corrupted Delta transaction log or files deleted manually/outside of Delta. Check the table history and verify that no user or automated process removed data files. If issues are found, restore the table ...
- 0 kudos
- 1782 Views
- 9 replies
- 2 kudos
databricks api to create a serverless job
I am trying to follow your documentation on how to create serverless job via API https://docs.databricks.com/api/workspace/jobs/create#environments-spec-environment_version So i see that sending the json request resulted for me to see serverless clus...
- 1782 Views
- 9 replies
- 2 kudos
- 2 kudos
It looks like you’re hitting one of the current limitations of Databricks serverless jobs. Even though the API supports passing an environments object, only certain fields are honored right now. In particular:The environment_version parameter will de...
- 2 kudos
- 7297 Views
- 3 replies
- 1 kudos
i have created a materialized view table using delta live table pipeline and its not appending data
i have created a materialized view table using delta live table pipeline , for some reason it is overwriting data every day , i want it to append data to the table instead of doing full refresh suppose i had 8 million records in table and if irun the...
- 7297 Views
- 3 replies
- 1 kudos
- 1 kudos
[expect_table_not_view.no_alternative] 'insert' expects a table but dim_airport_unharmonised is a view can you please help how to reslove this.thanksuma devi
- 1 kudos
- 749 Views
- 1 replies
- 2 kudos
Best practices : Silver Layer to Salesforce
Need community view to evaluate my solution based best practice Problem i am solving is reading match data from a CSV, this was uploaded into a volume , then i clean and transfo...
- 749 Views
- 1 replies
- 2 kudos
- 2 kudos
- skip the pandas conversion- persist the transformed data in a databricks table and then write to salesforce.
- 2 kudos
- 1871 Views
- 10 replies
- 4 kudos
Resolved! sync delta table to Nosql
Hello Guys,Whats is best way to build sync process which sync data for two engine database like delta table and Nosql table ( Mongo) ?Thanx Cordially,
- 1871 Views
- 10 replies
- 4 kudos
- 4 kudos
The other option I can think of is change streams. Here is a blogpost on it.https://contact-rajeshvinayagam.medium.com/mongodb-changestream-spark-delta-table-an-alliance-a70962133b95
- 4 kudos
- 2090 Views
- 7 replies
- 5 kudos
Resolved! timestamp date filter does not work
HelloI have a column called LastUpdated defined as timestampIf I select from the table it displays as (e.g.) 2025-08-27T10:50:31.610+00:00How do I filter on this without having to be specific with the year, month, day, ... This does not work:select *...
- 2090 Views
- 7 replies
- 5 kudos
- 5 kudos
Hello @collierd ,The way I would tackle this would involve data time specifiers. Because your value is likely stored as a timestamp which you can see via the catalog explorer, you cannot compare it to a string value such as "2025-08-27T10:50:31.610+0...
- 5 kudos
- 787 Views
- 3 replies
- 1 kudos
Resolved! Silver layer to Salesforce - Need Help Debugging - IllegalArgumentException: Secret does not exist
I have ingested raw data Converted into Bronze TableSubsequently have saved the DataFrame as a Delta table in the 'silver' schemaAS part of sending data from silvertable to salesforceInstall & authenticate the Databricks CLI - DoneCreate the secret s...
- 787 Views
- 3 replies
- 1 kudos
- 1 kudos
@szymon_dybczak Resolved it now it I had to use commands specific to Databricks CLI v0.265.0
- 1 kudos
- 693 Views
- 1 replies
- 0 kudos
Static Table Creation in DLT
We're encountering a specific issue in our DLT pipeline and would appreciate some advice. Here's an example to illustrate the challenge we're facing:Tables OverviewMaterial Master: Contains comprehensive material data updated daily with new records. ...
- 693 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @Anubhav2011 ,From your question, it means that you want the output to appear in the Catalog UI as an actual Table, not a Materialized View (MV). In DLT, datasets derived from other DLT datasets are shown as MVs (or Streaming Table). They’re st...
- 0 kudos
- 1266 Views
- 4 replies
- 3 kudos
Can I get more details on the performance differences between pyodbc and SQL Connector for Python?
This article (Connect Python and pyodbc to Databricks | Databricks on AWS) states the following"However pyodbc may have better performance when fetching queries results above 10 MB."This is a bit vague. The word "may" implies "maybe not". Also, "bett...
- 1266 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi @Travis84, Hi,I came across an article that might help you, which makes the following comparison:A blog on high-bandwidth connections using Databricks’ Cloud Fetch optimization (leveraging parallel data transfer via pre-signed URLs) reported up to...
- 3 kudos
- 1590 Views
- 2 replies
- 1 kudos
Resolved! In Unity Catalog repartition method issue
We are in the process of upgrading our notebooks to Unity Catalog. Previously, I was able to write data to an external Delta table using df.repartition(8).write. Save('path'), which correctly created multiple files. However, during the upgrade, in te...
- 1590 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Shiva3,Maybe you can try this option in Delta Lake in Unity Catalog may have optimizedWrites enabled by default, which can reduce the number of files by automatically coalescing partitions during writes. # Disable auto-compaction and optimized wr...
- 1 kudos
- 1391 Views
- 2 replies
- 5 kudos
Resolved! Databricks Docs removed/hidden File Metadata documentation?
Hey everyone, Hopefully this is a quick one to resolve (and it's probably me being behind-the-times or slightly stupid ). I've been looking at getting metadata into my SQL query (when I'm ingesting files). This article is fantastic for solving this v...
- 1391 Views
- 2 replies
- 5 kudos
- 5 kudos
Hi Bro!Yes — this page doesn’t show up in search because it’s marked Unlisted, so it’s only available to people with the direct link (or via a few internal links). You can confirm this by viewing the page source and searching for “noindex”, as shown ...
- 5 kudos
- 1786 Views
- 5 replies
- 2 kudos
Resolved! Access task level parameters along with parameters passed by airflow job
I have a airflow DAG which calls databricks job that has a task level parameters defined as job_run_id (job.run_id) and has a type as python_script. When I try to access it using sys.argv and spark_python_task, it only prints the json that has passed...
- 1786 Views
- 5 replies
- 2 kudos
- 2 kudos
Hey @divyab7 Sorry, now I understand better what you actually need. I got confused at first and thought you only wanted to access the parameters you pass through Airflow.I think the dynamic identifiers that Databricks generates at runtime (like run I...
- 2 kudos
- 3131 Views
- 6 replies
- 4 kudos
ModuleNotFound when running DLT pipeline
My new DLT pipeline gives me a ModuleNotFound error when I try to request data from an API. For some more context, I develop in my local IDE and then deploy to databricks using asset bundles. The pipeline runs fine if I try to write a static datafram...
- 3131 Views
- 6 replies
- 4 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 553 | |
| 349 | |
| 287 |
