- 5591 Views
- 5 replies
- 5 kudos
Databricks workspace unstable
Our company's databricks workspace is unstable lately. It can't launch any compute cluster. I have never seen this issue. In addition to this issues, I have seen storage credential error on main unity catalog. Why would this happen AWS Databricks ins...
- 5591 Views
- 5 replies
- 5 kudos
- 5 kudos
Hello @Retired_mod and @jose_gonzalez ,I couldn't locate the support ticket we opened. How can we track that ticket down? It came from the peopletech.com domain. If it is more efficient to create another ticket, please let me know. Let us know the UR...
- 5 kudos
- 2668 Views
- 1 replies
- 6 kudos
Webinar: Accelerate Data and AI Projects With Databricks Notebooks
Register now: October 24, 2023 | 8:00 AM PT Use new capabilities in Databricks Notebooks to speed up innovation. This webinar will walk you through the features that are designed to take the manual effort and delays out of building and deploying dat...
- 2668 Views
- 1 replies
- 6 kudos
- 6 kudos
This will help databricks user to speed up development.
- 6 kudos
- 5997 Views
- 3 replies
- 3 kudos
Application extracting Data from Unity Catalogue
Dear Databricks community,I'm seeking advice on the best method for applications to extract data from the Unity catalogue. One suggested approach is to use JDBC, but there seems to be a dilemma. Although using a job cluster has been recommended due t...
- 5997 Views
- 3 replies
- 3 kudos
- 3 kudos
What exacltly do you mean by 'extracting'? If you want to load tables defined in Unity into a database, I would indeed do this using job clusters and a notebook.If you want to extract some data once in a while into a csv f.e., you could perfectly do...
- 3 kudos
- 5131 Views
- 4 replies
- 2 kudos
Resolved! Error on Workflow
Hi , I have some mysteries situation here My workflow (job) ran and got an error -> [INVALID_IDENTIFIER] The identifier transactions-catalog is invalid. Please, consider quoting it with back-quotes as `transactions-catalog`.(line 1, pos 12) == SQL ==...
- 5131 Views
- 4 replies
- 2 kudos
- 2 kudos
Jobs are just notebooks executed in background, so if the notebook is the same between interactive (manual) and job run, there should be no difference.So I don't see what is wrong. Is the job using DLT perhaps?
- 2 kudos
- 5834 Views
- 1 replies
- 1 kudos
DataBricks Cluster
Hi All,I am curious to know the difference between a spark cluster and a DataBricks one.As per the info I have read Spark Cluster creates driver and Workers when the Application is submitted whereas in Databricks we can create cluster in advance in c...
- 5834 Views
- 1 replies
- 1 kudos
- 1755 Views
- 0 replies
- 0 kudos
Model serving is not available for trial workspaces. Please contact Databricks
Hi, as mentioned in the title, I'm getting this error when I try to use model serving, despite being on the premium plan, my trial account ends on 28th September 2023, is there a way to use model serving immediately or am i stuck until 28th September...
- 1755 Views
- 0 replies
- 0 kudos
- 4649 Views
- 0 replies
- 0 kudos
SQL Warehouse - several issues
Hi there,I am facing several issues while trying to run SQL warehouse-starter on Azure databricks.Please note I am new to this data world, Azure & Databricks . while starting SQL starter warehouse in Databricks Trail version and I am getting these ...
- 4649 Views
- 0 replies
- 0 kudos
- 5372 Views
- 2 replies
- 2 kudos
Resolved! Is it not needed to preserve the data in its original format anymore with the usage of medallion?
Hi Community I have a doubt. The bronze layer always causes confusion for me. Someone mentioned, "File Format: Store data in Delta Lake format to leverage its performance, ACID transactions, and schema evolution capabilities" for bronze layers.Then, ...
- 5372 Views
- 2 replies
- 2 kudos
- 10637 Views
- 5 replies
- 0 kudos
Exam got suspended Databricks Certified Data Engineer Associate exam
Hi team,My Databricks Certified Data Engineer Associate exam got suspended within 20 minutes. My exam got suspended due to eye movement without any warning. I was not moving my eyes away from laptop screen. Some questions are so big in the exam so I...
- 10637 Views
- 5 replies
- 0 kudos
- 0 kudos
@rajib_bahar_ptg funny, not funny, right?! I just posted that tip today in this post: https://community.databricks.com/t5/certifications/minimize-the-chances-of-your-exam-getting-suspended-tip-3/td-p/45712
- 0 kudos
- 8267 Views
- 6 replies
- 3 kudos
Cluster policy not showing while creating delta live table pipeline
Hi all!!!I have created a cluster policy but when i want to use that while creating dlt pipeline, It is showing none. I have checked I have all the necessary permissions to create cluster policies. Still, in dlt ui it is showing none.
- 8267 Views
- 6 replies
- 3 kudos
- 3 kudos
@btafur Can we also set the auto_terminate minutes with the policy? (for the dlt cluster type)
- 3 kudos
- 1845 Views
- 1 replies
- 0 kudos
How to automate creating notebooks when i have multiple .html or .py files
Hi all,I have 50+ .html and .py files, for which I have to create separate notebooks for each and every single one of them. For this, manually creating a notebook using the UI and importing the .html/.py file is a bit tedious and time consuming. Is t...
- 1845 Views
- 1 replies
- 0 kudos
- 0 kudos
Depending on your use case and requirements, one alternative would be to create a script that loops through your files and uploads them using the API. You can find more information about the API here: https://docs.databricks.com/api/workspace/workspa...
- 0 kudos
- 2905 Views
- 0 replies
- 0 kudos
move files and folder from workspace/repo to external location.
I would like to move the folder from my repo under /Workspace/Repos/ar... to the external Azure blob location.I tried dbutils.fs.mv(repo_path, az_path) but this gave me an error for the file not found.Also, I am not able to see workspace -> repo usin...
- 2905 Views
- 0 replies
- 0 kudos
- 6550 Views
- 2 replies
- 3 kudos
Resolved! Limitations of committing ipynb notebook output with Repos
During my experimentation with the latest feature that allows including notebook output in a commit, I ran into a specific issue. While attempting to commit my recent changes, I encountered an error message stating "Error fetching Git status." Intere...
- 6550 Views
- 2 replies
- 3 kudos
- 3 kudos
I've found that the restriction I've encountered isn't related to the file size within Repos, but rather the maximum file size that can be shown in the Azure Databricks UI. You can find this limitation documented at https://learn.microsoft.com/en-us/...
- 3 kudos
- 15624 Views
- 2 replies
- 2 kudos
workspace
How can I do to access the Workspace ?
- 15624 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi, you can try checking https://docs.databricks.com/en/administration-guide/workspace/index.html , please let us know if this helps. Also please tag @Debayan​ with your next response which will notify me, Thank you!
- 2 kudos
- 9227 Views
- 1 replies
- 1 kudos
Structured Streaming of S3 source
I am trying to setup s3 as a structured streaming source. The bucket receives ~17K files/day and the original load to the bucket was ~54K files. The bucket was first loaded 3 months ago and we haven't started reading from it since. So let's say there...
- 9227 Views
- 1 replies
- 1 kudos
- 1 kudos
Thanks,We were able to make things work by increasing the driver instance size so it has more memory for the initial load. After initial load we scaled the instance down for subsequent runs. We're still testing, if we aren't able to make it work we'l...
- 1 kudos
