- 4044 Views
- 2 replies
- 1 kudos
Unity Catalog Table/View Column Data Type Changes
When changing a delta table column data type in Unity Catalog, we noticed a view that is referencing that table did not automatically update to reflect the new data type.Is there a way to update the delta table column data type so that it also update...
- 4044 Views
- 2 replies
- 1 kudos
- 1 kudos
Can you try refreshing the view by running the command: REFRESH TABLE <viewname>
- 1 kudos
- 2375 Views
- 1 replies
- 0 kudos
Soft stop a Streaming Job
I have an auto loader streaming job with the continuous run, I want to stop that job on weekend for some time and restart again.
- 2375 Views
- 1 replies
- 0 kudos
- 7028 Views
- 5 replies
- 4 kudos
Resolved! Cluster Performance
Facing an issue with cluster performance, in event log can see - cluster is not responsive likely due to GC. Number of pipeline (databricks notebooks) running and cluster configuration is same as it used to be before but started seeing this issue sin...
- 7028 Views
- 5 replies
- 4 kudos
- 4 kudos
Hi @Vibhor Sethi​ ,Do you see any other error messages? did you data volume increase? what kind of job are you running?
- 4 kudos
- 2842 Views
- 1 replies
- 1 kudos
what is the best way to optimize historical data tables in databricks?
Hi There,Looking for recommendations on how to optimize history data tables in databricks?
- 2842 Views
- 1 replies
- 1 kudos
- 1 kudos
Thanks Kaniz!
- 1 kudos
- 26957 Views
- 5 replies
- 10 kudos
Resolved! SFTP Connect
How I can connect sftp server from databricks. So I can write files into tables directly?
- 26957 Views
- 5 replies
- 10 kudos
- 10 kudos
The classic solution is to copy data from FTP to ADLS storage using Azure Data Factory, and after the copy is done in the ADF pipeline, trigger the databricks notebook.
- 10 kudos
- 2852 Views
- 1 replies
- 0 kudos
Resolved! Databricks lakehouse fundamental certification page is not working
While going to give Databricks lakehouse fundamental certification quizIt is showing that you don't have permission to access this. Could you please guide me on that or can provide a link through which I can get permission to access this and complet...
- 2852 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Souvikng, Here is the link for the Accrediition.
- 0 kudos
- 5445 Views
- 1 replies
- 0 kudos
How to Get spark history server logs using python REST API
Hello,I am trying to load logs from spark job in remote location using Python Rest API.I want to collect those logs for particular job runs using runID field, that log should contain Errors, Exceptions, and print details.I have tried "/api/2.1/jobs/r...
- 5445 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello,How can I access endpoint (http URL) of spark history server for a particular cluster? Thanks,Parimal
- 0 kudos
- 9673 Views
- 2 replies
- 2 kudos
Resolved! Cannot delete file in dbfs
Hello, I am trying to delete a folder in /dbfs/mnt, but I am unable to do so. The folder was an old mounted storage account, which was deleted. The folder contains a single file 'mount.err'.In command line I tried rm -rf my_folder, with and without s...
- 9673 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @erigaud The issue you're facing might be due to the fact that the folder was a mounted storage account that has been deleted. This could cause some inconsistencies in the file system view and hence, you're unable to delete the folder.You can try...
- 2 kudos
- 5355 Views
- 3 replies
- 1 kudos
Resolved! Running DBT via Jobs Compute
Is it possible to run a DBT job using Jobs Compute? If not, is there a reason for that, and is there any plans to change that?
- 5355 Views
- 3 replies
- 1 kudos
- 1 kudos
Hello @brendanc19 ,Running DBT with Jobs compute cluster is actually supportedYou will need to create a job task with type DBTKindly check documentation here for step by step on how to run DBT transformation Databricks jobsRegards
- 1 kudos
- 2628 Views
- 1 replies
- 0 kudos
Error creating repo
I have this error when trying to clone a repo, using aos sparse checkout mode. Error creating repoGit operation is terminated because of out of memory or CPU, possibly the repository is too big. Need help? See our Repos Limitations and FAQs docs How...
- 2628 Views
- 1 replies
- 0 kudos
- 0 kudos
From the error message, you could exceed below limit :Working branches are limited to 200 MB.Individual files are limited to 200 MB.Files larger than 10 MB can’t be viewed in the Databricks UI.Databricks recommends that in a repo:The total number of ...
- 0 kudos
- 13937 Views
- 8 replies
- 3 kudos
How to stop continuous running streaming job over weekend
I have an continuous running streaming Job, I would like to stop this over weekend and start again on Monday. Here is my streaming job code.(spark.readStream.format("delta").load(input_path).writeStream.option("checkpointLocation", input_checkpoint_p...
- 13937 Views
- 8 replies
- 3 kudos
- 3 kudos
@sanjay Any luck on that, I am also looking for the solution for the same issue
- 3 kudos
- 2508 Views
- 0 replies
- 0 kudos
Connect a SQL warehouse to an Azure Blob container
Hi all,I was wondering if it was possible to configure a single SQL warehouse to a specific Blob container?I know you can add configuration properties to the Workspace from the Admin settings, but it appears that those configurations will be shared b...
- 2508 Views
- 0 replies
- 0 kudos
- 75776 Views
- 14 replies
- 9 kudos
How to import one databricks python notebook into another?
I have a python notebook A in Azure Databricks having import statement as below: import xyz, datetime,... I have another notebook xyz being imported in notebook A as shown in above code. When I run notebook A, it throws the following error: ImportEr...
- 75776 Views
- 14 replies
- 9 kudos
- 9 kudos
Create a repository containing an __init__.py fileAdd your library as .py file(s). Let's imagine that our library is composed by multiple sub-folders consolidated in "my_folder", one of sub-folders is named as "math_library" and contains my_awesome_l...
- 9 kudos
- 10791 Views
- 9 replies
- 2 kudos
- 10791 Views
- 9 replies
- 2 kudos
- 2 kudos
Hi @Amardeep Singh​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...
- 2 kudos
- 12254 Views
- 14 replies
- 0 kudos
Create a Hive db in Azure Databricks with the LOCATION parameter set to an ADLS GEN2 account fails
Hello,I am trying to create a database in Azure Databricks using the abfss Location in the create database statement and it throws an exception.%sqlCREATE DATABASE IF NOT EXISTS test COMMENT "Database for Test Area" LOCATION "abfss://test@storagetemp...
- 12254 Views
- 14 replies
- 0 kudos
- 0 kudos
ok.Can you check this link?Because I suspect something is wrong, and so dbrx uses the account key as a default (fs.azure.account.key)
- 0 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 566 | |
| 349 | |
| 287 |
