- 1527 Views
- 1 replies
- 0 kudos
Constraint options usage
I am trying to use the constraints options:NOT ENFORCEDDEFERRABLEINITIALLY DEFERREDNORELYHowever it seems I am not able to use them successfully. When I try to use them with PRIMARY KEYS (not sure if it is possible), I am not able to enforce any key....
- 1527 Views
- 1 replies
- 0 kudos
- 0 kudos
BTW the forum is bugged I can't paste code..
- 0 kudos
- 1697 Views
- 1 replies
- 0 kudos
Multi Task Job creation through Pulumi
I am trying to create a multi-task Databricks Job in Azure Cloud with its own cluster.Although I was able to create a single task job without any issues, the code to deploy the multi-task job fails due to the following cluster validation error:error:...
- 1697 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @Retired_mod, thanks for your answer, but the problem keeps the same. I had already tested with different cluster configurations, single-node and multi-node, including those cluster configurations which worked with single task jobs, but the err...
- 0 kudos
- 9024 Views
- 2 replies
- 2 kudos
Databricks Asset bundle
Hi,I am new to databricks, We are trying to use Databricks asset bundles for code deployment .I have spect a lot of time but still so many things are not clear to me.Can we change the target path of the notebooks deployed from /shared/.bundle/* to so...
- 9024 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Retired_mod,Thank you for your post, I thought it will solve my issues too, however after reading your suggestion it was nothing new for me, because l already have done exactly that.. Here is what I have dome so you or anyone can replicate it:1. ...
- 2 kudos
- 544 Views
- 0 replies
- 0 kudos
Auto Loader notebook for multiple tables
Hi Team,My requirement is ,i do have File A from source A which needs to write into Multiple Delta tables i.e DeltaTableA,DeltaTableB,DeltaTableC. Is it possible to have a single instance of an autoloader script. (multiple write streams). Could you p...
- 544 Views
- 0 replies
- 0 kudos
- 965 Views
- 0 replies
- 0 kudos
Error from Knime trought proxy
I want to connect to Databricks from Knime on a company computer that uses a proxy. The error I'm encountering is as follows: ERROR Create Databricks Environment 3:1 Execute failed: Could not open the client transport with JDBC URI: jdbc:hive2://adb-...
- 965 Views
- 0 replies
- 0 kudos
- 3055 Views
- 2 replies
- 0 kudos
'NotebookHandler' object has no attribute 'setContext' in pyspark streaming in AWS
I am facing issue while calling dbutils.notebook.run() inside of pyspark streaming with concurrent.executor. At first the error is "pyspark.sql.utils.IllegalArgumentException: Context not valid. If you are calling this outside the main thread,you mus...
- 3055 Views
- 2 replies
- 0 kudos
- 0 kudos
The error message you're encountering in PySpark when using dbutils.notebook.run() suggests that the context in which you are attempting to call the run() method is not valid. PySpark notebooks in Databricks have certain requirements when it comes to...
- 0 kudos
- 687 Views
- 0 replies
- 0 kudos
Databricks to make a machine learning model
Hey all,I've been using a voice cloning AI and it's working well. I'm thinking of using Databricks to make a machine learning model for speech tech. I want to start with personal content creation. Any tips or advice would be great!
- 687 Views
- 0 replies
- 0 kudos
- 323 Views
- 0 replies
- 0 kudos
thinking of using Databricks to make a machine learning model
Hey community,I've been using this voice cloning AI and it's working well. I'm thinking of using Databricks to make a machine learning model for speech tech. I want to start with personal content creation.Any tips or advice would be great!
- 323 Views
- 0 replies
- 0 kudos
- 1493 Views
- 0 replies
- 0 kudos
Billing usage per user
Hi Team ,Unity catalog is not enabled in our workspace, We would like to know the billing usage information per user ,could you please help us how to get these details( by using notebook level script).Regards,Phanindra
- 1493 Views
- 0 replies
- 0 kudos
- 4634 Views
- 1 replies
- 0 kudos
unpersist doesn't clear
from pyspark.sql import SparkSessionfrom pyspark import SparkContext, SparkConffrom pyspark.storagelevel import StorageLevelspark = SparkSession.builder.appName('TEST').config('spark.ui.port','4098').enableHiveSupport().getOrCreate()df4 = spark.sql('...
- 4634 Views
- 1 replies
- 0 kudos
- 0 kudos
Thank you so much for taking time and explaining the concepts
- 0 kudos
- 919 Views
- 0 replies
- 0 kudos
Question for exam project
For my exam i have to do a small project for the company im interning at. I am creating a datawarehouse where i will have to transfer data from another database, and then transforming it to a star schema. would databricks be good for this, or is it t...
- 919 Views
- 0 replies
- 0 kudos
- 968 Views
- 2 replies
- 0 kudos
checklist for : process to move and deploy in the prod
Hi Team,Could you please help me with best practices to move and deploy (code, workspace, notebooks, etc) in the prod?Regards,Phanindra
- 968 Views
- 2 replies
- 0 kudos
- 0 kudos
the most important is to use Repos!Link your workspace with git and use feature branches and pull requests to promote code/notebooks.Check the databricks docs on Repos. If you have further questions; shoot.
- 0 kudos
- 2170 Views
- 1 replies
- 0 kudos
Archival Strategy for Delta tables
Hi Team, We would like to define the archival strategy for data. Could you please share best practices /guide me on the below are the 3 use cases Case-1: On-Prem SQL and Oracle Data which is more than 20 years and they wanted to bring them into clou...
- 2170 Views
- 1 replies
- 0 kudos
- 0 kudos
case 1: I'd extract the data from the db to a data lake (cold storage if that is possible, that is cheaper) using an ETL tool like Data Factory, Glue etc. Then the archiving can take place. Perhaps also create a backup of the data on a 2nd data lak...
- 0 kudos
- 2593 Views
- 2 replies
- 0 kudos
Databricks setup/deployment checklist/best practices
Hi Team, could you please share or guide us on any checklist/best practices for Databricks setup/deployment?
- 2593 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Phani1 , here are some best practices https://github.com/Azure/AzureDatabricksBestPractices/tree/master and you could take these points as your "checklist".Choose the right Databricks Workspace:Decide on the appropriate Azure region for your Data...
- 0 kudos
- 2421 Views
- 2 replies
- 0 kudos
Databricks Certified Associate Developer for Apache Spark 3.0 - Python sample paper
I am trying to get sample papers for Databricks Certified Associate Developer for Apache Spark 3.0 - Python before I attempt my exam
- 2421 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi, there, This exam has a Practice Exam, I think that's what you are looking for.
- 0 kudos
