- 3378 Views
- 2 replies
- 0 kudos
Resolved! Error during setup of cluster
Unexpected Launch Failure: An unexpected error was encountered while setting up the cluster. Please retry and contact Azure Databricks if the problem persists. Internal error message: Timeout while placing node.
- 3378 Views
- 2 replies
- 0 kudos
- 0 kudos
Ensure that CNO$ account has Full Control on the CNO and The computers container;Add CNO$ account (CNO computer object) in Local Admins group;Finally, add CNO$ in Domain Admins group.Regards,Willjoe
- 0 kudos
- 103555 Views
- 6 replies
- 13 kudos
Can databricks be used locally to learn it or is it cloud only
I'm tired of telling clients or referrals I don't know databricks but it seems like the only option is to have a big AWS account and then use databricks on that data. Can I download it locally for training, upskilling with python or is it only for cl...
- 103555 Views
- 6 replies
- 13 kudos
- 13 kudos
Thanks for linking directly to the docker image @Hubert Dudek​ ! And thanks for the info @Prabakar Ammeappin​ and @Amit Nainawati​ @Andrew Schell​ Let us know if you have more questions! If not, choose a best answer in this thread and let us know how...
- 13 kudos
- 9042 Views
- 2 replies
- 3 kudos
How do I move existing workflows and jobs running on an all-purpose cluster to a shared jobs cluster?
A Databricks cluster is a set of computation resources that performs the heavy lifting of all of the data workloads you run in Databricks. Databricks provides a number of options when you create and configure clusters to help you get the best perform...
- 9042 Views
- 2 replies
- 3 kudos
- 3 kudos
@Doug Harrigan​ Thanks for your question! @Prabakar Ammeappin​ linked above to our Docs page that mentions a bit more about the recent (April) version update/change: "This release fixes an issue that removed the Swap cluster button from the Databrick...
- 3 kudos
- 13887 Views
- 7 replies
- 5 kudos
Resolved! Job fails after runtime upgrade
I have a job running with no issues in Databricks runtime 7.3 LTS. When I upgraded to 8.3 it fails with error An exception was thrown from a UDF: 'pyspark.serializers.SerializationError'... SparkContext should only be created and accessed on the driv...
- 13887 Views
- 7 replies
- 5 kudos
- 5 kudos
Adding to @Sean Owen​ comments, The only reason this is working is that the optimizer is evaluating this locally rather than creating a context on executors and evaluating it.
- 5 kudos
- 10147 Views
- 1 replies
- 1 kudos
Resolved! query array in SQL
Hello I have a databricks question I was not able to answer myselfI have this queryselect count(*) from tablewhere object[0].value is not null and object[0].value.value1 = "s"and created_year = 2022 and created_month = 7 and created_day = 4you can se...
- 10147 Views
- 1 replies
- 1 kudos
- 1 kudos
SELECT count(*)FROM ( SELECT explode(mmycolumn) FROM table WHERE created_year = 2022 and created_month = 7 and created_day = 5)WHERE col.field is not null and col.field.field! = "signal"
- 1 kudos
- 2867 Views
- 0 replies
- 2 kudos
Delta Table Merge Operation logs Output is not correct number of updated records?
Hi all,I am performing merge operation on my delta table in spark. I have existing delta table , it already has some records. Now I created another dataframe of csv file, and added one new record and updated one records in that. Please check below sn...
- 2867 Views
- 0 replies
- 2 kudos
- 11120 Views
- 7 replies
- 2 kudos
Resolved! Unable to read file from S3
I tried to read a file from S3, but facing the below error:org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 53.0 failed 4 times, most recent failure: Lost task 0.3 in stage 53.0 (TID 82, xx.xx.xx.xx, executor 0): com...
- 11120 Views
- 7 replies
- 2 kudos
- 2 kudos
Which DBR version are you using? Could you please test it with a different DBR version probably DBR 9.x?
- 2 kudos
- 4238 Views
- 4 replies
- 6 kudos
Resolved! why this not able to go through?
https://textdoc.co/index.php/UFEQdwxWn60LtOVfError:https://textdoc.co/index.php/3JisnHKGkvLIaAOF
- 4238 Views
- 4 replies
- 6 kudos
- 6 kudos
It would be best if you used Databricks ML runtime (in cluster settings), not the standard one.
- 6 kudos
- 3126 Views
- 2 replies
- 0 kudos
Resolved! Save data from Spark DataFrames to TFRecords
https://docs.microsoft.com/en-us/azure/databricks/_static/notebooks/deep-learning/tfrecords-save-load.htmlI could not run the Cell # 2java.lang.ClassNotFoundException: --------------------------------------------------------------------------- Py4JJ...
- 3126 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @THIAM HUAT TAN​,Which DBR version are you using? are you using the ML runtime?
- 0 kudos
- 3387 Views
- 1 replies
- 1 kudos
Unity Catalog will allow you to bring your own HMS
Anyone know more about how the Unity Catalog will allow you to bring your own HMS (eg Glue)?Will this be treated as a separate 'catalog', which you can access but you can't use the other features of Unity Catalog on eg ABAC etcAny reading on this top...
- 3387 Views
- 1 replies
- 1 kudos
- 1 kudos
@Kunal Gaurav​ yes, it is treated as a synthetic catalog. You can query it by using the convention "hive_metastore" as the catalog name. i.e. SELECT * FROM hive_metastore.schema_name.table_nameThis will work for internal HMS, external HMS and Glue.Yo...
- 1 kudos
- 5224 Views
- 2 replies
- 2 kudos
Resolved! How would you integrate Unity Catalog with existing Data governance solutions like Collibra?
- 5224 Views
- 2 replies
- 2 kudos
- 2 kudos
@Prashanth P​ We offer a fully featured REST API with Unity Catalog that provides the ability to CRUD objects such as catalogs/schemas/tables/acls/lineage etc.Companies like Colliba/Alation/MS Purview etc use these in middleware integrations to integ...
- 2 kudos
- 2815 Views
- 2 replies
- 1 kudos
Dbt vs databricks sql
Can I use db sql to replace dbt?
- 2815 Views
- 2 replies
- 1 kudos
- 3841 Views
- 2 replies
- 1 kudos
Parallelize spark jobs on the same cluster?
Whats the best way to parallelize multiple spark jobs on the same cluster during a backfill?
- 3841 Views
- 2 replies
- 1 kudos
- 1 kudos
In the past I used direct multi-threaded orchestration inside of driver applications, but that was prior to Databricks supporting multi-task jobs.If you create a job through the workflows tab, you can set up multiple notebooks, python, or jar tasks t...
- 1 kudos
- 4508 Views
- 3 replies
- 3 kudos
Resolved! Databricks Workflows
How do I get started with Databricks Workflows?
- 4508 Views
- 3 replies
- 3 kudos
- 6336 Views
- 4 replies
- 4 kudos
Resolved! SQL Serverless Endpoint failing to start with Instance Profile
Hi allSuper stoked about the PP of SQL Serverless, but it does seem that the instance profile Im using doesnt have the required trust relationship for it to work with the Sererless Endpoint. Altough on "classic" mode it works fine. Does Serverless re...
- 6336 Views
- 4 replies
- 4 kudos
- 4 kudos
Thank you for sharing your valuable solution, it's work properly.mymilestonecard
- 4 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 582 | |
| 349 | |
| 287 |
