- 919 Views
- 0 replies
- 0 kudos
www.activdigital.marketing
https://www.activdigital.marketing/website-design/
- 919 Views
- 0 replies
- 0 kudos
Forum Posts
- 6702 Views
- 3 replies
- 6 kudos
Resolved! Unable to change the index, when writing to a Azure SQL Data Warehouse
Hi,I have some data in a spark data frame and I am trying to write it to a table in Azure SQL Data Warehouse. If I use df.write.mode(saveMode="overwrite") I get this error:com.microsoft.sqlserver.jdbc.SQLServerException: The statement failed. Column ...
- 6702 Views
- 3 replies
- 6 kudos
Latest Reply
- 6 kudos
Thanks @Hubert Dudek​ . Do you know if there is a way to run a create table statement on Azure Synapse/Azure SQL Datawarehouse from Databricks?
- 6 kudos
- 9227 Views
- 5 replies
- 5 kudos
Resolved! Recommended database when using R in databricks
I'm new to integrating the sparklyr / R interface in databricks. In particular it appears that sparklyr and R commands and functions are dependent upon the type of dataframe one is working with (hive, Spark R etc). Is there a recommend best practice...
- 9227 Views
- 5 replies
- 5 kudos
Latest Reply
- 5 kudos
Recommended is delta format in data lake. Here is code example https://docs.databricks.com/delta/quick-start.html#language-r
- 5 kudos
- 10884 Views
- 8 replies
- 10 kudos
Resolved! dbutils.fs.mv taking too long with delta table
I have a folder which contains multiple delta tables and some parquet tables. I want to move that folder to another path. When I use dbutils.fs.mv(), it takes an absurd amount of time.
- 10884 Views
- 8 replies
- 10 kudos
Latest Reply
- 10 kudos
Thanks @Hubert Dudek​ !Actually I want to delete the folder. But when I try to do that, I get this error: shaded.databricks.org.apache.hadoop.fs.azure.AzureException: hadoop_azure_shaded.com.microsoft.azure.storage.StorageException: This operation is...
- 10 kudos
- 1190 Views
- 0 replies
- 1 kudos
Hi folks, I am trying a particular use case, where I need to schedule a run of three different notebooks(pyspark,sql code) in sequence ​I need to use ...
Hi folks, I am trying a particular use case, where I need to schedule a run of three different notebooks(pyspark,sql code) in sequence ​I need to use date field as a common parameter in all three (date is part of the sql query in each nb where clause...
- 1190 Views
- 0 replies
- 1 kudos
- 9372 Views
- 6 replies
- 3 kudos
Resolved! Random error related to dynamic variables in SQL - at least one column must be specified (shallowclone table)
Environment: AzureA data transformation template that take s table name as variable. The variable is set in separate notebook %run ../_configShallow clone is used to generate a staging table with exact same columns of target table.A random error occ...
- 9372 Views
- 6 replies
- 3 kudos
Latest Reply
- 3 kudos
@lizou​ - That's great! I'm so glad. Would you be happy to share your solution with us and then mark the answer as best? That will help others find the solution more quickly.
- 3 kudos
- 5835 Views
- 2 replies
- 2 kudos
Resolved! How to find the identity column seed value?
How to find the identity column seed value? A seed value is required when we need specifically like start generating new values from a number (most likely we need to keep the original key values when data is reloaded from another source, and any new ...
- 5835 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
found it, thanks!of course, it will be nice to have a sql function available to query the value.example\"delta.identity.start\":984888,\"delta.identity.highWaterMark\":1004409,\"comment\":\"identity\",\"delta.identity.step\":1}
- 2 kudos
- 12952 Views
- 8 replies
- 3 kudos
Resolved! Clone the data from Prod Workspace to Test workspace
Hi there, I have requirement to deep clone the data from Prod workspace to Test Workspace. I know we can do in the same workspace, but I am not aware of between workspaces.I heard some key is needed and have some firewall issues when linking between ...
- 12952 Views
- 8 replies
- 3 kudos
Latest Reply
- 4136 Views
- 3 replies
- 3 kudos
Resolved! DataBricks Job Orchestration in PROD
Can I suggest customer about databricks Job orchestration (public preview) ? Can we use this feature in Production as well?
- 4136 Views
- 3 replies
- 3 kudos
Latest Reply
- 3 kudos
@ANOOP V​ , At present, we don’t have these features. I assume we do plan to include it by Q4 FY2022.What is next for Multitask Jobs?After GA we will be working on some highly-requested features during the private preview:- Job cluster reuse: make it...
- 3 kudos
- 29604 Views
- 5 replies
- 6 kudos
Resolved! Unable to make Databricks API calls from an HTML iframe rendered by a notebook's `displayHTML()` call, due to the browser enforcing CORS policy.
My GoalI want to make my Databricks Notebooks more interactive and have custom HTML/JS UI widgets that guide non-technical people through a business/data process. I want the HTML/JS widget to be able to execute a DB job, or execute some python code t...
- 29604 Views
- 5 replies
- 6 kudos
Latest Reply
- 6 kudos
displayHTML content is served in production from an iframe on the domain databricksusercontent.comThis iframe is sandboxed and it seems to prevent any callsSo even by creating a CORS-friendly server in the notebook exposed through the driver-proxy-ap...
- 6 kudos
- 5195 Views
- 3 replies
- 3 kudos
Resolved! Cluster terminated.Reason:Unexpected launch failure
Using Community Edition. Cluster starts up and immediately terminates with the following error message.Why would that be?MessageCluster terminated.Reason:Unexpected launch failureAn unexpected error was encountered while setting up the cluster. Pleas...
- 5195 Views
- 3 replies
- 3 kudos
Latest Reply
- 1836 Views
- 0 replies
- 1 kudos
- 1836 Views
- 0 replies
- 1 kudos
- 6403 Views
- 6 replies
- 4 kudos
Resolved! Learn Apache Spark
I want to learn Apache Spark for Developer, where do I start and want materials are recommended.
- 6403 Views
- 6 replies
- 4 kudos
Latest Reply
- 4 kudos
If you are a databricks customer (any paid subscription like Azure databricks), please register through https://databricks.com/learn/training/home to Academy using email from subscription.Course there are the best on the internet.If you will not see ...
- 4 kudos
- 23520 Views
- 4 replies
- 5 kudos
Resolved! How to provide UPSERT condition in PySpark
I have a table `demo_table_one` in which I want to upsert the following valuesdata = [ (11111 , 'CA', '2020-01-26'), (11111 , 'CA', '2020-02-26'), (88888 , 'CA', '2020-06-10'), (88888 , 'CA', '2020-05-10'), (88888 , 'WA', '2020-07-10'), ...
- 23520 Views
- 4 replies
- 5 kudos
Latest Reply
- 5 kudos
@John Constantine,​ can you additionally share what data is in demo_table_one? as we have only df (alias update_table) in that example
- 5 kudos
- 13364 Views
- 3 replies
- 2 kudos
Resolved! java.lang.NoSuchMethodError in databricks
I have created a package in scala. Now, I am calling a method from that package and using it in my notebook. During run time, it throws me an error java.lang.NoSuchMethodError. The method exists in the package but still, I am getting this error. Plea...
- 13364 Views
- 3 replies
- 2 kudos
Latest Reply
- 2 kudos
Hi! @Kaniz Fatma​ . I am using scala version 2.11 with spark 2.4.3. According to Apache spark official website https://spark.apache.org/docs/2.4.3/#:~:text=For%20the%20Scala%20API%2C%20Spark,x.) Spark 2.4.3 uses Scala 2.12. (https://spark.apache.org/...
- 2 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 584 | |
| 349 | |
| 287 |
