IntelliMindz is the best IT Training in Chennai with Placement, offering 200 and more software courses with 100% Placement Assistance. Start learning with us intellimindz, and became an expert in Online Training. Contact 9655877577 for more details.S...
Everyone help me solve bug No module named 'dependencies.spark'source code:from pyspark.sql import Rowfrom pyspark.sql.functions import col, concat_ws, litfrom dependencies.spark import start_spark
What are you trying to do?
I am looking for more information around the security design around how JDBC connections to clusters function:What security controls are in operation to safeguard the Databricks clusters?Is the API gateway abstracted from the Databricks cluster in th...
In fact, I have no idea what kind of design we are talking about, it causes associations with Cloud Ceilings
https://www.activdigital.marketing/website-design/
Hi,I have some data in a spark data frame and I am trying to write it to a table in Azure SQL Data Warehouse. If I use df.write.mode(saveMode="overwrite") I get this error:com.microsoft.sqlserver.jdbc.SQLServerException: The statement failed. Column ...
Thanks @Hubert Dudek​ . Do you know if there is a way to run a create table statement on Azure Synapse/Azure SQL Datawarehouse from Databricks?
I'm new to integrating the sparklyr / R interface in databricks. In particular it appears that sparklyr and R commands and functions are dependent upon the type of dataframe one is working with (hive, Spark R etc). Is there a recommend best practice...
Recommended is delta format in data lake. Here is code example https://docs.databricks.com/delta/quick-start.html#language-r
I have a folder which contains multiple delta tables and some parquet tables. I want to move that folder to another path. When I use dbutils.fs.mv(), it takes an absurd amount of time.
Thanks @Hubert Dudek​ !Actually I want to delete the folder. But when I try to do that, I get this error: shaded.databricks.org.apache.hadoop.fs.azure.AzureException: hadoop_azure_shaded.com.microsoft.azure.storage.StorageException: This operation is...
Hi folks, I am trying a particular use case, where I need to schedule a run of three different notebooks(pyspark,sql code) in sequence ​I need to use date field as a common parameter in all three (date is part of the sql query in each nb where clause...
Environment: AzureA data transformation template that take s table name as variable. The variable is set in separate notebook %run ../_configShallow clone is used to generate a staging table with exact same columns of target table.A random error occ...
@lizou​ - That's great! I'm so glad. Would you be happy to share your solution with us and then mark the answer as best? That will help others find the solution more quickly.
How to find the identity column seed value? A seed value is required when we need specifically like start generating new values from a number (most likely we need to keep the original key values when data is reloaded from another source, and any new ...
found it, thanks!of course, it will be nice to have a sql function available to query the value.example\"delta.identity.start\":984888,\"delta.identity.highWaterMark\":1004409,\"comment\":\"identity\",\"delta.identity.step\":1}
Hi there, I have requirement to deep clone the data from Prod workspace to Test Workspace. I know we can do in the same workspace, but I am not aware of between workspaces.I heard some key is needed and have some firewall issues when linking between ...
Can I suggest customer about databricks Job orchestration (public preview) ? Can we use this feature in Production as well?
@ANOOP V​ , At present, we don’t have these features. I assume we do plan to include it by Q4 FY2022.What is next for Multitask Jobs?After GA we will be working on some highly-requested features during the private preview:- Job cluster reuse: make it...
My GoalI want to make my Databricks Notebooks more interactive and have custom HTML/JS UI widgets that guide non-technical people through a business/data process. I want the HTML/JS widget to be able to execute a DB job, or execute some python code t...
displayHTML content is served in production from an iframe on the domain databricksusercontent.comThis iframe is sandboxed and it seems to prevent any callsSo even by creating a CORS-friendly server in the notebook exposed through the driver-proxy-ap...
Using Community Edition. Cluster starts up and immediately terminates with the following error message.Why would that be?MessageCluster terminated.Reason:Unexpected launch failureAn unexpected error was encountered while setting up the cluster. Pleas...
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 580 | |
| 349 | |
| 287 |
