- 5483 Views
- 3 replies
- 3 kudos
Resolved! Pyspark installation issue
When I try to start pyspark session in pycharm. It throws me this error "RuntimeError("Java gateway process exited before sending its port number"). Could anyone help me to solve this?
- 5483 Views
- 3 replies
- 3 kudos
- 1052 Views
- 0 replies
- 0 kudos
IntelliMindz is the best IT Training in Chennai with Placement, offering 200 and more software courses with 100% Placement Assistance. Start learning ...
IntelliMindz is the best IT Training in Chennai with Placement, offering 200 and more software courses with 100% Placement Assistance. Start learning with us intellimindz, and became an expert in Online Training. Contact 9655877577 for more details.S...
- 1052 Views
- 0 replies
- 0 kudos
- 3187 Views
- 1 replies
- 2 kudos
No module named 'dependencies.spark'
Everyone help me solve bug No module named 'dependencies.spark'source code:from pyspark.sql import Rowfrom pyspark.sql.functions import col, concat_ws, litfrom dependencies.spark import start_spark
- 3187 Views
- 1 replies
- 2 kudos
- 2 kudos
What are you trying to do?
- 2 kudos
- 1808 Views
- 2 replies
- 0 kudos
Looking for information on security design on how JDBC connections to clusters function
I am looking for more information around the security design around how JDBC connections to clusters function:What security controls are in operation to safeguard the Databricks clusters?Is the API gateway abstracted from the Databricks cluster in th...
- 1808 Views
- 2 replies
- 0 kudos
- 0 kudos
In fact, I have no idea what kind of design we are talking about, it causes associations with Cloud Ceilings
- 0 kudos
- 740 Views
- 0 replies
- 0 kudos
www.activdigital.marketing
https://www.activdigital.marketing/website-design/
- 740 Views
- 0 replies
- 0 kudos
- 5533 Views
- 3 replies
- 6 kudos
Resolved! Unable to change the index, when writing to a Azure SQL Data Warehouse
Hi,I have some data in a spark data frame and I am trying to write it to a table in Azure SQL Data Warehouse. If I use df.write.mode(saveMode="overwrite") I get this error:com.microsoft.sqlserver.jdbc.SQLServerException: The statement failed. Column ...
- 5533 Views
- 3 replies
- 6 kudos
- 6 kudos
Thanks @Hubert Dudek​ . Do you know if there is a way to run a create table statement on Azure Synapse/Azure SQL Datawarehouse from Databricks?
- 6 kudos
- 8144 Views
- 5 replies
- 5 kudos
Resolved! Recommended database when using R in databricks
I'm new to integrating the sparklyr / R interface in databricks. In particular it appears that sparklyr and R commands and functions are dependent upon the type of dataframe one is working with (hive, Spark R etc). Is there a recommend best practice...
- 8144 Views
- 5 replies
- 5 kudos
- 5 kudos
Recommended is delta format in data lake. Here is code example https://docs.databricks.com/delta/quick-start.html#language-r
- 5 kudos
- 9167 Views
- 8 replies
- 10 kudos
Resolved! dbutils.fs.mv taking too long with delta table
I have a folder which contains multiple delta tables and some parquet tables. I want to move that folder to another path. When I use dbutils.fs.mv(), it takes an absurd amount of time.
- 9167 Views
- 8 replies
- 10 kudos
- 10 kudos
Thanks @Hubert Dudek​ !Actually I want to delete the folder. But when I try to do that, I get this error: shaded.databricks.org.apache.hadoop.fs.azure.AzureException: hadoop_azure_shaded.com.microsoft.azure.storage.StorageException: This operation is...
- 10 kudos
- 951 Views
- 0 replies
- 1 kudos
Hi folks, I am trying a particular use case, where I need to schedule a run of three different notebooks(pyspark,sql code) in sequence ​I need to use ...
Hi folks, I am trying a particular use case, where I need to schedule a run of three different notebooks(pyspark,sql code) in sequence ​I need to use date field as a common parameter in all three (date is part of the sql query in each nb where clause...
- 951 Views
- 0 replies
- 1 kudos
- 7756 Views
- 6 replies
- 3 kudos
Resolved! Random error related to dynamic variables in SQL - at least one column must be specified (shallowclone table)
Environment: AzureA data transformation template that take s table name as variable. The variable is set in separate notebook %run ../_configShallow clone is used to generate a staging table with exact same columns of target table.A random error occ...
- 7756 Views
- 6 replies
- 3 kudos
- 3 kudos
@lizou​ - That's great! I'm so glad. Would you be happy to share your solution with us and then mark the answer as best? That will help others find the solution more quickly.
- 3 kudos
- 5020 Views
- 2 replies
- 2 kudos
Resolved! How to find the identity column seed value?
How to find the identity column seed value? A seed value is required when we need specifically like start generating new values from a number (most likely we need to keep the original key values when data is reloaded from another source, and any new ...
- 5020 Views
- 2 replies
- 2 kudos
- 2 kudos
found it, thanks!of course, it will be nice to have a sql function available to query the value.example\"delta.identity.start\":984888,\"delta.identity.highWaterMark\":1004409,\"comment\":\"identity\",\"delta.identity.step\":1}
- 2 kudos
- 11020 Views
- 8 replies
- 3 kudos
Resolved! Clone the data from Prod Workspace to Test workspace
Hi there, I have requirement to deep clone the data from Prod workspace to Test Workspace. I know we can do in the same workspace, but I am not aware of between workspaces.I heard some key is needed and have some firewall issues when linking between ...
- 11020 Views
- 8 replies
- 3 kudos
- 3363 Views
- 3 replies
- 3 kudos
Resolved! DataBricks Job Orchestration in PROD
Can I suggest customer about databricks Job orchestration (public preview) ? Can we use this feature in Production as well?
- 3363 Views
- 3 replies
- 3 kudos
- 3 kudos
@ANOOP V​ , At present, we don’t have these features. I assume we do plan to include it by Q4 FY2022.What is next for Multitask Jobs?After GA we will be working on some highly-requested features during the private preview:- Job cluster reuse: make it...
- 3 kudos
- 25939 Views
- 5 replies
- 6 kudos
Resolved! Unable to make Databricks API calls from an HTML iframe rendered by a notebook's `displayHTML()` call, due to the browser enforcing CORS policy.
My GoalI want to make my Databricks Notebooks more interactive and have custom HTML/JS UI widgets that guide non-technical people through a business/data process. I want the HTML/JS widget to be able to execute a DB job, or execute some python code t...
- 25939 Views
- 5 replies
- 6 kudos
- 6 kudos
displayHTML content is served in production from an iframe on the domain databricksusercontent.comThis iframe is sandboxed and it seems to prevent any callsSo even by creating a CORS-friendly server in the notebook exposed through the driver-proxy-ap...
- 6 kudos
- 4259 Views
- 3 replies
- 3 kudos
Resolved! Cluster terminated.Reason:Unexpected launch failure
Using Community Edition. Cluster starts up and immediately terminates with the following error message.Why would that be?MessageCluster terminated.Reason:Unexpected launch failureAn unexpected error was encountered while setting up the cluster. Pleas...
- 4259 Views
- 3 replies
- 3 kudos
| User | Count |
|---|---|
| 1629 | |
| 790 | |
| 511 | |
| 349 | |
| 287 |
