- 4564 Views
- 2 replies
- 3 kudos
- 4564 Views
- 2 replies
- 3 kudos
- 3 kudos
@Hemanth​ , example :spark.sql("select substring(25.45678,0, instr(25.45678,'.')+2) as out").show() will give
- 3 kudos
- 4610 Views
- 3 replies
- 0 kudos
Resolved! Bug Report: "Unsubscribed from" emails for deleted jobs have bad templating
I guess someone inverted the tokens in the template, because the emails look like this:Subject: "[user@company.com] Unsubscribed from 'Job'"Body: "This job has been deleted by dbc-12345678-1234."But it should look like this instead:Subject: "[dbc-123...
- 4610 Views
- 3 replies
- 0 kudos
- 0 kudos
The bug reported has been fixed and merged. It will be deployed in the next release, which is planned for tomorrow in the PST time zone. !!!! Thanks to @Marco Roy​
- 0 kudos
- 6838 Views
- 3 replies
- 0 kudos
Resolved! SSO with Auth0?
Do you support SSO with any IdP which supports SAML 2.0 (e.g. Auth0) or is it limited to https://docs.databricks.com/administration-guide/users-groups/single-sign-on/index.html#supported-identity-providers?
- 6838 Views
- 3 replies
- 0 kudos
- 0 kudos
I am currently having few applications (say App1, App2) along with databricks all integrated with auth0. Now what I wanted to achieve is that when we login to say databricks and then access other apps url in another tab it should not ask for login in...
- 0 kudos
- 8507 Views
- 3 replies
- 3 kudos
Resolved! Pyspark installation issue
When I try to start pyspark session in pycharm. It throws me this error "RuntimeError("Java gateway process exited before sending its port number"). Could anyone help me to solve this?
- 8507 Views
- 3 replies
- 3 kudos
- 1571 Views
- 0 replies
- 0 kudos
IntelliMindz is the best IT Training in Chennai with Placement, offering 200 and more software courses with 100% Placement Assistance. Start learning ...
IntelliMindz is the best IT Training in Chennai with Placement, offering 200 and more software courses with 100% Placement Assistance. Start learning with us intellimindz, and became an expert in Online Training. Contact 9655877577 for more details.S...
- 1571 Views
- 0 replies
- 0 kudos
- 3631 Views
- 1 replies
- 2 kudos
No module named 'dependencies.spark'
Everyone help me solve bug No module named 'dependencies.spark'source code:from pyspark.sql import Rowfrom pyspark.sql.functions import col, concat_ws, litfrom dependencies.spark import start_spark
- 3631 Views
- 1 replies
- 2 kudos
- 2 kudos
What are you trying to do?
- 2 kudos
- 2420 Views
- 2 replies
- 0 kudos
Looking for information on security design on how JDBC connections to clusters function
I am looking for more information around the security design around how JDBC connections to clusters function:What security controls are in operation to safeguard the Databricks clusters?Is the API gateway abstracted from the Databricks cluster in th...
- 2420 Views
- 2 replies
- 0 kudos
- 0 kudos
In fact, I have no idea what kind of design we are talking about, it causes associations with Cloud Ceilings
- 0 kudos
- 6851 Views
- 3 replies
- 6 kudos
Resolved! Unable to change the index, when writing to a Azure SQL Data Warehouse
Hi,I have some data in a spark data frame and I am trying to write it to a table in Azure SQL Data Warehouse. If I use df.write.mode(saveMode="overwrite") I get this error:com.microsoft.sqlserver.jdbc.SQLServerException: The statement failed. Column ...
- 6851 Views
- 3 replies
- 6 kudos
- 6 kudos
Thanks @Hubert Dudek​ . Do you know if there is a way to run a create table statement on Azure Synapse/Azure SQL Datawarehouse from Databricks?
- 6 kudos
- 9486 Views
- 5 replies
- 5 kudos
Resolved! Recommended database when using R in databricks
I'm new to integrating the sparklyr / R interface in databricks. In particular it appears that sparklyr and R commands and functions are dependent upon the type of dataframe one is working with (hive, Spark R etc). Is there a recommend best practice...
- 9486 Views
- 5 replies
- 5 kudos
- 5 kudos
Recommended is delta format in data lake. Here is code example https://docs.databricks.com/delta/quick-start.html#language-r
- 5 kudos
- 11163 Views
- 8 replies
- 10 kudos
Resolved! dbutils.fs.mv taking too long with delta table
I have a folder which contains multiple delta tables and some parquet tables. I want to move that folder to another path. When I use dbutils.fs.mv(), it takes an absurd amount of time.
- 11163 Views
- 8 replies
- 10 kudos
- 10 kudos
Thanks @Hubert Dudek​ !Actually I want to delete the folder. But when I try to do that, I get this error: shaded.databricks.org.apache.hadoop.fs.azure.AzureException: hadoop_azure_shaded.com.microsoft.azure.storage.StorageException: This operation is...
- 10 kudos
- 1246 Views
- 0 replies
- 1 kudos
Hi folks, I am trying a particular use case, where I need to schedule a run of three different notebooks(pyspark,sql code) in sequence ​I need to use ...
Hi folks, I am trying a particular use case, where I need to schedule a run of three different notebooks(pyspark,sql code) in sequence ​I need to use date field as a common parameter in all three (date is part of the sql query in each nb where clause...
- 1246 Views
- 0 replies
- 1 kudos
- 9876 Views
- 6 replies
- 3 kudos
Resolved! Random error related to dynamic variables in SQL - at least one column must be specified (shallowclone table)
Environment: AzureA data transformation template that take s table name as variable. The variable is set in separate notebook %run ../_configShallow clone is used to generate a staging table with exact same columns of target table.A random error occ...
- 9876 Views
- 6 replies
- 3 kudos
- 3 kudos
@lizou​ - That's great! I'm so glad. Would you be happy to share your solution with us and then mark the answer as best? That will help others find the solution more quickly.
- 3 kudos
- 6008 Views
- 2 replies
- 2 kudos
Resolved! How to find the identity column seed value?
How to find the identity column seed value? A seed value is required when we need specifically like start generating new values from a number (most likely we need to keep the original key values when data is reloaded from another source, and any new ...
- 6008 Views
- 2 replies
- 2 kudos
- 2 kudos
found it, thanks!of course, it will be nice to have a sql function available to query the value.example\"delta.identity.start\":984888,\"delta.identity.highWaterMark\":1004409,\"comment\":\"identity\",\"delta.identity.step\":1}
- 2 kudos
- 13212 Views
- 8 replies
- 3 kudos
Resolved! Clone the data from Prod Workspace to Test workspace
Hi there, I have requirement to deep clone the data from Prod workspace to Test Workspace. I know we can do in the same workspace, but I am not aware of between workspaces.I heard some key is needed and have some firewall issues when linking between ...
- 13212 Views
- 8 replies
- 3 kudos
- 4235 Views
- 3 replies
- 3 kudos
Resolved! DataBricks Job Orchestration in PROD
Can I suggest customer about databricks Job orchestration (public preview) ? Can we use this feature in Production as well?
- 4235 Views
- 3 replies
- 3 kudos
- 3 kudos
@ANOOP V​ , At present, we don’t have these features. I assume we do plan to include it by Q4 FY2022.What is next for Multitask Jobs?After GA we will be working on some highly-requested features during the private preview:- Job cluster reuse: make it...
- 3 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 586 | |
| 349 | |
| 287 |
