- 1267 Views
- 1 replies
- 0 kudos
cross join issue generating surrogate keys in delta table
I used below code to populate target table, it is working as expected but expect for surrogatekey column. After I inserted dummy entry -1 and did run merge code, it is generating the numbers in surrogatekey column like this 1,3,5,7,9 odd numbers, it ...
- 1267 Views
- 1 replies
- 0 kudos
- 0 kudos
This is expected. Identity columns can have gaps, or per the documentation: Values assigned by identity columns are unique and increment in the direction of the specified step, and in multiples of the specified step size, but are not guaranteed to b...
- 0 kudos
- 272 Views
- 4 replies
- 2 kudos
Expose Databricks Cluster Logs to Grafana
I am trying to send Databricks cluster logs to Grafana using an init_script where I define the system journals to consume. The issue I am facing is that I cannot get the driver logs, standard error, and output to reach Grafana. Is there something spe...
- 272 Views
- 4 replies
- 2 kudos
- 3584 Views
- 3 replies
- 1 kudos
Spatial Queries
Hi,I'm trying to execute the following code:%sqlSELECT LSOA21CD, ST_X(ST_GeomFromWKB(Geom_Varbinary)) AS STX, ST_Y(ST_GeomFromWKB(Geom_Varbinary)) AS STYFROM ordnance_survey_lsoas_december_2021_population_weighted_centroidsWHERE LSOA21CD ...
- 3584 Views
- 3 replies
- 1 kudos
- 1 kudos
we also need this function without the private preview, any solutions?
- 1 kudos
- 674 Views
- 5 replies
- 2 kudos
Issue with Non-Bulk Inserts Using JDBC on Databricks Runtime 14.1
Hello team,I am experiencing an issue with insert operations on Databricks using the JDBC driver. In my SAS Viya system, the DatabricksJDBC42.jar driver version 2.6.40 is configured. I’ve noticed that, up to Databricks Runtime version 13.1, insert op...
- 674 Views
- 5 replies
- 2 kudos
- 2 kudos
Dear @Walter_C , After an initial test with a moderate number of rows, we believed it to be the optimal solution.However, upon increasing the row count beyond a million, we realized the dataset is divided into chunks of unclear size. Sometimes chunks...
- 2 kudos
- 247 Views
- 3 replies
- 0 kudos
Supporting Material for self phased Data Analysis with Databricks Course
Hi All,Newbie here, Any idea where I can find the supporting materials that are used in the online "Data Analysis with Databricks Course" that the instructor is using?. It seems to be having the scripts to create schema, tables, etcThanks in advance
- 247 Views
- 3 replies
- 0 kudos
- 0 kudos
Hello @BSalla, here is our suggested learning path. I hope it helps you!
- 0 kudos
- 275 Views
- 2 replies
- 1 kudos
Finding Materials for Databricks Course
Hi All,Where can I find the supporting materials used by the instructor in the online "Data Analysis with Databricks" course? It appears to include scripts for creating schemas, tables, and other database structures.Thanks in advance.
- 275 Views
- 2 replies
- 1 kudos
- 1 kudos
Hello @RoseCliver1 and @BSalla! To access the course supporting materials, please raise a support ticket here.
- 1 kudos
- 3663 Views
- 2 replies
- 0 kudos
Databricks Notebook says "Connecting.." for some users
For some users, after clicking on a notebook the screen says "connecting..." and the notebook does not open.The users are using Chrome browser and the same happens with Edge as well.What could be the reason?
- 3663 Views
- 2 replies
- 0 kudos
- 0 kudos
Same issue here, just tried to get set up with community edition but no luck
- 0 kudos
- 471 Views
- 1 replies
- 1 kudos
Resolved! read file from local machine(my computer) and create a dataframe
I want to create a notebook and add a widget which will allow the user to select a file from local machine(my computer) and read the contents of the file and create a dataframe. is it possible? and how? in dbutils.widgets i dont have any options for ...
- 471 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @khaansaab ,Currently, there is now out of the box mechanism that will allow you to that. As a workaround,you can create a UC volume and tell your users to upload files into that volume. Then you can create notebook that will have file_name parame...
- 1 kudos
- 3409 Views
- 2 replies
- 1 kudos
Resolved! Configure Databricks in VSCode through WSL
Hi,I am having a hard time configuring my Databricks workspace when working in VSCode via WSL. When following the steps to setup Databricks authentication I am receiving the following error on the Step 5 of "Step 4: Set up Databricks authentication"....
- 3409 Views
- 2 replies
- 1 kudos
- 1 kudos
What worked for me was NOT opening the browser using the pop-up (which generated the 3-legged-OAuth flow error), but clicking on the link provided by the CLI (or copy paste the link on the browser)
- 1 kudos
- 5852 Views
- 1 replies
- 0 kudos
SAP Successfator
Hi Team,We are working on a new Data Product onboarding to the current Databricks Lakehouse Platform.The first step is foundation where we should get data from SAP success factors to S3+ Bronze layer and then do the initial setup of Lakehouse+Power B...
- 5852 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi Phani1, Were you able to get any success with this?
- 0 kudos
- 7947 Views
- 3 replies
- 0 kudos
Space in Column names when writing to Hive
All,I have the following code.df_Warehouse_Utilization = ( spark.table("hive_metastore.dev_ork.bin_item_detail") .join(df_DIM_Bins,col('bin_tag')==df_DIM_Bins.BinKey,'right') .groupby(col('BinKey')) .agg(count_distinct(when(col('serial_lo...
- 7947 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi,I have faced this issue a few times. When we are overwriting the dataframes to hive catalog in databricks, it doesn't naturally allow for column names to have spaces or special characters. However, you can add an option statement to bypass that ru...
- 0 kudos
- 4537 Views
- 18 replies
- 4 kudos
Mounting Data IOException
Hello,I am currently taking a course from Coursera for data science using SQL. For one of our assignments we need to mount some data by running a script that has been provided to us by the class. When I run the script I receive the following error. I...
- 4537 Views
- 18 replies
- 4 kudos
- 4 kudos
I have resolved this issue. Please see the Databricks notebook below. Thanks to Tawfeeq for the helpful method.https://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/2108650195107345/4043517972039275/874892524...
- 4 kudos
- 611 Views
- 1 replies
- 0 kudos
Connect to SQL Developer using Custom JDBC
Hello,I'm trying to connect databricks SQL to the SQL Developer using custom JDBC.I'm getting errorjdbc:databricks:<server>:443;HttpPath=<HttpPath>;UID=token;PWD=<password> RegardsNaga
- 611 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi,We are trying to test if we can connect sql developer to databricks. did it work for you?Regards, shalini
- 0 kudos
- 616 Views
- 6 replies
- 0 kudos
Unity Catalog About Metastore
Registered on 2024/10 from AWS marketplace.We have created a customer management VPC and manually created the workspace.No specific metastore settings were made when the workspace was created.In the catalog screen of the account console,unity catalog...
- 616 Views
- 6 replies
- 0 kudos
- 0 kudos
Hello @tyorisoo,I hope you are doing well!Metastore manages metadata, not catalog information, schema information, table information, function information, access control information, etc. In the current state, the metastore configuration is not done...
- 0 kudos
- 1042 Views
- 3 replies
- 0 kudos
Databricks CI/CD
Hi Team,How can we implement CI/CD in Databricks and its best practices, and tools to consider?Regards,Janga
- 1042 Views
- 3 replies
- 0 kudos
