- 18395 Views
- 3 replies
- 3 kudos
Facing Issues with Databricks JDBC Connectivity after Idle time
Hello team, I am using commons(commons-dbcp2) Datasource which supports default connection pooling in Spring Java application (rest services to fetch databricks data via JDBC template).Initially all works fine and can get the data from databricks via...
- 18395 Views
- 3 replies
- 3 kudos
- 3 kudos
I am seeing the same issue with hikari. When a pooled connection is created then the databricks cluster is terminated (or restarted), the HikariDataSource retains a stale session handle.Why does connection.isValid() returns true then executing any qu...
- 3 kudos
- 1524 Views
- 1 replies
- 0 kudos
validating record count at SQL server database tabbles with migrated azure data lake gen2
we are migrating out project from on-premise to azure , so on-premise database is the SQL server that we are using and azure data lake gen2 is the storage location where store data currently and so far we are currently validating record count of each...
- 1524 Views
- 1 replies
- 0 kudos
- 0 kudos
we are migrating out project from on-premise to azure , so on-premise database is the SQL server that we are using and azure data lake gen2 is the storage location where store data currently and so far we are currently validating record count of each...
- 0 kudos
- 2035 Views
- 1 replies
- 0 kudos
Databricks Certified Data Engineer Associate Exam suspended
Initially, My exam was suspended after 20 mins on me into the exam. I stated that there was some internet issue from my end(however there wasn't any). However, I immediately reached out to Kryterion folks but they responded after 50 minutes and allow...
- 2035 Views
- 1 replies
- 0 kudos
- 0 kudos
Adding my username - akhil.inala@tredence.comexam - Databricks Certified Data Engineer Associate Exam
- 0 kudos
- 3322 Views
- 0 replies
- 0 kudos
Databricks job runs - metadata tables
Hello experts,Do you know if there are available metadata/log tables with information on databricks job runs instead of using the Workflows UI?Thank you!
- 3322 Views
- 0 replies
- 0 kudos
- 911 Views
- 0 replies
- 0 kudos
Cloning Github Repository Accessibility
Hi all,I am trying to clone GitHub repository on the databricks notebook, but there are some accessibility issues faced here.Already linked the GitHub in the User Settings but still not working in the command on the notebook.
- 911 Views
- 0 replies
- 0 kudos
- 953 Views
- 0 replies
- 0 kudos
Pandas API on Spark creates huge query plans
Hello,I have a piece of code written in Pyspark and Pandas API on Spark. On comparing the query plans, I see Pandas API on Spark creates huge query plans whereas Pyspark plan is a tiny one. Furthermore, with Pandas API on spark, we see a lot of incon...
- 953 Views
- 0 replies
- 0 kudos
- 2510 Views
- 1 replies
- 1 kudos
How connect connection ORACLE onpremise in Databricks for extract data
Hello, How Connect in Databricks Enterprise connection Oracle Onpremise and that permissions is necessary.Thank you
- 2510 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello, How Connect in Databricks Enterprise connection Oracle Onpremise and that permissions is necessary.Thank you
- 1 kudos
- 872 Views
- 0 replies
- 0 kudos
Using map_filter / filter to return only map elements for a given predicate
I want to apply a filter to a map structure (on a column called "ActivityMap") for elements only where a given predicate holds. Representative data is below. Applying both "map_filter" and "array_contains" will return rows where a predicate holds, ho...
- 872 Views
- 0 replies
- 0 kudos
- 1350 Views
- 0 replies
- 0 kudos
Questions about the new Serverless SQL
I have a list of questions about the Serverless SQL option:As per the docs, it says it's hosted by Databricks. Is there any exception to that? Will it ever create ec2 instances on AWS for the serverless option?Are any serverless asset stored on custo...
- 1350 Views
- 0 replies
- 0 kudos
- 4538 Views
- 2 replies
- 4 kudos
Can we apply multiple exam discount voucher to one exam?
I have a 75% off certification exam voucher using my working email, and i also noticed that you can redeem a 25% off voucher from the rewards store. I am wondering if you can add these 2 vouchers together at the same time so that you can get 1 exam 1...
- 4538 Views
- 2 replies
- 4 kudos
- 4 kudos
Hi chenqian0562,Could you please share the solution with the community? I have the same issue.
- 4 kudos
- 1287 Views
- 0 replies
- 0 kudos
How can I resolve a problem between AWS and Databricks platform?
This is Algograp Co., Ltd., a partner of Databricks.There was a problem in the process of subscribing to Databricks and linking AWS account.I had a problem using my existing platform and canceled my subscription. Account activation is not possible du...
- 1287 Views
- 0 replies
- 0 kudos
- 6266 Views
- 0 replies
- 0 kudos
DLT use case
Is Delta Live Tables (DLT) appropriate for data that is in the millions of rows and GB sized? Or is DLT only optimal for larger data with billions of rows and TB sized?Please consider the Total Cost of Ownership.Development costs (engineering time)O...
- 6266 Views
- 0 replies
- 0 kudos
- 5637 Views
- 1 replies
- 1 kudos
Set default database thru Cluster Spark Configuration
Set the default catalog (AKA default SQL Database) in a Cluster's Spark configuration. I've tried the following :spark.catalog.setCurrentDatabase("cbp_reporting_gold_preprod") - this works in a Notebook but doesn't do anything in the Cluster.spark.sq...
- 5637 Views
- 1 replies
- 1 kudos
- 1 kudos
I've tried different commands in the Cluster's Spark Config, none work, they execute at Cluster startup w/o any errors shown in the logs, but once you run a notebook attached to the cluster Default catalog is still set to 'default'.
- 1 kudos
- 2936 Views
- 4 replies
- 1 kudos
Resolved! My exam has suspended by an unprofessional proctor- need help to reschedule
case #00378268 I passed Spark Developer associate exam about 6 month ago with great experience. However, this time the proctor did not even bother to show up to start the exam - checking ID, the room and the surroundings. Somehow, I was able to st...
- 2936 Views
- 4 replies
- 1 kudos
- 3205 Views
- 2 replies
- 0 kudos
How to Create a DataBricks Notebook using API
import requestsimport json# Databricks workspace API URLdatabricks_url = "https://dbc-ab846cbe-f48b.cloud.databricks.com/api/2.0/workspace/import"# Databricks API token (generate one from your Databricks account)databricks_token = "xxxxxxxxxxxxxxxxxx...
- 3205 Views
- 2 replies
- 0 kudos
- 0 kudos
import requestsimport json# Databricks workspace API URLdatabricks_url = "https://dbc-ab846cbe-f48b.cloud.databricks.com/api/2.0/workspace/import"# Databricks API token (generate one from your Databricks account)databricks_token = "xxxxxxxxxxxxxxxxxx...
- 0 kudos
