cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

databrick53
by New Contributor
  • 296 Views
  • 5 replies
  • 0 kudos

can't execute the code

When I was executing the code, I was getting this error:"Notebook detached×Exception when creating execution context: java.net.SocketTimeoutException: Connect Timeout"Can someone help me?

  • 296 Views
  • 5 replies
  • 0 kudos
Latest Reply
Lakshay
Esteemed Contributor
  • 0 kudos

Hi @databrick53 , Could you please share what was the resolution?

  • 0 kudos
4 More Replies
hossein_kolahdo
by New Contributor
  • 50 Views
  • 0 replies
  • 0 kudos

Accessing data from a legacy hive metastore workspace on a new Unity Catalog workspace

Hello,For the purposes of testing I'm interested in creating a new workspace with Unity Catalog enabled, and from there I'd like to access (external - S3) tables on an existing legacy hive metastore workspace (not UC enabled). The goal is for both wo...

Data Engineering
hivemetastore
unitycatalog
Workspaces
  • 50 Views
  • 0 replies
  • 0 kudos
Meshynix
by New Contributor II
  • 165 Views
  • 3 replies
  • 0 kudos

Not able to create external table in a schema under a Catalog.

Problem StatementCluster 1 (Shared Cluster) is not able to read the file location at "dbfs:/mnt/landingzone/landingzonecontainer/Inbound/" and hence we are not able to create an external table in a schema inside Enterprise Catalog.Cluster 2 (No Isola...

  • 165 Views
  • 3 replies
  • 0 kudos
Latest Reply
YuliyanBogdanov
New Contributor
  • 0 kudos

Hi @Meshynix,Can you provide the code snippet you execute to create your tables, this would give us a better insight for both use cases. Also can you provide the error that is being returned in the first use case. This would help a lot.

  • 0 kudos
2 More Replies
Cheryl
by New Contributor
  • 82 Views
  • 3 replies
  • 0 kudos

Query example for databricks Query History API

Hi I am trying to get query history data from my SQL warehouse. Following previous examples is not working. databricks_workspace_url = "xxx"token = "xxx"start_time = 1707091200end_time = 1707174000api_endpoint = f"{databricks_workspace_url}/api/2.0/s...

  • 82 Views
  • 3 replies
  • 0 kudos
Latest Reply
shan_chandra
Honored Contributor III
  • 0 kudos

@Cheryl - you can use query_start_time=2023-01-01T00:00:00Z  as a parameter to filter for the time frame. available filter criteria are given below - https://docs.databricks.com/api/workspace/queryhistory/list#filter_by-query_start_time_range    

  • 0 kudos
2 More Replies
Kibour
by New Contributor II
  • 77 Views
  • 0 replies
  • 0 kudos

Import from repo

Hi all,I am trying the new "git folder" feature, with a repo that works fine from the "Repos". In the new folder location, my imports from my own repo don't work anymore. Anyone faced something similar?Thanks in advance for sharing your experience

  • 77 Views
  • 0 replies
  • 0 kudos
AxelBrsn
by New Contributor II
  • 59 Views
  • 2 replies
  • 0 kudos

Use DLT from another pipeline

Hello, I have a question.Context :I have a Unity Catalog organized with three schemas (bronze, silver and gold). Logically, I would like to create tables in each schemas.I tried to organize my pipelines on the layers, which mean that I would like to ...

  • 59 Views
  • 2 replies
  • 0 kudos
Latest Reply
YuliyanBogdanov
New Contributor
  • 0 kudos

Hey @AxelBrsn,Unfortunately this is a limitation with DLT as far as my experience goes. You should organize the pipelines in a way that they encompass the full Bronze/Silver/Gold flow, since you don't have control over the schema if you want to make ...

  • 0 kudos
1 More Replies
Kibour
by New Contributor II
  • 38 Views
  • 0 replies
  • 0 kudos

git reset

Hi all,Since the last update of Databricks (21.03.2024), I can't find the option to git reset my repo anymore. Where did it go?Thanks in advance for your guidance!

  • 38 Views
  • 0 replies
  • 0 kudos
HelloDatabricks
by New Contributor
  • 283 Views
  • 5 replies
  • 7 kudos

Connect Timeout - Error when trying to run a cell

Hello everybody.Whenever I am trying to run a simple cell I receive the following error message now:Notebook detached. Exception when creating expectation context: java.net.SocketTimeoutException: Connect Timeout.After that error message the cluster ...

  • 283 Views
  • 5 replies
  • 7 kudos
Latest Reply
MarijaS
New Contributor
  • 7 kudos

today is ok

  • 7 kudos
4 More Replies
Geoff123
by New Contributor
  • 109 Views
  • 7 replies
  • 0 kudos

Trouble on Accessing Azure Storage from Databricks (Python)

I used the same accessing method shown in https://community.databricks.com/t5/data-engineering/to-read-data-from-azure-storage/td-p/32230 but kept get the error below.org.apache.spark.SparkSecurityException: [INSUFFICIENT_PERMISSIONS] Insufficient pr...

  • 109 Views
  • 7 replies
  • 0 kudos
Latest Reply
Wojciech_BUK
Contributor III
  • 0 kudos

Hi,you can find storage account firewall information by accessing resource in azure portal Please mind that if you are using Unity Catalog you should NOT mount Storage Account, you should rather use abstraction of Storage Creadentials and External Lo...

  • 0 kudos
6 More Replies
satishnavik
by New Contributor II
  • 540 Views
  • 3 replies
  • 0 kudos

How to connect Databricks Database with Springboot application using JPA

facing issue with integrating our Spring boot JPA supported application with Databricks.Below are the steps and setting we did for the integration.When we are starting the spring boot application we are getting a warning as :HikariPool-1 - Driver doe...

  • 540 Views
  • 3 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @satishnavik, It seems you’re encountering issues while integrating your Spring Boot JPA application with Databricks. Let’s address the warnings and exceptions you’re facing. Warning: Driver Does Not Support Network Timeout for Connections The...

  • 0 kudos
2 More Replies
RajNath
by New Contributor II
  • 64 Views
  • 2 replies
  • 0 kudos

Traversing to previous rows and getting the data based on condition

Sample Input data setClusterIdEventEventTime1212-18-r9u1kzn1RUNNING2024-02-02T11:38:30.168+00:001212-18-r9u1kzn1TERMINATING2024-02-02T13:43:33.933+00:001212-18-r9u1kzn1STARTING2024-02-02T15:50:05.174+00:001212-18-r9u1kzn1RUNNING2024-02-02T15:54:21.51...

  • 64 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @RajNath , Handling event times and aggregations in large datasets can be challenging, but Structured Streaming in Databricks provides powerful tools to address this. Let’s break down your requirements and explore how you can achieve them: Ru...

  • 0 kudos
1 More Replies
RajNath
by New Contributor II
  • 347 Views
  • 2 replies
  • 0 kudos

Cost of using delta sharing with unity catalog

I am new to databricks delta sharing. In case of delta sharing, i don't see any cluster running. Tried looking for documentation but only hint i got is, it usage delta sharing server but what is the cost of it and how to configure and optimize for la...

  • 347 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @RajNath, Let’s dive into the world of Delta Sharing and explore how it works, its cost implications, and optimization strategies. What is Delta Sharing? Delta Sharing is a secure data-sharing platform developed by Databricks. It allows you to ...

  • 0 kudos
1 More Replies