cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
Madhawa
by • New Contributor II
  • 2270 Views
  • 1 replies
  • 0 kudos

Unable to access AWS S3 - Error : java.nio.file.AccessDeniedException

Reading file like this "Data = spark.sql("SELECT * FROM edge.inv.rm") Getting this error org.apache.spark.SparkException: Job aborted due to stage failure: Task 10 in stage 441.0 failed 4 times, most recent failure: Lost task 10.3 in stage 441.0 (TID...

Community Platform Discussions
Reply
  • 2270 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
<div style= This widget could not be displayed.
" src="" />
This widget could not be displayed.
This widget could not be displayed.
This widget could not be displayed.
  • 0 kudos

This widget could not be displayed.
Reading file like this "Data = spark.sql("SELECT * FROM edge.inv.rm") Getting this error org.apache.spark.SparkException: Job aborted due to stage failure: Task 10 in stage 441.0 failed 4 times, most recent failure: Lost task 10.3 in stage 441.0 (TID...

This widget could not be displayed.
  • 0 kudos
This widget could not be displayed.
NarenderKumar
by • New Contributor III
  • 2232 Views
  • 2 replies
  • 0 kudos

Resolved! Unable to generate account level PAT for service principle

I am trying to generate PAT for a service principle.I am following the documentation as shown below:https://docs.databricks.com/en/dev-tools/auth/oauth-m2m.html#create-token-in-accountI have prepared the below curl command:I am getting below error:Pl...

NarenderKumar_0-1715695724302.png NarenderKumar_1-1715695859890.png NarenderKumar_2-1715695895738.png
Community Platform Discussions
Reply
  • 2232 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
NarenderKumar
New Contributor III
  • 0 kudos

I was able to generate the workspace level token using the databricks cli.I set the following details in the databricks cli profile(.databrickscfg) file: host  = https://myworksapce.azuredatabricks.net/ account_id = (my db account id)client_id     = ...

  • 0 kudos
1 More Replies
NhanNguyen
by • Contributor III
  • 3507 Views
  • 2 replies
  • 1 kudos

[Delta live table vs Workflow]

Hi Community Members,I have been using Databricks for a while, but I have only used Workflow. I have a question about the differences between Delta Live Table and Workflow. Which one should we use in which scenario?Thanks,

Community Platform Discussions
Reply
  • 3507 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
Hkesharwani
Contributor II
  • 1 kudos

Hi, Delta Live Tables focuses on managing data ingestion, transformation, and management of Delta tables using a declarative framework. Job Workflows are designed to orchestrate and schedule various data processing and analysis tasks, including SQL q...

  • 1 kudos
1 More Replies
kazinahian
by • New Contributor III
  • 3164 Views
  • 1 replies
  • 0 kudos

Enable or disable Databricks Assistant in the Community Edition.

Hello,Good afternoon great people. I was following the step-by-step instructions to enable or disable Databricks Assistant in my Databricks Community Edition to enable the AI assistance. However, I couldn't find the option and was unable to enable it...

Community Platform Discussions
datbricks community
Reply
  • 3164 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
kazinahian
New Contributor III
  • 0 kudos

Thank you @Retired_mod 

  • 0 kudos
RobsonNLPT
by • Contributor II
  • 939 Views
  • 0 replies
  • 0 kudos

Databricks UC Data Lineage Official Limitations

Hi all.I have a huge data migration project using medallion architecture,  UC, notebooks and workflows . One of the relevant requirements we have is to capture all data dependencies (upstreams and downstreams) using data lineage. I've followed all re...

Community Platform Discussions
Reply
  • 939 Views
  • 0 replies
  • 0 kudos
paritosh_sharma
by • New Contributor
  • 2089 Views
  • 0 replies
  • 0 kudos

DAB template dbt-sql not working

Hi,We are trying to use the dbt-sql template provided for databricks asset bundles but getting error as follows: Looks like its regarding default catalog configuration. Has anyone faced this previously or can help with the same  

Screenshot 2024-05-17 at 10.25.38.png
Community Platform Discussions
Reply
  • 2089 Views
  • 0 replies
  • 0 kudos
NandiniN
by • Databricks Employee
  • 3353 Views
  • 1 replies
  • 2 kudos

How to collect a thread dump from Databricks Spark UI.

If you observe a hung job, thread dumps are crucial to determine the root cause. Hence, it would be a good idea to collect the thread dumps before cancelling the hung job. Here are the Instructions to collect the Spark driver/executor thread dump:  ​...

Community Platform Discussions
Reply
  • 3353 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 2 kudos

Thank you for sharing @NandiniN

  • 2 kudos
traillog
by • New Contributor
  • 1172 Views
  • 0 replies
  • 0 kudos

Response code 400 received when using VSCode on Windows 10 but no issue while using Ubuntu

I use VSCode on Windows 10 for building and deploying a workflow from my system and always encounter response code 400 when trying to deploy it. I am able to deploy the workflows via Ubuntu, but not via Windows. Has anyone encountered this issue befo...

Community Platform Discussions
Reply
  • 1172 Views
  • 0 replies
  • 0 kudos
zgreen
by • New Contributor
  • 630 Views
  • 0 replies
  • 0 kudos

jobs.python_wheel_task.enty_point can't find entry points defined in dependency packages

Let's say I have packageA with no entry points, packageA dependents on dependencyA package, which has entry-points.In order to be able to use those entrypoints, i.e.```yamlpython_wheel_task:  package_name: packageA  entry_point:dependencyA_entry```I ...

Community Platform Discussions
Reply
  • 630 Views
  • 0 replies
  • 0 kudos
Verr
by • New Contributor II
  • 1356 Views
  • 2 replies
  • 0 kudos

child notebook is not displaying output.

I have built pipeline to execute databricks notebook having SQL scripts. It is executing notebook but not able to see output for each cell. I am executing child notebook through driver notebook.

Community Platform Discussions
Reply
  • 1356 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
koushiknpvs
New Contributor III
  • 0 kudos

Hi Verr,In short it depends on how your child notebook is configured. But I would start with the following points -Output Logging Settings: Check the logging settings for your notebook cells. Ensure that the cells are configured to display output. In...

  • 0 kudos
1 More Replies
GeKo
by • New Contributor III
  • 3967 Views
  • 6 replies
  • 2 kudos

Resolved! column "storage_sub_directory" is now always NULL in system.information_schema.tables

Hello,I am running a job that depends on the information provided in column storage_sub_directory in system.information_schema.tables .... and it worked until 1-2 weeks ago.Now I discovered in the doc that this column is deprecated and always null , ...

Community Platform Discussions
Unity Catalog
unitycatalog
Reply
  • 3967 Views
  • 6 replies
  • 2 kudos
View Replies
Latest Reply
GeKo
New Contributor III
  • 2 kudos

Many thanks for the update @NandiniN 

  • 2 kudos
5 More Replies
Lucifer
by • New Contributor
  • 1036 Views
  • 1 replies
  • 0 kudos

displaying unity catalog metadata and other information in sharePoint

Is there any connectors or api which we can use to display metadata information stored in Unity catalog to business users using SharePoint.

Community Platform Discussions
Reply
  • 1036 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Ajay-Pandey
Esteemed Contributor III
  • 0 kudos

Hi @Lucifer As metadata are stored in system schema as table that means you can use databricks to extract the data from databricks and display it to Sharepoint Docs - Statement Execution API: Run SQL on warehouses | Databricks on AWS

  • 0 kudos
_databreaks
by • New Contributor II
  • 933 Views
  • 0 replies
  • 0 kudos

DLT to push data instead of a pull

I am relatively new to Databricks, and from my recent experience it appears that every step in a DLT Pipeline, we define each LIVE TABLES (be it streaming or not) to pull data upstream.I have yet to see an implementation where data from upstream woul...

Community Platform Discussions
dlt
DLT pipeline
Reply
  • 933 Views
  • 0 replies
  • 0 kudos
NarenderKumar
by • New Contributor III
  • 3357 Views
  • 1 replies
  • 1 kudos

Resolved! Unable to provide access in unity catalog using SQL commands

I am trying to provide access in unity catalog using the SQL commands.I am following the below documentation:https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/manage-privileges/It suggests to create SQL commands in belo...

NarenderKumar_2-1715253228379.png NarenderKumar_1-1715253206943.png NarenderKumar_3-1715253369442.png
Community Platform Discussions
Reply
  • 3357 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
mhiltner
Databricks Employee
  • 1 kudos

Try this. For some reason the quotes are crazy when using these commands.  GRANT USAGE ON CATALOG `uda_dev` TO `your-group`   GRANT SELECT ON SCHEMA uda_dev.default TO `your-group` (without quotes for the schema) 

  • 1 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels
Top Kudoed Authors
View all