cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

rudyevers
by New Contributor III
  • 882 Views
  • 1 replies
  • 0 kudos

Unity catalog internal error - quality monitoring

I try to get my head around the quality monitoring functionality in Unity Catalog. I configured one of the tables in our unity catalog. My assumption is that the profile and drift metrics tables are automatically created. But when I get an internal e...

rudyevers_0-1702049219646.png rudyevers_0-1702049692739.png
  • 882 Views
  • 1 replies
  • 0 kudos
Latest Reply
jreddy
New Contributor II
  • 0 kudos

Hi, were you able to resolve this, am having a similar issue - thanks

  • 0 kudos
hayden_blair
by New Contributor III
  • 2791 Views
  • 0 replies
  • 0 kudos

Error authenticating databricks.sdk.WorkspaceClient with external workspace via Azure Native Auth

I am referencing this doc to initialize a databricks.sdk.WorkspaceClient object instance via Azure Native Authentication. I am initializing this WorkspaceClient within a databricks notebook, but I am trying to use the client to access the Jobs api of...

error.png
Community Platform Discussions
authentication
azure
WorkspaceClient
  • 2791 Views
  • 0 replies
  • 0 kudos
Sujitha
by Databricks Employee
  • 6626 Views
  • 0 replies
  • 3 kudos

Unity Catalog Governance Value Levers

What makes Unity Catalog a game-changer? The blog intricately dissects five main value levers: mitigating data and architectural risks, ensuring compliance, accelerating innovation, reducing platform complexity and costs while improving operational e...

Screenshot 2024-01-29 at 11.48.55 AM.png
  • 6626 Views
  • 0 replies
  • 3 kudos
Lackshu
by New Contributor II
  • 976 Views
  • 1 replies
  • 0 kudos

Workspace Assignment Issue via REST API

I’m relying on workspace assignment via REST API to have the account user created in the workspace. This is like the workspace assignment screen at account level or adding existing user screen at workspace level. The reference URL is below.Workspace ...

  • 976 Views
  • 1 replies
  • 0 kudos
Latest Reply
Lackshu
New Contributor II
  • 0 kudos

It turns out, the problem is the documentation. It says that the permission parameter (that's supplied in) is an array of strings. It really just expects a string, either UNKNOWN, USER, or ADMIN. It would be great if the team could fix the documentat...

  • 0 kudos
JohnsonBDSouza
by New Contributor II
  • 2227 Views
  • 1 replies
  • 0 kudos

Unable to create Iceberg tables pointing to data in S3 and run queries against the tables.

I need to to set up Iceberg tables in Databricks environment, but the data resides in an S3 bucket. Then read these tables by running SQL queries.Databricks environment has access to S3. This is done bysetting up the access by mapping the Instance Pr...

JohnsonBDSouza_0-1705982713662.png JohnsonBDSouza_1-1705982713665.png JohnsonBDSouza_2-1705982713667.jpeg JohnsonBDSouza_3-1705982713676.png
  • 2227 Views
  • 1 replies
  • 0 kudos
Latest Reply
shan_chandra
Databricks Employee
  • 0 kudos

@JohnsonBDSouza  - could you please let me know if you had a chance to review the Uniform feature that allows to create iceberg tables from the delta format.  Based on what i could understand from the above, you can create a delta table and use the b...

  • 0 kudos
gb
by New Contributor
  • 1414 Views
  • 1 replies
  • 0 kudos

Write stream to Kafka topic with DLT

Hi,Is it possible to write stream to Kafka topic with Delta Live Table?I would like to do something like this:@dlt.view(name="kafka_pub",comment="Publish to kafka")def kafka_pub():return (dlt.readStream("source_table").selectExpr("to_json (struct (*)...

  • 1414 Views
  • 1 replies
  • 0 kudos
Latest Reply
shan_chandra
Databricks Employee
  • 0 kudos

@gb  - Currently, DLT does not support writing to non-delta sink.  we can divide the pipeline currently into two by writing to a materialized view or delta sink and build a non-DLT job to write out to Kafka sink. Databricks DLT engineering team is wo...

  • 0 kudos
kiko_roy
by Contributor
  • 833 Views
  • 2 replies
  • 0 kudos

IsBlindAppend config change

Hello Allcan someone please suggest me how can I change the config IsBlindAppend true from false.I need to do this not for a data table but a custom log table .Also is there any concern If I toggle the value as standard practises. pls suggest

  • 833 Views
  • 2 replies
  • 0 kudos
Latest Reply
Lakshay
Databricks Employee
  • 0 kudos

IsBlindAppend is not a config but an operation metrics that is used in Delta Lake History. The value of this changes based on the type of operation performed on Delta table. https://docs.databricks.com/en/delta/history.html

  • 0 kudos
1 More Replies
crowley
by New Contributor III
  • 2310 Views
  • 0 replies
  • 0 kudos

How are Struct type columns stored/accessed (interested in efficiency)?

Hello, I've searched around for awhile and didn't find a similar question here or elsewhere, so thought I'd ask...I'm assessing the storage/access efficiency of Struct type columns in delta tables.  I want to know more about how Databricks is storing...

  • 2310 Views
  • 0 replies
  • 0 kudos
JohnsonBDSouza
by New Contributor II
  • 1989 Views
  • 1 replies
  • 0 kudos

Connection Error Python databricks-sql-connector

Hello Databricks Community, I'm trying to connect to Databricks via the Python library (databricks-sql-connector-3.0.1). This below code was working few months ago. Now, it is failing to connect.hostname, http_path and access_token are valid values a...

  • 1989 Views
  • 1 replies
  • 0 kudos
Latest Reply
" src="" />
This widget could not be displayed.
This widget could not be displayed.
This widget could not be displayed.
  • 0 kudos

This widget could not be displayed.
Hello Databricks Community, I'm trying to connect to Databricks via the Python library (databricks-sql-connector-3.0.1). This below code was working few months ago. Now, it is failing to connect.hostname, http_path and access_token are valid values a...

This widget could not be displayed.
  • 0 kudos
This widget could not be displayed.
OlekNV
by New Contributor
  • 1458 Views
  • 1 replies
  • 0 kudos

Enable system schemas

Hello All,I'm new with Databricks,Have an issue within enable system schemas. When run api call to check system schemas status in metastores -I see that all schemas in "Unavailable" state (except "information_schema", which is "ENABLE_COMPLETED").Is ...

  • 1458 Views
  • 1 replies
  • 0 kudos
Latest Reply
" src="" />
This widget could not be displayed.
This widget could not be displayed.
This widget could not be displayed.
  • 0 kudos

This widget could not be displayed.
Hello All,I'm new with Databricks,Have an issue within enable system schemas. When run api call to check system schemas status in metastores -I see that all schemas in "Unavailable" state (except "information_schema", which is "ENABLE_COMPLETED").Is ...

This widget could not be displayed.
  • 0 kudos
This widget could not be displayed.
udi_azulay
by New Contributor II
  • 1938 Views
  • 1 replies
  • 0 kudos

local filesystem access is forbidden

Hi,When i running this command over my private cluster (Single User) it works well :dbutils.fs.cp(ituff_file, protocol_local_file_path) When i try to run it over a shared cluster, i am getting : java.lang.SecurityException: Cannot use com.databricks....

  • 1938 Views
  • 1 replies
  • 0 kudos
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi, Could you please refer to the limitations here: https://docs.databricks.com/en/compute/access-mode-limitations.html#shared-access-mode-limitations-on-unity-catalog  (Volumes do not support dbutils.fs commands distributed to executors.) , Also, fo...

  • 0 kudos
Mado
by Valued Contributor II
  • 3191 Views
  • 1 replies
  • 0 kudos

How to update a value in a column in a delta table with Map of Struct datatype?

I have a delta table in Databricks named "prod.silver.control_table". It has a few columns including "table_name" with string data type and "transform_options" with the below structure:  |-- transform_options: map (nullable = true) | |-- key: str...

Community Platform Discussions
MAP
Struct
Update_table
  • 3191 Views
  • 1 replies
  • 0 kudos
Latest Reply
" src="" />
This widget could not be displayed.
This widget could not be displayed.
This widget could not be displayed.
  • 0 kudos

This widget could not be displayed.
I have a delta table in Databricks named "prod.silver.control_table". It has a few columns including "table_name" with string data type and "transform_options" with the below structure:  |-- transform_options: map (nullable = true) | |-- key: str...

This widget could not be displayed.
  • 0 kudos
This widget could not be displayed.
Godhuli
by New Contributor II
  • 1377 Views
  • 0 replies
  • 0 kudos

Unable to login to Databricks Community edition

I signed up to Databricks Community Edition with gmail and verified my account and used it to create a notebook as well but having issue in re-logging in even though email and password provided are correct. I did  try  "Forget Password" step but I am...

  • 1377 Views
  • 0 replies
  • 0 kudos
arun949290
by New Contributor II
  • 1693 Views
  • 3 replies
  • 0 kudos

Unable to login to community edition

Hello there,I have successfully created a databricks account and went to login to the community edition with the exact same login credentials as my account, but it tells me that the email/password are invalid. I can login with these same exact creden...

  • 1693 Views
  • 3 replies
  • 0 kudos
Latest Reply
Erik_L
Contributor II
  • 0 kudos

Databricks Community Edition and Databricks are separate services. You have to create an account specific to community.

  • 0 kudos
2 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Top Kudoed Authors