cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

VGS777
by New Contributor III
  • 1769 Views
  • 2 replies
  • 2 kudos

Resolved! Regarding cloning my gitrepo under workspace/Users/user_name

Hi all,I am recently started using databricks. I want to my git repo under workspace/Users/user_name path which I can't able to do it. But i can able to clone only under repo directory by default.Can anyone pls advice me regarding this Thank you

  • 1769 Views
  • 2 replies
  • 2 kudos
Latest Reply
VGS777
New Contributor III
  • 2 kudos

Thanks for this advice . 

  • 2 kudos
1 More Replies
Surajv
by New Contributor III
  • 2545 Views
  • 4 replies
  • 0 kudos

Connect my spark code running in AWS ECS to databricks cluster

Hi team, I wanted to know if there is a way to connect a piece of my pyspark code running in ECS to Databricks cluster and leverage the databricks compute using Databricks connect?I see Databricks connect is for connecting local ide code to databrick...

Community Platform Discussions
AWS
databricks connect
ecs
pyspark
  • 2545 Views
  • 4 replies
  • 0 kudos
Latest Reply
Surajv
New Contributor III
  • 0 kudos

Noted @Retired_mod @RonDeFreitas. I am currently using Databricks runtime v12.2 (which is < v13.0). I followed this doc (Databricks Connect for Databricks Runtime 12.2 LTS and below) and connected my local terminal to Databricks cluster and was able ...

  • 0 kudos
3 More Replies
BabuMahesh
by New Contributor
  • 641 Views
  • 0 replies
  • 0 kudos

Databricks & Bigquery

Databricks is packaging a old version of big-query jar(Databricks also repackaged and created a fat jar), and our application needs a latest jar. Now the latest jar depends on spark-bigquery-connector.properties  file for a property scala.binary.vers...

  • 641 Views
  • 0 replies
  • 0 kudos
rudyevers
by New Contributor III
  • 1013 Views
  • 1 replies
  • 0 kudos

Unity catalog internal error - quality monitoring

I try to get my head around the quality monitoring functionality in Unity Catalog. I configured one of the tables in our unity catalog. My assumption is that the profile and drift metrics tables are automatically created. But when I get an internal e...

rudyevers_0-1702049219646.png rudyevers_0-1702049692739.png
  • 1013 Views
  • 1 replies
  • 0 kudos
Latest Reply
jreddy
New Contributor II
  • 0 kudos

Hi, were you able to resolve this, am having a similar issue - thanks

  • 0 kudos
hayden_blair
by New Contributor III
  • 3181 Views
  • 0 replies
  • 0 kudos

Error authenticating databricks.sdk.WorkspaceClient with external workspace via Azure Native Auth

I am referencing this doc to initialize a databricks.sdk.WorkspaceClient object instance via Azure Native Authentication. I am initializing this WorkspaceClient within a databricks notebook, but I am trying to use the client to access the Jobs api of...

error.png
Community Platform Discussions
authentication
azure
WorkspaceClient
  • 3181 Views
  • 0 replies
  • 0 kudos
Sujitha
by Databricks Employee
  • 6686 Views
  • 0 replies
  • 3 kudos

Unity Catalog Governance Value Levers

What makes Unity Catalog a game-changer? The blog intricately dissects five main value levers: mitigating data and architectural risks, ensuring compliance, accelerating innovation, reducing platform complexity and costs while improving operational e...

Screenshot 2024-01-29 at 11.48.55 AM.png
  • 6686 Views
  • 0 replies
  • 3 kudos
Lackshu
by New Contributor II
  • 1149 Views
  • 1 replies
  • 0 kudos

Workspace Assignment Issue via REST API

I’m relying on workspace assignment via REST API to have the account user created in the workspace. This is like the workspace assignment screen at account level or adding existing user screen at workspace level. The reference URL is below.Workspace ...

  • 1149 Views
  • 1 replies
  • 0 kudos
Latest Reply
Lackshu
New Contributor II
  • 0 kudos

It turns out, the problem is the documentation. It says that the permission parameter (that's supplied in) is an array of strings. It really just expects a string, either UNKNOWN, USER, or ADMIN. It would be great if the team could fix the documentat...

  • 0 kudos
JohnsonBDSouza
by New Contributor II
  • 2840 Views
  • 1 replies
  • 0 kudos

Unable to create Iceberg tables pointing to data in S3 and run queries against the tables.

I need to to set up Iceberg tables in Databricks environment, but the data resides in an S3 bucket. Then read these tables by running SQL queries.Databricks environment has access to S3. This is done bysetting up the access by mapping the Instance Pr...

JohnsonBDSouza_0-1705982713662.png JohnsonBDSouza_1-1705982713665.png JohnsonBDSouza_2-1705982713667.jpeg JohnsonBDSouza_3-1705982713676.png
  • 2840 Views
  • 1 replies
  • 0 kudos
Latest Reply
shan_chandra
Databricks Employee
  • 0 kudos

@JohnsonBDSouza  - could you please let me know if you had a chance to review the Uniform feature that allows to create iceberg tables from the delta format.  Based on what i could understand from the above, you can create a delta table and use the b...

  • 0 kudos
gb
by New Contributor
  • 1559 Views
  • 1 replies
  • 0 kudos

Write stream to Kafka topic with DLT

Hi,Is it possible to write stream to Kafka topic with Delta Live Table?I would like to do something like this:@dlt.view(name="kafka_pub",comment="Publish to kafka")def kafka_pub():return (dlt.readStream("source_table").selectExpr("to_json (struct (*)...

  • 1559 Views
  • 1 replies
  • 0 kudos
Latest Reply
shan_chandra
Databricks Employee
  • 0 kudos

@gb  - Currently, DLT does not support writing to non-delta sink.  we can divide the pipeline currently into two by writing to a materialized view or delta sink and build a non-DLT job to write out to Kafka sink. Databricks DLT engineering team is wo...

  • 0 kudos
kiko_roy
by Contributor
  • 959 Views
  • 2 replies
  • 0 kudos

IsBlindAppend config change

Hello Allcan someone please suggest me how can I change the config IsBlindAppend true from false.I need to do this not for a data table but a custom log table .Also is there any concern If I toggle the value as standard practises. pls suggest

  • 959 Views
  • 2 replies
  • 0 kudos
Latest Reply
Lakshay
Databricks Employee
  • 0 kudos

IsBlindAppend is not a config but an operation metrics that is used in Delta Lake History. The value of this changes based on the type of operation performed on Delta table. https://docs.databricks.com/en/delta/history.html

  • 0 kudos
1 More Replies
JohnsonBDSouza
by New Contributor II
  • 2166 Views
  • 1 replies
  • 0 kudos

Connection Error Python databricks-sql-connector

Hello Databricks Community, I'm trying to connect to Databricks via the Python library (databricks-sql-connector-3.0.1). This below code was working few months ago. Now, it is failing to connect.hostname, http_path and access_token are valid values a...

  • 2166 Views
  • 1 replies
  • 0 kudos
Latest Reply
" src="" />
This widget could not be displayed.
This widget could not be displayed.
This widget could not be displayed.
  • 0 kudos

This widget could not be displayed.
Hello Databricks Community, I'm trying to connect to Databricks via the Python library (databricks-sql-connector-3.0.1). This below code was working few months ago. Now, it is failing to connect.hostname, http_path and access_token are valid values a...

This widget could not be displayed.
  • 0 kudos
This widget could not be displayed.
udi_azulay
by New Contributor II
  • 2351 Views
  • 1 replies
  • 0 kudos

local filesystem access is forbidden

Hi,When i running this command over my private cluster (Single User) it works well :dbutils.fs.cp(ituff_file, protocol_local_file_path) When i try to run it over a shared cluster, i am getting : java.lang.SecurityException: Cannot use com.databricks....

  • 2351 Views
  • 1 replies
  • 0 kudos
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi, Could you please refer to the limitations here: https://docs.databricks.com/en/compute/access-mode-limitations.html#shared-access-mode-limitations-on-unity-catalog  (Volumes do not support dbutils.fs commands distributed to executors.) , Also, fo...

  • 0 kudos
Mado
by Valued Contributor II
  • 3415 Views
  • 1 replies
  • 0 kudos

How to update a value in a column in a delta table with Map of Struct datatype?

I have a delta table in Databricks named "prod.silver.control_table". It has a few columns including "table_name" with string data type and "transform_options" with the below structure:  |-- transform_options: map (nullable = true) | |-- key: str...

Community Platform Discussions
MAP
Struct
Update_table
  • 3415 Views
  • 1 replies
  • 0 kudos
Latest Reply
" src="" />
This widget could not be displayed.
This widget could not be displayed.
This widget could not be displayed.
  • 0 kudos

This widget could not be displayed.
I have a delta table in Databricks named "prod.silver.control_table". It has a few columns including "table_name" with string data type and "transform_options" with the below structure:  |-- transform_options: map (nullable = true) | |-- key: str...

This widget could not be displayed.
  • 0 kudos
This widget could not be displayed.
Godhuli
by New Contributor II
  • 1456 Views
  • 0 replies
  • 0 kudos

Unable to login to Databricks Community edition

I signed up to Databricks Community Edition with gmail and verified my account and used it to create a notebook as well but having issue in re-logging in even though email and password provided are correct. I did  try  "Forget Password" step but I am...

  • 1456 Views
  • 0 replies
  • 0 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Top Kudoed Authors