cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

YevheniiY
by New Contributor II
  • 5008 Views
  • 2 replies
  • 0 kudos

Empty xml tag

 <ItemMaintenance> <Batch> <BathInfo>info</BathInfo> <Item attr1="tekst" attr2="Tekst2"> <ItemId type="Type" id="id"/> <Dates> <Start>2023-11-09</Start> <End>2024-01-02</End> </Dates> <MoreData> More data </MoreData> <...

Data Engineering
empty xml tag
public preview
XML
xml auto loader
xml read
  • 5008 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @YevheniiY,  I understand that you’re encountering issues with reading an XML file using the new feature XML auto-loader. Let’s troubleshoot this together. Spark-XML Library Installation: First, ensure that you have installed the Spark-XML OSS...

  • 0 kudos
1 More Replies
Yashir
by New Contributor III
  • 1767 Views
  • 4 replies
  • 4 kudos

Is there a way to add Features descriptions for each of the features in a Feature Store table?

 If not, then I believe that it will be beneficial because the feature tables contain engineered features that its a good idea to document their calc logic for the benefit of other data scientists. Also, even non-engineered features are many times no...

  • 1767 Views
  • 4 replies
  • 4 kudos
Latest Reply
ShaneVanHeerden
New Contributor II
  • 4 kudos

Hi thereHas there perhaps been an update on this functionality? I am also attempting to programmatically update an individual feature's description and tag(s).

  • 4 kudos
3 More Replies
elgeo
by Valued Contributor II
  • 451 Views
  • 1 replies
  • 2 kudos

Resolved! Interactive Notebook with widgets

Dear experts,We need to create a notebook in order the users to insert/update values in a specific table. We have created one using widgets. However the code performed per action selected is visible to the users. Is there a way to have in a the dropd...

  • 451 Views
  • 1 replies
  • 2 kudos
Latest Reply
Kaniz
Community Manager
  • 2 kudos

Hi @elgeo, You can achieve this by separating the user interface (UI) and the underlying code logic in different notebooks. Let’s break down the steps: UI Notebook: Create a notebook specifically for the user interface. In this notebook, you’ll d...

  • 2 kudos
Eren_DE
by New Contributor
  • 225 Views
  • 1 replies
  • 0 kudos

legacy Git integration has been removed from Notebook

How to integrate a notebook saved in workspace folder with git repos with feature branch? 

  • 225 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @Eren_DE, Integrating notebooks from your Databricks workspace with Git repositories, especially when working with feature branches, can streamline collaboration and version control. Here are the steps you can follow: Clone Your Git Repositor...

  • 0 kudos
Learnit
by New Contributor II
  • 817 Views
  • 2 replies
  • 0 kudos

Databricks Vs ADLS for reporting.

Hi everyone, I'm a business analyst currently facing a decision on how best to develop a report. I need to choose between using Databricks or Azure Data Lake Storage (ADLS) as the data source on the transformed data in csv or excel file format from d...

  • 817 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hey there! Thanks a bunch for being part of our awesome community!  We love having you around and appreciate all your questions. Take a moment to check out the responses – you'll find some great info. Your input is valuable, so pick the best solution...

  • 0 kudos
1 More Replies
yvuignie
by Contributor
  • 3305 Views
  • 11 replies
  • 3 kudos

Resolved! Unity catalog - How do you modify groups properly ?

Hello,What is the best practice to modify/delete/recreate groups properly ?In order to rename a group, the only mean was to delete/recreate. But after deletion in the account console, the permissions granted to the deleted groups in the tables were i...

  • 3305 Views
  • 11 replies
  • 3 kudos
Latest Reply
RobinK
New Contributor III
  • 3 kudos

Hello,I have exactly the same issue - I am also using terraform.I deleted a group and the catalog permissions are in bad state.  I am not able to revoke access to this group using the Databricks UI nor REST API. I also tried to recreate the group wit...

  • 3 kudos
10 More Replies
weldermartins
by Honored Contributor
  • 3119 Views
  • 5 replies
  • 10 kudos

Resolved! Spark - API Jira

Hello guys. I use pyspark in my daily life. A demand has arisen to collect information in Jira. I was able to do this via Talend ESB, but I wouldn't want to use different tools to get the job done. Do you have any example of how to extract data from ...

  • 3119 Views
  • 5 replies
  • 10 kudos
Latest Reply
Marty73
New Contributor II
  • 10 kudos

Hi,There is also a new Databricks for Jira add-on on the Atlassian Marketplace. It is easy to setup and exports are directly created within Jira. They can be one-time, scheduled, or real-time. It can also export additional Jira data such as Assets, C...

  • 10 kudos
4 More Replies
Nixon
by New Contributor II
  • 192 Views
  • 1 replies
  • 0 kudos

Resonable long running time

Hi there, I have a block of code which can be executed around a month ago within 20 mins. But I came back recently and try to execute it again. It takes over 50 minutes still cannot complete (finally got kick out). Any advise and hints is appreciated...

  • 192 Views
  • 1 replies
  • 0 kudos
Latest Reply
Nixon
New Contributor II
  • 0 kudos

Haha...    Should be "unreasonable long" 

  • 0 kudos
Reza
by New Contributor III
  • 5698 Views
  • 13 replies
  • 3 kudos

Resolved! How can search in a specific folder in Databricks?

There is a keyword search option in Databricks that searches for a command or word in the entire workspace. How can search for a command in a specific folder or repository?

  • 5698 Views
  • 13 replies
  • 3 kudos
Latest Reply
nelsoncardenas
New Contributor II
  • 3 kudos

I would not consider this resolved. When I visit a forum, it's to find a solution, and I'm not completely sure if this feature has still not been created in Databricks since 2022. Regardless of that, I have submitted the feature request in Databricks...

  • 3 kudos
12 More Replies
pferreira
by New Contributor II
  • 652 Views
  • 2 replies
  • 2 kudos

MongoDB Spark Connector v10.x read error on Databricks 14.3 LTS

Im facing an error when updating DBR from 13.3 LTS to 14.3LTSIm using the spark:mongo-spark-connector:10.2.1 and running the following script   connectionString = ****** database = ***** collection = ***** spark = SparkSession \ .builder \ ...

  • 652 Views
  • 2 replies
  • 2 kudos
Latest Reply
feiyun0112
Contributor III
  • 2 kudos

The following notebook shows you how to read and write data to MongoDB Atlas, https://docs.databricks.com/en/_extras/notebooks/source/mongodb.html 

  • 2 kudos
1 More Replies
vsharma
by New Contributor
  • 1742 Views
  • 3 replies
  • 0 kudos

Where to move cluster-init scripts after latest message (Storing initialization scripts on DBFS is being deprecated. We recommend using a different storage location)?

Recently Databricks has started showing "Storing initialization scripts on DBFS is being deprecated. We recommend using a different storage location" . Is there an alternative of still keep using DBFS or do we need to move to ABSFS ? I could not find...

  • 1742 Views
  • 3 replies
  • 0 kudos
Latest Reply
vkeziv
New Contributor II
  • 0 kudos

It is suggested that we can use Workspace but databricks CLI not supporting importing shell scripts, but we can import shell script using web page? 

  • 0 kudos
2 More Replies
Rik
by New Contributor III
  • 2341 Views
  • 4 replies
  • 7 kudos

Resolved! File information is not passed to trigger job on file arrival

We are using the UC mechanism for triggering jobs on file arrival, as described here: https://learn.microsoft.com/en-us/azure/databricks/workflows/jobs/file-arrival-triggers.Unfortunately, the trigger doesn't actually pass the file-path that is gener...

Data Engineering
file arrival
trigger file
Unity Catalog
  • 2341 Views
  • 4 replies
  • 7 kudos
Latest Reply
Panda
New Contributor II
  • 7 kudos

Could you please provide an update on the status of this particular request? Additionally, do we have any ETA for it?

  • 7 kudos
3 More Replies
peta
by New Contributor II
  • 1327 Views
  • 3 replies
  • 0 kudos

Slow read using Snowflake connector in Databricks.

Hi,I am trying to read table from Snowflake with Databricks native Snowflake jdbc connector. It is going well for small amount of data (100 rows), but if I am adding more (even just 1000 rows) the query does not finish. I was checking if the query fi...

  • 1327 Views
  • 3 replies
  • 0 kudos
Latest Reply
sri123
New Contributor II
  • 0 kudos

Hi @peta Have your issue got resolved, I'm facing the similar issue. If your issue got resolved can you please post the steps that you followed to resolve your issue?Thanks & RegardsSri

  • 0 kudos
2 More Replies
bergmaal
by New Contributor III
  • 660 Views
  • 2 replies
  • 1 kudos

Workflows 7 second delay between tasks

When you have a job in Workflows with multiple tasks running after one another, there seems to be a consistent 7 seconds delay between execution of the tasks. Or, more precisely, every task has an approximate 7 second overhead before the code actuall...

Data Engineering
delay
overhead
tasks
Workflows
  • 660 Views
  • 2 replies
  • 1 kudos
Latest Reply
JensH
New Contributor III
  • 1 kudos

Hi @bergmaal , I am experiencing the same issue.My Databricks consultant suggested opening a support ticket as this should not be normal behavior.Did you solve this issue yet?We observed these delays do not seem to occur in workflows that use noteboo...

  • 1 kudos
1 More Replies
NotARobot
by New Contributor III
  • 646 Views
  • 2 replies
  • 1 kudos

Resolved! Delta Live Tables UDFs and Versions

Trying to do a url_decode on a column, which works great in development, but running via DLT fails when trying multiple ways.1. pyspark.sql.functions.url_decode - This is new as of 3.5.0, but isn't supported using whatever version running a DLT pipel...

  • 646 Views
  • 2 replies
  • 1 kudos
Latest Reply
NotARobot
New Contributor III
  • 1 kudos

Thanks @Kaniz, for reference if anybody finds this, the DLT release docs are here: https://docs.databricks.com/en/release-notes/delta-live-tables/index.htmlThis shows which versions are running for CURRENT and PREVIEW channels. In this case, was runn...

  • 1 kudos
1 More Replies
Labels
Top Kudoed Authors