- 2606 Views
- 1 replies
- 1 kudos
All my replies has been deleted and new one is being moderated (deleted) - why?
I noticed that when I Reply to post and trying to help solve community problem - my post are being either moderated (deleted) or just not being saved. Old post/replies has been deleted.Is there any reason for that?I kind of lost my will to participat...
- 2606 Views
- 1 replies
- 1 kudos
- 1 kudos
Currently, I'm also facing same issue, my comments are automatically deleted and also not receiving email for same, which is happening to specific post.
- 1 kudos
- 4096 Views
- 2 replies
- 2 kudos
Error accessing file from dbfs inside mlflow serve endpoint
Hi,I have mlflow model served using serverless GPU which takes audio file name as input and then file will be passed as parameter to huggiung face model inside predict method. But I am getting following errorHFValidationError(\nhuggingface_hub.utils....
- 4096 Views
- 2 replies
- 2 kudos
- 2 kudos
I have the same issue.I have a large file that I cannot access from an MLFlow service.Things I have tried (none of these work):Read-only from DBFS`dbfs:/myfolder/myfile.chroma` does not work`/dbfs/myfolder/myfile.chroma` does not workRead-only from U...
- 2 kudos
- 1018 Views
- 0 replies
- 0 kudos
PowerBI reports migration from Synapse to Databricks sql
Hello Techie,Do someone please share detailed design consideration when migrating 250 powerBI reports from Azure Synapse, to Databricks (DB SQL).How to plan for cutover and strategy for PowerBI. what are the consideration to be done , is there any da...
- 1018 Views
- 0 replies
- 0 kudos
- 7842 Views
- 5 replies
- 1 kudos
Query results in csv file include 'null' string for blank cell
After running a sql script, when downloading the results to a csv file, the file includes a null string for blank cells (see screenshot). Is ther a setting I can change to simply get empty cells instead?
- 7842 Views
- 5 replies
- 1 kudos
- 1 kudos
I understand, however this is more on CSV file format. Save your data in Delta format instead of CSV or text-based formats. Delta tables handle empty strings and NULL values more effectively, ensuring that empty strings are preserved during data ins...
- 1 kudos
- 8185 Views
- 6 replies
- 1 kudos
Materialized Views Without DLT?
I'm curious, is DLT *required* to use Materialized Views in Databricks? Is it not possible to create and refresh a Materialized view via a standard Databricks Workflow?
- 8185 Views
- 6 replies
- 1 kudos
- 1 kudos
Hi @ChristianRRL ,When creating a materialized view in Databricks, the data is stored in DBFS, cloud storage, or Unity Catalog volume. You can still create a materialized view by overwriting the same table each time, instead of using Append, Update, ...
- 1 kudos
- 5485 Views
- 4 replies
- 1 kudos
Purpose of DLT Table table_properties > quality:medallion
Hi there, silly question here but can anyone help me understand what practical purpose does labelling the table_properties with "quality":"<specific_medallion>"? For example: @Dlt.table( comment="Bronze live streaming table for Test data", name="...
- 5485 Views
- 4 replies
- 1 kudos
- 1 kudos
I'm with the same doubt @ChristianRRL, did you figured out something related to it?My doubt is to check if it's possible to apply any kind of access control based on this property.
- 1 kudos
- 4588 Views
- 6 replies
- 3 kudos
Resolved! Plotly Express not rendering in Firefox but fine in Safari
Using a basic example of plotly express i see no output in firefox but is fine in Safari. Any ideas why this may occur? import plotly.express as px import pandas as pd # Create a sample dataframe df = pd.DataFrame({ 'x': range(10), 'y': [2, 3, 5, 7...
- 4588 Views
- 6 replies
- 3 kudos
- 3 kudos
UPDATE: I reached out further to Databricks support and they have since deployed a fix. Works fine for me now!
- 3 kudos
- 9147 Views
- 8 replies
- 3 kudos
Unity catalog enabled workspace -Is there any way to disable workflow/job creation for certain users
Currently in unity catalog enabled workspace users with "Workspace access" can create workflows/jobs, there is no access control available to restrict users from creating jobs/workflows.Use case: In production there is no need for users, data enginee...
- 9147 Views
- 8 replies
- 3 kudos
- 3 kudos
@Lakshay Databricks offers a robust platform with a variety of features, including data ingestion, engineering, science, dashboards, and applications. However, I believe that some features, such as workflow/job creation, alerts, dashboards, and Genie...
- 3 kudos
- 4366 Views
- 3 replies
- 0 kudos
String to date conversion errors
Hi,I am getting data from CDC on SQL Server using Informatica which is writing parquet files to ADLS. I read the parquet files using DLT and end up with the date data as a string such as this'20240603164746563' I couldn't get this to convert using m...
- 4366 Views
- 3 replies
- 0 kudos
- 0 kudos
Checking on my current code, this is what I am using, which works for me because we don't use daylight savings time. from_utc_timestamp(date_time_utc, 'UTC-7') as date_time_local
- 0 kudos
- 30148 Views
- 5 replies
- 1 kudos
Insufficient privileges:User does not have permission SELECT on any file
Hello,after switching to "shared cluster" usage a python job is failing with error message: Py4JJavaError: An error occurred while calling o877.load. : org.apache.spark.SparkSecurityException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: User...
- 30148 Views
- 5 replies
- 1 kudos
- 1 kudos
Hi @GeKo The checkpoint directory, is that set on cluster level or how do we set that ? Can you please help me with this ?
- 1 kudos
- 2099 Views
- 1 replies
- 0 kudos
Databricks UC Data Lineage Official Limitations
Hi all.I have a huge data migration project using medallion architecture, UC, notebooks and workflows . One of the relevant requirements we have is to capture all data dependencies (upstreams and downstreams) using data lineage. I've followed all re...
- 2099 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @RobsonNLPT , Yes SQL CTE are supported by the data lineage service. You can track table that were created using CTEs. Here is an example that demonstrate the feature. CREATE TABLE IF NOT EXISTS mpelletier.dbdemos.menu ( recipe_id INT, ...
- 0 kudos
- 4164 Views
- 3 replies
- 1 kudos
Resolved! Ingesting and Transforming NetCDF Data in Delta Table on Databricks Cluster
Hi,I need to ingest and transform historical climate data into a Delta table. The data is stored in .nc format (NetCDF). To work with this format, specific C libraries for Python are required, along with particular versions of Python libraries (e.g.,...
- 4164 Views
- 3 replies
- 1 kudos
- 3512 Views
- 5 replies
- 0 kudos
Service Principal Access to Users Directory in Databricks - Creating Git Folders
I am trying to automate the creation of git folders in user workspace directories triggered by GitHub feature branch creation. When developers create feature branches in GitHub, we want a service principal to automatically create corresponding git fo...
- 3512 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi @Brianhourigan, Can you please DIM your suggestions? I can add it to our internal AHA idea.
- 0 kudos
- 6019 Views
- 5 replies
- 0 kudos
Restore deleted databricks jobs and job runs
Hi All,Is there a way to restore deleted databricks jobs?Thank you.
- 6019 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi @iptkrisna ,Currently, there is no option to recover deleted items. In architectures, it not necessary to control or manage the final code available in the system. Instead, the focus should be controlling and managing how code and jobs are deploye...
- 0 kudos
- 4522 Views
- 1 replies
- 1 kudos
Resolved! Unity Catalog : RDD Issue
In our existing notebooks, the scripts are reliant on RDDs. However, with the upgrade to Unity Catalog, RDDs will no longer be supported. We need to explore alternative approaches or tools to replace the use of RDDs. Could you suggest the best practi...
- 4522 Views
- 1 replies
- 1 kudos
- 1 kudos
To transition from using RDDs (Resilient Distributed Datasets) to alternative approaches supported by Unity Catalog, you can follow these best practices and migration strategies: Use DataFrame API: The DataFrame API is the recommended alternative to...
- 1 kudos
