- 4001 Views
- 3 replies
- 2 kudos
Connect databricks delta table to Apache Kyln using JDBC
I am using Apache Kylin for Data Analytics and Databricks for data modelling and filtering. I have my final data in gold tables and I would like to integrate this data with Apache Kylin using JDBC where the gold table will be the Data Source. I would...
- 4001 Views
- 3 replies
- 2 kudos
- 2 kudos
@Sidhant07 also using Kylin4 and Sqoop I am getting: "Unsupported transaction isolation level: 2" when running sqoop list-tables.
- 2 kudos
- 2397 Views
- 6 replies
- 1 kudos
FeatureEngineeringClient and Databricks Connect
Hello everyone, I want to use Databricks Connect to connect externally to my clusters and run code, and while Databricks connect works without any issue, like this: ```from databricks.sdk.core import Config config = Config(cluster_id="XXXX")spark = S...
- 2397 Views
- 6 replies
- 1 kudos
- 1 kudos
I just accomplished by modifying the DatabricksClient by adding the feature_store_uri parameter to it and passing it to each call of get_host_creds. Lets see if databricks releases this simple change soon...
- 1 kudos
- 7744 Views
- 4 replies
- 0 kudos
Can anyone provide support on streamlit connectivity with databricks delta table/sql end point
Can anyone provide support on streamlit connectivity with databricks delta table/sql end point
- 7744 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @ananthakrishna raikar​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best...
- 0 kudos
- 2171 Views
- 6 replies
- 1 kudos
Issue in getting system.compute.warehouses table in some workspaces
In some workspaces, I can get system.compute.warehouses table But in some other workspaces, it is not available how can i enable it?Both are in same account but assigned to different metastore.
- 2171 Views
- 6 replies
- 1 kudos
- 551 Views
- 1 replies
- 0 kudos
Cluster policy "personal compute"
Hello! can you help me? In the police "personal compute" how can I remove the permission for all users? I am facing this message when I go to edit: "All users can use the Personal Compute default policy to create compute. This is enabled at the accou...
- 551 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @ViniciusFC54! Did you try following the steps mentioned here: Managing access to the Personal Compute policy? It shows how admins can manage access to the Personal Compute policy and control who can use it.
- 0 kudos
- 919 Views
- 2 replies
- 0 kudos
Docker image for runtime 16.4 LTS with Scala 2.13
I'm trying to test a custom python package with new 16.4 LTS runtime, pulling the official Databricks docker image fromhttps://hub.docker.com/layers/databricksruntime/standard/16.4-LTS/images/sha256-604b73feeac08bc902ab16110218ffc63c1b24ac31f5f04a679...
- 919 Views
- 2 replies
- 0 kudos
- 0 kudos
Thanks @Renu_ . That makes sense, I understand that in the DBX workspaces, you can choose between two 16.4 spark versions when creating a compute.My confusion was with using the docker image in local environment. I pull it from the registry but then ...
- 0 kudos
- 898 Views
- 2 replies
- 1 kudos
Displaying images hosted in a Unity Catalog volume in a notebook
I have an image dataset in a Volume in the Unity Catalog.As part of exploratory analysis, I want to run some queries and display the resulting image set.Yet, including dynamic references to images in the Catalog doesn't seem to work on a Databricks N...
- 898 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Walter_C , Thanks for your suggestion. Exporting the images (=> duplicating the storage) is a no-go in our case. We are talking about very large automotive datasets.Is there another option or should we consider moving this data out of the catalog...
- 1 kudos
- 2501 Views
- 3 replies
- 0 kudos
Error: Unable to locate package winehq-stable
Hi,I am running a notebook which has mix of python and scala codes. Below code was working fine until May 27, 2025. However, since yesterday, I’ve been encountering the following error: "E: Unable to locate package winehq-stable".Could someone please...
- 2501 Views
- 3 replies
- 0 kudos
- 0 kudos
add me https://www.linkedin.com/in/aviralb/ we will do live debugging
- 0 kudos
- 2364 Views
- 2 replies
- 1 kudos
Resolved! Error: Community edition spark job skewnees
I am trying to run the spark job in community edition, but when I noticed in the spark UI the whole data is reading on the driver node, instead of reading it on the worker node.does community version will not support for the worker node?
- 2364 Views
- 2 replies
- 1 kudos
- 1 kudos
In Databricks Community Edition, the compute environment is set up as a single-node cluster. This means there is only one node, which serves both as the driver and the worker. Because of this, all data processing—including reading data—is performed o...
- 1 kudos
- 8682 Views
- 9 replies
- 1 kudos
Configuration spark.sql.sources.partitionOverwriteMode is not available.
Dear, In the current setup, we are using dbt as a modeling tool for our data lakehouse.For a specific use case, we want to use the insert_overwrite strategy, where dbt will replace all data for a specific partition:Databricks configurations | dbt Dev...
- 8682 Views
- 9 replies
- 1 kudos
- 1 kudos
An approach that works well when using a Databricks SQL Warehouse is to use the replace_where strategy - I've just tested this. It also works with partitioned tables:{{ config( materialized='incremental', incremental_strategy='replace_where', ...
- 1 kudos
- 3704 Views
- 6 replies
- 1 kudos
Medallion Architecture and Databricks Assistant
I am in the process of rebuilding the data lake at my current company with databricks and I'm struggling to find comprehensive best practices for naming conventions and structuring medallion architecture to work optimally with the Databricks assistan...
- 3704 Views
- 6 replies
- 1 kudos
- 1 kudos
If we talk about traditional data warehouse way of building the architecture, we can consider Silver layer as Data mart with star schema kind of relations for dimensions and fact. Can we build entire DWH enterprise scale using databrikcs? I see in pr...
- 1 kudos
- 540 Views
- 1 replies
- 0 kudos
JDBC Oracle Connection change Container Statement
Hey,Im running into a weird issue while running the following code:def getDf(query, preamble_sql=None): jdbc_url = f"jdbc:oracle:thin:@//{host}:{port}/{service_name}" request = spark.read \ .format("jdbc") \ .o...
- 540 Views
- 1 replies
- 0 kudos
- 0 kudos
Here is something to consider: The issue you're experiencing likely stems from differences in behavior when accessing Oracle database objects via Spark JDBC versus other database clients like DBeaver. Specifically, Spark's JDBC interface may perform ...
- 0 kudos
- 6430 Views
- 6 replies
- 7 kudos
Resolved! Auto Loader for Shape File
Hello: As you can see from below link, that it support 7 file formats. I am dealing with GeoSpatial Shape files and I want to know if Auto Loader can support Shape Files ???Any help on this is greatly appreciated. Thanks. https://docs.microsoft.com/...
- 6430 Views
- 6 replies
- 7 kudos
- 7 kudos
You could try to use the binary file type. But the disadvantage of this is that the content of the shape files will be put into a column, that might not be what you want.If you absolutely want to use the autoloader, maybe some thinking outside the b...
- 7 kudos
- 6921 Views
- 6 replies
- 6 kudos
Error While Rendering Visualization -- Map (Markers)
I have a table with latitude and longitude for a few addresses (no more than 10 at the moment) but when I select the appropriate columns in the visualization editor for Map (Markers) I get an message that states "error while rendering visualization"....
- 6921 Views
- 6 replies
- 6 kudos
- 6 kudos
Having the same issue. Every time I update my SQL, all the widgets drop and show the error 'Unable to render visualization'. The only way I found to fix is to manually duplicate all my widgets and delete the old ones with errors, which is a pain and ...
- 6 kudos
- 515 Views
- 1 replies
- 0 kudos
What happens when you change from .ipynb to .py as default fileformat for notebooks
Hi, I was struggling to do Pull Requests with the "new" default fileformat for Notebooks and wanted to change it back to source(.py). My questions are:1) Does this affect the whole workspace for all users?2) Does this change the format of old .ipynb ...
- 515 Views
- 1 replies
- 0 kudos
- 0 kudos
Changing the default notebook file format from .ipynb to .py in Databricks has several implications based on current implementations and user scenarios: User Experience: The .ipynb format captures more comprehensive data, including environment setti...
- 0 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 485 | |
| 349 | |
| 287 |
