- 858 Views
- 2 replies
- 1 kudos
Resolved! Natural language queries through REST API?
Natural language queries provided by Genie are really powerful and a compeling tool.Is there any way to execute these natural language queries through the REST API to integrate them into in-house applications?
- 858 Views
- 2 replies
- 1 kudos
- 1 kudos
@Gusman wrote:Natural language queries provided by Genie are really powerful and a compeling tool.Is there any way to execute these natural language queries through the REST API to integrate them into in-house applications?While there's no direct RES...
- 1 kudos
- 444 Views
- 1 replies
- 1 kudos
Retrieve data older than the one year window : system.access.table_lineage
Hello,I am currently using table_lineage from system.access.table_lineage. It is a great feature but I am experiencing missing data. After some search I have seen that "Because lineage is computed on a one-year rolling window, lineage collected more ...
- 444 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Clara ,I don't think so. But you can build history such history tables by yourself. Desing ETL process that will extract data from system tables and store them in your own data tables.
- 1 kudos
- 573 Views
- 2 replies
- 1 kudos
Can we create Materialized view n exsting view and table?
Dear All,Is it possible to create Materialized view on view and table (Joining view and table)?I suspect it is not possible. Please suggest.Also please provide best way to schedule the refresh of Materialized view. Regards,Surya
- 573 Views
- 2 replies
- 1 kudos
- 5459 Views
- 4 replies
- 1 kudos
Resolved! Help - Can't create table from tutorial. Is my setup wrong?
Trying out databricks for the first time and followed the Get Started steps. I managed to successfully create a cluster and ran the simple sql tutorial to query data from a notebook. However, got the following error:Query:DROP TABLE IF EXISTS diamond...
- 5459 Views
- 4 replies
- 1 kudos
- 1 kudos
It seems as though you're doing great with your Databricks arrangement, however this sort of mistake could be connected with a couple of expected issues. In light of the subtleties you've shared, here are a few things you should check:Group Setup: Gu...
- 1 kudos
- 1763 Views
- 2 replies
- 2 kudos
Resolved! applying column tags
Can anyone tell me the correct syntax for applying a column tag to a specific tableThese are what I tried ALTER TABLE accounts_and_customer.bronze.BB1123_loans ALTER/CHANGE COLUMN loan_number SET TAGS ('classification' = 'confidential')<p>I got thi...
- 1763 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi there @Takuya-Omi ,I agree. The syntax was correct. I was facing some completely different problems with schemas and I solved it. Thanks though. Or I would have spent hours banging my head to find the reason for error.
- 2 kudos
- 599 Views
- 1 replies
- 1 kudos
CDF table partition - Real time Data
Hi team,We are currently working in loading CDF table using data events from Kafka. The table is going to hold data across geographies. When we tried partitioning it is slowing down the ingestion time. But without partition the downstream application...
- 599 Views
- 1 replies
- 1 kudos
- 1 kudos
1. Instead of using many small partitions (e.g., country or region), opt for larger partitions, such as continent or time-based partitions (e.g., weekly or monthly). This will reduce the number of partitions and improve performance.2. Write data to ...
- 1 kudos
- 1239 Views
- 3 replies
- 2 kudos
Resolved! UNPIVOT VARIANT data in SQL
Hi All,Have a VARIANT column with the following data;CREATE TABLE unpivot_valtype AS SELECT parse_json( '{ "Id": 1234567, "Result": { "BodyType": "NG", "ProdType": "Auto", "ResultSets": [ { "R1": { "AIn...
- 1239 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @binsel ,You need to use variant_explode function.Here is the working code:WITH first_explode AS ( SELECT uv.rowData:Id AS Id, uv.rowData:Result:BodyType AS BodyType, uv.rowData:Result:ProdType AS ProdType, v.value AS result_se...
- 2 kudos
- 520 Views
- 1 replies
- 0 kudos
Error Querying Shallow Clones: Couldn't Initialize File System for Path
Hi,We are offering data products through a central catalog for our users. To minimize data duplication and to display relationships between tables, we use shallow clones to provide access to the data.However, since implementing this approach, we occa...
- 520 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @schluca ,I’ve encountered an issue where an error occurred when trying to reference a table after deleting and recreating the source table for a Shallow Clone, and then performing the Shallow Clone again. As a solution, try deleting the destinati...
- 0 kudos
- 998 Views
- 2 replies
- 1 kudos
Databricks table lineage
Hi,I wanted to know if it is possible to edit the lineage that we see in databricks, like the one shown below.Can I edit this lineage graph, like add other ETL tools (at the start of the tables) that I have used to get data in aws and then in databri...
- 998 Views
- 2 replies
- 1 kudos
- 1 kudos
This will be extremely beneficial. We have certain use cases where we do not leverage Spark in our pipelines and lose the lineage. I would prefer to set an extra parameter when writing a table to specify the lineage.
- 1 kudos
- 502 Views
- 2 replies
- 0 kudos
Error while running a notebook job using git repo (Gitlab)
I am trying to run a notebook job with a git repo hosted on Gitlab. I have Linked my Gitlab account using Gitlab tokenYet i am getting the following error on running the job How to resolve this?
- 502 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @vinitkhandelwal, Looks like the token could be missing required permissions for the operation. Please refer to: You can clone public remote repositories without Git credentials (a personal access token and a username). To modify a public remote r...
- 0 kudos
- 1168 Views
- 1 replies
- 0 kudos
Resolved! how to pass using DABs the parameters to use
Hello Community, I want to pass parameters to my Databricks job through the DABs CLI. Specifically, I'd like to be able to run a job with parameters directly using the command: databricks bundle run -t prod --params [for example: table_name="client"...
- 1168 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @jeremy98 ,You can pass parameters using CLI in following way: databricks bundle run -t ENV --params Param1=Value1,Param2=Value2 Job_Name And in your yml file you should define parameters in similar way to following:You can find more info in follo...
- 0 kudos
- 655 Views
- 1 replies
- 0 kudos
Pydeequ is not working with shared mode clusters on a unity catalog enabled databricks account
Hi folks,Recently i was doing a poc on pydeequ. I found that it is not working with shared mode clusters in a databricks account where unity catalog is enabled. It throws the below error :It works fine with the single mode clusters. Does anyone have ...
- 655 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @parasupadhyay ,I don't think there is much you can do. For security reasons, you can't use shared access mode in UC to access sparkContext. Your only option to make it work is to use single mode cluster.
- 0 kudos
- 969 Views
- 2 replies
- 0 kudos
Get user who ran a job
From the databricks API/ CLI is it possible to get the user who triggered a job run programatically?The information can be found in the job "event log" and can be queried in the "audit log" but neither of these seem to have a API option. Is there a w...
- 969 Views
- 2 replies
- 0 kudos
- 0 kudos
with the databricks cli you can get all infos about a job run with this command:databricks jobs get-run <run-id> replace <run-id> with your actual run-id
- 0 kudos
- 21189 Views
- 19 replies
- 10 kudos
Resolved! Lakehouse Fundamentals Certificate/Badge not appearing
Hello! I just passed the Lakehouse Fundamentals Accreditation and I haven't received any badge or certificate for it. I understand that I need to go to credentials.databricks.com but it is not there. How long before it appears? Need help
- 21189 Views
- 19 replies
- 10 kudos
- 10 kudos
Hello can you help me, I have the same problem. I don't have the certificate available on credentials.databricks.com.
- 10 kudos
- 1770 Views
- 3 replies
- 0 kudos
Databricks Genie AI Next Level Q&A
Hi Team Could Please you answer these three questions to onboard Genie Space for Analytics?Do you know if we can use GenieSpace in our web application through API or SDK?Is there any way to manage access control other than the Unity catalog?Can we ad...
- 1770 Views
- 3 replies
- 0 kudos
- 0 kudos
@AH my loyola wrote:Hi Team Could Please you answer these three questions to onboard Genie Space for Analytics?Do you know if we can use GenieSpace in our web application through API or SDK?Is there any way to manage access control other than the Uni...
- 0 kudos
| User | Count |
|---|---|
| 1615 | |
| 779 | |
| 349 | |
| 287 | |
| 267 |
