- 2084 Views
- 2 replies
- 0 kudos
There is no certification number in my Databricks certificate that i had received after passing the
I enrolled myself for the Databricks data engineer certification recently and gave a shot at the exam and i did clear it successfully. I have received the certificate in the form of a pdf file along with a URL in which i can see my certificate and ba...
- 2084 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @vinay076 Thanks for asking! Our support team can provide you with a credential ID. Please file a ticket with our support team, give them your email associated with your certification, and they can get you the credential ID.
- 0 kudos
- 8375 Views
- 5 replies
- 4 kudos
Resolved! How obtain a list of workflows in Databricks?
I need to obtain a list of my Databricks workflows with their job IDs in a notebook Databricks
- 8375 Views
- 5 replies
- 4 kudos
- 4 kudos
Hi @VabethRamirez , Also, instead of using directly the API, you can use databricks Python sdk : %pip install databricks-sdk --upgrade dbutils.library.restartPython()from databricks.sdk import WorkspaceClient w = WorkspaceClient() job_list = w.jobs...
- 4 kudos
- 1717 Views
- 1 replies
- 0 kudos
Can api for query history /api/2.0/sql/history/queries return data which is older than 30 days?
I am using this api but it is returning the data for only last 30 days. Can this api return data which is older than 30 days?
- 1717 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @RahulChaubey, The query history system table was announced during the Q1 roadmap webinar (see the recording, 32:25). There is a chance that it will provide data with a horizon beyond 30 days. Meanwhile, you can enable system tables - I hope some ...
- 0 kudos
- 2770 Views
- 2 replies
- 0 kudos
Does Delta Table can be the source of streaming/auto loader?
Hi,Since the Auto Loader only accept "append-only" data as the source, I am wondering if the "Delta Table" can also be the source.Does VACCUM(deleting stale files) or _delta_log(creating nested and different file format than parquet) going to break A...
- 2770 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @QPeiran, Auto-loader is a feature that allows to integrate files into the Data Platform. Once your data is stored into the Delta Table, you can rely on spark.readStream.table("<my_table_name>") to continuously read from the table. Take a look at ...
- 0 kudos
- 1579 Views
- 1 replies
- 0 kudos
Handling large volumes of streamed transactional data using DLT
We have a data stream from event hub with approximately 10 million rows per day (into one table) - these records are insert only (no update). We are trying to find a solution to aggregate / group by the data based on multiple data points and our requ...
- 1579 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi, please find below a set of resources I believe relevant for you. Success stories You can find the success stories of companies leveraging the streaming on Databricks here. Videos Introduction to Data Streaming on the Lakehouse : Structured Stream...
- 0 kudos
- 4541 Views
- 2 replies
- 0 kudos
Resolved! Rearrange tasks in databricks workflow
Hello,There is anyway to rearrange tasks in databricks workflow?.I would like that line that join the two marked tasks doesn't pass behind the other tasks. It is posible that this line by one side?Thanks.
- 4541 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @chemajar, Take a look at Databricks Asset Bundles. It allows you to streamline the development of complex workflows using a yaml definition. In case you need to change the task dependencies, you can rearrange the flow as you need just change the ...
- 0 kudos
- 2821 Views
- 2 replies
- 0 kudos
Do we pay just for qurery run duration while using databricks serverless sql ?
While using databricks serverless sql to run queries does we only pay for the compute resources during the run duration of the query ?
- 2821 Views
- 2 replies
- 0 kudos
- 1742 Views
- 1 replies
- 0 kudos
- 1742 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi, can you clatify what is your aim ? Maybe there is no need to use DB SDK at all ?
- 0 kudos
- 3240 Views
- 3 replies
- 0 kudos
Unity Catalog view access in Azure Storage account
Hi,I have my unity catalog in Azure Storage account and I can able to access table objects but I couldn't find my views that were created on top of those table. 1. I can can access Delta tables & related views via Databricks SQL and also find the tab...
- 3240 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi, Couple of options are possible : Use Databricks to do the complex SQL queries (joins, unions, etc) and write to a staging Delta Table. Then use DataFlow to read from that staged table. Orchestrate all of this using ADF or even Databricks Workflo...
- 0 kudos
- 2992 Views
- 2 replies
- 1 kudos
PowerBI Tips
Does anyone have any tips for using PowerBI on top of databricks? Any best practices you know of or roadblocks you have run into that should be avoided?Thanks.
- 2992 Views
- 2 replies
- 1 kudos
- 1 kudos
Hey, Use Partner Connect to establish a connection to PBI Consider to use Databricks SQL Serverless warehouses for the best user experience and performance (see Intelligent Workload Management aka auto-scaling and query queuing, remote result cache, ...
- 1 kudos
- 1753 Views
- 1 replies
- 0 kudos
Connecting to DataBricks Sql warehouse from .net
Hi,How can I connect to DataBricks sql warehouse from .net application. Kr
- 1753 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey, Please, take a look at Statement Execution API Best,
- 0 kudos
- 1963 Views
- 1 replies
- 1 kudos
Resolved! Can we get SQL Serverless warehouses monitoring data using APIs or logs ?
I am looking for a possible way to get the autoscaling history data for SQL Serverless Warehouses using API or logs.I want something like what we see in monitoring UI.
- 1963 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi Rahul, you need to perform two actions : Enable system tables schema named "compute" (how-to, take a look on the page, it's highly possible that you'll find other schemas useful too)Explore system.compute.warehouse_events table Hope this helps. B...
- 1 kudos
- 11481 Views
- 4 replies
- 1 kudos
column "id" is of type uuid but expression is of type character varying.
Hello,I'm trying to write to Azure PostgreSQL flexible database from Azure Databricks, using PostgreSQL connector in Databricks Runtime in 12.2LTS.I'm using df.write.format("postgresql").save() to write to PostgreSQL database, but getting the follow...
- 11481 Views
- 4 replies
- 1 kudos
- 1 kudos
Yes, this stack overflow was my reference too and adding below option made load go with no error on UUID data type in postgres columnSpoiler.option(stringtype, "unspecified").option(stringtype, "unspecified")https://stackoverflow.com/questions/409739...
- 1 kudos
- 2358 Views
- 1 replies
- 3 kudos
Regional Group Request for Istanbul
Hello, I kindly request the formation of a regional group for Istanbul/Turkey. I would appreciate your assistance in this matter.Thank you,Can
- 2358 Views
- 1 replies
- 3 kudos
- 3 kudos
@kankotan Happy to help set it up for you. I have dropped an email for more information!
- 3 kudos
- 1310 Views
- 0 replies
- 0 kudos
Usage of SparkMetric_CL, SparkListenerEvent_CL and SparkLoggingEvent_CL
I am wondering If can retrieve any information from Azure Log Analytics custom tables (already set) for Azure Databricks. Would like to retrieve information about query and data performance for SQL Warehouse Cluster. I am not sure If I can get it fro...
- 1310 Views
- 0 replies
- 0 kudos
