- 6761 Views
- 4 replies
- 0 kudos
Fundamentals of the Databricks Lakehouse Platform Accreditation (V2)
Hi,After 48 hours after passing the exam, I didn´t receive any certification. My ID: E-03DK31. This is the prevous ticket Ticket URL: https://help.databricks.com/s/case/5008Y00002AAdImQAL
- 6761 Views
- 4 replies
- 0 kudos
- 2363 Views
- 4 replies
- 2 kudos
Resolved! Please provide more information about Photon
Please provide more information about Photon
- 2363 Views
- 4 replies
- 2 kudos
- 2 kudos
There is some information here, in case you haven't checked yet, this may help gaining some knowledge. https://www.databricks.com/product/photon
- 2 kudos
- 1039 Views
- 1 replies
- 0 kudos
Cross region DR setup.
Does anybody have any guidance on the best practices in setting up a DR env on a different region or on a different cloud ?
- 1039 Views
- 1 replies
- 0 kudos
- 0 kudos
@ariforu cross region you can setup, but different cloud looks as of now it is not supported. did you get a chacne to go through this https://docs.databricks.com/administration-guide/disaster-recovery.html
- 0 kudos
- 7831 Views
- 5 replies
- 1 kudos
How to register a JDBC Spark dialect in Python?
I am trying to read from a databricks table. I have used the url from a cluster in the databricks. I am getting this error: java.sql.SQLDataException: [Simba][JDBC](10140) Error converting value to int.After these statements:jdbcConnUrl= "jdbc:spark:...
- 7831 Views
- 5 replies
- 1 kudos
- 21136 Views
- 3 replies
- 11 kudos
How do I access the account console of Databricks in Azure?
To create a Unity metastore the docs refer me to the account console in Databricks, see:https://docs.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/create-metastoreHowever when I go to manage account, I get redirected to select wo...
- 21136 Views
- 3 replies
- 11 kudos
- 11 kudos
Please refer here - https://community.databricks.com/s/question/0D58Y000098lIqgSAE/unity-catalog-azure-account-console-how-to-accessYou must be an Azure Databricks account admin.The first Azure Databricks account admin must be an Azure Active Directo...
- 11 kudos
- 44331 Views
- 10 replies
- 12 kudos
How to efficiently process a 50Gb JSON file and store it in Delta?
Hi, I'm a fairly new user and I am using Azure Databricks to process a ~50Gb JSON file containing real estate data. I uploaded the JSON file to Azure Data Lake Gen2 storage and read the JSON file into a dataframe.df = spark.read.option('multiline', '...
- 44331 Views
- 10 replies
- 12 kudos
- 12 kudos
The spark connector is super slow. I found loading json into Azure cosmos dB then writing queries to get sections of data out was 25x times faster because cosmos dB indexes the json. You can stream read data from cosmosdb. You can find python code sn...
- 12 kudos
- 1933 Views
- 2 replies
- 1 kudos
Refresh delta
How refresh delta table with New raw from CDC Json file.
- 1933 Views
- 2 replies
- 1 kudos
- 1 kudos
To refresh a delta table with new raw data from a CDC JSON file, you can use change data capture (CDC) to update tables based on changes in source data. Here are the steps:1. Create a streaming table using the CREATE OR REFRESH STREAMING TABLE statem...
- 1 kudos
- 5525 Views
- 4 replies
- 1 kudos
Facing issue while creating Delta Live Table on top of csv file
Hello Everyone,I am trying to create Delta Live Table on top of csv file using below syntax:CREATE OR REFRESH LIVE TABLE employee_bronze_dltCOMMENT "The bronze employee dataset, ingested from /mnt/lakehouse/PoC/DLT/Source/."AS SELECT * FROM csv.`/mnt...
- 5525 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @Manasi_Sarang ,I believe the Delta is unable to infer the schema as you are using select statement to read entire content from csv file and I think the inferschema won't work here. Instead you can try to create a temp live table or live view wit...
- 1 kudos
- 4639 Views
- 2 replies
- 0 kudos
INTERNAL ERROR
I have the following query;select customer_id, first(if(name_type = 'Official', name, null),true) official_name, first(if(name_type = 'Preferred', name, null),true) preferred_namefrom( select customer_id, ifnull(name_type, 'Official...
- 4639 Views
- 2 replies
- 0 kudos
- 0 kudos
I experienced similar issues from time to time. What helped is to refresh the browser page.If that does not work, restart the sql warehouse.The internal error indeed is pretty vague, but my experience is that this is not related to a wrong SQL scrip...
- 0 kudos
- 13856 Views
- 1 replies
- 2 kudos
Resolved! Difference between using DBT and data bricks's lineage toolol
So my team is using DBT for a lot of data lineage items but then at the data summit it was shown that data bricks also has a similar tool that is in fact better because it does lineage on columns. So what's the main draw of DBT at this point?
- 13856 Views
- 1 replies
- 2 kudos
- 2 kudos
DBT (Data Build Tool) remains popular for its extensive templating capabilities, modularity, and open-source nature, which allows for customization and integration with various data platforms. While Databricks may offer more advanced lineage features...
- 2 kudos
- 788 Views
- 0 replies
- 1 kudos
- 788 Views
- 0 replies
- 1 kudos
- 800 Views
- 0 replies
- 0 kudos
2023 summit feedback
Event covered everything a data engineer would dream of.My favorite discussions:- SparkConnect- AI on top unity catalog- delta live tables pipelines for streaming #Summit23
- 800 Views
- 0 replies
- 0 kudos
- 853 Views
- 0 replies
- 0 kudos
Data +AI Summit 2023
Amazing summit event, exciting features LLM, Lakeshouse IQ.
- 853 Views
- 0 replies
- 0 kudos
- 676 Views
- 0 replies
- 0 kudos
Data ai training
The deep learning training by Joseph and Marie was pretty cool
- 676 Views
- 0 replies
- 0 kudos
- 618 Views
- 0 replies
- 0 kudos
Expo experience
Great partners and SaaS solutions to jump start on floor
- 618 Views
- 0 replies
- 0 kudos
| User | Count |
|---|---|
| 1616 | |
| 786 | |
| 366 | |
| 349 | |
| 287 |
