Certifications
Join dynamic discussions on Databricks certifications within the Community. Exchange insights, tips,...
764 Posts
Activity in Community Discussions
- 59 Views
- 1 replies
- 2 kudos
AutoLoader Pros/Cons When Extracting Data (Cross-Post)
Cross-posting from: https://community.databricks.com/t5/data-engineering/autoloader-pros-cons-when-extracting-data/td-p/127400Hi there, I am interested in using AutoLoader, but I'd like to get a bit of clarity if it makes sense in my case. Based on e...
- 59 Views
- 1 replies
- 2 kudos
Latest Reply
- 2 kudos
You’ve already identified data duplication as a potential con of landing the data first, but there are several benefits to this approach that might not be immediately obvious:Schema Inference and Evolution: AutoLoader can automatically infer the sche...
- 2 kudos
- 1140 Views
- 1 replies
- 0 kudos
Data load issue
I have a job in Databricks which completed successfully but the data is not been written into the target table, I have checked all the possible ways, each n every thing is correct in the code, target table name, source table name, etc etc. It is a Fu...
- 1140 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
@Srajole just checking in. Did you manage to resolve your issue?All the best,BS
- 0 kudos
- 64073 Views
- 177 replies
- 16 kudos
Suspended Exam? Don't worry! Our support team can help!
Taking exams is so stressful already, and if your exam gets suspended it is extra stressful. Don't worry! Our support team is awesome and will help you get back on track. The most important step is to file a ticket in our Help Center. Be sure to inc...
- 64073 Views
- 177 replies
- 16 kudos
Latest Reply
- 16 kudos
Hi,My exam has been suspended, without violating any code of conduct. I hope to be able to rewrite the exam, and get certified.Hope the support team can help as soon as possible .#Ticket number #0071471Proctored Assessment 1: Databricks Certified Dat...
- 16 kudos
- 721 Views
- 4 replies
- 2 kudos
Resolved! Databricks Certified Generative AI Engineer Associate : Result (Pass)
I have passed the Databricks Certified Generative AI Engineer Associate on 27 July 2025.Though, it has crossed 48 hrs., still did not get the certificate:Name: Rudra Prasad Martha
- 721 Views
- 4 replies
- 2 kudos
- 134 Views
- 3 replies
- 2 kudos
Resolved! Python module import with Dedicated access mode
I currently have a repo connected in databricks and I was able to correctly import a python module from src folder located in the same root.Since I am using a Machine Learning runtime, I am force to choose a Dedicated (formerly: Single user) access m...
- 134 Views
- 3 replies
- 2 kudos
Latest Reply
- 2 kudos
Thanks @szymon_dybczak ! I confirm that's a permission issue and assigning "CAN MANAGE" solves it.I still find it not really intuitive, since the goal is to use a shared cluster (with ML runtime) for development purposes. I mean, it would make sense ...
- 2 kudos
- 36 Views
- 0 replies
- 0 kudos
Automating technical documentation in ETL pipelines using LLMs
Generate pipeline documentation using LLMs and rich metadata extract As enterprise data environments expand, the complexity of maintaining accurate and current documentation across ETL pipelines has intensified. While modern platforms such as Databri...
- 36 Views
- 0 replies
- 0 kudos
- 259 Views
- 3 replies
- 1 kudos
add new column to a table and failing the previous jobs
Hello community! I’m new to Databricks and currently working on a project structured in Bronze / Silver / Gold layers using Delta Lake and Change Data Feed.I recently added 3 new columns to a table and initially applied these changes via PySpark SQ...
- 259 Views
- 3 replies
- 1 kudos
Latest Reply
- 1 kudos
Hello @leticialima__ Good dayCan you please share the error observed on the driver log. is it : [Errno 13] Permission denied or No such file or directory? Please let me know the error on the driver log. THank you.
- 1 kudos
- 146 Views
- 1 replies
- 1 kudos
Using merge Schema with spark.read.csv for inconsistent schemas
The Problem:A common data engineering challenge is reading a directory of CSV files where the schemas are inconsistent. For example, some files might have columns in a different order, or be missing certain columns altogether. The standard behavior o...
- 146 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
Very nice example and explanation helped me a lot, thanks !
- 1 kudos
- 40 Views
- 1 replies
- 0 kudos
Databricks - Subscription is not Active
I am getting Databricks - Subscription is not Active error after linking aws with databricks.i still have time for the same.pls help to resolve
- 40 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello @rajesh2025! Check your AWS Marketplace account to ensure the subscription is active and correctly linked to your billing account. Also, verify that your email isn’t already associated with another Databricks account. If the issue persists, con...
- 0 kudos
- 117 Views
- 3 replies
- 1 kudos
Resolved! Writing to data to a .csv file ( in the Databricks free edition)
I have been testing out a notebook. Is is possible to write any tabular outputs to a .csv file and if so would this be the filesystem or an S3 bucket.I get errrors when I try either approaches with the snippet below.#sort by match date to get the mo...
- 117 Views
- 3 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi @ilir_nuredini ,Thanks very much for the advice as this really helped esp DFS not available on the free edtions.I ran into an errror and I realised I needed to create a volume and schema first. So with with a little help from the AI assistant, -...
- 1 kudos
- 68 Views
- 1 replies
- 0 kudos
Unity Catalog tool function with custom parameters not being used
I have created a UC tool that takes in a few custom STRING parameters. I gave this tool to an ai agent using the mosaic ai agent framework with hardcoded parameter values for testing. The issue is my ai agent hallucinates and injects its own ai gener...
- 68 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @Boban12335,Can we get UC function definition to understand your problem better?Best Regards,Nivethan V
- 0 kudos
- 48 Views
- 1 replies
- 1 kudos
Connect langchain with databricks unity catalog
Team,Please help me connect langchain with databricks unity catalog
- 48 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
Steps Explained: Connect LangChain with Databricks Unity Catalog1. Install Required Packagespip install langchain databricks-sql-connector2. Configure Access to Databricks Unity CatalogUse Databricks SQL Connector to connect to Unity Catalog tables....
- 1 kudos
- 136 Views
- 3 replies
- 3 kudos
AutoLoader - Write To Console (Notebook Cell) Long Running Issue
Hi there,I am likely misunderstanding how to use AutoLoader properly while developing/testing. I am trying to write a simple AutoLoader notebook cell to *read* the contents of a path with json files, and *write* them to console (i.e. notebook cell) i...
- 136 Views
- 3 replies
- 3 kudos
Latest Reply
- 3 kudos
Hi @ChristianRRL ,It looks like spark.readStream with Auto Loader creates a continuous streaming job by default, which means it keeps running while waiting for new files.To avoid this, you can control the behaviour using trigger(availableNow=True), w...
- 3 kudos
- 411 Views
- 2 replies
- 1 kudos
Documentation for spatial SQL public preview - Where is it?
Hi everybody,since DBR 17.1 spatial sql functions (st_point(), st_distancesphere, ... ) are in public preview.The functionality is presented in this talk Geospatial Insights With Databricks SQL: Techniques and Applications or discussed here in the fo...
- 411 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
Is this what you were after?https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-st-geospatial-functions
- 1 kudos
- 32533 Views
- 60 replies
- 2 kudos
Resolved! Databricks exam got suspended. Need urgent help
Hello Team, I faced multiple interruptions and challenges while attempting my certification, The exam got paused 4 times where twice I had to show my desk and my entire room. Eventually my exam got suspended. I was not involved in any unfair means an...
- 32533 Views
- 60 replies
- 2 kudos
Latest Reply
- 2 kudos
Databricks team, I attended the examination for Databricks certified Data engineer associate exam on Aug 4th and I was reviewing my questions when my exam got paused and later after showing room my video stream stopped and they have suspended my ex...
- 2 kudos
