- 1323 Views
- 1 replies
- 0 kudos
autoloader data processing
Hi Team,Can you share the best practices for designing the autoloader data processing?We have data from 30 countries data coming in various files. Currently, we are thinking of using a root folder i.e country, and with subfolders for the individual ...
- 1323 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Phani1 ,Structure of folders that you are going to use make sense to me. Since you've mentioned that there will be thousands of files, the best practice will be to use autoloader with file notification mode. Also, you can read about databricks r...
- 0 kudos
- 4447 Views
- 4 replies
- 0 kudos
Accessing the S3 Files
I am using the Unity Catalog Cluster. I have a requirement to read the files placed by the source team in a specific location (landing) in S3. I am already using a metastore pointing to a different bucket. Do I need to use an external location pointi...
- 4447 Views
- 4 replies
- 0 kudos
- 0 kudos
did anyone get any solution on this topic? I am also facing the challenges reading the file from s3 using the boto3 with unity enabled cluster, created the s3 external location and granted the enough access. any help on this ?same path and data acce...
- 0 kudos
- 11853 Views
- 5 replies
- 1 kudos
databricks vscode profile
Hello,I'm trying to connect to our databricks instance using the vscode extension. However, when following this guide we cannot get the configuration to proceed past the point that it asks for our instance URL.when i am creating profile [DEFAULT]host...
- 11853 Views
- 5 replies
- 1 kudos
- 2386 Views
- 1 replies
- 0 kudos
Alert Notifications not firing. Inconsistent Success Rate
Hi there,I have a few alerts for a query where if the column is set to >= 1, it will send an email notification to me and 2 others. The 2 individuals are already added as a destination.The query works because the results are populating that column wi...
- 2386 Views
- 1 replies
- 0 kudos
- 0 kudos
I'm facing the same issue.Don't see anything missing but still no emails even when status of the alert is triggered.@DangEdmond did you find the solution?
- 0 kudos
- 8507 Views
- 9 replies
- 3 kudos
Unable to login to databricks community edition
Hi, I was using databricks and it was working fine. Im using community edition. Suddenly, it logged me out. I also clicked on forgot password, changed the password and when i tried to login again, i keeps me redirecting to login page without any erro...
- 8507 Views
- 9 replies
- 3 kudos
- 3 kudos
I am still facing this issue. Is there any update ? It says "Session expired, Log back in" and goes to login window.
- 3 kudos
- 1564 Views
- 2 replies
- 3 kudos
Session Expires every few seconds after logging into community edition
Session Expires every few seconds after logging into community edition
- 1564 Views
- 2 replies
- 3 kudos
- 3 kudos
It automatically disconnects and then connects after 1 min, destroying overall concentration.
- 3 kudos
- 3000 Views
- 8 replies
- 0 kudos
Databricks test has been suspended.
Hi @Cert-Team, My Databricks certification test has been suspended. Not even turned my head a bit or looked away from screen but got suspended. Can you guys assist here.Request no - #00520224 ref:!00D610JGc4.!500Vp0A9m4Y:ref
- 3000 Views
- 8 replies
- 0 kudos
- 0 kudos
@roshan_07 You could log into your test taker account to see if it has been rescheduled, just FYI.
- 0 kudos
- 1359 Views
- 1 replies
- 1 kudos
Your session has expired, please authenticate again
I get this message every fifteen seconds in the community edition. The platform is practically not usable by any means.
- 1359 Views
- 1 replies
- 1 kudos
- 1 kudos
Same issue. Its so annoying. Hope they rectify it asap.
- 1 kudos
- 5833 Views
- 6 replies
- 0 kudos
Running dbt in a Databricks task, the dbt_output from the Databricks jobs api is empty
I'm running a scheduled workflow with a dbt task on Azure Databricks. We want to export the dbt-output from the dbt task to a storage container for our Slim CI setup and data observability. The issue is, that the Databricks API (/api/2.1/jobs/runs/ge...
- 5833 Views
- 6 replies
- 0 kudos
- 0 kudos
I am running dbt on a databricks job. It saves all documentation: manifest.json, run_results.json, etc in "Download Artifacts" in a job. I am not able to find out a way to read those in codes, transform and save on databricks. Tried job API. The arti...
- 0 kudos
- 1235 Views
- 0 replies
- 0 kudos
Experiencing Performance Issues When Using Databricks as a Transactional System with .NET App
Hello everyone,We are currently facing performance issues when using Databricks as a transactional system with our .NET application via the Simba ODBC driver. Specifically, queries are taking 30 seconds to 1 minute to insert data into 8 to 10 tables,...
- 1235 Views
- 0 replies
- 0 kudos
- 663 Views
- 0 replies
- 0 kudos
Getting Error when add shell script in init script for job cluster to copy file from DBFS to local
Getting Error when add shell script in init script for job cluster to copy file from DBFS to local as below is not working for GCP Databricks and same thing is working for azure data bricks Verified DBFS location file is present there .shell script i...
- 663 Views
- 0 replies
- 0 kudos
- 1268 Views
- 1 replies
- 0 kudos
Databricks certificate exam got suspended, Need help to reschedule the exam
Hi @Cert-Team ,Today Mu Databricks certificate exam was suspended in the midway after approx 1 hour. Proctor asked me to show the room and after sometime he conveyed that my exam is being suspended and i will have to reach out to Databricks team for ...
- 1268 Views
- 1 replies
- 0 kudos
- 0 kudos
@Ajaykumar the support team has responded to your ticket.
- 0 kudos
- 2412 Views
- 6 replies
- 0 kudos
Getting an error while editing databrics metastore "databricks_metastore" using terraform (Azure)
Hi , I am getting an error (Metastore has reached the limit for metastores in region ) while updating metatsore using terraform , below the script i am using for updating metastore . There is already a metastore available i dont want to create the ne...
- 2412 Views
- 6 replies
- 0 kudos
- 0 kudos
Hi, Thank you for the suggestions, But i am still getting same error .
- 0 kudos
- 1702 Views
- 4 replies
- 0 kudos
Help to reschedule of my exam
Hi team,My Databricks Certified Data Engineer Associate exam got suspended within 40 minutes. I had also shown my exam room to the proctor. My exam got suspended due to watch. I was not using anything. Some questions are so big in the exam so I had t...
- 1702 Views
- 4 replies
- 0 kudos
- 2436 Views
- 1 replies
- 0 kudos
How to Resolve ConnectTimeoutError When Registering Models with MLflow
Hello everyone,I'm trying to register a model with MLflow in Databricks, but encountering an error with the following command: model_version = mlflow.register_model(f"runs:/{run_id}/random_forest_model", model_name) The error message is as follows:...
- 2436 Views
- 1 replies
- 0 kudos
- 0 kudos
@otara_geni if you are still struggling, try this - set the environmental variable in your code just before logging the model with the URL of the regional S3 endpoint (from the error it looks like MLFlow is attempting to use global one, which may not...
- 0 kudos
