- 1613 Views
- 1 replies
- 1 kudos
Resolved! Databricks community edition and cloud platform
I have been using databricks in work for a few years now and absolutely love it.I have been wanting to use it at home but I dont have a ton of money to spend.Any community members can advise a good cloud option to run databricks on that's relatively ...
- 1613 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Aidzillafont, Databricks can run on several cloud platforms, including AWS, Azure, and Google Cloud. The cost of running Databricks would largely depend on the pricing policies of these cloud platforms. However, if you are looking for a relative...
- 1 kudos
- 4849 Views
- 7 replies
- 8 kudos
Resolved! Cannot use docker container with runtime 13.1 and above
Hi,In our setup we use custom docker images on our job clusters. The setup of these images is in line with the docker images found on the databricks github page. The custom image works fine on runtimes, including 13.0. When using anything higher that...
- 4849 Views
- 7 replies
- 8 kudos
- 8 kudos
Interestingly enough I can use the exact same docker image on 13.3LTS Beta with no issues.
- 8 kudos
- 1035 Views
- 1 replies
- 0 kudos
My exam has suspended , Need help Urgently (21/08/2023)
Hello Team,I encountered Pathetic experience while attempting my 1st DataBricks certification. Abruptly, Proctor asked me to show my desk, after showing he/she asked multiple times.. wasted my time and then suspended my exam.I want to file a complain...
- 1035 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @sirishavemula20 , I'm sorry about your challenging experience during your Databricks certification attempt. Your frustration is entirely understandable, and I apologize for any inconvenience you faced. We take your concerns seriously, and we'll ...
- 0 kudos
- 944 Views
- 1 replies
- 0 kudos
The Voucher code provided to me by my organization is not working. Please help ASAP.
Hello,I have been recently given a voucher code by my organization to register for the Databricks Data Engineer Associate (Version 3) exam. However, when I tried to apply the code, the page kept showing "Candidate is not associated with the promotion...
- 944 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @GopiBollineni , I want to assure you that your query has been noted, and someone from our training team will contact you shortly to assist with this matter. We appreciate your patience and apologize for any inconvenience this may have caused. If...
- 0 kudos
- 7284 Views
- 8 replies
- 4 kudos
InternalError, error message: The output of the notebook is too large.
Hello guys. I am having the following annoying issue with DataBricks job executed from ADF. I have stopped all the logs, deleted all the print statements and display. And I am still getting this error. I don't understand what might cause it. I am att...
- 7284 Views
- 8 replies
- 4 kudos
- 4 kudos
Apologies for the late reply. I will try to upload the logs,cause it's been some time.Interesting is how I solve it. I put all the code in a single cell and It is running since then without issues. Although it was just few cells with python code. I e...
- 4 kudos
- 5146 Views
- 1 replies
- 1 kudos
S3 Permission errors
Hello Background -I have an S3 datalake set up prior to signing up with Databricks. I'm still in my evaluation period. I'm trying to read the contents of an S3 bucket but am getting all kinds of permission problems. Here is the command in the noteb...
- 5146 Views
- 1 replies
- 1 kudos
- 1 kudos
UPDATE:The permission problems only exist when the Cluster's (compute's) Access mode is "Shared No Isolation". When the Access Mode is either "Shared" or "Single User" then the IAM configuration seems to apply as expected. When set to "Shared No Is...
- 1 kudos
- 1501 Views
- 0 replies
- 0 kudos
move databricks workspace to another resource group
Hello Databricks Community,I have to issue related to Azure databricks workspace:- First: I have one azure databrick workspace in resource group A. But I want to move this to resource group B, but as I know Azure hasn't supported yet. So which is the...
- 1501 Views
- 0 replies
- 0 kudos
- 564 Views
- 0 replies
- 0 kudos
Job REST API CORS error
Hello guys,I want to integrate Databricks Job API to the share point site. But I have a CORS policy error that blocks all of the calls from the share point site. How can I white list my share point site ? Or is there any other way ? Thank you
- 564 Views
- 0 replies
- 0 kudos
- 2386 Views
- 3 replies
- 0 kudos
Cannot %run notebook using relative path
In the root of my workspace, I have the following folder structure set up:/foo / ETL / setup / utilitiesFrom a notebook in /foo/ETL or /foo/setup, if I try to %run a notebook in /foo/utilities by giving the relative path ../utilities/<x>, I get...
- 2386 Views
- 3 replies
- 0 kudos
- 0 kudos
Hello @DManowitz-BAH ,Hope you are doing great, I tried to replicate this behavior on 12.2 LTS runtime and didn't get any errors for the same folder/workspace structure..Can you share the cluster configuration details so I can mimic exactly the same ...
- 0 kudos
- 1224 Views
- 2 replies
- 2 kudos
Delta Live Tables: BAD_REQUEST: Pipeline cluster is not reachable.
Hello community: I don´t know why my Delta Live table workflow fails into this step. This is the configuration I have for the pipeline:{"id": "**","pipeline_type": "WORKSPACE","clusters": [{"label": "default","spark_conf": {***},"num_workers": 3},{"l...
- 1224 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @jorgemarmol, There could be several reasons why the pipeline cluster is unreachable. Some of the possible reasons include: 1. Infrastructural issues: The dev clusters are occasionally down due to infrastructural issues. Infra changes are often t...
- 2 kudos
- 2065 Views
- 5 replies
- 4 kudos
Databricks repos adds white spaces to notebook.
Hi,Databricks repos is making white spaces in the notebooks, without us opening the notebook. So seems to be automatically. Does someone has a solution how we can turn this off?An example:Can someone helps us with this?
- 2065 Views
- 5 replies
- 4 kudos
- 4 kudos
This bug is causing problems in our CI/CD pipeline too. I've had to manually clear things to keep it running as we get merge conflicts when no changes are made.
- 4 kudos
- 3997 Views
- 2 replies
- 0 kudos
Pyspark cast error
Hi All,hive> create table UK ( a decimal(10,2)) ;hive> create table IN ( a decimal(10,5)) ;hive> create view T as select a from UK union all select a from IN ;above all statements executes successfully in Hive and return results when select statement...
- 3997 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @anandreddy23, The error you're encountering, "Cannot upcast a from decimal(10,2) to decimal(10,5)", is typically due to an attempt to cast a decimal value with lower precision to a decimal value with higher accuracy. In PySpark, the casting oper...
- 0 kudos
- 5590 Views
- 0 replies
- 0 kudos
- 5590 Views
- 0 replies
- 0 kudos
- 4143 Views
- 1 replies
- 0 kudos
Resolved! Connecting to hive metastore as well as glue catalog at the sametime
Hi,Is there any way we can connect glue catalog as well as to hive metastore in the same warehouse?I can create a single instance profile and provide all the required access for buckets or for glue catalog.I tried with the below configuration,spark.s...
- 4143 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @RC, Based on the provided information, you cannot dynamically switch between Glue Catalog and a Hive metastore in the same warehouse. As per the limitations mentioned in the AWS Glue metastore documentation, you must restart the cluster for new ...
- 0 kudos
- 1033 Views
- 1 replies
- 0 kudos
Optuna results change when rerun on Databricks
The best trial results seem to change every time the same study is rerun. On Microsoft Azure this can be fixed by setting the sampler seed. However this solution doesn't seem to work on Databricks. Does anyone know why that is the case and how to mak...
- 1033 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Shwang, It's not uncommon for results to vary when re-running experiments, especially when using complex machine learning or optimization techniques. To better understand your situation, please provide more details about the changes you observed ...
- 0 kudos
