Hello Databricks Community,I have to issue related to Azure databricks workspace:- First: I have one azure databrick workspace in resource group A. But I want to move this to resource group B, but as I know Azure hasn't supported yet. So which is the...
Hello guys,I want to integrate Databricks Job API to the share point site. But I have a CORS policy error that blocks all of the calls from the share point site. How can I white list my share point site ? Or is there any other way ? Thank you
In the root of my workspace, I have the following folder structure set up:/foo / ETL / setup / utilitiesFrom a notebook in /foo/ETL or /foo/setup, if I try to %run a notebook in /foo/utilities by giving the relative path ../utilities/<x>, I get...
Hello @DManowitz-BAH ,Hope you are doing great, I tried to replicate this behavior on 12.2 LTS runtime and didn't get any errors for the same folder/workspace structure..Can you share the cluster configuration details so I can mimic exactly the same ...
What is the best way to be proficient in Apache spark?
Hello community: I don´t know why my Delta Live table workflow fails into this step. This is the configuration I have for the pipeline:{"id": "**","pipeline_type": "WORKSPACE","clusters": [{"label": "default","spark_conf": {***},"num_workers": 3},{"l...
Hi,Databricks repos is making white spaces in the notebooks, without us opening the notebook. So seems to be automatically. Does someone has a solution how we can turn this off?An example:Can someone helps us with this?
This bug is causing problems in our CI/CD pipeline too. I've had to manually clear things to keep it running as we get merge conflicts when no changes are made.
Hi,Is there any way we can connect glue catalog as well as to hive metastore in the same warehouse?I can create a single instance profile and provide all the required access for buckets or for glue catalog.I tried with the below configuration,spark.s...
The best trial results seem to change every time the same study is rerun. On Microsoft Azure this can be fixed by setting the sampler seed. However this solution doesn't seem to work on Databricks. Does anyone know why that is the case and how to mak...
I know this has probably been asked many times, although I have struggled to find the answer using sql CREATE STREAMING LIVE TABLE bronze_site_usage COMMENT "ingest latest snapshot from api"TBLPROPERTIES ("layer" = "bronze") AS SELECT * FROM cloud_fi...
I realize that the best practice would be to integrate our service with Kafka as a streaming source for Databricks, but given that the service already stores data into TimescaleDB, how can I stream data from TimescaleDB into DBX? Debezium doesn't wor...
When anyone (admins included) click on an alert task in a job run we see the error `No results were found: the query results may no longer be available or you may not have permissions`.Should we be seeing something else or is this a matter of a poor ...
The error message "No results were found: the query results may no longer be available or you may not have permissions" is designed to address a range of potential situations. This includes instances where data might not be accessible due to reasons ...
I have the following command that runs in my databricks notebook.spark.conf.get("spark.databricks.clusterUsageTags.managedResourceGroup")I have wrapped this command into a function (simplified).def get_info(): return spark.conf.get("spark.databri...
Hi,Whenever I try to signup for an account, I keep getting the following message in the first step - "an error has occurred. please try again later" Could you please let me know why this could be? I tried multiple emails and seems to be having same i...
Is it the same in all the browsers?
Recently I discovered an issue when creating a PARQUET table that contains a column of type STRUCT with more than 350 string subfields. Such a table can be successfully created via a standard DDL script nevertheless each subsequent attempt to work wi...
