- 9876 Views
- 6 replies
- 7 kudos
Is catalog a feature in the community version?
%sql create catalog if not exists catalog1I tried above, but it gives me error as below:com.databricks.backend.common.rpc.DatabricksExceptions$SQLExecutionException: org.apache.spark.sql.AnalysisException: Catalog namespace is not supported. at com.d...
- 9876 Views
- 6 replies
- 7 kudos
- 7 kudos
Hi @THIAM HUAT TAN​​, It would mean a lot if you could select the "Best Answer" to help others find the correct answer faster.This makes that answer appear right after the question, so it's easier to find within a thread.It also helps us mark the que...
- 7 kudos
- 1398 Views
- 2 replies
- 0 kudos
Databricks workflow migration to higher environments
How do we migrate the databricks workflows to higher environment ? I do see an option for calling the tasks (notebooks,python) from the github repositories. But as such how do we migrate the entire workflow jobs to other environment ?
- 1398 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Menaka Murugesan​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.
- 0 kudos
- 1049 Views
- 2 replies
- 0 kudos
numSourceRows greater than expected
HeyI am doing an upsert of a source DataFrame into a target table. Before said upsert, I print out the source DataFrame's row count, which is a bit smaller than what `numSourceRows` says after the operation completes and I check the operationMetrics....
- 1049 Views
- 2 replies
- 0 kudos
- 0 kudos
could you share your code snippet please? also share the expected output.
- 0 kudos
- 1705 Views
- 3 replies
- 3 kudos
Resolved! R code that works perfectly on Rstudio does not run here
Hi,I have a "simple" R script that I need to import into Databricks and am running into errors.For example:TipoB <- Techtb %>% dplyr::filter(grepl('being evaluated', Comentarios))#TipoB$yearsSpec <- NATipoB$yearsSpec <- str_replace(TipoB$Comentarios,...
- 1705 Views
- 3 replies
- 3 kudos
- 3 kudos
R studio version 2022.12.0.R latest version available on 08/FEB/2023. I don't know where to find the DBR version and configuration. Can you direct me?
- 3 kudos
- 2135 Views
- 3 replies
- 0 kudos
Resolved! Importing irregularly formatted json files
HiI'm importing a large collection of json files, the problem is that they are not what I would expect a well-formatted json file to be (although probably still valid), each file consists of only a single record that looks something like this (this i...
- 2135 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Michael Johnson​,I would like to share the following notebook which contains examples on how to process complex data types, like JSON. Please check the following link and let us know if you still need help https://docs.databricks.com/optimization...
- 0 kudos
- 1603 Views
- 1 replies
- 4 kudos
- 1603 Views
- 1 replies
- 4 kudos
- 4 kudos
You can now use cluster policies to restrict the number of clusters a user can create. For more information https://docs.databricks.com/administration-guide/clusters/policies.html#cluster-limit
- 4 kudos
- 1747 Views
- 1 replies
- 2 kudos
Resolved! Can I clone Apache Iceberg Tables ?
- 1747 Views
- 1 replies
- 2 kudos
- 2 kudos
Clone can now be used to create and incrementally update Delta tables that mirror Apache Parquet and Apache Iceberg tables. You can update your source Parquet table and incrementally apply the changes to their cloned Delta table with the clone comman...
- 2 kudos
- 939 Views
- 1 replies
- 2 kudos
- 939 Views
- 1 replies
- 2 kudos
- 2 kudos
You can now use OAuth to authenticate to Power BI and Tableau. For more information, see Configure OAuth (Public Preview) for Power BI and Configure OAuth (Public Preview) for Tableau.https://docs.databricks.com/integrations/configure-oauth-powerbi.h...
- 2 kudos
- 3292 Views
- 6 replies
- 3 kudos
Resolved! Is 'run_as' user available from jobs api 2.1?
I know that the run_as user generally defaults to the creator_user, but I would like to find the defined run_as user for each of our jobs. Unfortunately, I'm unable to locate that field in the api.
- 3292 Views
- 6 replies
- 3 kudos
- 3 kudos
Hi @Keller, Michael​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...
- 3 kudos
- 3530 Views
- 5 replies
- 2 kudos
Managing the permissions using MLFlow APIs
Hello All,I am trying to manage the permissions on the experiments using the MLFLow API. Do we have any MLFlow API which helps to manage the permissions of Can Read ,Can Edit , Can Manage.Example :I create the model using MLFlow APIs and through my c...
- 3530 Views
- 5 replies
- 2 kudos
- 2 kudos
Hey folks, did we get any workaround for this or what @Sean Owen​ said is true ?
- 2 kudos
- 1762 Views
- 1 replies
- 1 kudos
Resolved! Is it possible to restart a cluster from a Notebook without using the UI
I have some code that occasionally wrong executed, meaning that every n-th time a calculation in a table is wrong. If that happens, I want to be able to restart the cluster from the Notebook.- I'm therefore lookong for a piece of code that can accomp...
- 1762 Views
- 1 replies
- 1 kudos
- 1 kudos
@Lukas Goldschmied​ It is. You'll need to use Databricks API.Here you can find an example:https://learn.microsoft.com/en-us/azure/databricks/_extras/notebooks/source/clusters-long-running-optional-restart.html
- 1 kudos
- 1218 Views
- 3 replies
- 3 kudos
We are using the Databricks managed MLflow to log experiment runs for quite some time already and never experienced issues. However, now we seem to have encountered a bug in the associated Databricks UI.
We observe the following behavior when we keep adding new runs to an experiment:- In the beginning, the runs are still displayed correctly in the UI.- After a certain number of total runs, the following bug occurs in the UI: - In the UI, there are ...
- 1218 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @Timo Burmeister​ Apologies for the delay! I went through the video, does it happen all the time? I see after sorting it with different filter the list appears.
- 3 kudos
- 6344 Views
- 3 replies
- 0 kudos
Error loading MANAGED table in unity catalog delta lake on azure. Anyone seen this issue? "ErrorClass=INVALID_PARAMETER_VALUE] Input path <file system name>.dfs.core.windows.net overlaps with other external tables"
00007160: 2023-01-30T14:22:06 [TARGET_LOAD ]E: Failed (retcode -1) to execute statement: 'COPY INTO `e2underwriting_dbo`.`product` FROM(SELECT cast(_c0 as INT) as `ProductID`, _c1 as `ShortName`, cast(_c2 as INT) as `Status`, cast(_c3 as TIMESTA...
- 6344 Views
- 3 replies
- 0 kudos
- 0 kudos
we have solved this issue related to Qlik replicate copying data into delta table
- 0 kudos
- 937 Views
- 1 replies
- 1 kudos
- 937 Views
- 1 replies
- 1 kudos
- 1 kudos
You can ensure there is always an active run of your Databricks job with the new continuous trigger type. https://docs.databricks.com/workflows/jobs/jobs.html#continuous-jobs
- 1 kudos
- 7898 Views
- 9 replies
- 3 kudos
Resolved! Can't write / overwrite delta table with error: oxxxx.saveAsTable. (Driver Error: OutOfMemory)
Current Cluster Config:Standard_DS3_v2 (14GB, 4 Cores) 2-6 workersStandard_DS3_v2 (14GB, 4Cores) for driverRuntime: 10.4x-scala2.12We want to overwrite a temporary delta table with new records. The records will be load by another delta table and tran...
- 7898 Views
- 9 replies
- 3 kudos
- 3 kudos
Hi,thank you for your help!We tested the configuration settings and it runs without any errors.Could you give us some more information, where we can find some documentation about such settings. We searched hours to fix our problem. So we contacted th...
- 3 kudos