- 2225 Views
- 0 replies
- 0 kudos
New draft for every post I visit
When I visit my profile page, under the drafts section I see an entry for every post I visit in the discussions. Is this normal?
- 2225 Views
- 0 replies
- 0 kudos
Forum Posts
- 1504 Views
- 1 replies
- 1 kudos
Databricks Web Editor's Cell like UI in local IDE
I want to have databricks related developement locally.There is extension that allows to run local python file on remote databricks cluster.But I want to have cell like structure that is present in databricks UI for python files in local IDE as well....
- 1504 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
@swapnilmd You can use VSCode extension for Databricks.https://docs.databricks.com/en/dev-tools/vscode-ext/index.html
- 1 kudos
- 5492 Views
- 2 replies
- 0 kudos
Databricks jdbc driver connectiion issue with apache solr
Hi,databricks jdbc version - 2.6.34I am facing the below issue with connecting databricks sql from apache solr Caused by: java.sql.SQLFeatureNotSupportedException: [Databricks][JDBC](10220) Driver does not support this optional feature.at com.databri...
- 5492 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Databricks team recommended to set IgnoreTransactions=1 and autocommit=false in the connection string but that didn't resolve the issue .Ultimately I had to use solr update API for uploading documents
- 0 kudos
- 2241 Views
- 3 replies
- 1 kudos
[Memory utilization in Metrics Tab still display after terminate a cluster]
Hi All,Could you guys help me to check this?I run a cluster and then terminate that cluster but when i navigate to the Metrics tab of Cluster still see the Memory utilization show metrics.Thanks
- 2241 Views
- 3 replies
- 1 kudos
Latest Reply
- 3269 Views
- 3 replies
- 0 kudos
databricks-jdbc lists `spark_catalog` among catalogs for Standard tier Azure workspace
databricks-jdbc lists `spark_catalog` among catalogs for Standard tier Azure workspace. The UI lists `hive_metastore`. It would be better if these two were consistent.
- 3269 Views
- 3 replies
- 0 kudos
- 2111 Views
- 0 replies
- 0 kudos
Both `hive_metastore` and `spark_catalog`?
HiIs it possible for a workspace to have both a `hive_metastore` catalog and a `spark_catalog` catalog?
- 2111 Views
- 0 replies
- 0 kudos
- 3211 Views
- 0 replies
- 0 kudos
New to Spark
Hi all,I am new to Spark, trying to write below code but getting an error.Code:df1 = df.filter(df.col1 > 60 and df.col2 != 'abc') Any suggestion?
- 3211 Views
- 0 replies
- 0 kudos
- 9139 Views
- 3 replies
- 4 kudos
Resolved! Error not a delta table for Unity Catalog table
Is anyone able to advise why I am getting the error not a delta table? The table was created in Unity Catalog. I've also tried DeltaTable.forName and also using 13.3 LTS and 14.3 LTS clusters. Any advice would be much appreciated
- 9139 Views
- 3 replies
- 4 kudos
Latest Reply
- 4 kudos
@StogponI believe if you are using DeltaTable.forPath then you have to pass the path where the table is. You can get this path from the Catalog. It is available in the details tab of the table.Example:delta_table_path = "dbfs:/user/hive/warehouse/xyz...
- 4 kudos
- 1962 Views
- 0 replies
- 0 kudos
Best practices for working with external locations where many files arrive constantly
I have an Azure Function that receives files (not volumes) and dumps them to cloud storage. One-five files are received approx. per second. I want to create a partitioned table in Databricks to work with. How should I do this? E.g.: register the cont...
- 1962 Views
- 0 replies
- 0 kudos
- 8600 Views
- 9 replies
- 0 kudos
Performance issue while calling mlflow endpoint
Hi,I have pyspark dataframe and pyspark udf which calls mlflow model for each row but its performance is too slow.Here is sample codedef myfunc(input_text): restult = mlflowmodel.predict(input_text) return resultmyfuncUDF = udf(myfunc,StringType(...
- 8600 Views
- 9 replies
- 0 kudos
Latest Reply
- 2960 Views
- 1 replies
- 0 kudos
Resolved! Understanding Spark Architecture during Table Creation
Team ,I am trying understand how the parquet files and JSON under the delta log folder stores the data behind the scenesTable Creation:from delta.tables import *DeltaTable.create(spark) \.tableName("employee") \.addColumn("id", "INT") \.addColumn("na...
- 2960 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
@Ramakrishnan83 - Kindly go through the blog post - https://www.databricks.com/blog/2019/08/21/diving-into-delta-lake-unpacking-the-transaction-log.html which discuss in detail on delta's transaction log.
- 0 kudos
- 3392 Views
- 0 replies
- 0 kudos
How to use external locations
Hi,I am struggling with truly understanding how to work with external locations. As far as I am able to read, you have:1) Managed catalogs2) Managed schemas3) Managed tables/volumes etc.4) External locations that contains external tables and/or volum...
- 3392 Views
- 0 replies
- 0 kudos
- 8456 Views
- 1 replies
- 0 kudos
Databricks and streamlit and fast API combination
hello friends ! i have project where i need databricks to train eval model then put it to productioni trained model & eval in databricks i used mlflow everything is good now i have another two steps that i have zeroclue how they should be done : usag...
- 8456 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
This repo has examples that you can use in your Databricks workspace for FastAPI and Streamlit. I recommend only using these for development or lightweight use cases.
- 0 kudos
- 3437 Views
- 0 replies
- 0 kudos
can not set permission in table
In databricks database table I was able to set permissions to groups but Now I get this error when using a cluster:Error getting permissionssummary: SparkException: Trying to perform permission action on Hive Metastore /CATALOG/`hive_metastore`/DATAB...
- 3437 Views
- 0 replies
- 0 kudos
- 3643 Views
- 4 replies
- 3 kudos
BAD_REQUEST: ExperimentIds cannot be empty when checking ACLs in bulk
I was going through this tutorial https://mlflow.org/docs/latest/getting-started/tracking-server-overview/index.html#method-2-start-your-own-mlflow-server, I ran the whole script and when I try to open the experiment on the databricks website I get t...
- 3643 Views
- 4 replies
- 3 kudos
Latest Reply
