- 5162 Views
- 1 replies
- 1 kudos
- 5162 Views
- 1 replies
- 1 kudos
- 1 kudos
Please use Repos and in admin settings please enable "Files in Repo" than you will be able to import class in notebook:from repo_folder.subfolders.file import your_class
- 1 kudos
- 6380 Views
- 2 replies
- 2 kudos
need to move notebook file from workspace to dbfs.
Hi teami need to move the notebook file from workspace to dbfs. i have tried and getting an error there is no file ?
- 6380 Views
- 2 replies
- 2 kudos
- 2 kudos
Not sure what command are you using and what is error? As access to workspace is managed per user on databricks so %sh magic commands will not work as you could see other user files.Dbfs is data storage. Notebook code from Workspace can be moved to R...
- 2 kudos
- 2012 Views
- 1 replies
- 1 kudos
which table in external hive metastore stores the folder path for delta table's data?
which table in external hive metastore stores the folder path for delta table's data? Is it SDS table?
- 2012 Views
- 1 replies
- 1 kudos
- 1 kudos
Image here can be helpful https://analyticsanvil.files.wordpress.com/2016/08/hive_metastore_database_diagram.png
- 1 kudos
- 7841 Views
- 6 replies
- 2 kudos
Resolved! SQL endpoint is unable to connect to external hive metastore ( Azure databricks)
Using Azure databricks, I have set up SQL Endpoint with the connection details that match with global init script. I am able to browse tables from regular cluster in Data Engineering module but i get below error when trying a query using SQL Endpoint...
- 7841 Views
- 6 replies
- 2 kudos
- 2 kudos
@Prabakar Ammeappin​ @Kaniz Fatma​ Also I found out that after delta table is created in external metastore (and the table data resides in ADLS) then in the sql end point settings I do not need to provide ADLS connection details. I only provided...
- 2 kudos
- 1131 Views
- 0 replies
- 0 kudos
Custom real estate printing service for agents, realtors and brokers. Agent Print offers fast production, unmatchable quality and the best prices onli...
Custom real estate printing service for agents, realtors and brokers. Agent Print offers fast production, unmatchable quality and the best prices online. Shop Now.
- 1131 Views
- 0 replies
- 0 kudos
- 8011 Views
- 11 replies
- 2 kudos
Resolved! Databricks SQL Connector
I started to explore Databricks SQL Connector. And I'm wondering can I execute query from Databricks SQL Connector from notebook or job?sample code:'test_query' lives in Databricks SQL. It is ready to use query!from databricks import sql hostname =...
- 8011 Views
- 11 replies
- 2 kudos
- 2 kudos
@Borislav Blagoev​ - There is a "Select as Best" link underneath each reply. All you need to do, if you choose, is to click on that link for whichever reply best answered your question. The purpose is to help other members find helpful solutions more...
- 2 kudos
- 1161 Views
- 0 replies
- 2 kudos
- 1161 Views
- 0 replies
- 2 kudos
- 3940 Views
- 3 replies
- 2 kudos
Do Databricks support XLA compilation for TensorFlow models?
I am defining a sequential Keras model using tensorflow.kerasRuntime: Databricks ML 8.3Cluster: Standard NC24 with 4 GPUs per node.To enable XLA compilation, I set the following flag:tf.config.optimizer.set_jit(True)Here is the output when I try to t...
- 3940 Views
- 3 replies
- 2 kudos
- 2 kudos
I don't think this is specific to Databricks, but rather Tensorflow. See https://stackoverflow.com/questions/68614547/tensorflow-libdevice-not-found-why-is-it-not-found-in-the-searched-path for a possibly relevant solution.I don't see evidence that t...
- 2 kudos
- 8918 Views
- 5 replies
- 2 kudos
Resolved! Hello Experts - I am facing one technical issue with Databricks SQL - IF-ELSE or CASE statement implementation when trying to execute two separate set of queries based on a valued of a column of the Delta table.
Hi Experts,I'm reading a pipe delimited source file where first row does not contain the data but contains the - REPLACE or UPDATE values which indicate the refresh type if it is Full refresh Or Upsert. The second row is nothing but header and the a...
- 8918 Views
- 5 replies
- 2 kudos
- 2 kudos
Thanks a ton Gurpreet, Suggested approach was helpful in resolution of the issue. Appreciate your help !!
- 2 kudos
- 11416 Views
- 2 replies
- 1 kudos
Resolved! Notebooks font size
Hi,Is there a way to increase the cells' font size and set it as a default setting ?Thanks.Best Damien
- 11416 Views
- 2 replies
- 1 kudos
- 8357 Views
- 5 replies
- 1 kudos
Hive Serde table vs Delta table
This might be stupid question. Does the Hive Serde table have the same features (e.g. transactions) comparing to the Delta table?I tried to find the information in the Databricks documentation but I cannot find a clear answer.I create the Hive Serde ...
- 8357 Views
- 5 replies
- 1 kudos
- 1 kudos
AFAIK Hive SerDe is just Serializer and Deserializer (write and read data to/from storage).Hive uses SerDe (and FileFormat) to read and write table rows. So it is not an actual file format like parquet, orc and also delta lake (which I consider a se...
- 1 kudos
- 14085 Views
- 3 replies
- 4 kudos
Resolved! How do you use cloud fetch?
We're trying to pull a big amount of data using databricks sql and seem to have a bottleneck on network throughput when fetching the data.I see there's a new feature called cloud fetch and this seems to be the perfect solution for our issue. But I do...
- 14085 Views
- 3 replies
- 4 kudos
- 4 kudos
Trying to get an idea of what you are trying:so you query directly on a database of +100GB or is it parquet/delta source?Also, where is the result fetched to? File download, BI tool, ...?
- 4 kudos
- 2602 Views
- 1 replies
- 2 kudos
Do we have delta table access logs ?
I have delta tables on databricks with AWS s3.Are there any logs or anything else to figure out who all are accessing a particular DB or tables.
- 2602 Views
- 1 replies
- 2 kudos
- 2 kudos
The thing that comes closest are Audit logs. Here is a list of log triggers.
- 2 kudos
- 13939 Views
- 17 replies
- 7 kudos
Resolved! unzip twice the same file not executing
Hi, I need to unzip some files that are ingested but when I unzip twice the same zipped file, the unzip command does not execute :As suggesgted in the documentation I did :import urllib urllib.request.urlretrieve("https://resources.lendingclub.com/L...
- 13939 Views
- 17 replies
- 7 kudos
- 7 kudos
Another problem is that dbfs storage doesn't support random writes (used by zip):Does not support random writes. For workloads that require random writes, perform the operations on local disk first and then copy the result to/dbfssource: https://docs...
- 7 kudos
- 5780 Views
- 0 replies
- 19 kudos
www.lunasec.io
Log4J: part 2Apparently log4j 2.15 is still vulnerable:https://www.lunasec.io/docs/blog/log4j-zero-day-update-on-cve-2021-45046/So beter use version 2.16.But as mentioned in several topics: Databricks does not use an impacted version.
- 5780 Views
- 0 replies
- 19 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 566 | |
| 349 | |
| 287 |
