- 4327 Views
- 1 replies
- 1 kudos
Resolved! TIMESTAMP(NANOS,false) error
Hi,I'm getting Illegal Parquet type: INT64 (TIMESTAMP(NANOS,false)) error while trying to read a parquet file (generated outside of DataBricks). Unfortunately, due to security configuration, I do not have the ability to read it with pandas or similar...
- 4327 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @staskh That error happens because data type is not supported natively. You can try with below spark setting: spark.conf.set(“spark.sql.legacy.parquet.nanosAsLong”, “true”)
- 1 kudos
- 15739 Views
- 11 replies
- 6 kudos
Resolved! Bootstrap Timeout during cluster start
sometimes while starting a cluster I am facing bootstrap timeout error, what is the reason? when I try the next time it starts the cluster.
- 15739 Views
- 11 replies
- 6 kudos
- 6 kudos
i have this issue, can someone help on that ?Instance bootstrap failed command: Bootstrap_e2e Instance bootstrap inferred timeout reason: Command_UpdateWorker_Slow Failure message (may be truncated): Bootstrap is terminated spontaneously by VM becaus...
- 6 kudos
- 3829 Views
- 1 replies
- 0 kudos
What is the best approach to use Delta tables without Unity Catalog enabled?
Hello!I would like to work with delta tables outside of Databricks UI notebook. I know that the best option would be to use databricks-connect but I don’t have Unity Catalog enabled.What would be the most effective way to do so? I know that via JDBC ...
- 3829 Views
- 1 replies
- 0 kudos
- 0 kudos
Programatically, you can go for DeltaTables.forPath ( not forName which require Unity Catalog )... This works
- 0 kudos
- 1535 Views
- 3 replies
- 1 kudos
Transition from partitioned table to Liquid clustered table
Hi all,I have a table called classes, which is already partitioned on three different columns. I want to create a Liquid Clustered Table, but as far as I understand from the documentation—and from Dany Lee and his team—it was not possible as of 2024 ...
- 1535 Views
- 3 replies
- 1 kudos
- 1 kudos
I'm sorry, I missed that your tables are partitioned, in this case easiest way will be to recreate it.
- 1 kudos
- 1418 Views
- 1 replies
- 0 kudos
databricks
Hi Team,Having excel file in sharepoint folder, and has to insert excel data into SQL table from databricks notebook . can i have clear steps on it. Dont have access to Azure Active Directory. can anyone gives solution without using AZURE Active Dir...
- 1418 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi KSBYou could read direkt with Databricks from Excel Sharepoint with Graph API. Here is one possible way: https://community.databricks.com/t5/data-engineering/load-data-from-sharepoint-site-to-delta-table-in-databricks/td-p/16410However, you need t...
- 0 kudos
- 1881 Views
- 1 replies
- 0 kudos
When Sign in databricks partner-academy i can not see the courses
Dear partner academy team, I am writing to report an issue I am experiencing when trying to access the partner academy courses. Despite using my credentials, I am unable to view any of the courses. Could you please look into this and assist me in res...
- 1881 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @Sergio_Linares! Please file a ticket with the Databricks support team to get assistance with this issue.
- 0 kudos
- 690 Views
- 2 replies
- 0 kudos
DAB for_each_task python wheel fail
using python_wheel_wrapper experimental true allows me to use python_wheel_task on an older cluster.However, if I embed the python_wheel_task in a for_each_task it fails at runtime with: "Library installation failed for library due to user error. Er...
- 690 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @BillBishop, I will check on this internally as outcome does not seem to be correct. If possible, upgrade your cluster to DBR 14.1 or later. This would resolve the issue without relying on the experimental feature
- 0 kudos
- 4486 Views
- 5 replies
- 0 kudos
sparkContext in Runtime 15.3
Hello All, Our Azure databricks cluster is running under "Legacy Shared Compute" policy with 15.3 runtime. One of the python notebooks is used to connect to an Azure SQL database to read/insert data. The following snippet of code is responsible for r...
- 4486 Views
- 5 replies
- 0 kudos
- 0 kudos
@rushi29 @GangsterI ended up implementing pyodbc with the mssql driver using init scripts.Spark context is no longer usable in shared compute so that was the only approach we could take.
- 0 kudos
- 4160 Views
- 7 replies
- 1 kudos
Resolved! databricks workspace import_dir not working without any failure message
Morning everyone!I`m trying to copy from the repo into the databricks workspace all the notebooks. I`m using the command: databricks workspace import_dir . /Shared/Notebooks, it will just print all the info regarding the Workspace API.If I launch dat...
- 4160 Views
- 7 replies
- 1 kudos
- 1 kudos
Hi all,I`ve disinstalled and installed again databricks-cli and now worked.Is not a real solution but still it worked after one week...
- 1 kudos
- 627 Views
- 1 replies
- 0 kudos
Autoloader is not able to infer schema from json
Hi, I have json files and it contains json array and only one object (payload below), I have set in autoloader inferSchema to true, however autoloader throws:"Failed to infer schema for format json from existing files ..."I have also check option to ...
- 627 Views
- 1 replies
- 0 kudos
- 0 kudos
Yep, my mistake, json file was corrupted.
- 0 kudos
- 1111 Views
- 2 replies
- 0 kudos
External connection to Azure ADLS Gen2 storage
Hello, I have a problem trying to make an external connection to a blob storage configured as ADLS Gen2 with hierarchical namespace (HNS) enabled. I have setuup the storage account with a container wirh HNS enabled as in the image attached:Next I hav...
- 1111 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi, would it work if you try to remove "landing" at the end? Seems the error is complaining that the external location should be a directory.
- 0 kudos
- 1285 Views
- 1 replies
- 0 kudos
Resolved! Hostname not resolving using Spark JDBC
Hey guys,Ive ran into a weird error this morning. Last week I was testing a new Oracle Connector and it ran through smooth the whole last week!This morning at 7 it ran again and following it was showing a "SQLRecoverableException: IO Error: Unknown h...
- 1285 Views
- 1 replies
- 0 kudos
- 0 kudos
I have even cloned the cluster and it worked on the new one. But after turning the cluster off over night, it started working again the next morning. This is really weird.
- 0 kudos
- 985 Views
- 4 replies
- 0 kudos
Does DBR 14.3 not support Describe history command?
Hello, We have recently updated DBR version from 9.1 LTS to 14.3 LTS and observed that DESCRIBE HISTORY is not supported in 14.3 LTS. Could you please suggest any alternative to be used for table history?
- 985 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi, I'm still not able to recreate this issue with Standard_DS3_v2. I'm not sure if this is relevant, but do you also have this issue on an old High Concurrency cluster with custom access mode for the Standard_DS3_v2 cluster too?
- 0 kudos
- 572 Views
- 1 replies
- 1 kudos
External Location Naming Issue & Impact of Renaming in Unity Catalog
Hey,I created an external location in my test environment using a mount point . Now, when I try to create the same external location in prod, it doesn’t allow me to use the same name. Is there any specific reason for this restriction in Unity Catalog...
- 572 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello @Faizan_khan8171, Thanks for your question. In Unity Catalog, external location names must be unique within the metastore. This restriction prevents naming conflicts and ensures that every external location is easily identifiable and manageable...
- 1 kudos
- 2386 Views
- 6 replies
- 0 kudos
Move Databricks service to another resource group
Hello,Is it possible to move in another resource group the databricks service without any problem?I have a resource group where there are two workspaces the prod and staging environment, I created another resource group to maintain only the databrick...
- 2386 Views
- 6 replies
- 0 kudos
- 0 kudos
I'm running into the same problem, what's the procedure to create a feature request for this? It seems to me that when DB is running in Azure that I should be able to move it to a different resource group.
- 0 kudos
| User | Count |
|---|---|
| 1629 | |
| 790 | |
| 511 | |
| 349 | |
| 287 |
