- 8527 Views
- 2 replies
- 0 kudos
Resolved! How to use dateadd in databricks sql ?
I am trying to something like this but getting error like :Error in SQL statement: AnalysisException: Undefined function: 'DATEADD'. This function is neither a registered temporary function nor a permanent function registered in the database 'default...
- 8527 Views
- 2 replies
- 0 kudos
- 0 kudos
Dateadd was added in DBR 10.4 and is in DBSQL current.SELECT DATEADD(HOUR,IFNULL(100, 0),current_date) AS Date_Created_Local=> 2022-05-31T04:00:00.000+0000.You can also use one of these casts to turn any wellformed string into an interval:SELECT curr...
- 0 kudos
- 1060 Views
- 0 replies
- 0 kudos
Pattern Matching not working as expected with 10.4 LTS databricks runtime version
Hi everyone, I recently upgraded the runtime version of one of the databricks job to 10.4 LTS but Pattern Matching is not working as expected the same code is working in 7.3 LTS.Basically doing this and returning Left or Right: val result = spark.sql...
- 1060 Views
- 0 replies
- 0 kudos
- 8125 Views
- 0 replies
- 0 kudos
How to install wheel package from git repo
Using VS code for development and a wheel package is created for shipment.We put this wheel package in Azure data lake storage and ADB notebook accessed this wheel package and installed it in the cluster. It is working fine. But instead of keeping th...
- 8125 Views
- 0 replies
- 0 kudos
- 1484 Views
- 0 replies
- 0 kudos
Issue in creating workspace - Custom AWS Configuration
Hello,We have tried to created new Databricks Workspace using "Custom AWS Configuration" and we have given our own VPC (Customer managed VPC) and tried but workspace failed to launch. There are Documents Available for resolve this error , According t...
- 1484 Views
- 0 replies
- 0 kudos
- 1521 Views
- 1 replies
- 0 kudos
How to run sparkStream for earlier (not future messages)
Hi, I'm listening to a stream for kinesis, don't need the data in real-time, so I could run it on an hourly basis looking to achieve two things:-Save money by don't have a cluster up 24/7-Have bigger files saved for each readThe stream is constant so...
- 1521 Views
- 1 replies
- 0 kudos
- 4735 Views
- 3 replies
- 0 kudos
New Databricks Driver gives SQLNonTransientConnectionException when trying to connect to Databricks Instance
import com.databricks.client.jdbc.DataSource; import java.sql.*; public class testDatabricks { public static void main(String[] args) throws SQLException { String dbUrl = "jdbc:databricks://<hostname>:443;HttpPath=<HttpPath>;"; // Cop...
- 4735 Views
- 3 replies
- 0 kudos
- 0 kudos
This looks like due to maintenance on US . Are you still facing the issue @Sriramkumar Thamizharasan​ Is your workspace on eastus and eastus2 ?
- 0 kudos
- 8014 Views
- 4 replies
- 3 kudos
Resolved! Delta Live Tables with EventHub
Hello,I would like to integrate Databricks Delta Live Tables with Eventhub, but i cannot install com.microsoft.azure:azure-eventhubs-spark_2.12:2.3.17 on delta live cluster.I tried installed in using Init script (by adding it in Json cluster settings...
- 8014 Views
- 4 replies
- 3 kudos
- 3 kudos
I think this has some details https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-kafka-spark-tutorial @Kacper Mucha​ is the issue resolved ?
- 3 kudos
- 4793 Views
- 5 replies
- 6 kudos
Resolved! Start Up Notebook in Databricks
Hi Team,Is it possible to have a startup notebook in databricks similar to init sql script ( This need to run on start of every cluster )
- 4793 Views
- 5 replies
- 6 kudos
- 4153 Views
- 0 replies
- 0 kudos
Spark Failure Error Unable to download spark docker Image
Cluster terminated. Reason: Spark Image Download Failure "reason": { "code": "SPARK_IMAGE_DOWNLOAD_FAILURE", "type": "SERVICE_FAULT", "parameters": { "instance_id": "6565aa39b0ae4fe69c7fe6f313e3ca2a", "databricks_error_message": "Failed to set up th...
- 4153 Views
- 0 replies
- 0 kudos
- 2377 Views
- 1 replies
- 2 kudos
Resolved! Is Databricks Light Runtime Discontinued?
The last Databricks Light runtime release was 2.4 Extended Support. There was no Light version for Spark 3.x. Is Databricks Light runtime discontinued? If not, when we can expect the next DBR Light version?
- 2377 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @Venkadeshwaran K​, I looked around, and it does look like there won't be future light runtimes. We can't hire enough engineers to maintain and develop everything, and light is one of the casualties of that.
- 2 kudos
- 892 Views
- 0 replies
- 0 kudos
Startup Names is the place to find best brand names for sale completed with logo. Use our free business name generator for industry wise creative bran...
Startup Names is the place to find best brand names for sale completed with logo. Use our free business name generator for industry wise creative brand names.
- 892 Views
- 0 replies
- 0 kudos
- 10349 Views
- 5 replies
- 6 kudos
Resolved! connection to databricks data source in Visual Studio fails with SSL error
Hi,I'm new to databricks but am positively surprised by the product. We use databricks delta tables as source to build a tabular model, which will serve as data source for Power Bi. To develop our tabular model we use Visual studio to import tables ...
- 10349 Views
- 5 replies
- 6 kudos
- 6 kudos
In Visual Studio do you use databricks-connect? Maybe it is enough to rerun "databricks-connect configure"
- 6 kudos
- 2606 Views
- 3 replies
- 3 kudos
AWS + Cluster with custom docker image = export metrics to CloudWatch?
Is there a working setup on setting up metrics export to CloudWatch while using custom docker images for cluster creation? I've tried to set up the CloudWatch agent manually, but launching `amzon-cloudwatch-agent-ctl` in the bootstrap script fails wi...
- 2606 Views
- 3 replies
- 3 kudos
- 3 kudos
We do not support gangila with custom docker too. but let me cross verify if we are supporting cloudwatch for the same. Sorry for the inconvenience @Sergey Ivanychev​
- 3 kudos
- 3983 Views
- 1 replies
- 2 kudos
Arrow R package fails to install
# Databricks notebook source .libPaths() # COMMAND ---------- dir("/databricks/spark/R/lib") # COMMAND ---------- ## Add current working directory to library paths .libPaths(c(getwd(), .libPaths())) # COMMAND ---------- ## The latest vers...
- 3983 Views
- 1 replies
- 2 kudos
- 2 kudos
@Anthony McGrath​ can you please download and upload to DBFS and see if the issue still persists?You can check if any global initscript is reinstalling this to your cluster.
- 2 kudos
- 4115 Views
- 2 replies
- 2 kudos
Notebook metadata
HelloI would like to view metadata about the notebooks in the Workspace folder hierarchy, for example date created, modified, by user, etc.Is this possible?
- 4115 Views
- 2 replies
- 2 kudos
- 2 kudos
@Martin Aronsson​, you need to follow the notebook revision history (https://docs.databricks.com/notebooks/notebooks-use.html#revision-history) I believe. Also you can try https://docs.microsoft.com/en-us/azure/databricks/administration-guide/account...
- 2 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 553 | |
| 349 | |
| 287 |
