- 5386 Views
- 4 replies
- 1 kudos
Resolved! How to connect via JDBC to SAP-HANA in a Databricks Notebook?
I have a set of connection credentials for SAP-HANA, how can I retrieve data from that location using JDBC?I have already installed in my cluster the ngdbc.jar (for the driver), but this simple Query has already taken more than 5 minutes and I don't ...
- 5386 Views
- 4 replies
- 1 kudos
- 1 kudos
It worked changing the port to: 30041, the port for the next tenant (reference: https://community.sap.com/t5/technology-q-a/hana-connectivity-and-ports/qaq-p/12193927 ).jdbcQuery = '(SELECT * FROM DUMMY)' df_sap_hana_dummy_table = (spark.read .form...
- 1 kudos
- 520 Views
- 1 replies
- 0 kudos
Dynamic scheduling again and again
Hi Community,Is it possible to dynamic scheduling a databricks job definition as is possible to do it on Airflow Dags? If not, which could be a way to handle it?
- 520 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @jeremy98, Databricks does not natively support dynamic scheduling of job definitions in the same way that Apache Airflow does with its Directed Acyclic Graphs (DAGs). However, there are ways to achieve similar functionality using Databricks Jobs:...
- 0 kudos
- 1853 Views
- 6 replies
- 0 kudos
Resolved! Connection from BigQuery to Databricks populating dictionary keys as "v"
I was able to connect our Bigquery account to our Databricks catalog. However, all the keys in the nested dictionary columsn populate as 'v'. For example:{"v":[{"v":{"f":[{"v":"engagement_time_msec"},{"v":{"f":[{"v":null},{"v":"2"},{"v":null},{"v":nu...
- 1853 Views
- 6 replies
- 0 kudos
- 0 kudos
@szymon_dybczak I couldn't run select TO_JSON_STRING(event_params) as event_params FROM ...I don't think thats a built-in Databricks. Is there another way you've had success?error:[UNRESOLVED_ROUTINE] Cannot resolve routine `TO_JSON_STRING` on search...
- 0 kudos
- 1128 Views
- 2 replies
- 0 kudos
How to connect to Tableau Server FROM within Databricks Notebooks and publish data to Tableau Serv?
My company is having trouble connecting Databricks to Tableau Server. We need to be able to publish Hyper Files that are developed using Python on Databricks Notebooks to our Tableau Server, but it seems impossible to get a connection established des...
- 1128 Views
- 2 replies
- 0 kudos
- 948 Views
- 1 replies
- 0 kudos
Resolved! Is there a INTERVAL data type?
Hi community,I was using a column in postgresSQL that is a DATETIME.TIMEDELTA, is it possible to have the same data type also in Databricks?
- 948 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @jeremy98, You can use the TIMESTAMP and TIMESTAMP_NTZ data types to handle date and time values, similar to the DATETIME type in PostgreSQL. However, Databricks does not have a direct equivalent to PostgreSQL's TIMEDELTA type https://docs.databri...
- 0 kudos
- 2248 Views
- 1 replies
- 0 kudos
How to resolve SSL_connect error when VPN is enabled
I am trying to connect to Databricks using ODBC protocol with Simba Driver DSN. I am able to successfully connect and access the data when our corporate VPN is turned OFF. but when it's turned ON , I am getting the following error "[Simba][ThriftExte...
- 2248 Views
- 1 replies
- 0 kudos
- 0 kudos
The error you are encountering, " (14) Unexpected response from server during a HTTP connection: SSL_connect:", when trying to connect to Databricks using the ODBC protocol with the Simba Driver DSN while the corporate VPN is turned on, is likely rel...
- 0 kudos
- 1777 Views
- 2 replies
- 1 kudos
Liquid Cluster enabled table - concurrent writes
I am trying to insert rows into a Liquid cluster enabled delta table using multiple threads. This link, states that liquid clustering is used for : Tables with concurrent write requirements.I get this error: [DELTA_CONCURRENT_APPEND] ConcurrentAppend...
- 1777 Views
- 2 replies
- 1 kudos
- 1 kudos
We encountered a similar issue as well, and the workaround we tried was partitioning those columns, as Liquid clustering can sometimes trigger this error.
- 1 kudos
- 1312 Views
- 2 replies
- 0 kudos
Databricks inconsistent count and select
Hi, I have a table with 2 versions:1. Add txn: path = "a.parquet" numRecords = 10 deletionVector = null2. Add txn: path = "a.parquet" numRecords = 10 deletionVector = (..., cardinality = 2)Please note both transactions point to the same physical path...
- 1312 Views
- 2 replies
- 0 kudos
- 0 kudos
Hello the behavior observed indeed seems to be inconsistent with the expected behavior in delta, do you have a support contract to open a support ticket so this can be further analyzed?
- 0 kudos
- 3009 Views
- 1 replies
- 0 kudos
Avoid scientific values
I am trying to insert data into catalog tables on data bricks but the values are being displayed in scientific notation which i am trying to avoid how do I view the data in standard formatfor example - 0.0000000 is being displayed as 0E-7
- 3009 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Tej_04, Can you try with format_number Function. SELECT format_number(column_name, decimal_places) AS column_name FROM table_name; https://docs.databricks.com/en/sql/language-manual/functions/format_number.html
- 0 kudos
- 18478 Views
- 15 replies
- 12 kudos
Resolved! Error when create an external location using code
I'm trying to create an external location from notebook, and I got this kind of error[PARSE_SYNTAX_ERROR] Syntax error at or near 'LOCATION'(line 1, pos 16) == SQL == CREATE EXTERNAL LOCATION IF NOT EXISTS test_location URL 's3://test-bronze/db/tes...
- 18478 Views
- 15 replies
- 12 kudos
- 12 kudos
Hey everyone,I'm facing an issue with retrieving data from a volume or table that contains a string with a symbol, for example, 'databricks+'. Whenever I try to retrieve this data, I encounter a syntax error. Can anyone help me resolve this issue?
- 12 kudos
- 3306 Views
- 8 replies
- 5 kudos
Resolved! DLT Pipleline with only views
I'm trying to create a pipeline containing a view from a federated source. In this case, I'd like to just create materialized views from the federation and and schedule the pipeline for execution. If I define a pipeline with only something like the ...
- 3306 Views
- 8 replies
- 5 kudos
- 3595 Views
- 7 replies
- 0 kudos
How to enable row tracking on Delta Live Tables?
We are encountering a scenario where we need to enable support for Incremental Processing on Materialized views having DLT base tables. However, we have observed that the compute is being executed with the COMPLETE_RECOMPUTE mode instead of INCREMENT...
- 3595 Views
- 7 replies
- 0 kudos
- 0 kudos
Moreover, we have CDF enabled DLT tables, but as per documentation we see a limitation if CDF is enabled then row Tracking won't be possible. Use row tracking for Delta tables | Databricks on AWSBut as per our use case we need incremental processing ...
- 0 kudos
- 2106 Views
- 1 replies
- 1 kudos
Create table from json and flatten in the same SQL
Any help in writing SQL to create the table using JSON file and flatten in the same step? As I'm new to JSON it would be nice if someone can give the heads up by referencing to any document or help provide the recommended the solution.Sample JSON fil...
- 2106 Views
- 1 replies
- 1 kudos
- 1 kudos
To create a table directly from a JSON file and flatten it using SQL in Databricks, you can use the CREATE TABLE statement with the USING JSON clause. However, SQL alone does not provide a direct way to flatten nested JSON structures. You would typic...
- 1 kudos
- 1482 Views
- 3 replies
- 1 kudos
What does durationMs.commitBatch measure?
With a structured streamin job from Kafka, we have a metric in durationMs called commitBatch. There is also an example of this in this databricks documentation. I can not find any description of what this measures, and how it relates to the other met...
- 1482 Views
- 3 replies
- 1 kudos
- 1 kudos
The commitBatch metric is a part of the overall triggerExecution time, which encompasses all stages of planning and executing the microbatch, including committing the batch data and updating offsets. The commitBatch metric may not always be present i...
- 1 kudos
- 8770 Views
- 5 replies
- 3 kudos
Resolved! Why I am not receiving any mail sent to the Azure AD Group mailbox when databricks job fails ?
I have created an Azure AD Group in "Microsoft 365" type with its own email address, which being added to the Notification of a Databricks Job (on failure). But there is no mail sent to the Azure Group mailbox when the job fails.I am able to send a d...
- 8770 Views
- 5 replies
- 3 kudos
- 3 kudos
Hello Guys, I have setup ses receive email for databricks notification. When i send email message from google mail or yahoo mail, it gets to the SES email receiving rule. However, notification from databricks doesn't get to the same SES email receivi...
- 3 kudos
| User | Count |
|---|---|
| 1644 | |
| 791 | |
| 553 | |
| 349 | |
| 287 |
