- 2965 Views
- 2 replies
- 3 kudos
SQL Notebook Tab Spacing
My SQL notebooks in databricks suffer from having at least several different counts of spaces between tab marks. It makes it very difficult to maintain pretty code spacing. What sets the tab spacing in SQL language notebooks, and how is it set/adju...
- 2965 Views
- 2 replies
- 3 kudos
- 3 kudos
I’m experiencing the same issue with SQL cell indentation in Databricks notebooks. While editing, I’ve noticed that the indentation level can vary from one cell to another - sometimes it’s two spaces, other times it’s four. This inconsistency makes i...
- 3 kudos
- 9184 Views
- 22 replies
- 26 kudos
Resolved! Run selected text shortcut not working
The keyboard shortcut to run selected text (ctrl + shift + enter) has not been working for me since yesterday (10/31/24). Instead of running the selected text, databricks notebooks are treating it like shift + enter and running the entire cell. I hav...
- 9184 Views
- 22 replies
- 26 kudos
- 26 kudos
Folks, could you please double-check now, issue should be fixed now. Thanks!
- 26 kudos
- 1700 Views
- 2 replies
- 1 kudos
Resolved! Comparing two SQL notebooks from different Environments
Hello Everyone,I am part of data testing team which is responsible to verify data trends and insights generated from different sources. There are multiple schemas and tables in our platform. We use SQL queries in notebooks to verify all enrichment, m...
- 1700 Views
- 2 replies
- 1 kudos
- 1 kudos
Thank you Walter. I did thought about it doing it one by one but then it was not coming out to be very efficient way. I then found a way to do it in Python via iterating through a dataframe of table names.
- 1 kudos
- 22407 Views
- 7 replies
- 6 kudos
'File not found' error when executing %run magic command
I'm just walking through a simple exercise presented in the Databricks Platform Lab notebook, in which I'm executing a remote notebook from within using the %run command. The remote notebook resides in the same directory as the Platform Lab notebook,...
- 22407 Views
- 7 replies
- 6 kudos
- 6 kudos
I have seen this error popping up when you define a Python file without the header: # Databricks notebook source Databricks has a hard time running the file as a nested notebook and you can get some weird errors.
- 6 kudos
- 1228 Views
- 3 replies
- 0 kudos
What is "ExecuteGrpcResponseSender: Deadline reached, shutting down stream"
Hi, I have a delta table which is loaded by structured streaming job. When I tried to read this delta table and do a MERGE with foreachBatch, I found sometimes there is a big interval between streaming starts and MERGE starting to run and seems spar...
- 1228 Views
- 3 replies
- 0 kudos
- 0 kudos
It may not necessarily be a bug, but some tuning due to architectural differences. What the message says is: The system was processing a gRPC operation identified by opId=5ef071b7-xxx, and it set a deadline for that operation (likely 120 seconds).The...
- 0 kudos
- 3307 Views
- 1 replies
- 1 kudos
Creating a private connection with Google Sheets
How do I ingest sensitive data from a googlesheet doc to Databricks unity catalogue without making the googlesheet public.
- 3307 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @dtb_usr,You can share the Google Sheet with the Service Account and use Google Sheets API ClientOpen the Google Sheet you want to access.Click on Share and add the email address of the service account (it will look something like your-service-acc...
- 1 kudos
- 869 Views
- 1 replies
- 0 kudos
Databricks cluster pools with init scripts
Ability to submit a single job with cluster pools and init scriptsfor the following payload: { "run_name": "A multitask job run", "timeout_seconds": 86400, "tasks": [ { "task_key": "task_1", "depends_on": ...
- 869 Views
- 1 replies
- 0 kudos
- 0 kudos
Are you still facing issues with the job run submit API endpoint?
- 0 kudos
- 2434 Views
- 2 replies
- 1 kudos
Resolved! streaming vs batch, unbounded vs bound
Any one can help me understand why cannot we unify streaming with batch and unbounded with bounded, if we regard the streaming/unbounded as mini-version of batch/bounded, please?i.e., if I set one second as the frequency for batch processing, will it...
- 2434 Views
- 2 replies
- 1 kudos
- 1081 Views
- 2 replies
- 1 kudos
Resolved! Using instance profile for sns message publish with PassRole
Hi, I want to attach instance profile to compute and publish message on SNS without using credentials. Is that possible? has anyone used it. Will Boto3 support it?
- 1081 Views
- 2 replies
- 1 kudos
- 1137 Views
- 1 replies
- 0 kudos
Simba ODBC for ARM-based Linux
HiIs there an ARM build of the Simba ODBC driver available for Linux? I've seen this thread (for Mac)https://community.databricks.com/t5/data-engineering/problems-connecting-simba-odbc-with-a-m1-macbook-pro/td-p/20566but it seems that there are only ...
- 1137 Views
- 1 replies
- 0 kudos
- 0 kudos
@stiaangerber unfortunately not. The odbc driver for linux is technically CPU agnostic, as long as it's amd64/x86_64, it should work, but we don't have one for ARM.
- 0 kudos
- 2370 Views
- 3 replies
- 0 kudos
QuantileDiscretizer is not whiteliested erro!!
Dear Team,We observed that while attempting to use the following import:from pyspark.sql import functions as Ffrom pyspark.ml.feature import QuantileDiscretizerwe are encountering the following error:Py4JSecurityException: QuantileDiscretizer is not ...
- 2370 Views
- 3 replies
- 0 kudos
- 0 kudos
@sandeephenkel23 I've run the same code in DBR 13.3 LTS, and 1) It is successfully imported, 2) I can confirm it is in the whitelisted libs. Hence I'm wondering if there's anything else particular in your use case, triggering this? Is your use case a...
- 0 kudos
- 2218 Views
- 5 replies
- 0 kudos
Schema evolution in Autoloader not evolving beyond version 0
I am working through the current version of the standard AutoLoader demo, i.e. dbdemos.install('auto-loader')I.e. data gets read into a dataframe, but never written to a target table.Notebook is "01-Auto-loader-schema-evolution-Ingestion"Compute i...
- 2218 Views
- 5 replies
- 0 kudos
- 0 kudos
Hello @Brahmareddy,I have tried the above, without success.> enable detailed logging to trace schema evolution stepsPlease can you giude me with the steps or a URL? We are on AWS.Kind regards - Roelof
- 0 kudos
- 2155 Views
- 1 replies
- 1 kudos
Enhanced Cost Management for Serverless Compute
Budget policies include tags that are applied to serverless compute activities incurred by users assigned. These tags are recorded in your billing records, allowing you to allocate specific serverless usage to designated budgets. For more information...
- 2155 Views
- 1 replies
- 1 kudos
- 1 kudos
Thanks for sharing.
- 1 kudos
- 1661 Views
- 1 replies
- 0 kudos
Passing values
Hello how I can pass a value of a parameter between two jobs that are not nested and have separated terraform and notebooks
- 1661 Views
- 1 replies
- 0 kudos
- 0 kudos
Currently, there is no dedicated parameter store or variables section in Databricks for easily sharing values between jobs. One approach is to save the parameter in the Hive metastore or an S3 bucket, then retrieve it in each job. Alternatively, you ...
- 0 kudos
- 7533 Views
- 5 replies
- 7 kudos
How to access final delta table for web application or interface.
I have a final layer of the gold delta table, that has final aggregated data from silver data . I want to access this final layer of data through the WEB interfaceI think I need to write a web script that would run the spark SQL behind to get the d...
- 7533 Views
- 5 replies
- 7 kudos
- 7 kudos
You can also use direct statement execution from databricks: https://docs.databricks.com/api/workspace/statementexecution
- 7 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 485 | |
| 349 | |
| 287 |
