- 206 Views
- 1 replies
- 0 kudos
INSERT OVERWRITE DIRECTORY
I am using this query to create a csv in a volume named test_volsrr that i createdINSERT OVERWRITE DIRECTORY '/Volumes/DATAMAX_DATABRICKS/staging/test_volsrr'USING CSVOPTIONS ('delimiter' = ',', 'header' = 'true')SELECT * FROM staging.extract1gbDISTR...
- 206 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey,The issue you’re facing with the CSV file size being significantly larger than the original table is likely due to the serialization and formatting overhead when exporting the data. A good way to verify this would be to try exporting the same dat...
- 0 kudos
- 167 Views
- 1 replies
- 0 kudos
JDBC Invalid SessionHandle with dbSQL Warehouse
Connecting Pentaho Ctools dashboards to Databricks using JDBC to a serverless dbSQL Warehouse, it works fine on the initial load, but then if we leave it idle for awhile and come back we get this error:[Databricks][JDBCDriver](500593) Communication l...
- 167 Views
- 1 replies
- 0 kudos
- 0 kudos
@pdiamond wrote:Connecting Pentaho Ctools dashboards to Databricks using JDBC to a serverless dbSQL Warehouse, it works fine on the initial load, but then if we leave it idle for awhile and come back we get this error:[Databricks][JDBCDriver](500593)...
- 0 kudos
- 190 Views
- 2 replies
- 0 kudos
Create csv and upload on azure
Can some write a sql query , which queries a table like select * from stages.benefit , creates a csv and upload on azure
- 190 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @subhadeep ,You can achieve this in SQL similarly to how you write a dataframe into a table or blob path. We will create an external table pointing to the blob path or mounted blob path. Note that this table does not support ACID transactions and ...
- 0 kudos
- 517 Views
- 10 replies
- 2 kudos
01_demo_setup error
HelloI was following "Demo: Creating and Working with a Delta Table"while I have a community edition user.The first command in the Notebook is: %run ./setup/01_demo_setup But I got the following error:Notebook not found: Users/<my-email-was-here..>/s...
- 517 Views
- 10 replies
- 2 kudos
- 2 kudos
Hey!Sad news guys... if you go to Course Logistics Review you can read:"We are pleased to offer a version of this course that also contains hands-on practice via a Databricks Academy Labs subscription. With a Databricks Academy Labs subscription, you...
- 2 kudos
- 50 Views
- 1 replies
- 0 kudos
Databricks app giving 'upstream request timeout '
Hello all,We are developing an app which is based on flask, which is used to download logs from databricks dbfs location. For this useful case we are using databricks inbuilt App feature to deploy our app.While we pass a smaller file it is getting do...
- 50 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey!It looks like the issue you’re facing might be related to the proxy timeout when downloading large files from DBFS. Since modifying the proxy settings might not be an option, there are a couple of alternative approaches you could consider to miti...
- 0 kudos
- 3375 Views
- 3 replies
- 0 kudos
50%-off Databricks certification voucher
Hello Databricks Community Team, I am reaching out to inquire about the Databricks certification voucher promotion for completing the Databricks Learning Festival (Virtual) courses.I completed one of the Databricks Learning Festival courses July 2024...
- 3375 Views
- 3 replies
- 0 kudos
- 55 Views
- 1 replies
- 0 kudos
User Unable to Access Key Vault Secrets Despite Role Assignment in Terraform
Hi All,I'm encountering an issue where a user is unable to access secrets in an Azure Key Vault, even though the user has been assigned the necessary roles using Terraform. Specifically, the user gets the following error when trying to access the sec...
- 55 Views
- 1 replies
- 0 kudos
- 0 kudos
Are they accessing the Key Vault directly and not through Databricks? If so, based on your Terraform code, they should be able to directly read Secrets in the Azure Key Vault. You've configured the Key Vault with RBAC Authorization and assigned Key ...
- 0 kudos
- 65 Views
- 2 replies
- 0 kudos
Unity Catalog for Enterprise level governance
Can we import cataloguing information from other non Databricks workloads into unity catalog? Importing metadata information from Synapse, Redshift, ADF etc. into Unity catalog for end to end lineage and tracking?
- 65 Views
- 2 replies
- 0 kudos
- 0 kudos
Hello @ShankarM, Thanks for your question. You can use Lakehouse Federation to create a connection https://docs.databricks.com/en/query-federation/index.html to be able to use with UC.
- 0 kudos
- 46 Views
- 1 replies
- 0 kudos
Prevent users from running shell commands
Hi, is there any way to prevent users from running shell commands in Databricks notebooks? for example, "%%bash" I read that REVOKE EXECUTE ON SHELL command can be used. but i am unable to make it to work. Thanks in advance for any help.
- 46 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @bvraravind, You can use this spark setting "spark_conf.spark.databricks.repl.allowedLanguages": { "type": "fixed", "value": "python,sql" Over a cluster policy to prevent access to shell commands. https://docs.databricks.com/en/archive/compute/c...
- 0 kudos
- 2334 Views
- 15 replies
- 4 kudos
Unable to Login - Account Verification Loop
I'm having trouble logging in to my Databricks account at databricks.com. Here's what happens:I enter my email address and password.I receive an account verification code via email.I enter the verification code on the login page.Instead of logging me...
- 2334 Views
- 15 replies
- 4 kudos
- 12595 Views
- 10 replies
- 0 kudos
Resolved! databricks data engineer associate exam
Hello Team, I encountered Pathetic experience while attempting my 1st Databricks certification. I was giving the exam and Abruptly, Proctor asked me to show my desk, everything i showed every corner of my bed.. It was neat and clean with no suspiciou...
- 12595 Views
- 10 replies
- 0 kudos
- 0 kudos
i have completed my exam on december 1 2024 but still i didnt get my certificate.mail id : 927716@cognizant.comexam name: Data Engineering with Databricks
- 0 kudos
- 149 Views
- 0 replies
- 0 kudos
Creating a hierarchy without recursive statements
I am looking to build a hierarchy from a parent child relationship table, which I would typically use a recursive statement for in SQL Server / Azure SQL. This would mean setting an anchor, most commonly the top record of the tree, and then join back...
- 149 Views
- 0 replies
- 0 kudos
- 124 Views
- 2 replies
- 0 kudos
Terraform: Add Key Vault Administrator Role Assignment and Save Outputs to JSON Dynamically in Azure
Hi everyone,I am using Terraform to provision an OpenAI service and its modules along with a Key Vault in Azure. While the OpenAI service setup works as expected, I am facing two challenges:Role Assignment for Key VaultI need to assign the Key Vault ...
- 124 Views
- 2 replies
- 0 kudos
- 0 kudos
For question two, you can use the local_file resource in Terraform: output "openai_api_type" {value = module.openai.api_type} output "openai_api_base" {value = module.openai.api_base} output "openai_api_version" {value = module.openai.api_version} ou...
- 0 kudos
- 2683 Views
- 4 replies
- 1 kudos
Cluster policy type
Hi guys,I am creating a cluster policy through json. "runtime_engine": {"type": "fixed""value": "PHOTON"}When I run the above code... PHOTON option is getting enabled but graying out... What would I specify in type field so that the photon option sho...
- 2683 Views
- 4 replies
- 1 kudos
- 1 kudos
Hey, If you change the type to "allowlist" and provide "PHOTON" and "STANDARD" as options that should fix your issue.here is an example: "runtime_engine": {"type": "allowlist""value": ["PHOTON", "STANDARD"]"defaultValue": "PHOTON"}
- 1 kudos
- 8526 Views
- 10 replies
- 1 kudos
Resolved! My Databrick exam got suspended just for coming closer to laptop screen to read question and options
Hi team,My Databricks Certified Data Engineer Associate exam got suspended within 10 minutes.I had also shown my exam room to the proctor. My exam got suspended due to eye movement. I was not moving my eyes away from laptop screen. It's hard to focus...
- 8526 Views
- 10 replies
- 1 kudos
- 1 kudos
Hi Databricks team, I attended the examination for Databricks certified Data engineer associate exam on Jan 18th 2025 at 1pm and I was left with last 10 questions when my exam got paused and later after showing room my video stream stopped and the...
- 1 kudos
