- 1927 Views
- 1 replies
- 0 kudos
log delivery are not creating data in s3 bucket
Hiii, Does anyone have an idea about the typical duration for Databricks to create logs in an S3 bucket using the databricks_mws_log_delivery Terraform resource? I've implemented the code provided in the Databricks official documentation, but I've be...
- 1927 Views
- 1 replies
- 0 kudos
- 0 kudos
The issue has been resolved. There was no problem with the code or the API. However, it took over 12 hours for logs to start appearing in my bucket, despite Databricks documentation indicating that logs should appear within 1 hour..Thank you!
- 0 kudos
- 16346 Views
- 3 replies
- 1 kudos
Is there a (request-) size limit for the Databricks Rest Api Sql statements?
When inserting rows through the Sql Api (/api/2.0/sql/statements/), when more than a certain number of records (about 25 records with 8 small columns) are included in the statement, the call fails with the error:"The request could not be processed by...
- 16346 Views
- 3 replies
- 1 kudos
- 1 kudos
@TheIceBrick did you find out anything else about this?I am experiencing exactly the same, I can insert up to 35 rows but break at about 50 rows.The payload size is 42KB, I am passing parameters for each row.@Debayan This is no where near the 16MiB /...
- 1 kudos
- 12685 Views
- 2 replies
- 0 kudos
Long running jobs get lost
Hello,I tried to schedule a long running job and surprisingly it does seem to neither terminate (and thus does not let the cluster shut down), nor continue running, even though the state is still "Running":But the truth is that the job has miserably ...
- 12685 Views
- 2 replies
- 0 kudos
- 0 kudos
Have you looked at the sql plan to see what the spark job 72 was doing?
- 0 kudos
- 3423 Views
- 2 replies
- 0 kudos
Reading csv file with spark throws [insufficient privelage] error
Hello Community,I have some csv files saved in databricks workspace and want to read them with spark. I make use of the commanddf = spark.read.format('csv').load(r'filepath') However, it throws the error.org.apache.spark.SparkSecurityException: [INSU...
- 3423 Views
- 2 replies
- 0 kudos
- 0 kudos
If this a UC enabled workspace, you need to provide the right access.
- 0 kudos
- 5816 Views
- 3 replies
- 2 kudos
Resolved! Update regarding Community Reward Store
Hi Team,Is there any update on the Community Reward Store, as it's been discontinued from the old portal, and we still can't see the new portal for that.Is there any expected date when this will be available for community members?
- 5816 Views
- 3 replies
- 2 kudos
- 1860 Views
- 1 replies
- 0 kudos
Using SQL for Structured Streaming
Hi!I'm new to Databricks. I'm trying to create a data pipeline with structured streaming. A minimal example data pipeline would look like: read from upstream Kafka source, do some data transformation, then write to downstream Kafka sink. I want to do...
- 1860 Views
- 1 replies
- 0 kudos
- 0 kudos
Ok I figured out why I was getting an error on the usage of `read_kafka`. My default cluster was set up with the wrong Databricks runtime
- 0 kudos
- 2863 Views
- 0 replies
- 3 kudos
Error while encoding: java.lang.RuntimeException: org.apache.spark.sql.catalyst.util.GenericArrayDa
Hello:)we are trying to run an existing working flow that works currently on EMR, on databricks.we use LTS 10.4, and when loading the data we get the following error:at org.apache.spark.api.python.BasePythonRunner$WriterThread.run(PythonRunner.scala:...
- 2863 Views
- 0 replies
- 3 kudos
- 5043 Views
- 1 replies
- 0 kudos
if any user has only permission 'select table' in unityCatalog but not having permission to ext loc
Hi,Suppose one use having access 'Select' permission the table but user not having any permission to table external location in the 'external location'.. User will be able to read the data from table?? if yes how can user will be able to read the wh...
- 5043 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Retired_mod , thanks for response.. Why the hyperlink command not showing full?
- 0 kudos
- 4455 Views
- 5 replies
- 0 kudos
How to switch Workspaces via menue
Hello,In various webinars and videos featuring Databricks instructors, I have noticed that it is possible to switch between different workspaces using the top menu within a workspace. However, in our organization, we have three separate workspaces wi...
- 4455 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi @RobinK looking at screenshots provided i can see you have access to different workspaces but still the dropdown is not visible for you, i also checked if there is any setting for same but i didnt found it.you can raise a ticket to databricks and ...
- 0 kudos
- 4269 Views
- 1 replies
- 1 kudos
Resolved! Issue with creating cluster on Community Edition
I have recently signed up for Databricks Community Edition and have yet to succesfully create a cluster.I get this message when trying to create a cluster:"Self-bootstrap failure during launch. Please try again later and contact Databricks if the pro...
- 4269 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @dustint121 It's Databricks internal issue; wait for some time and it will resolve.
- 1 kudos
- 4903 Views
- 3 replies
- 1 kudos
Resolved! Databricks community edition down?
I am getting this error when trying to create a cluster: "Self-bootstrap failure during launch. Please try again later and contact Databricks if the problem persists. Node daemon fast failed and did not answer ping for instance"
- 4903 Views
- 3 replies
- 1 kudos
- 1 kudos
I still have this issue, and have yet to successfully create a cluster instance.Please advise on how this error was fixed.
- 1 kudos
- 4037 Views
- 3 replies
- 0 kudos
Autoloader update table when new changes are made
Hello,Everyday a new file of the same name gets sent to my storage account with old and new data appended at the end. Columns may also be added during one of these file updates. This file does a complete overwrite of the previous file. Is it possibl...
- 4037 Views
- 3 replies
- 0 kudos
- 0 kudos
This may be helpful - the bit on allow overwritehttps://docs.databricks.com/en/ingestion/auto-loader/faq.html
- 0 kudos
- 4141 Views
- 1 replies
- 0 kudos
System Tables - Billing schema
Hi Experts!We enabled UC and also the system table (Billing) to start monitoring usage and cost. We were able to create a dashboard where we can see the usage and cost for each workspace. The usage table in the billing schema has workspace_id but I'd...
- 4141 Views
- 1 replies
- 0 kudos
- 0 kudos
@Retired_mod Im also not seeing the compute names logged in the system billing tables. Is this located elsewhere?
- 0 kudos
- 1373 Views
- 0 replies
- 0 kudos
Azure Oauth Passthrough with the Go Driver
Can anyone point me towards some resources for achieving this? I already have the token.Trying with: dbsql.WithAccessToken(settings.Token)But I'm getting the following error:Unable to load OAuth Config: request error after 1 attempt(s): unexpected HT...
- 1373 Views
- 0 replies
- 0 kudos
- 3330 Views
- 1 replies
- 0 kudos
Can browse external Storage, but can not create a Table from there - VNET, ADLSGen2
Hi there!Hope somebody here can help me. We have created a new Databricks Account on Azure with the ARM template for VNET injection.We have all the subnets etc., unitiy catalog active and the connector for databricks.I want now to create my first tab...
- 3330 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi,To solve this problem, the following Microsoft documentation can be used to configure the NCC to enable the connection between the private Azure storage and the serverless resources.https://learn.microsoft.com/en-us/azure/databricks/security/netwo...
- 0 kudos
