I have a multi-task job that runs everyday where the first notebook in the job checks if the run should be continued based on the date that the job is run. The majority of the time the answer to that is no and I'm raising an exception for the job to ...
Hi Community members and Databricks Officials,Now a days i am seeing lot of spam post in our groups and discussions. Forum admins and databricks officials please take action on the users who are spamming the timeline with some promotional contents.As...
Yes @Databricks Forum Admin​ please take an action on this
I have lot of tables with 80% of columns being filled with nulls. I understand SQL sever provides a way to handle these kind of data during the data definition of the tables (with Sparse keyword). Do datalake provide similar kind of thing?
datalake itself not, but the file format you use to store data does.f.e. parquet uses column compression, so sparse data will compress pretty good.csv on the other hand: total disaster
We have a scenario where ideally we'd like to use Managed Identities to access storage but also secrets. Per now we have a setup with service principals accessing secrets through secret scopes, but we foresee a situation where we may get many service...
I have unofficial word that this is not supported, and docs don't mention it. I have the feeling that even if I got it to work it should not be trusted for now.
or Databricks only charges you whenever you are actually running the cluster, no matter how long you keep the cluster idle?Thanks!
If you not congifure your cluster auto terminate after period of idle time, yes you will be charged for that.
Goal is to try to revoke SELECT permissions from a user for a table in Data Explorer in the SQL Workspace.I've tried navigating to the Permissions tab of the tab in Data Explorer.Initially the Revoke button is greyed out and only the Grant button is ...
Hi @Christian Seberino​ connect with Databricks for the sameYou can also raise a request for the same
We need to be able to import a custom certificate (https://learn.microsoft.com/en-us/azure/databricks/kb/python/import-custom-ca-cert) in the same way as in the "data engineering" module but in the Databricks SQL module
You can try downloading it to DBFS and may be accessing it from there if you use case really needs that.
Using DBR 10.0 When calling toPandas() the worker fails with IndexOutOfBoundsException. It seems like ArrowWriter.sizeInBytes (which looks like a proprietary method since I can't find it in OSS) calls arrow's getBufferSizeFor which fails with this ...
I am also facing the same issue, I have applied the config: `spark.sql.execution.arrow.pyspark.enabled` set to `false`, but still facing the same issue. Any Idea, what's going on???. Please help me out....org.apache.spark.SparkException: Job aborted ...
Hi All,I am looking for some options to add the Client side encryption feature of azure to store data in adls gen2https://learn.microsoft.com/en-us/azure/storage/blobs/client-side-encryption?tabs=javaAny help will be highly appreciatedNote: Fernet si...
@Vidula Khanna​ We are going with fernet encryption as direct method is not available
we are trying install databricks cli on init scripts and in order to do this we need to autheticate with databricks token but it is not secure as anyone got access to cluster can get hold of this databricks token.we try to inject the secretes into se...
I think you don't need to install CLI. There is a whole API available via notebook. below is example:import requests ctx = dbutils.notebook.entry_point.getDbutils().notebook().getContext() host_name = ctx.tags().get("browserHostName").get() host_toke...
Can anyone suggest to take the best series of courses offered by Databricks to learn pyspark for ETL purpose either in Databricks partner learning portal or Databricks learning portal.
To learn Databricks ETL, I highy recommend videos made by Simon on that channel https://www.youtube.com/@AdvancingAnalytics
I am preparing for the data analyst exam and I need as many resources as I can get to fully prepare. Hands-on labs will be welcome as well
Hi,Kindly refer to the materials below:Videohttps://info.databricks.com/dc/kvtpV3WYob2etSFEoxuDGMYVc6afyrIMgIW50ZzIbvpUgj2uOQyz91VsFjIVPsTMDcYAQ8K0HTbFHGKunTHn_tZmFrrG7SaByl8pfwUNMIZfHhQHiMHwQEKzYSwtM9Vr6hKVl28RlEsSlOluDqaxKqoLcg8-qEwq4xtnrG8zKMEOSpQ...
We have service with AWS Databricks. We are doing the same replica on GCP Databricks. Here we required all the services and functionalities should be run in AWS and AWS Databricks. The only thing data should be stored on the GCP Storage. Simply funct...
no, right now i don't think they are supporting this type of architecture
Can we create a alias URL for databricks dashboard ?
yes it is possible , you have to create DNS for your databricks and map it to your workspace URL
Dear @Vidula Khanna​ Vidula, Databricks team, @nadia Elsayed​ @Jose Gonzalez​ @Aden Jaxson​ What is the SLA/ETA for normal priority ticket and HIGH priority ticket? I created tickets #00244807 on 7th Dec and #00245872 but haven't received any u...
you can only create high-priority tasks if you have an enterprise plan.as a normal user you can only create normal tasksif you have enterprise plan then you can escalate case .databricks team will revert you soon there.
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 586 | |
| 349 | |
| 287 |
