- 2179 Views
- 3 replies
- 0 kudos
Resolved! Are there any scenarios where it doesn't make sense to use Spot Instances?
It seems like using spot instances make a lot of sense for cost savings. But are there any risks to using them? Or things to consider before enabling them?
- 2179 Views
- 3 replies
- 0 kudos
- 0 kudos
On AWS, to further improve the chance of acquiring spot instances, you can use the newly GA'd feature Fleet instance types: https://docs.databricks.com/compute/aws-fleet-instances.html
- 0 kudos
- 1792 Views
- 1 replies
- 1 kudos
Limited number of workspaces vs workspace for department or line of business ?
We are just getting started with databricks currently we have 1 workspace for each environment (DEV, QA and PRD). we have started with 1 workspace but there is already getting flooded with new workspace requests? is there any check list/criteria for ...
- 1792 Views
- 1 replies
- 1 kudos
- 1 kudos
I believe DEV, QA and PRD are the right segregate . We may need this because this will be easy to categorize the production and QA workload - based upon the amount of data cluster processes (Meaning more DBU) which we can restrict by the company work...
- 1 kudos
- 1885 Views
- 1 replies
- 0 kudos
Delta Live Table tables in Data Tab
If I use this code (CREATE STREAMING LIVE TABLE Employee) in dlt pipeline, where does the Employee table gets created by default, if no storage location is specified? How can I create this table in Data tab within a database (a.k.a: Schema) lets say ...
- 1885 Views
- 1 replies
- 0 kudos
- 0 kudos
@Kris Koirala​ :When you create a streaming live table in Databricks Delta Lake using the code CREATE STREAMING LIVE TABLE Employee, the table is created in the default database called default. If no storage location is specified, the table is stored...
- 0 kudos
- 2865 Views
- 3 replies
- 0 kudos
If I come via Databricks Partner connect and subscribe a partner product then how is the billing done and what api is used for publishing usage information to databricks?
If I come via Databricks Partner connect and subscribe a partner product then how is the billing done and what api is used for publishing usage information to databricks?
- 2865 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Atul Karn​ We haven't heard from you since the last response from @Kaniz Fatma​ ​, and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to others....
- 0 kudos
- 9850 Views
- 1 replies
- 0 kudos
Spark Streaming: Checkpoint corrupted
Hi Everyone!Today 4 streaming jobs started to fail out of nowhere due to: StreamingQueryException: [STREAM_FAILED] Query [id = ####, runId = ####] terminated with exception: dbfs:/mnt/path/my_table/sources/0/0 doesn't exist (latestId: 8, compactInter...
- 9850 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Martin Riccardi​,We haven't heard from you since the last response from @Kaniz Fatma​ , and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to ot...
- 0 kudos
- 5205 Views
- 2 replies
- 3 kudos
Resolved! How can I set the data access for each SQL warehouse individually?
As far as the official documentation from Microsoft says and I understood it correctly, each SQL warehouse uses the same service principal for data access. But I want to control the access to the data (Azure Data Lake Storage) for each SQL warehouse ...
- 5205 Views
- 2 replies
- 3 kudos
- 3 kudos
It can be done but it's very tricky. You should create a service principal to whom you will grant permission over some tables then you can generate a token using the SP to access only this data.
- 3 kudos
- 16291 Views
- 2 replies
- 0 kudos
How to move from an embedded spark inside a spring boot microservices to azure databricks?
Hi, My question will be more about the architecture solutions and potential implementation of these solutions.The project :Multiple microservices each one deployed on a Tomcat and each Tomcat is running in one of the tree VM existing in Azure.Each mi...
- 16291 Views
- 2 replies
- 0 kudos
- 0 kudos
My main question is why you use spark in the first place, but that is another discussion.If you want to use databricks, you will have to launch a spark job in one way or another on the databricks cluster with spark-submit or by using the jobs api.How...
- 0 kudos
- 2614 Views
- 0 replies
- 0 kudos
Unity Catalog - Cluster Access Mode - Change Management
Hi all,I am curious to know how you handled change management when rolling out changes to cluster access modes (single user, shared) applicable to UC, especially when there is a centralized platform team offering Infrastructure Deployment services vi...
- 2614 Views
- 0 replies
- 0 kudos
- 8775 Views
- 2 replies
- 2 kudos
Resolved! What is the advantages and disadvantages of setting up log-delivery at account vs workspace level?
- 8775 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @janaki jagadeesh​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answ...
- 2 kudos
- 3248 Views
- 2 replies
- 0 kudos
Resolved! Does CVE-2020-13949 (vulnerability) affect the data plane and its security patch is included in Databricks 10.4 LTS?
Here is the previous discussion. https://community.databricks.com/s/question/0D58Y0000ACcIv2SQF/does-thrift-only-exist-in-databrick-control-planeI have the following questions.Does CVE-2020-13949 affect the data plane or not?Do you know from which ve...
- 3248 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Jimin Hsieh​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so w...
- 0 kudos
- 8722 Views
- 0 replies
- 0 kudos
Is CVE-2023-22946 managed by databricks?
Hi Databricks,We want to understand a vulnerability issue - CVE-2023-22946.does this CVE affect Azure databricks users? Does it affect data plane? Is it managed by databricks? Should end users do anything to mitigate it if this issue is a shared res...
- 8722 Views
- 0 replies
- 0 kudos
- 2679 Views
- 1 replies
- 0 kudos
Number of DBU's utilized in Azure
We would like to know the number of DBU's utilized cluster level/workspace level which was hosted in Azure. We were able to see the amount spend but not able to see the DBU's. If we get any idea of no.of DBU's it would help us to think of reserved DB...
- 2679 Views
- 1 replies
- 0 kudos
- 0 kudos
@Mani Teja G​ we don't have screen like aws for azure, but in azure you can monitor everything in cost management --> cost analysis and select tags as Databricks
- 0 kudos
- 1757 Views
- 1 replies
- 0 kudos
How can I set up a DBU usage alert
Hi,I would like to set up a DBU usage alert, such that I will get an email notification when the DBU usage of last X hour exceeds the budget.How can I do that?Thanks,Kit
- 1757 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @Kit Yam Tse​ You can download and analyse usage logs to monitor the DBUs.Check this out: https://docs.databricks.com/administration-guide/account-settings-e2/usage.htmlwe also have usage analysis dashboard notebook:https://docs.databricks.com/...
- 0 kudos
- 1346 Views
- 1 replies
- 1 kudos
How to get Azure Databricks Workspaces databricks unit (DBU)?
Is there a way to get the databricks units (DBU) for an existing Azure databricks workspace?
- 1346 Views
- 1 replies
- 1 kudos
- 3582 Views
- 2 replies
- 0 kudos
Access Ganglia metrics via endpoint
How to access the raw Ganglia metrics. Is there an endpoint or a DBFS storage it belongs to?
- 3582 Views
- 2 replies
- 0 kudos
Connect with Databricks Users in Your Area
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group-
Access control
1 -
Access Delta Tables
2 -
ActiveDirectory
1 -
AmazonKMS
1 -
Apache spark
1 -
App
1 -
Availability
1 -
Availability Zone
1 -
AWS
5 -
Aws databricks
1 -
AZ
1 -
Azure
8 -
Azure Data Lake Storage
1 -
Azure databricks
6 -
Azure databricks workspace
1 -
Best practice
1 -
Best Practices
2 -
Billing
2 -
Bucket
1 -
Cache
1 -
Change
1 -
Checkpoint
1 -
Checkpoint Path
1 -
Cluster
1 -
Cluster Pools
1 -
Clusters
1 -
ClustersJob
1 -
Compliance
1 -
Compute Instances
1 -
Cost
1 -
Credential passthrough
1 -
Data
1 -
Data Ingestion & connectivity
6 -
Data Plane
1 -
Databricks Account
1 -
Databricks Control Plane
1 -
Databricks Error Message
2 -
Databricks Partner
1 -
Databricks Repos
1 -
Databricks Runtime
1 -
Databricks SQL
3 -
Databricks SQL Dashboard
1 -
Databricks workspace
1 -
DatabricksJobs
1 -
DatabricksLTS
1 -
DBFS
1 -
DBR
3 -
Dbt
1 -
Dbu
3 -
Deep learning
1 -
DeleteTags Permissions
1 -
Delta
4 -
Delta Sharing
1 -
Delta table
1 -
Dev
1 -
Different Instance Types
1 -
Disaster recovery
1 -
DisasterRecoveryPlan
1 -
DLT Pipeline
1 -
EBS
1 -
Email
2 -
External Data Sources
1 -
Feature
1 -
GA
1 -
Ganglia
3 -
Ganglia Metrics
2 -
GangliaMetrics
1 -
GCP
1 -
GCP Support
1 -
Gdpr
1 -
Gpu
2 -
Group Entitlements
1 -
HIPAA
1 -
Hyperopt
1 -
Init script
1 -
InstanceType
1 -
Integrations
1 -
IP Addresses
1 -
IPRange
1 -
Job
1 -
Job Cluster
1 -
Job clusters
1 -
Job Run
1 -
JOBS
1 -
Key
1 -
KMS
1 -
KMSKey
1 -
Lakehouse
1 -
Limit
1 -
Live Table
1 -
Log
2 -
LTS
3 -
Metrics
1 -
MFA
1 -
ML
1 -
Model Serving
1 -
Multiple workspaces
1 -
Notebook Results
1 -
Okta
1 -
On-premises
1 -
Partner
74 -
Pools
1 -
Premium Workspace
1 -
Public Preview
1 -
Redis
1 -
Repos
1 -
Rest API
1 -
Root Bucket
2 -
SCIM API
1 -
Security
1 -
Security Group
1 -
Security Patch
1 -
Service principal
1 -
Service Principals
1 -
Single User Access Permission
1 -
Sns
1 -
Spark
1 -
Spark-submit
1 -
Spot instances
1 -
SQL
1 -
Sql Warehouse
1 -
Sql Warehouse Endpoints
1 -
Ssh
1 -
Sso
2 -
Streaming Data
1 -
Subnet
1 -
Sync Users
1 -
Tags
1 -
Team Members
1 -
Thrift
1 -
TODAY
1 -
Track Costs
1 -
Unity Catalog
1 -
Use
1 -
User
1 -
Version
1 -
Vulnerability Issue
1 -
Welcome Email
1 -
Workspace
2 -
Workspace Access
1
- « Previous
- Next »
User | Count |
---|---|
42 | |
26 | |
21 | |
13 | |
9 |