cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
weldermartins
by • Honored Contributor
  • 6962 Views
  • 6 replies
  • 10 kudos

Resolved! Spark - API Jira

Hello guys. I use pyspark in my daily life. A demand has arisen to collect information in Jira. I was able to do this via Talend ESB, but I wouldn't want to use different tools to get the job done. Do you have any example of how to extract data from ...

Data Engineering
Reply
  • 6962 Views
  • 6 replies
  • 10 kudos
View Replies
Latest Reply
Marty73
New Contributor II
  • 10 kudos

Hi,There is also a new Databricks for Jira add-on on the Atlassian Marketplace. It is easy to setup and exports are directly created within Jira. They can be one-time, scheduled, or real-time. It can also export additional Jira data such as Assets, C...

  • 10 kudos
5 More Replies
ivanychev
by • Contributor II
  • 1941 Views
  • 1 replies
  • 1 kudos

Is there a way to avoid using EBS drives on workers with local NVMe SSD?

The Databricks on AWS docs claim that 30G + 150G EBS drives are mounter to every node by default. But if I use instance type like r5d.2xlarge, it already has local disk so I want to avoid mounting the 150G EBS drive to it. Is there a way to do it?We ...

Data Engineering
Reply
  • 1941 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
pabloanzorenac
New Contributor II
  • 1 kudos

Hey Ivan, did you find a way to do this?

  • 1 kudos
User16826992783
by • New Contributor II
  • 1266 Views
  • 1 replies
  • 0 kudos

Why are some of my AWS EBS volumes in my workspace unencrypted?

I noticed that 30GB of my EBS volumes are unencrypted, is there a reason for this, and is there a way to encrypt these volumes?

Data Engineering
Reply
  • 1266 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Abishek
Databricks Employee
  • 0 kudos

https://docs.databricks.com/security/keys/customer-managed-keys-storage-aws.html#introductionThe Databricks cluster’s EBS volumes (optional) - For Databricks Runtime cluster nodes and other compute resources in the Classic data plane, you can option...

  • 0 kudos
Mohit_m
by • Valued Contributor II
  • 2178 Views
  • 2 replies
  • 1 kudos

Resolved! Why EC2 or EBS volumes are not getting tagged for the instances pool

Why EC2 or EBS volumes are not getting tagged for the instances pool when the tags are present for the Clusters

Data Engineering
Reply
  • 2178 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 1 kudos

"If a cluster is created from a pool, its EC2 instances inherit only the custom and default pool tags, not the cluster tags. Therefore if you want to create clusters from a pool, make sure to assign all of the custom cluster tags you need to the pool...

  • 1 kudos
1 More Replies
AjayHN
by • New Contributor II
  • 3322 Views
  • 1 replies
  • 2 kudos

Resolved! Notebook failing in job-cluster but runs fine in all-purpose-cluster with the same configuration

I have a notebook with many join and few persist operations (which runs fine on all-purpose-cluster (with worker nodes - i3.xlarge and autoscale enabled), but the same notebook failing in job-cluster with the same cluster definition (to be frank the ...

job-cluster all-purpose-cluster
Data Engineering
Reply
  • 3322 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 2 kudos

Hi @Ajay Nanjundappa​ ,Check "Event log" tab. Search for any spot terminations events. It seems like all your nodes are spot instances. The error "FetchFailedException" is associated with spot termination nodes.

  • 2 kudos
Mohit_m
by • Valued Contributor II
  • 852 Views
  • 1 replies
  • 2 kudos

docs.databricks.com

Are EBS volumes used by Databricks Clusters are encrypted especially the root volumes

Data Engineering
Reply
  • 852 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
Mohit_m
Valued Contributor II
  • 2 kudos

Yes these EBS volumes are encrypted. Earlier root volume encryptions were not supported but recently this encryption is also enabled (since Apr, 2021)please find more details on the below docs pagehttps://docs.databricks.com/clusters/configure.html#e...

  • 2 kudos
Labels