cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

weldermartins
by Honored Contributor
  • 3579 Views
  • 5 replies
  • 10 kudos

Resolved! Spark - API Jira

Hello guys. I use pyspark in my daily life. A demand has arisen to collect information in Jira. I was able to do this via Talend ESB, but I wouldn't want to use different tools to get the job done. Do you have any example of how to extract data from ...

  • 3579 Views
  • 5 replies
  • 10 kudos
Latest Reply
Marty73
New Contributor II
  • 10 kudos

Hi,There is also a new Databricks for Jira add-on on the Atlassian Marketplace. It is easy to setup and exports are directly created within Jira. They can be one-time, scheduled, or real-time. It can also export additional Jira data such as Assets, C...

  • 10 kudos
4 More Replies
ivanychev
by Contributor
  • 1103 Views
  • 2 replies
  • 1 kudos

Is there a way to avoid using EBS drives on workers with local NVMe SSD?

The Databricks on AWS docs claim that 30G + 150G EBS drives are mounter to every node by default. But if I use instance type like r5d.2xlarge, it already has local disk so I want to avoid mounting the 150G EBS drive to it. Is there a way to do it?We ...

  • 1103 Views
  • 2 replies
  • 1 kudos
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Hi @ivanychev, Based on the provided information, if you want to avoid mounting the 150G EBS drive to a node with the local disk, you can set ebs_volume_count it to 0 in the Clusters API when creating the cluster. Another option could be manually det...

  • 1 kudos
1 More Replies
User16826992783
by New Contributor II
  • 824 Views
  • 1 replies
  • 0 kudos

Why are some of my AWS EBS volumes in my workspace unencrypted?

I noticed that 30GB of my EBS volumes are unencrypted, is there a reason for this, and is there a way to encrypt these volumes?

  • 824 Views
  • 1 replies
  • 0 kudos
Latest Reply
Abishek
Valued Contributor
  • 0 kudos

https://docs.databricks.com/security/keys/customer-managed-keys-storage-aws.html#introductionThe Databricks cluster’s EBS volumes (optional) - For Databricks Runtime cluster nodes and other compute resources in the Classic data plane, you can option...

  • 0 kudos
Mohit_m
by Valued Contributor II
  • 1389 Views
  • 2 replies
  • 1 kudos

Resolved! Why EC2 or EBS volumes are not getting tagged for the instances pool

Why EC2 or EBS volumes are not getting tagged for the instances pool when the tags are present for the Clusters

  • 1389 Views
  • 2 replies
  • 1 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 1 kudos

"If a cluster is created from a pool, its EC2 instances inherit only the custom and default pool tags, not the cluster tags. Therefore if you want to create clusters from a pool, make sure to assign all of the custom cluster tags you need to the pool...

  • 1 kudos
1 More Replies
AjayHN
by New Contributor II
  • 2093 Views
  • 2 replies
  • 2 kudos

Resolved! Notebook failing in job-cluster but runs fine in all-purpose-cluster with the same configuration

I have a notebook with many join and few persist operations (which runs fine on all-purpose-cluster (with worker nodes - i3.xlarge and autoscale enabled), but the same notebook failing in job-cluster with the same cluster definition (to be frank the ...

job-cluster all-purpose-cluster
  • 2093 Views
  • 2 replies
  • 2 kudos
Latest Reply
jose_gonzalez
Moderator
  • 2 kudos

Hi @Ajay Nanjundappa​ ,Check "Event log" tab. Search for any spot terminations events. It seems like all your nodes are spot instances. The error "FetchFailedException" is associated with spot termination nodes.

  • 2 kudos
1 More Replies
Mohit_m
by Valued Contributor II
  • 402 Views
  • 1 replies
  • 2 kudos

docs.databricks.com

Are EBS volumes used by Databricks Clusters are encrypted especially the root volumes

  • 402 Views
  • 1 replies
  • 2 kudos
Latest Reply
Mohit_m
Valued Contributor II
  • 2 kudos

Yes these EBS volumes are encrypted. Earlier root volume encryptions were not supported but recently this encryption is also enabled (since Apr, 2021)please find more details on the below docs pagehttps://docs.databricks.com/clusters/configure.html#e...

  • 2 kudos
Labels