For dummies: How to avoid 'bill shock' & control AWS charges while learning to use Databricks?

[hantha]

Hi, I'm an out-of-work data analyst wanting to re-skill as a 'citizen data engineer'.  By following how-to guides I was able to set up my own Databricks account etc along with a personal VPC in AWS. After 2 weeks of problem-free training I checked my month-to-date billing info in AWS. The "NAT Gateway Hour" cost (337 hours!) is the 2nd largest cost item but seems rather unnecessary, given how everything I need is co-located in the same AWS regional data center (Sydney). What would be the likely impact on my Databricks training experience supposing I just delete my NAT gateway? Or is that billing item just included as a notional charge, one of several that will be waived by AWS provided I stay within the 'free tier' limits? (The AWS invoice status is "pending" so I guess I'll find out soon enough 🙂  

[holly]

Hi Hantha

• Databricks needs VPCs to work, but there are the default ones and customer managed ones: https://docs.databricks.com/en/security/network/classic/customer-managed-vpc.html
• Customer managed ones are optional, but many tutorials include them as that's standard security for lots of companies. I have made the mistake of this and cost Databricks $1200 🤡
• If you start deleting your network pieces, it'll delete things associated with it 
• Recommendation: take a look at the docs linked. If it looks like you set up a managed one, you'll have to recreate everything with the default in a new Databricks workspace (sorry)
• As for the AWS charges, I couldn't guarantee what's part of the free tier and what isn't. I do know their support team is quite good so if you drop them an email they should be able to give you guidance. 

[Kaniz_Fatma]

Hi @hantha, Thanks for reaching out! Please review the response and let us know if it answers your question. Your feedback is valuable to us and the community.

If the response resolves your issue, kindly mark it as the accepted solution. This will help close the thread and assist others with similar queries.

We appreciate your participation and are here if you need further assistance!