Databricks Community

csmcpherson · 09-05-2024

With respect to the file watch trigger in workflows, how can we capture what files and or path was identified as raising the trigger? I'd like to use this information to set parameters based upon the file name and the file path Thank you! https://...

csmcpherson · 06-17-2024

Hi folks, Our company typically uses Databrick during a 12 hour block, however the AWS NAT for elastic compute is up 24 hours, and I'd rather not pay for those hours.I gather AWS lambda and cloudwatch can be used to schedule / trigger NAT destruction...

csmcpherson · 07-30-2024

For interest, this is how I ended up solving the situation, with pointers from AWS support:<< CREATE NAT >>import boto3 import logging from datetime import datetime ec2 = boto3.client('ec2') cloudwatch = boto3.client('logs') def lambda_handler(even...

csmcpherson · 07-15-2024

@Retired_mod Thanks for your reply. I created some Lambda functions to execute the NAT delete / create approach, factoring in route tables, elastic IP details and security groups per the forum guide. However, there is a problem with Databricks not be...

Databricks Community

User Stats

User Activity

Workflow file watch - capture filename trigger

AWS NAT (Network Address Translation) Automated On-demand Destruct / Create

Re: AWS NAT (Network Address Translation) Automated On-demand Destruct / Create

Re: AWS NAT (Network Address Translation) Automated On-demand Destruct / Create