https://community.databricks.com/t5/administration-architecture/instances-are-not-being-terminated-in-time-extra-aws-costs/m-p/65529#M1047 <P>Nevermind, this was actually due to a reservation that expired. 🤦‍</P> Thu, 04 Apr 2024 16:39:50 GMT mroy 2024-04-04T16:39:50Z https://community.databricks.com/t5/administration-architecture/instances-are-not-being-terminated-in-time-extra-aws-costs/m-p/65253#M1036 <P>For a few days we have been trying to figure out why our AWS costs suddenly went up around March 20th, and we just found the answer: the EC2 instances are left in an unterminated state for a couple of minutes at the end of each run! <span class="lia-unicode-emoji" title=":face_screaming_in_fear:">😱</span></P><P>This is a very serious bug, and I'm sure we're not the only customers being affected by this. <span class="lia-unicode-emoji" title=":money_with_wings:">💸</span></P><P>We have many jobs which run very often (every 15 minutes, 30 minutes, hourly, etc.), and all of these incur an extra cost because of this bug (more than double the cost in some cases). It affects all jobs, but the impact is much less for those which run less often.</P><P>This screenshot is very explicit (y axis is the costs). It is from <A href="https://www.apptio.com/products/cloudability/" target="_blank" rel="noopener">Cloudability</A>, which we use to manage our AWS costs.</P><P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="mroy_0-1712023586396.png" style="width: 400px;"><img src="https://community.databricks.com/t5/image/serverpage/image-id/6878i0B16EA083C4D55CA/image-size/medium/is-moderation-mode/true?v=v2&amp;px=400" role="button" title="mroy_0-1712023586396.png" alt="mroy_0-1712023586396.png" /></span></P><P>Please fix this!</P> Tue, 02 Apr 2024 02:17:40 GMT https://community.databricks.com/t5/administration-architecture/instances-are-not-being-terminated-in-time-extra-aws-costs/m-p/65253#M1036 mroy 2024-04-02T02:17:40Z https://community.databricks.com/t5/administration-architecture/instances-are-not-being-terminated-in-time-extra-aws-costs/m-p/65529#M1047 <P>Nevermind, this was actually due to a reservation that expired. 🤦‍</P> Thu, 04 Apr 2024 16:39:50 GMT https://community.databricks.com/t5/administration-architecture/instances-are-not-being-terminated-in-time-extra-aws-costs/m-p/65529#M1047 mroy 2024-04-04T16:39:50Z