azure cost analysis skill told me to kill the A100 pool mid-eval
Azure Cost Analysis
Analyze Azure spend, bill spikes, and AKS or AI cost breakdowns
plugged the Azure Cost Analysis skill into Cursor yesterday because finance pinged me about a $1.8k spike on rg-evals-westus.
it wrote a tidy markdown plan: deallocate the A100 pool, rightsize the jump box, cancel a reservation renewal. the A100s were mid-benchmark for a 70B latency run that finishes tonight.
i almost hit approve because the chart looked clean. anyone else running these Microsoft skills against live GPU jobs, or do you keep them read-only?
4 comments
Join the discussion
Log in to comment.
Did it even look at Reservations, or just the on-demand burn curve? Every time I tried a cost skill it treated a reserved A100 as "waste" because utilization dipped between eval batches.
Read-only until it can show the commitment end date next to the delete button.
same energy here. last week it flagged my
stg-redisas idle while a cron sleeps 20 min between jobs. almost deleted it from staging before i checked the schedule.i keep the skill installed but approvals stay human. the markdown plan looks too confident.
the markdown confidence is the trap. we started requiring the skill to attach the reservation end date AND the last 24h of GPU minutes before any deallocate step is even allowed in the plan.
without those two fields a dumb pre-check rejects it. cut our almost-approve rate a lot.
We saw the same on a 70B latency sweep. Skill recommended deallocate while utilization sat at 12% between batches — true on the chart, useless in practice. Cold-starting those A100s cost us ~$40 and 18 min; the "savings" evaporated.
I keep it read-only plus a hard block on any delete/deallocate touching a resource tagged
eval-active. Finance can wait for the run to finish.