Hello guys,
Today I’d like to share one of the issues that I had in Azure of one of the company customers I work for.
The customer has a script that turns VMs ON/OFF based on the time stated on the VM’s tag. For example, a VM has a tag “StartTime: 06:00” and “StopTime: 23:00”, this Azure Function runs every 1 hour and compares the current time with the time stated on the VM’s tag, if the time matches, the VM will be turned ON/OFF.
Everything was working very well, but when the customer decided to increase the number of tagged VMs, the Azure Function started returning with the timeout error. We initially thought it was the “Consumption plan”, which by default has a timeout of 5 minutes. We changed the plan, which allowed unlimited timeout (1 hour recommended by MS). And yet the function continues to give the timeout error.
We decided to split the VMs with a trigger for each subscription (The initial Azure function varies all subscriptions by looking for the tag and comparing the time), it didn’t work either.
We raised a ticket with MS, it took several days of troubleshoot and nothing to find the root cause of the problem. Until I decided to look deeper into the function code and realized that every time the function was executed and the script turned ON/OFF a VM, the function had a significant pause in between turning ON/OFF VMs.
So I decided to add to the code -NoWait and guess what? It worked!! The timeout stopped happening and the function started to be executed in less than 2 minutes, even with several VMs being tagged.
Initial code: Start-AzVM -Name $vm.Name -ResourceGroupName $vm.ResourceGroupName
Final code: Start-AzVM -Name $vm.Name -ResourceGroupName $vm.ResourceGroupName –NoWait
Ps. I don’t want to share the whole code, as I took it from the customer environment.
Apparently the function worker was waiting for a return from the previous command (Stop or Start the tagged VM), but in some cases the return wasn’t happening and the function was getting stuck.
In the moment I added the command to ignore the return (-NoWait) the function started working perfectly again.
That’s all for today guys, see you later!
Joao Costa
Get Practical 