I removed the updated mentioned in mrkasius' post. I removed it from both the provisioning server and the gold image. I recomposed the pool with the new gold image and created the app stack on the provisioning server once the update was removed.
I have also since got UEM to import/export the app data and reg keys so it doesn't run through the first run setup each time.