This runs and works fine for a short period (couple minutes to a couple days), but then it starts failing.
$ systemctl status oem-cloudinit
● oem-cloudinit.service – Cloudinit from EC2-style metadata
Loaded: loaded (/run/systemd/system/oem-cloudinit.service; static; vendor preset: disabled)
Active: failed (Result: exit-code) since Wed 2018-04-11 21:53:14 UTC; 1min 30s ago
Process: 767 ExecStart=/usr/bin/coreos-cloudinit –oem=ec2-compat (code=exited, status=1/FAILURE)
Main PID: 767 (code=exited, status=1/FAILURE)
Apr 11 21:53:14 ip-10-0-0-180.ec2.internal coreos-cloudinit[767]: 2018/04/11 21:53:14 Wrote unit “teamcity-agent.service”
Apr 11 21:53:14 ip-10-0-0-180.ec2.internal coreos-cloudinit[767]: 2018/04/11 21:53:14 Ensuring runtime unit file “etcd.service” is unmasked
Apr 11 21:53:14 ip-10-0-0-180.ec2.internal coreos-cloudinit[767]: 2018/04/11 21:53:14 Ensuring runtime unit file “etcd2.service” is unmasked
Apr 11 21:53:14 ip-10-0-0-180.ec2.internal coreos-cloudinit[767]: 2018/04/11 21:53:14 Ensuring runtime unit file “fleet.service” is unmasked
Apr 11 21:53:14 ip-10-0-0-180.ec2.internal coreos-cloudinit[767]: 2018/04/11 21:53:14 Ensuring runtime unit file “locksmithd.service” is unmasked
Apr 11 21:53:14 ip-10-0-0-180.ec2.internal coreos-cloudinit[767]: 2018/04/11 21:53:14 Calling unit command “start” on “teamcity-server.service”
Apr 11 21:53:14 ip-10-0-0-180.ec2.internal coreos-cloudinit[767]: 2018/04/11 21:53:14 Failed to apply cloud-config: Unit format-mnt-data.service is not loaded properly: Exec format error.
Apr 11 21:53:14 ip-10-0-0-180.ec2.internal systemd[1]: oem-cloudinit.service: Main process exited, code=exited, status=1/FAILURE
Apr 11 21:53:14 ip-10-0-0-180.ec2.internal systemd[1]: oem-cloudinit.service: Failed with result ‘exit-code’.
Apr 11 21:53:14 ip-10-0-0-180.ec2.internal systemd[1]: Failed to start Cloudinit from EC2-style metadata.
$ systemctl status format-mnt-data
● format-mnt-data.service
Loaded: error (Reason: Exec format error)
Active: inactive (dead)
Apr 11 21:53:09 ip-10-0-0-180.ec2.internal systemd[1]: format-mnt-data.service: Cannot add dependency job, ignoring: Unit format-mnt-data.service is not loaded properly: Exec format error.
Apr 11 21:53:14 ip-10-0-0-180.ec2.internal systemd[1]: /etc/systemd/system/format-mnt-data.service:8: Failed to resolve unit specifiers on /usr/sbin/pvcreate /dev/xvdg && /usr/sbin/vgcreate app /dev/xvdg && /usr/sbin/lvcreate -l 100%FREE -n data app && /usr/sbin
Apr 11 21:53:14 ip-10-0-0-180.ec2.internal systemd[1]: /etc/systemd/system/format-mnt-data.service:8: Failed to resolve unit specifiers on /usr/sbin/pvcreate /dev/xvdg && /usr/sbin/vgcreate app /dev/xvdg && /usr/sbin/lvcreate -l 100%FREE -n data app && /usr/sbin