The following issue occurred in mdx between 5/31 and 6/2. The issue has been resolved, but during the period of the disruption, the affected hosts were unable to communicate. Additionally, there may still be hosts affected by this issue, although we have not observed any specific cases. Please review the information below, and if you have any questions or concerns about the impact or recovery methods, contact us via email at <mdx-help[at]mdx.jp>.
[Issue Period]
May 31 (Fri) 11:34 AM to June 2 (Sun) 12:34 PM (approximately 49 hours).
[Affected Hosts]
Among hosts with IPv4 addresses allocated by DHCP, those that started or renewed their DHCP lease during this period.
[Issue Details]
・DHCP failed to allocate IPv4 addresses. As a result,
・Hosts that started during this period did not receive an IP address.
・Hosts that renewed their DHCP lease during this period failed to renew and lost their IP address, stopping IPv4 communication.
[Cause]
Configuration errors associated with the OS update of the DHCP server.
[Notes]
-
- Hosts that did not renew their lease during the period were not directly affected but might have experienced indirect effects if dependent on affected hosts.
- The DHCP lease renewal interval is 72 hours. Considering the 47 hour issue period, approximately 68% of DHCP hosts were likely affected (49/72).
- IPv4 addresses should have been reassigned automatically after the issue ended, but please confirm this and check for any impacts.
[How to Check Current IP Address Allocation]
On the mdx User Portal, select the machine you want to check, check if a valid IPv4 address is displayed “Service Network 1 | IPv4 Address.”
If not, a message ” Please stay tuned until the IP address is assigned. It may take around 5 mins” will be displayed.[How to Recover Hosts with Invalidated IP Addresses]
- If you can’t log in with a sudo-enabled user from the console of the VM available in the mdx User Portal, reboot the VM from the mdx User Portal.
- If you can, open the console screen of the VM from the mdx User Portal and follow these steps. Exact command differs depending on which templates your VM is using:
For VMs using Templates by University of Tokyo
- 1. Identify the network interface name with an IPv6 address starting with 2001:2f8:1041 (
ip -br addr). - 2. Stop the network interface:
$ sudo nmcli con down [interface name] - 3. Start the network interface:
$ sudo nmcli con up [interface name] - 4. Confirm IPv4 address allocation:
$ ip -br addr
Confirm the line with the IPv6 address also shows an IPv4 address.
For VMs using Vendor-Provided Templates
- 1. Identify the network interface name with an IPv6 address starting with 2001:2f8:1041 (
ip -br addr). - 2. Stop the network interface:
$ sudo ip link set [interface name] down - 3. Start the network interface:
$ sudo ip link set [interface name] up - 4. Confirm IPv4 address allocation:
$ ip -br addr
Confirm the line with the IPv6 address also shows an IPv4 address.