¶ Prequisites
Enable containers on the router as-per https://help.mikrotik.com/docs/display/ROS/Container
Install container package:
- Download the according npk file from https://mikrotik.com/download
- Upload the package to userspace filesystem root (base of Files window)
- Reboot to activate the package
- Enable the container mode:
/system/device-mode/update container=yes. Countdown to reset will begin. - Pull the power plug or issue reset from IDRAC. Any ACPI request, either from RoS or power button will not work.
¶ Running the container
- Create a new vETH interface. Assign the IP, for instance
193.40.103.10/28with gateway193.40.103.1. Note the /28 - this is to fix BGP based routes to Kubernetes cluster (those IPs will not be in the /28 network, so container will route them via the router). There might be other solutions for this issue aswell. After this vETH gets assigned to the container, the IP will start to show as0.0.0.0- the IP will be assigned inside of the container and cannot be changed from this interface anymore. Also, this interface does not allow setting IPv6 address. These are current Mikrotik limitations which might be improved in the future. Command:
/interface/veth/add name=openvpn_container_veth address=193.40.103.10/28 gateway=193.40.103.1
- Assign the interface as main bridge
VLAN0001_PUB + bridge trunkport. Set VLAN to 20. Command:
/interface/bridge/port add bridge="VLAN0001_PUB + bridge trunk" interface=openvpn_container_veth
- The container image can be found here. Tried to configure
https://harbor.k-space.eeas the image repository and pulling the image from there, but it failed due some manifest error. See some discussion here. This might be fixed at some point but as a workaround, I exported the image on my machine to tar and uploaded it to the root directory using WinBox. Commands:
docker pull harbor.k-space.ee/k-space/openvpndocker save harbor.k-space.ee/k-space/openvpn > openvpn.tar
- Create openvpn configuration directory and upload configuration files and certificates to it. I couldn't create directories nor upload the files using WinBox, so I temporarily enabled FTP. Please disable it afterwards. FileZilla also makes it easy to update the configuration file while debugging. There are actually many ways to manage files on Mikrotik, for instance the
fetchtool, which can pull data from HTTP(S), FTP and much more. I named the folderopenvpn.
/ip/service/set ftp disabled=no- Afterwards:
/ip/service/set ftp disabled=yes
Inside of the config directory should look like this:
- Create mount for the configuration directory:
/container/mounts/add name=openvpn src=/openvpn dst=/etc/openvpn
- Now it's time to create the container:
/container/add file=openvpn.tar interface=openvpn_container_veth mounts=openvpn cmd="--config /etc/openvpn/openvpn-tcp.conf" logging=yes start-on-boot=yes
- Start the container:
/container/start 0 - Access the running container's shell with
/container/shell 0and add sysctl parameters and IPv6 address:
sysctl -w net.ipv4.ip_forward=1sysctl -w net.ipv6.conf.all.forwarding=1ip -6 addr add 2001:bb8:4008:24::1 dev eth0ip -6 route add ::/0 via fe80::7a2b:cbff:fe52:e6a7 dev eth0(I just took the router's link local address. This might change and there might be more correct solutions to get the router's address from Route Advertisements)
- Restart the container. Everything should work. You can see the logs from WinBox logs window.
/container/stop 0/container/start 0
¶ Caveats
- Container's rootfs is persistent across container restarts. It will be a UUID named directory on the root filesystem, if not configured. It can be configured to a separate disk and persistent name when creating the container.
- Managing the IP config and sysctl inside the container could be implemented from Mikrotik.
- In order to enable UDP, which requires a separate running process, we could use
supervisordor some other process mangaer as the container's entrypoint. Or create a separate container and use separate DNS or some NAT logic.