I've been running Xen on Debian 8 at home for a couple of year in bridged mode , mostly to try PCI passthrough capabilities for gaming and still having local linux envs at reach. I've started building xen on a regular basis, from 4.2 unstable to 4.5.1 theses days, and i'm eager to try qxl accelerated drivers in 4.6 right after this summer.
But today, my problems are far away from passthrough. I've got this dedicated server rented since months, it only has a single IP and i've never sucessfully managed to setup vm's internal network config. A lot of scripts i've found on the web are for inferiors versions of xen, and networking & vif scripts changed quite a bit.
All I want is a clean way to get my VM's get adressed either based on MAC adress or statically into the 192.168.88.0/24 subnet, and being able to forward a list of ports (tcp or udp) toward specific vm's
So here are my config files :
/etc/network/interfaces
auto lo
iface lo inet loopback
allow-hotplug eth0
iface eth0 inet dhcp
auto dummy0
iface dummy0 inet manual
pre-up ifconfig $IFACE up
post-down ifconfig $IFACE down
auto xenbr0
iface xenbr0 inet static
bridge_ports dummy0
address 192.168.88.254
broadcast 192.168.88.255
netmask 255.255.255.0
bridge_maxwait 0
bridge_stp off
bridge_fd 0
netstat -rn
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 62.210.115.1 0.0.0.0 UG 0 0 0 eth0
62.210.115.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
192.168.88.0 0.0.0.0 255.255.255.0 U 0 0 0 xenbr0
iptables -L && iptables -t nat -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere PHYSDEV match --physdev-out vif1.0 --physdev-is-bridged
ACCEPT udp -- anywhere anywhere PHYSDEV match --physdev-in vif1.0 --physdev-is-bridged udp spt:bootpc dpt:bootps
ACCEPT all -- anywhere anywhere PHYSDEV match --physdev-out vif1.0 --physdev-is-bridged
ACCEPT all -- 192.168.88.2 anywhere PHYSDEV match --physdev-in vif1.0 --physdev-is-bridged
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
--
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
DNAT tcp -- anywhere anywhere tcp dpt:2222 to:192.168.88.2:22
DNAT tcp -- anywhere anywhere tcp dpt:8888 to:192.168.88.2:80
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
/etc/network/interfaces [domU]
auto lo
iface lo inet loopback
allow-hotplug eth0
iface eth0 inet dhcp
address 192.168.88.2
broadcast 192.168.88.255
netmask 255.255.255.0
gateway 192.168.88.254
For networking script in xen, i've created a copy of the vif-bridge script and i've added 2 lines into it to use a small script that handle iptables rules (found on the internet, and probably incomplete iptable rules)
/etc/xen/script/portmapper.py
#!/usr/bin/env python
netdev='eth0'
# {'domU'_ip:[(domU_port, dom0port, ['tcp'|'udp']), (domU_port, dom0port), ..}
# 3rd param - protocol is optional - if not specified, tcp is default
portmap={'192.168.88.2': [(22, 2222), (80, 8888)],
'192.168.88.3': [(8081, 10001), (22, 10002)],
'192.168.88.4': [(6697, 6697)],
}
# do not edit below this line
ip_tables_proto='iptables -%s PREROUTING -t nat -p %s -i %s --dport %d -j DNAT --to %s:%d\n'
import sys
is_delete=False
def usage():
print >>sys.stderr, 'Usage: %s [-d] domU_ip' % sys.argv[0]
sys.exit(1)
def is_ip(adr):
ip_list=adr.split('.')
if len(ip_list)!=4:
usage()
for i in ip_list:
try:
if int(i)>255 or int(i)<0:
usage()
except ValueError:
usage()
args_no=len(sys.argv)
if args_no==3:
if sys.argv[1]=='-d':
is_delete=True
ip=sys.argv[2]
is_ip(ip)
elif args_no==2:
ip=sys.argv[1]
is_ip(ip)
else:
usage()
if is_delete:
action="D"
else:
action="A"
mapping=portmap.get(ip, [])
cmds=''
for port_map in mapping:
if len(port_map)==3 and port_map[2] in ('tcp', 'udp'):
from_port, to_port, proto=port_map
elif len(port_map)==2:
from_port, to_port=port_map
proto='tcp'
cmds+=ip_tables_proto % (action, proto, netdev, to_port, ip, from_port)
import os
os.system(cmds)
/etc/xen/scripts/vif-bridge-nat
#!/bin/bash
#============================================================================
# ${XEN_SCRIPT_DIR}/vif-bridge-nat
# Script for configuring a vif in bridged mode.
#
# Usage:
# vif-bridge (add|remove|online|offline)
#
# Environment vars:
# vif vif interface name (required).
# XENBUS_PATH path to this device's details in the XenStore (required).
#
# Read from the store:
# bridge bridge to add the vif to (optional). Defaults to searching for the
# bridge itself.
# ip list of IP networks for the vif, space-separated (optional).
#
# up:
# Enslaves the vif interface to the bridge and adds iptables rules
# for its ip addresses (if any).
#
# down:
# Removes the vif interface from the bridge and removes the iptables
# rules for its ip addresses (if any).
#============================================================================
dir=$(dirname "$0")
. "$dir/vif-common.sh"
bridge=${bridge:-}
bridge=$(xenstore_read_default "$XENBUS_PATH/bridge" "$bridge")
ip=${ip:-}
if [ -z "$bridge" ]
then
bridge=$(brctl show | awk 'NR==2{print$1}')
if [ -z "$bridge" ]
then
fatal "Could not find bridge, and none was specified"
fi
else
#
# Old style bridge setup with netloop, used to have a bridge name
# of xenbrX, enslaving pethX and vif0.X, and then configuring
# eth0.
#
# New style bridge setup does not use netloop, so the bridge name
# is ethX and the physical device is enslaved pethX
#
# So if...
#
# - User asks for xenbrX
# - AND xenbrX doesn't exist
# - AND there is a ethX device which is a bridge
#
# ..then we translate xenbrX to ethX
#
# This lets old config files work without modification
#
if [ ! -e "/sys/class/net/$bridge" ] && [ -z "${bridge##xenbr*}" ]
then
if [ -e "/sys/class/net/eth${bridge#xenbr}/bridge" ]
then
bridge="eth${bridge#xenbr}"
fi
fi
fi
RET=0
ip link show dev $bridge 1>/dev/null 2>&1 || RET=1
if [ "$RET" -eq 1 ]
then
fatal "Could not find bridge device $bridge"
fi
case "$command" in
online)
setup_virtual_bridge_port "$dev"
set_mtu $bridge $dev
add_to_bridge "$bridge" "$dev"
$dir/portmap.py $ip
;;
offline)
do_without_error brctl delif "$bridge" "$dev"
do_without_error ifconfig "$dev" down
$dir/portmap.py -d $ip
;;
add)
setup_virtual_bridge_port "$dev"
set_mtu $bridge $dev
add_to_bridge "$bridge" "$dev"
;;
esac
handle_iptable
call_hooks vif post
log debug "Successful vif-bridge $command for $dev, bridge $bridge."
if [ "$type_if" = vif -a "$command" = "online" ]
then
success
fi
ATM, my VM's don't even have acesss to the outside. I've been playing with tcpdump, pings, and wget to try to see what's happening, and tweaking the config, managed sometimes to get request to the ouside, coming back to the dom0 and getting droped, routed on the wrong interface, not finding his way to the VM. I'm feeling like i'm just missing sommes little pieces or screws... I've also tried to use vif-nat with isc-dhcp-server, without any sucess. It could be usefull to be able to name the vm's and then map'em to a dns, so if you have any informations on this idea.
Thanks for your time.
Nevermind, i was just missing 2 iptables lines, here is what i was missing
I've put this in my /etc/rc.local so it's get executed right after startup and only once (i've put it in /etc/network/interfaces at first, but rules get multiplicated for each vm)
My script for port-forwarding do it's job, so ports get mapped correctly, rules are added uppon vm creation and removed when destroyed.
I'll probably take a deeper look into my network rules cause i doubt they are really perfect, but it does the job.