What is the principle behind OpenVPN tunnels?
Okay, here is a brief summary of the principle behind OpenVPN:
Imagine you had a direct physical wire (i.e. a long cable) connecting two computers (A and B) at different locations. On each computer there would be a /dev/longcable which would be a network device. You could route IP traffic over it, and do everything you could normally do with a network device.
Basically a tun device is like having a /dev/longcable except the OpenVPN daemon is the program that connects the /dev/longcable on computer A with the /dev/longcable on computer B so that you can use the internet rather than a real physical cable. But in this case it is called /dev/tun or whatever your OS prefers to call them.
Now the mechanism by which OpenVPN connects /dev/tun on computer A with /dev/tun on computer B is this: It simply creates an encrypted UDP connection over the internet between A and B and forwards traffic between /dev/tun on A with /dev/tun on B. Because of the clever way in which the tun and tap drivers were designed, it is possible for a program running entirely in user-space to effect this link, allowing OpenVPN to be a portable cross-platform daemon (like SSH), rather than an OS-specific kernel module (like IPSec).
The difference between a tun and tap device is this: a tun device is a virtual IP point-to-point device and a tap device is a virtual ethernet device. So getting back to the "long cable" analogy, using a tun device would be like having a T1 cable connecting the computers and using a tap device would be like having an ethernet network connecting the two computers. People who are running applications that need the special features of ethernet (which won't work on an IP-only network) will often bridge their physical local ethernet with a tap device (using a utility such as brctl on Linux), then VPN the tap device to another similar setup at the other end. This allows OpenVPN to route ethernet broadcasts and non-IP protocols such as Windows NetBios over the VPN. If you don't need the special features of ethernet (such as bridging capability), it's better to use a tun device.
Tun and tap devices can be interconnected to create a complex routing topology. Some people have created multi-node WAN networks over tap devices and actually run DHCP over the VPN so that clients can log into the virtual ethernet and request an IP address. I've even heard of people using Linux advanced routing to run OSPF (a kind of dynamic routing protocol) over the VPN WAN to allow for dynamic, fault-tolerant routing. They sky is the limit as far as the complexity of network you can build, but the basic building block is a VPN daemon such as OpenVPN connecting tun or tap devices on two different machines.