IPsec standard predominantly talks about policy based VPN. Policy based VPN contains multiple policy records, with each policy record having source, destination networks/hosts, service port and security paramters such as encryption, authentication algorithms etc.. These records are arranged in ordered list. Packets are matched against the policy table. If there is a policy record match, appropriate secuirty is enforced on the packets such as encryption and adding additional headers to reach the remote gateway. Remote gateway decrypts the packets and sends clear packets to intended host.
Thought policy based VPN gives granular control for administrators, it has its own disadvantages. If pair of security gateways have multiple networks and services for which data security need to be applied, then policy record(s) must be configured with these networks on the gateways. Every time new network is added in a site, policy records should be updated. Configuration update should be done not only on local gateway, but also in remote gateways. Source IP selector of policy record gets modified with new network on local gateway and Destination IP selector needs to get modified with new network on remote gateway. Admins not only need to configure routes to reach new networks in remote gateways, but also add or modify IPsec policy records. Since networks are to be added to the policy records explicitly, dynamic routing protocols also can't be used across sites.
Route based VPN solves above problems. In route based VPN, a point-to-point L3 interface is created and all traffic sent to this interface are tunneled to the remote gateway. As many interfaces are created as number of remote gateways. For a given pair of gateways, only one tunnel is created. Once this is done, administrator only needs to add routes to remote networks via tunnel interfaces. If dynamic routing protocols are used, admin need not even create routes explicitly.
Multiple types of route based VPNs are implemneted by appliance vendors. They are:
IP-in-IP route based VPN: Interfaces are created with IP-in-IP. IP packets are encapsulated with new IP header. Outer IP header IP addresses are gateway IP addresses. One IPsec policy record is created with these two gateway IP addresses as selectors. Any packets sent to this interface is encapsulated with outer IP header and then IPsec processing with ESP/AH encapsulation is done. When the encrypted packets are received, packets are decrypted, outer IP header is removed and then internal packet is routed to intended host.
GRE Route based VPN: It is similar to IP-in-IP route based VPN, except that it gets encapsulated with GRE+IP header. IPsec policy is created with GRE protocol and gateway IP addresses.
Encapsulation-less route based VPN: In this mode, there is no additional encapsulation such as IP-in-IP or GRE. Only IPsec ESP tunnel is used to encapsulate the packets with gateway IP addresses. In this case, Ipsec policy record is created with Source IP and destination IP selectors as 0.0.0.0 (ALL). So, selector negotiation happens with 0.0.0.0 IP addresses.
Since L3 interface is created, any dynamic routing protocols such as RIP, OSPF and BGP work on these interfaces and don't require any changes to the routing protocol implementations.