How to Connect VMs Between Two CMP Projects Using Private IPs Through a Site-to-Site VPN?

Networking
byPavitra

Introduction

When virtual machines are deployed in different CMP projects, their private networks and virtual routers are isolated at the project level. As a result, VMs in separate projects cannot communicate directly through a shared virtual router.

A Site-to-Site IPsec VPN can be used to securely connect the private networks of two CMP projects. Once the VPN is configured, traffic between the private networks can be routed through the VPN instead of using the public network.

This guide explains how to create private networks, configure virtual routers, establish a Site-to-Site IPsec VPN, configure Linux VM routing, and verify private-to-private connectivity between VMs in separate CMP projects.

Example Network Architecture

This guide uses the following example configuration.

Component Default 1 Default 2
Project Default 1 Default 2
Private Network 10.10.10.0/24 10.10.20.0/24
Router Private IP 10.10.10.1 10.10.20.1
VPN Public IP <DEFAULT-1-VPN-PUBLIC-IP> <DEFAULT-2-VPN-PUBLIC-IP>
Test VM 10.10.10.93 10.10.20.178

 

The expected traffic path is:

Default 1 VM

10.10.10.93

      |

      | Private Network

      v

10.10.10.1

      |

      | Default 1 Router

      |

      | IPsec Site-to-Site VPN

      |

      | Default 2 Router

      v

10.10.20.1

      |

      | Private Network

      v

Default 2 VM

10.10.20.178

The final configuration allows the VMs to communicate using their private IP addresses. The public IP addresses are used only as VPN endpoints.

Prerequisites

Before configuring the Site-to-Site VPN, ensure that:

  • You have administrative access to both CMP projects.
  • Both projects are in the same region or in regions where VPN connectivity is supported.
  • Each project has a private network.
  • Each private network uses a different, non-overlapping CIDR.
  • Each project has a virtual router.
  • Each router has access to the public network.
  • You have administrative/root access to the VMs.
  • The required VPN ports/protocols are permitted by the network/security configuration.

For this example:

Default 1 Private Network:
10.10.10.0/24

Default 2 Private Network:
10.10.20.0/24

Important: Do not use the same subnet on both sides of the VPN. The private networks must use different, non-overlapping CIDRs.

Step 1: Create a Private Network

If a project does not have a private network, create one before configuring the router or VPN.

Default 1

Navigate to:

Project → Networks → Private Networks → Create Private Network

Example:

Network Name: Default-1-Private
CIDR: 10.10.10.0/24
DHCP: Enabled

The network gateway will be:

10.10.10.1

Default 2

Switch to the second project and create another private network:

Network Name: Default-2-Private
CIDR: 10.10.20.0/24
DHCP: Enabled

The network gateway will be:

10.10.20.1

Important

The private networks must not overlap.

Correct:

Default 1: 10.10.10.0/24
Default 2: 10.10.20.0/24

Incorrect:

Default 1: 10.10.10.0/24
Default 2: 10.10.10.0/24

Step 2: Configure the Virtual Router

Because CMP projects are isolated, configure a separate virtual router in each project.

Default 1 Router

Go to:

Default 1 → Networks → Routers → Create Router

Example:

Router Name: Default-1-Router
External Network: public
SNAT: Enabled

After creating the router, add the Default 1 private network as an internal interface.

The router should have:

External:
Public IP: <DEFAULT-1-VPN-PUBLIC-IP>

Internal:
10.10.10.1

Network:
10.10.10.0/24

Default 2 Router

Switch to Default 2 and create:

Router Name: Default-2-Router
External Network: public
SNAT: Enabled

Attach the Default 2 private network.

The router should have:

External:
Public IP: <DEFAULT-2-VPN-PUBLIC-IP>

Internal:
10.10.20.1

Network:
10.10.20.0/24

At this point, the two projects should look like:

Default 1

10.10.10.0/24

      |

10.10.10.1

      |

Default-1-Router

      |

Public IP

<DEFAULT-1-VPN-PUBLIC-IP>

 

Default 2

10.10.20.0/24

      |

10.10.20.1

      |

Default-2-Router

      |

Public IP

<DEFAULT-2-VPN-PUBLIC-IP>

Step 3: Configure the Site-to-Site VPN

CMP provides an IPsec VPN configuration using IKE and IPsec policies, endpoint groups, and peer information. The local and remote subnets define the networks that should communicate through the VPN.

3.1 Create the IKE Policy

In Default 1, go to:

Networks → VPNs → Create VPN

Create a new IKE policy.

Example:

Policy Name: Default1-Default2-IKE
Key Lifetime: 3600
Authentication: SHA-256
Encryption: AES-256
IKE Version: IKEv2
DH Group: Group 14

Use matching IKE parameters on both sides.

3.2 Create the IPsec Policy

Create a new IPsec policy:

Policy Name: Default1-Default2-IPsec
Key Lifetime: 3600
Authentication: SHA-256
Encryption: AES-256
DH Group: Group 14

The IPsec settings must be compatible on both VPN peers.

3.3 Create Default 1 Endpoint Groups

Select:

Router:
Default-1-Router

Local Endpoint

Group Name:
Default-1-Local

Subnet:
10.10.10.0/24

Remote Endpoint

Group Name:
Default-2-Remote

Subnet:
10.10.20.0/24

Therefore:

Default 1

Local Network = 10.10.10.0/24
Remote Network = 10.10.20.0/24

3.4 Create the Default 1 VPN

Example:

VPN Name:
Default1-to-Default2

Peer Public IP:
<DEFAULT-2-VPN-PUBLIC-IP>

Pre-Shared Key:
<Strong PSK>

The peer IP must be the public IP of the VPN router in Default 2.

Step 4: Configure the VPN in Default 2

Repeat the VPN configuration in Default 2.

Use the same IKE and IPsec policy parameters.

Router

Default-2-Router

Local Endpoint

Group Name:
Default-2-Local

Subnet:
10.10.20.0/24

Remote Endpoint

Group Name:
Default-1-Remote

Subnet:
10.10.10.0/24

Therefore:
Default 2

Local Network = 10.10.20.0/24
Remote Network = 10.10.10.0/24

VPN Configuration

VPN Name:
Default2-to-Default1

Peer Public IP:
<DEFAULT-1-VPN-PUBLIC-IP>

Pre-Shared Key:
<SAME PSK USED ON DEFAULT 1>

Important: The PSK must be identical on both sides.

The final configuration is:

Default 1

Local: 10.10.10.0/24
Remote: 10.10.20.0/24
Peer: <DEFAULT-2-VPN-PUBLIC-IP>

Default 2

Local: 10.10.20.0/24
Remote: 10.10.10.0/24
Peer: <DEFAULT-1-VPN-PUBLIC-IP>

Step 5: Verify the VPN Status

After creating both VPN configurations, check:

Default 1 → Networks → VPNs

and:

Default 2 → Networks → VPNs

Both VPN connections should show:

Status: Active

Important: The tunnel should not be considered ready until both sides are active.

Step 6: Attach the Private Network to the VM

The VM in each project must have an interface on its project’s private network.

Default 1 VM

Example:

Private Interface:
10.10.10.93

Private Gateway:
10.10.10.1

Default 2 VM

Example:

Private Interface:
10.10.20.178

Private Gateway:
10.10.20.1

Verify the interfaces inside Linux:

Bash
ip -br addr

Example Default 1:

enp1s0 UP 10.10.10.93/24
enp8s0 UP <PUBLIC-IP>

Example Default 2:

enp1s0 UP 10.10.20.178/24
enp8s0 UP <PUBLIC-IP>

Step 7: Configure the Linux VM Routing

This step is important.

Having an Active VPN does not automatically mean that the VM will select the VPN for traffic to the remote private subnet.

Initially, Linux may use the public default route.

For example:

Bash
ip route get 10.10.20.178

If the result uses the public gateway/interface, the VM is not routing the traffic through the private network.

Default 1

Add a route for the Default 2 private network:

Bash
sudo ip route add 10.10.20.0/24 via 10.10.10.1 dev enp1s0

Verify:

Bash
ip route get 10.10.20.178

Expected result:

10.10.20.178 via 10.10.10.1 dev enp1s0

Default 2

Add the reverse route:

Bash
sudo ip route add 10.10.10.0/24 via 10.10.20.1 dev enp1s0

Verify:

Bash
ip route get 10.10.10.93

Expected result:

10.10.10.93 via 10.10.20.1 dev enp1s0

Important

The example interface names may differ.

Before adding the route, identify the private interface using:

Bash
ip -br addr

Warning: Do not use enp1s0 blindly if your VM uses a different interface name.

Step 8: Verify Private-to-Private Connectivity

Test from Default 1

Ping the Default 2 private IP:

Bash
ping -c 4 10.10.20.178

Expected:

4 packets transmitted, 4 received, 0% packet loss

Test SSH:

Bash
nc -zv 10.10.20.178 22

Expected:

Connection to 10.10.20.178 22 port [tcp/ssh] succeeded!

You can also test SSH directly:

Bash
ssh root@10.10.20.178

Test from Default 2

Ping the Default 1 private IP:

Bash
ping -c 4 10.10.10.93

Expected:

4 packets transmitted, 4 received, 0% packet loss

Step 9: Verify the Routing Path

From Default 1:

Bash
ip route get 10.10.20.178

The route should use:

10.10.10.1 dev enp1s0

and not the public default gateway.

You can also use:

Bash
traceroute 10.10.20.178

A working private-to-private path should show the traffic leaving through the private gateway and reaching the remote private network.

Example:

10.10.10.1
10.10.10.x
10.10.20.x

This confirms that traffic is traversing the routed VPN path rather than using the public default route.

Troubleshooting

VPN Shows Inactive

Check the following on both sides:

  • Peer public IP
  • IKE version
  • Encryption algorithm
  • Authentication algorithm
  • DH group
  • IPsec policy
  • Pre-shared key
  • Local subnet
  • Remote subnet

Ensure that the settings are compatible on both VPN peers.

Ping Fails but VPN Is Active

Check the VM route:

Bash
ip route get <REMOTE_PRIVATE_IP>

For example:

Bash
ip route get 10.10.20.178

If the result uses the public gateway, the VM is using the public network.

The route should instead use:

10.10.10.1

One Direction Works but the Reverse Direction Fails

Check the route on the second VM.

Default 1 must have:

10.10.20.0/24 via 10.10.10.1

Default 2 must have:

10.10.10.0/24 via 10.10.20.1

Both directions are required for normal TCP communication.

Ping Works but SSH Does Not

Check whether SSH is listening:

Bash
ss -lntp | grep :22

Check the firewall:

Bash
sudo ufw status

If CMP security groups are being used, verify that the required port is allowed from the remote private subnet.

For example:

Source: 10.10.10.0/24
Destination Port: 22
Protocol: TCP

The Route Disappears After Reboot

Routes added with:

Bash
ip route add

are runtime configuration and may not survive a reboot or network restart.

For production use, configure the route persistently using the networking method appropriate for the Linux distribution, such as Netplan on Ubuntu.

Verification Checklist

After completing the configuration, verify:

  • Default 1 private network created.
  • Default 2 private network created.
  • CIDRs are non-overlapping.
  • Router created in Default 1.
  • Router created in Default 2.
  • Private networks attached to routers.
  • IKE policy configured.
  • IPsec policy configured.
  • VPN configured on Default 1.
  • VPN configured on Default 2.
  • VPN status is Active on both sides.
  • Default 1 VM is attached to 10.10.10.0/24.
  • Default 2 VM is attached to 10.10.20.0/24.
  • Route configured on Default 1 VM.
  • Route configured on Default 2 VM.
  • Ping succeeds between private IPs.
  • Application port connectivity succeeds.

Summary

To connect VMs located in different CMP projects using private networking:

  1. Create a private network in each project.
  2. Ensure the private CIDRs do not overlap.
  3. Create a virtual router in each project.
  4. Attach each private network to its corresponding router.
  5. Configure compatible IKE and IPsec policies.
  6. Configure the Site-to-Site VPN on both routers.
  7. Verify that both VPN connections show Active.
  8. Attach the private networks to the VMs.
  9. Configure Linux routes for the remote private subnet.
  10. Verify private-to-private connectivity.

The final configuration is:

Default 1                                    Default 2

 

10.10.10.0/24                               10.10.20.0/24

      |                                           |

    VM-A                                        VM-B

10.10.10.93                               10.10.20.178

      |                                           |

  10.10.10.1                                 10.10.20.1

      |                                           |

Default-1-Router                            Default-2-Router

      |                                           |

      +————- IPsec VPN ——————+

This provides private-to-private communication between VMs in separate CMP projects, while the public IP addresses are used only as VPN endpoints.

Conclusion

A Site-to-Site IPsec VPN provides a secure way to connect private networks belonging to separate CMP projects when the projects cannot share a common virtual router.

The key components are the private networks, project-specific routers, IPsec VPN configuration, and correct VM routing.

After configuration, always verify that the VM sends traffic to the remote private subnet through its private gateway, rather than through the public default route.

FAQ’s

Can I connect two projects using a single router?

No. In this configuration, each CMP project uses its own router, and the two routers are connected through the Site-to-Site VPN.

Do both projects need a private network?

Yes. Each project needs a private subnet for private VM-to-VM communication.

Can both private networks use the same CIDR?

No. The local and remote networks must use different, non-overlapping CIDRs.

Does an Active VPN automatically change the VM’s routing?

No. An Active VPN does not automatically mean that the VM will use the VPN for remote private-subnet traffic. The VM must have a route directing the remote private subnet to its local private gateway.

Should VM-to-VM traffic use the public IP?

No. Application traffic should use the private IP addresses. The public IP addresses are used only as VPN peer endpoints.

How can I verify that the VM is using the VPN?

Run:

Bash
ip route get <REMOTE_PRIVATE_IP>

The route should point to the VM’s private gateway/interface, not the public gateway.

How can I verify application connectivity?

Use the appropriate test, for example:

Bash
ping <REMOTE_PRIVATE_IP>
nc -zv <REMOTE_PRIVATE_IP> 22
ssh user@<REMOTE_PRIVATE_IP>

Are the Linux routes persistent?

Routes added with ip route add are normally runtime routes. For production deployments, configure them persistently through the operating system’s network configuration.