How We Test VPNs: Methodology
We’ve outlined the metrics we look at when evaluating VPNs so you know exactly what our thought process is. We start by taking a close look at what features the VPN offers (like encryption standards for security, IP addresses for privacy, etc.) to make sure it covers the bases of what a VPN should provide. We do this by putting it through speed and security tests.
We test all of our VPNs on a private Optimum network from our Brooklyn office. Our Internet speed without a VPN serves as an objective control, and we only connect one device at a time. We also take steps to make sure we are covering all our readers by testing all of our VPNs on both a Macbook Air and a Windows Vivobook. We use the website SpeedTest.net to test download speed, upload speed, and ping (latency). First, we perform tests with the VPN, and then, without. Download and upload speed are measured in megabits per second, while latency is measured in milliseconds. Once we gauge these two measures, we identify the difference in terms of a percentage to account for discrepancies between the greatly varied natural speeds of Macs and Windows computers.
Lots of factors impact speed, like distance to the server, operating system, device, etc., but it’s ideal that VPNs have no more than a 40 percent difference in any of our categories.
One of the main reasons why people use VPNs is to protect their web traffic. This consists of domain name servers (website names), and their IP addresses. We also need to know that users’ private IP addresses aren’t being leaked due to WebRTC, which allows browsers to communicate directly with each other. It’s also the default on browsers like Chrome, Microsoft Edge, and Opera.
We test for DNS leaks by using DNSLeakTest.com. We compare our computer’s IP address with the IP addresses that the website provides, we can tell if there was a DNS link while using the VPN or not. Then, we test for WebRTC leaks by leveraging a tool available on ExpressVPN’s website. We use a similar tactic where we look at the local and Public IPv4 IP addresses to see if there were any WebRTC leaks.
We also look into a company’s privacy jurisdiction and data-logging policy. Where a company is based matters a lot, because their headquarter location dictates if they are members of Five Eyes, Nine Eyes, and 14 Eyes, international surveillance alliances that have the potential to legally force companies to hand over customer data. Ideally, companies would be based in countries that are not impacted by such agreements, but we also prefer companies that keep a very minimal amount of users’ account information, like their name, email, and payment information. We don’t see the reason why VPNs would keep any additional information about when customers use their VPNs, like how long they are using it for, how much data they’ve transferred, what servers they use, what websites they visit, and more.
Our full reviews also include analysis of VPN’s encryption methods and Internet protocols to make sure they meet industry standards. We prefer VPNs with AES-256 encryption and OpenVPN, the most secure methods available.
Additionally, we look to see if a company offers anonymous and unique IP addresses to increase the likelihood that we can’t be tracked. It’s better if a company offers dynamic IP addresses because that means they will shuffle each time we log on to the VPN, making it harder to track our activity. Static IP addresses don’t change and are less desirable. Lastly, we check that each VPN has a kill switch, which will close out Internet browsers if the VPN fails.
Another primary motivator of getting a VPN is using it for media like movies and television. So, we pay special attention to which VPNs have access to torrenting and Netflix. The Netflix portion is a bit of a curveball; it constantly updates its code to block VPNs, but we still test our ability to access. so we can’t guarantee that it will always work.
We keep an eye out for split tunneling too, which allows users to use the VPN and a public network simultaneously. The lower bandwidth can create faster speeds. We prefer double or multi-hop VPNs, as they encrypt data multiple times through multiple servers.
We give our readers straightforward cost information and also leverage our expertise in looking at so many VPNs to know if that price is reasonable and worthwhile. Some VPNs are super cheap, costing less than a dollar a month with long-term commitments, while some can be as expensive as $15 a month. We also take into consideration if a company offers flexibility in their pricing options, like a month-to-month price point, or a longer-term subscription for a cheaper monthly rate. We look for the option to take the VPN for a test drive through a free trial or money-back guarantee. Lastly, we spell out what a subscription actually entails in terms of everything from the number of server switchers to the number of devices we’re allowed to run at once. It’s most common for VPNs to allow for unlimited server switches and devices, and some even offer unlimited simultaneous connections.
We check what resources are available in the event we run into issues. Typically, companies offer a live chat feature as well as an online help center. If a VPN service offers a phone line, we view it as a big bonus, since not many companies do. In addition, most services offer a robust FAQ database on their sites, where users’ questions are answered and then published, in case other users run into similar issues.
Finally, we review the app for both iPhone and Android, as well as its app store ratings. A rating below three stars disqualifies an app from being recommended to readers.