Version 24 (modified by Samuli Seppänen, 2 years ago) (diff)



Microsoft has some documentation about HLK testing and WHQL signing, but it is quite incomplete, and there is lots of room for speculation and anecdotes. Practical testing is often required to understand the requirements fully. Therefore some of the requirements documented in this article are bound to change.

Different Windows versions have different kernel-mode signing options:

  • Windows 7/8/8.1/Server 2012r2
    • Cross-signing
    • WHQL-certified (HCR)
  • Windows 10 desktop
    • Attestation signing
    • WHQL-certified (HLK)
  • Windows Server 2016/2019
    • WHQL-certified (HLK)

In this article we focus on HLK testing, which allows getting a WHQL signature for a kernel-mode driver. This is the only way to make a driver load on Windows Server 2016 and later.

Getting source code for tap-windows6

Sgstair patched tap-windows6 to pass the HLK tests. At the moment (3rd May 2019) his work is not fully merged yet, so use his branch as a basis.

HLK test environment overview

HLK testing always requires a HLK Controller/Studio node, plus one or more HLK clients.

According to practical testing done by wintun developers it is possible to get a code signature that is valid for all Windows 10 platforms using the following HLK clients:

  • HLK controller: Windows Server 2016
  • HLK clients
    • Windows Server 2019 (64-bit)
    • Windows Server 2019 core (64-bit)
    • Windows 10 desktop (32-bit)

Wintun was able to pass HLK testing without any physical HLK clients. But due to wintun's narrower scope it had to pass much fewer HLK tests (~50 in total) than tap-windows6.

For tap-windows6 testing a couple of extra nodes are needed:

  • OpenVPN server (for "Run tests" in HLK, see below)
  • Support machine: required by some of the HLK tests

Lan testing prerequisites

There are some additional requirements for tap-windows6 that stem from generic LAN testing prerequisites:

  • HLK client needs at least 4 virtual processor cores (unverified)
  • HLK clients need to be physical computers, not virtualized (unverified)

Installing HLK software

For HLK software installation please refer to the official MS documentation, check out puppet-hlk or try out the Windows Virtual Hardware Lab Kit.

Preparing HLK clients for test-signed drivers

Installation of HLK client software automatically enables test signing mode in Windows. Tap-windows6 build system supports test-signing the driver automatically. You need to put the automatically generated test certificate to the Windows certificate store on the HLK clients. After that you can install the test-signed driver without signature errors.

Preparing HLK studio/controller

Loading compatibility playlists

Make sure to get the HLK Hardware Compatibility Playlists (on the main HLK download page), and apply the one for the correct context, e.g. HLK Version 1809 CompatPlaylist? x64 Server.xml. The playlist narrows down the list of tests to the set required to get an HLK certification, removes some extra stress/failure verification type tests designed to help find driver crashes. There’s a “Load Playlist” option in the tests panel in the HLK studio app that you can use.

Setting up OpenVPN for HLK tests


The "Run tests" in HLK fail consistently unless the tap-windows6 adapter has an IPv6 gateway address. This can be resolved by installing an OpenVPN server and joining all HLK clients connect to it. Launching OpenVPN client on the HLK client creates an instance of tap-windows6 adapter on the system with proper IPv6 gateway address.

Overview of the steps:

  • Install the latest OpenVPN 2.x on OpenVPN (Linux) server
  • Install the latest OpenVPN 2.x on the HLK clients
  • Install test-signed (to-be-HLK-tested) tap-windows6 driver on the HLK clients
  • Generate certificates and keys for OpenVPN with EasyRSA 3 and "openvpn --genkey"
  • Create and install configs for OpenVPN server and clients with embedded keys/certificates
  • Ensure that OpenVPN is enabled and running on server and clients
  • Verify OpenVPN connectivity

OpenVPN configuration files

NOTE: using p2p mode in OpenVPN would be simplest. The configuration files below do not use p2p, so please do not use them at this point.

OpenVPN server configuration is fairly simple:

# OpenVPN server test configuration for Windows Hardware Lab Kit test server
# We need this for HLK "Run tests" which ping6 the gateway of the interface
# bound to the driver being tested.
server-ipv6 2001:db8:6666::1/64
port 1194
proto udp
dev tun5
keepalive 10 120
verb 4
max-clients 15
status hlk-openvpn-status.log
# CA certicate here

# Server certificate here

# Server private key here

key-direction 0
# TLS auth key here

OpenVPN setup on HLK clients:

# OpenVPN client test configuration for Windows Hardware Lab Kit test clients
# We need this for HLK "Run tests" which ping6 the gateway of the interface
# bound to the driver being tested.

# Replace "x.x.x.x" with OpenVPN server's (public) IPv4 address 
remote x.x.x.x 1194
proto udp
dev tun
keepalive 10 120
verb 4
status hlk-openvpn-status.log

# CA certificate here

# Client certificate here

# Client private key here

key-direction 1
# TLS auth key here

Enabling OpenVPN on server and clients

Once OpenVPN server and clients are configured properly make sure that OpenVPN is running and automatically starts on boot. On OpenVPN server built on recent distros (e.g. Ubuntu 18.04) you'd do

$ systemctl enable openvpn-server@hlk
$ systemctl start openvpn-server@hlk

On OpenVPN clients you'd do this from an administrator Powershell session:

PS> Set-Service OpenVPNService -StartupType Automatic -Status Running

Testing OpenVPN connectivity

Assuming the above OpenVPN config you can very correct OpenVPN / tap-windows6 operation easily:

  • Verify that the TAP adapter has an IPv6 address (e.g. using ipconfig)
  • Ensure that the HLK client can ping the following VPN server addresses:
    • 2001:db8:6666::2

Configuring and running the HLK tests

Multi-machine tests

Setting up multi-machine tests is pretty straightforward to do. You just need to name some of the devices on the test client (that actually runs the tests) and support client (second machine that is connected “back to back” through the VPN). Details in LAN testing prerequisites.

Then when you go to schedule the multimachine test, you can change the “role” dropdown to the support machine, and select the support machine (should be there if you have two ready machines in the pool).

Tests that require a working network connection

Some of the non-NDIS tests that require “a working network connection” can just be given a valid IPV6 address on the VPN network and they will be happy with that. Whenever you see the “WDTFREMOTESYSTEM” parameter when scheduling a test, set it to an IPv6 address the system can ping over the VPN link. This could be the ipv6 address of the “support” system, or it could be some other arbitrary system on the VPN.

Cable unplug test

One of the HLK tests requires plugging/unplugging of the virtual cable. Use the tapdiag tool to manipulate the link state when the test asks for it.

NDIS QoS test (2c_Priority)

Use Tapdiag to enable 802.1Q on both machines before running the NDIS QoS test (2c_Priority), and use it to change link state on the test system for the LinkCheck? (plug/unplug ethernet) test. Follow the interactive messagebox prompts, may need some delay after “reattaching” the cable.

The process for using tapdiag is:

  1. tapdiag /enable # Sets a registry key that enables the .tapdiag endpoint in the driver
  2. Restart the TAP driver, reboot or disable/enable in device manager. May need to stop the openvpn service to avoid a reboot – Reconnect VPN
  3. tapdiag /link:[on|off] # Use to unplug/plug ethernet cable
  4. tapdiag /setq:[on|off|always] # to set the 802.1Q handling. Driver now disables 802.1Q by default.

Note that the tapdiag configuration is runtime only – if you reboot the test machine, you will lose the 802.1Q state.

Test issues and resolutions

Packet transmission too slow

This can result in test errors like "Expected minimum of 237 packets but we received 200 packets". The test allows one second after all the sends have been completed for all the packets to be received.

Disabling verbose log printing in the server makes it more reliable.

Packets reordering

Packets can (rarely) be reordered in flight, which causes an assertion in the test driver. Hints are errors such as

  • "1 total breakpoints were hit in the protocol driver while this test was executing"
  • "Out of order indication"
  • "Dropped indications"

HLK doesn't complain if some packets are lost, instead these errors are raised when out of order packets are received.

To fix just rerun the test. This kind of errors happen rarely.


Firewall rules for HLK server and clients

Installing HLK software automatically opens ports in the Windows firewall for HLK traffic. In case HLK controller and HLK clients are not in the same switch some firewall (e.g. EC2 security group rules) might block HLK traffic. Here is a reference for the ports which need to be open for HLK tests to work:

  • HLK clients -> OpenVPN server udp/1194
  • HLK clients -> HLK controller tcp/1771 (HLK Server Receiver Port)
  • HLK clients -> HLK controller tcp/1782 (HLKSvc Receiver Port)
  • HLK clients -> HLK controller tcp/445 (HLKInstall Samba share)
  • HLK controller -> HLK clients tcp/1771 (HLK Server Receiver Port)

Outbound traffic is assumed to be unrestricted. If not, adjust egress rules accordingly. Also note that IPv6 traffic needs to flow properly in the OpenVPN virtual network as HLK tests require IPv6.

External links