NDIS LWF questions

Question

1. My reading of the FilterReceiveNetBufferLists documentation for the flags is that one generally passes them up to the next layer. Given most of the flags represent the batch and can't be reasoned about (e.g, SWITCH_SINGLE_SOURCE) at this level, this makes sense. This is also what the sample LWF driver does.

   The problem I'm encountering is that verifier is complaining about the presence of FLAGS_PERFECT_FILTERED, faulting it as an "invalid flag".

   Is there maybe more responsibility on the filter driver to ensure these flags correctly reflect the batch I'm indicating up than I'm assuming? Outside of the special handling for FLAGS_RESOURCES the only other flag I believe a filter could rectify is SINGLE_ETHER_TYPE

   In practice I've noticed flags that I'd expect to be set (e.g, SINGLE_ETHER_TYPE) aren't. Maybe this is just the lower-edge misbehaving or the miniport simply not implementing it?

2. My understanding is that, FilterSendNetBufferListsComplete and FilterReturnNetBufferLists are only required if a filter originates send and receive operations itself, respectively. That is, if my filter is exclusively inspecting (and maybe dropping) NBLs, then neither of these routines need be implemented in my driver.

   Again, the problem is verifier is yelling at me; claiming that send/receive operations are timing out (always in a pair).

   This doesn't really much sense to me at all given the only sort of handling I could do is just passthrough, as I lack any NBLs of my own to pick out. I'll also note it doesn't appear to be an actual interruptions in dataflow; the test machine has very little traffic, and doing a simple ping -t doesn't suggest anything is getting lost in the stack/.

3. Is there a guarentee that NdisFRestartFilter and NdisFRestartComplete will not reenter my filter? The wording of "request" in their documentation somewhat suggests this is the case, and appears to be in practice, however I'd like to be sure.

4. Are filters obligated to return NBLs in the same order within the linked-list?

Note that with 1 and 2, I've used the sample driver to reproduce this behavior to rule out a bug in my own.

Answer 1

There's no one-size-fits-all story for all these flags. But I can generalize a bit about the four NDIS_RECEIVE_FLAGS*_SINGLE_XXX flags and NDIS_RECEIVE_FLAGS_PERFECT_FILTERED.

In practice I've noticed flags that I'd expect to be set (e.g, SINGLE_ETHER_TYPE) aren't. Maybe this is just the lower-edge misbehaving or the miniport simply not implementing it?

First, you should know that these five flags are always optional: even if the chain of NBLs is a single EtherType, you don't have to set the NDIS_RECEIVE_FLAGS_SINGLE_ETHER_TYPE flag. You should consider them to be a performance optimization: if you can cheaply determine the property is true, then the system might save a few CPU cycles if you set the corresponding flag.

Indeed it would be a performance pessimization to require drivers to set these flags if they apply. Many NICs do not compute these flags in hardware; forcing the NIC driver to compute it in software would just tax the CPU, and the OS might not necessarily even benefit from that cost. For example, NDIS currently ignores NDIS_RECEIVE_FLAGS_SINGLE_ETHER_TYPE if there's zero or exactly one protocol driver bound to the NIC; NDIS only starts considering the EtherType if there are multiple protocol drivers bound to that NIC. If the NIC driver did a bunch of work to compare EtherTypes, that work could all be wasted effort.

Tldr: don't compute these flags the hard way. Only set them if you have a cheap way to know they're true. Corollary: these flags might not be set, even if they could legally be set.

Is there maybe more responsibility on the filter driver to ensure these flags correctly reflect the batch I'm indicating up than I'm assuming?

These five flags are making a statement about the chain of NBLs that are batched into a single function call. It stands to reason that if your LWF is adding NBLs to that batch, then you do need to consider stripping out these flags. In general, if you add NBLs to the batch, just delete all five flags, unless you're sure that the NBLs you're adding are homogenous with the existing NBLs. Likewise, if you're changing the NBLs in a batch, e.g., changing the EtherType, then you'd better clear the corresponding flags unless you're sure that you're preserving the property. For example, if your LWF rewrites all VLAN tags to have VLAN ID 42, then you could set NDIS_RECEIVE_FLAGS_SINGLE_VLAN. But if you rewrite the VLAN IDs to be different values depending on (say) the source MAC address, then you'd better clear NDIS_RECEIVE_FLAGS_SINGLE_VLAN.

The problem I'm encountering is that verifier is complaining about the presence of FLAGS_PERFECT_FILTERED, faulting it as an "invalid flag".

This doesn't sound quite right. Are you getting NDIS_BUGCHECK_PACKET_MISUSE / NDIS_BUGCHECK_UNDEFINED_FLAG_VALUE? Arg1=0x7C, Arg2=0x32, Arg3=0x01, Arg4=your flags. That's coded to allow all defined receive flags (in current OS builds, 0xff03) in NdisFIndicateReceiveNetBufferLists. NDIS' datapath verifier does special per-flag work to verify that if a flag is set, it's actually semantically allowed to be set; but currently it doesn't implement any special checks for NDIS_RECEIVE_FLAGS_PERFECT_FILTERED.

As a guess — make sure you're not passing the receive flags to NdisFReturnNetBufferLists. That's a fairly common error, and that would bugcheck in Verifier, since 0x400 is not currently meaningful on the return path.

My understanding is that, FilterSendNetBufferListsComplete and FilterReturnNetBufferLists are only required if a filter originates send and receive operations itself, respectively. That is, if my filter is exclusively inspecting (and maybe dropping) NBLs, then neither of these routines need be implemented in my driver.

This is incorrect. Even if your driver is meant to be a simple passthrough, NDIS doesn't know that. You need to actually pass the NBLs through. You can do so by implementing these handlers in the trivial way:

void FilterSendNetBufferListsComplete(NDIS_HANDLE filter, NET_BUFFER_LIST* nblchain, ULONG flags) {
    MY_FILTER* my = (MY_FILTER*)filter;
    NdisFSendNetBufferListsComplete(my->ndishandle, nblchain, flags);
}

If you literally do nothing at all in these handlers, then NDIS assumes that your LWF has diverted them for some internal processing, but they inevitably need to be reintroduced into the datapath. The reason that ping and other things seem to work is because TCPIP and the NIC are simply allocating more and more NET_BUFFER_LIST datastructures and associated memory to replace what your driver is accumulating. Eventually someone will fail to allocate more memory, and the network datapath will seize up.

Is there a guarentee that NdisFRestartFilter and NdisFRestartComplete will not reenter my filter?

Yes.

Are filters obligated to return NBLs in the same order within the linked-list?

There are 3 cases to consider here:

• NdisFReturnNetBufferLists
• NdisFSendNetBufferListsComplete
• your FilterReceiveNetBufferLists when NDIS_RECEIVE_FLAGS_RESOURCES is set (which implicitly returns the NBLs when your function returns)

In the first 2 cases, there's no obligations on you at all. You can reorder NBLs however you like. It's actually somewhat common to use a stack/FIFO to hold completions, since it's easier to implement FIFO semantics with a singly-linked list, and this would naturally reverse the original order of the NBLs.

(This stands in sharp contrast to the first half of the datapath: you are not allowed to gratuitously reorder NBLs in the send and receive paths, since this would put TCP into a slow path for stream reassembly. [You are allowed to occasionally reorder a few NBLs, if it's during an infrequent transitional moment, like a pause or power transition, but you cannot do this all the time.] So that convenient FIFO you have in the completion path doesn't work here. The NBL_QUEUE works quite well, though.)

For the 3rd case, you are technically allowed to reorder the NBLs, but you are taking the risk of tripping over bugs in NIC drivers or other filter drivers. A sloppily-authored NIC driver might incorrectly write code like this:

NET_BUFFER_LIST* FirstNbl = AllocNbl(MyPool);
FirstNbl->Next = AllocNbl(MyPool);
NdisMIndicateReceiveNetBufferLists(..., FirstNbl, NDIS_RECEIVE_FLAGS_RESOURCES);
for each (Nbl in FirstNbl) {
    FreeNbl(Nbl);
}

The bug here is that the NIC driver assumes that if FirstNbl is the head of the list when indicating the receive, then FirstNbl would still be the head of the list after indicating the receive. If someone had swapped the order of the NBLs on the chain, then an NBL gets leaked.

For this reason, the most commonly-used parts of the OS are careful to put NBLs back into the same order after a NDIS_RECEIVE_FLAGS_RESOURCES indication. You aren't required to do this, but if you want to interop with the hundreds of NIC drivers out there, some of which are 15 years old and no longer maintained, you'd be wise to ensure the NBL chain is returned in the same order for NDIS_RECEIVE_FLAGS_RESOURCES indications.