Mangled packets from libnetfilter_queue do not reach destination

Question

I'm trying to use libnetfilter_queue to add custom options to the IP headers of certain packets.

As a test, I've tried modifying the sample program from the libnetfilter_queue documentation (nf-queue.c) to change the TTL value in the header. The packets appear in Wireshark with the modified contents, but they don't seem to actually reach the destination.

This is the modified callback function from the libnetfilter_queue example. It changes the TTL value from the default 64 to 88.

static int queue_cb(const struct nlmsghdr *nlh, void *data)
{
    struct nfqnl_msg_packet_hdr *ph = NULL;
    struct nlattr *attr[NFQA_MAX+1] = {};
    uint32_t id = 0, skbinfo;
    int queue_num;
    struct nfgenmsg *nfg;

    uint16_t plen;
    void *payload;
    struct pkt_buff *pktb;
    uint8_t new_ttl = 88;

    char buf[MNL_SOCKET_BUFFER_SIZE];
    struct nlmsghdr *nlh_verdict;
    struct nlattr *nest;

    /* Parse netlink message received from the kernel, the array of
     * attributes is set up to store metadata and the actual packet.
     */
    if (nfq_nlmsg_parse(nlh, attr) < 0) {
        perror("problems parsing");
        return MNL_CB_ERROR;
    }

    nfg = mnl_nlmsg_get_payload(nlh);

    if (attr[NFQA_PACKET_HDR] == NULL) {
        fputs("metaheader not set\n", stderr);
        return MNL_CB_ERROR;
    }

    /* Access packet metadata, which provides unique packet ID, hook number
     * and ethertype. See struct nfqnl_msg_packet_hdr for details.
     */
    ph = mnl_attr_get_payload(attr[NFQA_PACKET_HDR]);
    id = ntohl(ph->packet_id);
    queue_num = ntohs(nfg->res_id);

    /* Access actual packet data length. */
    plen = mnl_attr_get_payload_len(attr[NFQA_PAYLOAD]);

    /* Access actual packet data */
    payload = mnl_attr_get_payload(attr[NFQA_PAYLOAD]);

    /* Copy to packet buffer with extra space for mangling */
    pktb = pktb_alloc(AF_INET, payload, plen, 255);

    /* Change TTL */
    nfq_ip_mangle(
        pktb,
        0,
        offsetof(struct iphdr, ttl),
        sizeof(((struct iphdr *)0)->ttl),
        &new_ttl,
        sizeof(new_ttl)
    );

    /* Accept mangled packet */
    nlh_verdict = nfq_nlmsg_put(buf, NFQNL_MSG_VERDICT, ntohs(nfg->res_id));
    if (pktb_mangled(pktb)) {
        nfq_nlmsg_verdict_put_pkt(nlh_verdict, pktb_data(pktb), pktb_len(pktb));
    }
    nfq_nlmsg_verdict_put(nlh_verdict, id, NF_ACCEPT);
    if (mnl_socket_sendto(nl, nlh_verdict, nlh_verdict->nlmsg_len) < 0) {
        perror("mnl_socket_send");
        exit(EXIT_FAILURE);
    }

    return MNL_CB_OK;
}

To test, I've set up an nftables rule to queue packets like this:

nft add table inet test-table
nft add chain inet test-table test-chain '{ type filter hook output priority 0; }'
nft add rule inet test-table test-chain counter queue

Then I run the program and use nc to send UDP packets like this:

• Receiver: nc -u -l -p 4444
• Sender: nc -u localhost 4444 <<< Hello

When I do this, the modified packet appears in Wireshark, but nothing is output from the receiver's nc command. Wireshark displays the packet like this:

Frame 1: 48 bytes on wire (384 bits), 48 bytes captured (384 bits) on interface lo, id 0
Ethernet II, Src: 00:00:00_00:00:00 (00:00:00:00:00:00), Dst: 00:00:00_00:00:00 (00:00:00:00:00:00)
Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
    Total Length: 34
    Identification: 0xd911 (55569)
    010. .... = Flags: 0x2, Don't fragment
    ...0 0000 0000 0000 = Fragment Offset: 0
    Time to Live: 88
    Protocol: UDP (17)
    Header Checksum: 0x4bb7 [correct]
    [Header checksum status: Good]
    [Calculated Checksum: 0x4bb7]
    Source Address: 127.0.0.1
    Destination Address: 127.0.0.1
User Datagram Protocol, Src Port: 42078, Dst Port: 4444
    Source Port: 42078
    Destination Port: 4444
    Length: 14
    Checksum: 0xfe21 [correct] (matches partial checksum, not 0x2839, likely caused by "UDP checksum offload")
    [Checksum Status: Good]
    [Stream index: 0]
    [Timestamps]
    UDP payload (6 bytes)
Data (6 bytes)
    Data: 48656c6c6f0a
    [Length: 6]

When I run the same nc commands without the nftables rules and the program running, the packet also appears in Wireshark, and the receving nc command outputs Hello as expected.

I've also tried modifying the packet by changing the TTL to the same value (64 -> 64) using nfq_ip_mangle, but it ends up the same: the packet appears in Wireshark, but the receiving process never prints a message.

What could be the cause of this?

UPDATE: It seems that adding a call to nfq_udp_mangle_ipv4 before sending the verdict makes the receiving nc process receive the packet and print the message. This works even if nfq_udp_mangle_ipv4 doesn't actually change any data.

From what I can tell looking at the source code for nfq_udp_mangle_ipv4, it doesn't seem to be doing anything fundamentally different from what I have already. It calls nfq_ip_mangle, then updates the UDP checksum.

Why is the call to nfq_udp_mangle_ipv4 necessary? What is it doing that causes the packet to go through?