How to maintain a large, dynamic React state that defines children?

Question

I have a React component with a socket that receives information that updates its state. The state ends up being ~10,000 JSON objects, each of which represents a pointPrimitive to render in a cesium/resium app. Editing this list a few times per second is expensive, and even evaluating whether the memoized children need re-rendering is expensive at this scale. What can I do to make this component either run at a reasonable speed or re-organize it?

import React from 'react';
import { useState, useEffect } from 'react';

import { io } from "socket.io-client";
import PointCollection from '../TrackPoints/PointCollection';

import { getServerURL } from '../../lib/getServerURL';

const enoughPoints = 1;
const delayMSec = 1000 * .5;
const expireSec = 45; //This should depend on the class of item. Maybe anomalous things should last longer
const serverURL = getServerURL();


const PointManager = (props: {modes: any, onClick: Function, viewRef}) => {
    
    const [pointsDict,   setPointsDict] = useState({});
    const mode = props.modes.ADSB;
    
    //TODO: figure out whether to use a list of JSONs or a proper list of components
    //There may be performance consequences to either. For now, this is cleaner to code
    const addPoint = (JSONEntry) => {
        const k = JSONEntry.react_key;
        const oldJSON = pointsDict[k] === undefined ? [] : pointsDict[k];
        
        let newJSON = JSONEntry;
        if (oldJSON.length > 0) {
            newJSON.isAnomalous = oldJSON[oldJSON.length-1].isAnomalous;

        } else {
            newJSON.isAnomalous = false;
        }

        setPointsDict((pointsDict) => {
            const copyFoo = { ...pointsDict};
            if(copyFoo[k] === undefined) {
                copyFoo[k] = [newJSON];
                return copyFoo;
            }
            copyFoo[k].push(newJSON);
            globalThis.SCJJ = copyFoo;
            return copyFoo;
        });
    };



    const staleOutPoints = (expireSec: number) => {
        setPointsDict((pointsDict) => {
            const keys = Object.keys(pointsDict);
            const t = Date.now()/1000;
            let newPoints = {};
            keys.map((trackID) => {
                const pts = pointsDict[trackID];
                let ptsFilter = pts.filter((pt,idx) => t - pt.TIMESTAMP < expireSec);

                newPoints[trackID] = ptsFilter;
            });
            return newPoints;
        });
    };

    //Control point time-out
    useEffect(() => {
        const interval = setInterval(() => staleOutPoints(expireSec), delayMSec);
        return () => clearInterval(interval);
    }, []);
    
    //points socket
    useEffect(() => {
        const format = (mode == "live" ? "LIVE" : "DEMO");
        console.log("Setting up stream with format " + format);
        
        const socketIOSocket = io(serverURL,{});


    async function fetchStream() {
        socketIOSocket.on('connect', function() {
            console.log('connected to points server!')
            socketIOSocket.emit('please_send_JSON', {
                "format_requested": format,
        })
    });
        
        socketIOSocket.on('JSON_entry', function(msg) {
            addPoint(msg);
        });

    }

    fetchStream();

    return () => {
        console.log("cleanup called");
        socketIOSocket.disconnect();
    };
    }, [mode]);


    return(
        <>
            {/* These are all rendering with every new point. Maybe there's a way to prevent this*/} 
            <PointCollection points={pointsDict}></PointCollection>
        </>
    )
};


export default PointManager;

I've done a lot to decrease the render time of the children and to make sure this component's parent is re-rendered minimally. I've tried flattening the pointsDict. I've tried changing data types to immutable.js, but this wasn't really an ideal use case.