how to do a JSON diff/patch between 2 big nested json from .net backend to js frontend?

Question

i have a very big json that has a very small changes each second.

im using asp.net core 3.1 and signalr core. and clients are using browsers running pure js and jquery.

right now for every change im re sending all the json again. i know that its not a good way.

what i want to do is just sending the Changes to client and then patch the changes to the main JSON in javascript.

by google search i understood that its a JSON diff/patch or diff/Merge operation, but i did not found anything on how to implement this.

so, how can i achieve a diff/pach opration in signalr ?

J_m and j_m_lit maybe 100 keys.

the data struncture

[
   {
      "id":0,
      "stdid":3808003,
      "lid":10533,
      .
      .
      (40 root keys)
      .
      .
      "J_ps":null,
      "J_ms":null,
      "J_m":{
         "SA_H2 0":{
            "c":"SA_H2",
            "a":"0",
            "b":0,
            "r":"0",
            "w":0,
            "en":"2nd",
            "o":{
               "2:0":{
                  "c":"2:0",
                  "v":1.3,
                  "pv":0.0,
                  "b":0
               },
               "Tie":{
                  "c":"Tie",
                  "v":3.2,
                  "pv":0.0,
                  "b":0
               }
            }
         },....
       
      },
      "J_m_Lit":{
         "OE 0":{
            "c":"OE",
            "a":"0",
            "b":0,
            "r":"0",
            "w":0,
            "en":"Total ",
            "o":{
               "xxx":{
                  "c":"Odd",
                  "v":1.833,
                  "pv":0.0,
                  "b":0
               },
               "zzz":{
                  "c":"Even",
                  "v":1.833,
                  "pv":0.0,
                  "b":0
               }
            }
         },  ....

      },
    
      "J_fre":{
         "tq":[
            0,
            1
         ],
         "m1":[
            23,
            17
         ]
      },
     
   }
]

Answer 1

Yes, I think you are on the right track.

JSON-Patch (RFC6902), at its core, tries to solve the very problem you facing here: partial updates. JSON Patch offers an atomic update request that patches your JSON using a series of "add", "remove", "replace", "move" and "copy" operations. To quote one of the authors of the JSON Patch RFC, Mark Nottingham (IETF):

This has a bunch of advantages. Since it’s a generic format, you can write server- and client-side code once, and share it among a number of applications. You can do QA once, and make sure you get it right. Furthermore, your API will become less complex, because it has less URI conventions, leading to more flexibility and making it easier to approach for new developers. Using this approach will also make caches operate more correctly; since modifications to a resource will “travel” through its URL, rather than some other one, the right stored responses will get invalidated.

ASP.NET Core offers JSON-Patch support. JSON Patch can be added by installing Microsoft JSON Patch library (Nuget) from the Package Manager console:

Install-Package Microsoft.AspNetCore.JsonPatch

Then you apply patches using the JsonPatchDocument class and its methods like ApplyTo, Add, Copy, Move, Remove, Replace.

In an ASP.NET Core Web API context, we can also make use of the Microsoft.AspNetCore.Mvc.NewtonsoftJson NuGet package.

And then use the PUT and PATCH methods to update an existing resource with JSON Patch operations.

This MSDN article has some good explanations and a sample project.

---

In a JavaScript/Node.js project, you could also choose from a larger list of JS implementations, notably fast-json-patch.

Resources:

• Official JSON Patch documentation.
• .NET CORE Tutorials: JSON Patch With ASP.NET Core

---

Side note: Another option is JSON Merge Patch (RFC7396), which can be used to send partial updates as well but works more like an actual diff/patch that simply contains the changes instead of using mutating operations. In this respect, JSON Merge Patch is simpler but more limited than JSON Patch, e.g. you cannot set a key to null (since null means remove), merging only works with objects {..}, arrays [..] can only be replaced as a whole, and merging never fails, could cause side-effects and is therefore not transactional. This renders Merge Patch (overly) simplistic and impractical for certain real-world applications. There are more related projects named like JSON-Diffpatch, etc that take similar approaches. Arguably, JSON-Patch is more flexible but both have advantages and disadvantages.