How to improve time precision of two independent time series?

Question

Imagine we have two merged timeseries (by asof), one of them has very precise clocks (micro/nanoseconds) second has millisecond precision, assuming that timedrift is relatively low and clocks are synced (~10 microseconds precision), how to interpolate and improve time precision of the second series? In another words, i want to fill gaps of second time series and improve precision to precision of first series.

|    |             ts_us |             ts_ms |
|---:|------------------:|------------------:|
|  0 | 1668276142.555387 | 1668276142.555000 |
|  1 | 1668276142.563293 |        nan        |
|  2 | 1668276142.579397 |        nan        |
|  3 | 1668276142.580617 | 1668276142.581000 |
|  4 | 1668276142.600136 |        nan        |
|  5 | 1668276142.612303 |        nan        |
|  6 | 1668276142.625071 | 1668276142.625000 |
|  7 | 1668276142.636279 |        nan        |
|  8 | 1668276142.653311 |        nan        |
|  9 | 1668276142.670604 | 1668276142.671000 |
| 10 | 1668276142.690874 |        nan        |
| 11 | 1668276142.708148 |        nan        |
| 12 | 1668276142.724188 | 1668276142.724000 |

i want to have ts_ms without gaps and microsecond precision (*approx)

here is the code to simulate initial state of merged df

import pandas  as pd 
import time

data=[]
for x in range(100):
    ts={'ts_us':time.time()}
    
    if x%3==0:
        ts['ts_ms']=round(time.time(),3)
        
    data.append(ts.copy())
    time.sleep(1e-6)
    
df=pd.DataFrame(data)

not sure if df.interpolate doing required thing. May be there are better solutions/libraries which are doing probailistic estimation of a time drift and making better than simple linear interpolation?