Sending updated files across a network using Python Sockets

Question

I am trying to send some data across a network in a JSON file. This file will be updated periodically (every 10 seconds) and I need to read this and make it available in my application. Currently, i have a client / server scenario where i can send a single file across the network, but i am struggling with what to do when the file is updated after 10 seconds. The file is sent ok initially, but i'm not sure how to deal with an updated file.

My code is as follows: Server.py:

import socket

# File to transfer
json_file = r'position-info.json'


# Initialize Socket Instance
sock = socket.socket()
print ("Socket created successfully.")

# Defining port and host
port = 8800
host = ''

# binding to the host and port
sock.bind((host, port))

# Listening
sock.listen(10)
print('Socket is listening...')

while True:
    # Establish connection with the clients.
    con, addr = sock.accept()
    print('Connected with ', addr)

    # Get data from the client
    data = con.recv(1024)
    print(data.decode())
    # Read File in binary
    file = open(json_file, 'rb')
    line = file.read(1024)
    # Keep sending data to the client
    while(line):
        con.send(line)
        line = file.read(1024)
    
    file.close()
    print('File has been transferred successfully.')

    con.close()

On the client side I have this: client.py:

import socket

# Initialize Socket Instance
sock = socket.socket()
print ("Socket created successfully.")

# Defining port and host
port = 8800
host = 'localhost'

# Connect socket to the host and port
sock.connect((host, port))
print('Connection Established.')
# Send a greeting to the server
sock.send('A message from the client'.encode())

# Write File in binary
file = open('client-file.txt', 'wb')

# Keep receiving data from the server
line = sock.recv(1024)

while(line):
    file.write(line)
    line = sock.recv(1024)
    print(line)
print('File has been received successfully.')
print(file)
file.close()
sock.close()
print('Connection Closed.')

For sending a single file this works ok. However, i would like to send files every 10 seconds when the files have been updated. How do i do this? Many thanks

Answer 1

A socket is a byte stream, so a protocol is needed to interpret the bytes returned as complete messages.

Buffer data received until a complete message is received.

A complete message could be determined by using a known delimiter such as a newline or null, or prefix the message with its size. It's up to you. The size could be "4-byte little-endian integer" or "ASCII digits followed by newline" or whatever you decide.

socket.makefile is a way to buffer the data. It wraps a socket in a file-like object where methods such as .read() and .readline() are available to simplify buffering.

Here is an example sending JSON as a single newline-terminated line per message. Note that the input JSON could have newlines, but the file is parsed and sent without any indentation or newlines so that a message can be determined by the newline added when the JSON is sent.

Server:

import socket
import json
import time

with socket.socket() as sock:
    sock.bind(('', 8800))
    sock.listen()
    con, addr = sock.accept()
    print(f'{addr}: connected')
    with con:
        while True:
            with open('position-info.json', 'rb') as file:
                data = json.load(file)
            con.sendall(json.dumps(data).encode() + b'\n')
            print(f'{addr}: sent updated file')
            time.sleep(10)

Client:

import socket
import json

with socket.socket() as sock:
    sock.connect(('localhost', 8800))
    with sock, sock.makefile('rb') as infile:
        while True:
            line = infile.readline()
            if not line: break
            data = json.loads(line)
            print(data)

The version below send the size first as newline-terminated text, then the JSON data. In this case the JSON is sent with all indentation and newlines intact since the exact size is known:

Server:

import socket
import time

with socket.socket() as sock:
    sock.bind(('', 8800))
    sock.listen()
    con, addr = sock.accept()
    print(f'{addr}: connected')
    with con:
        while True:
            with open('position-info.json', 'rb') as file:
                data = file.read()
            con.sendall(f'{len(data)}\n'.encode() + data)
            time.sleep(10)

Client:

import socket
import json

with socket.socket() as sock:
    sock.connect(('localhost', 8800))
    with sock, sock.makefile('rb') as infile:
        while True:
            header = infile.readline()
            if not header: break
            length = int(header)
            message = infile.read(length)
            if len(message) != length:
                raise RuntimeError('incomplete message')
            data = json.loads(message)
            print(data)