I've created a simple Python HTTP + WS server for a remote touchpad control app. The server allows users to control the mouse on their PC by interacting with a <div /> acting as a touchpad on their mobile phones. It sends touchmove events to the server, which then moves the actual mouse using the deltaX and deltaY values.
However, I'm facing an issue where, during the initial touchpad movement, there's noticeable jitter or lag in the mouse, almost like some noise. Once the user continues to move their finger on the touchpad, everything works smoothly.
To address this, I'm considering implementing a Kalman filter to reduce noise and improve the initial mouse movement. I would appreciate any guidance or code examples on how to integrate a Kalman filter into my existing Python server for this purpose.
Additionally, even after the user has been moving for a while and everything is relatively smooth, there is still a notable sensitivity issue. Even a slight movement of the top of the finger results in significant mouse movement. Are there strategies or adjustments I can make to fine-tune the sensitivity?
I'm also interested in optimizing the data transfer since my end goal is to implement this over Bluetooth Low Energy, which has a smaller bandwidth compared to WiFi. Is there a way to reduce the rate and amount of data being sent without compromising the user experience?
Any insights, code snippets, or resources related to implementing a Kalman filter for mouse movement and optimizing data transfer in a Python server for a touchpad control app would be greatly appreciated.
main.py
from flask import Flask, render_template # pip install Flask
from flask_socketio import SocketIO # pip install flask_socketio
import autopy # pip install autopy. python3.8 recommended for precompiled wheel
import logging
import socket
import time
logging.basicConfig(level=logging.INFO)
app = Flask(__name__, template_folder='./')
socketio = SocketIO(app)
@app.get('/')
def index():
return render_template('index.html')
@socketio.on('message')
def handle_message(data):
# delta x, delta y
dx, dy = data['dx'], data['dy']
print(dx, dy)
# Move actual mouse
x, y = autopy.mouse.location()
autopy.mouse.move(x + dx, y + dy)
if __name__ == '__main__':
host_ip = socket.gethostbyname(socket.gethostname())
socketio.run(app, port=8000, host="0.0.0.0", debug=True, use_reloader=True)
index.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Document</title>
<style>
html,body {
overflow: hidden; width: 100vw; height: 100vh; margin: 0; padding: 0; box-sizing: border-box;
}
body {
display: flex; justify-content: center; align-items: center; width: 100vw; height: 100vh;
}
.touchpad {
width: 80vw; height: 65vh; background: rgb(92, 92, 92); border-radius: 25px;
}
</style>
</head>
<body>
<div class="touchpad"></div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/4.0.1/socket.io.js" integrity="sha512-q/dWJ3kcmjBLU4Qc47E4A9kTB4m3wuTY7vkFJDTZKjTs8jhyGQnaUrxa0Ytd0ssMZhbNua9hE+E7Qv1j+DyZwA==" crossorigin="anonymous"></script>
<script type="text/javascript" charset="utf-8">
const touchpad = document.querySelector(".touchpad");
const socket = io();
const deltaScale = 1.8
let previousTouch = null;
function onTouchMove(event) {
const { clientX, clientY } = event.touches[0];
// claculate delta X and delta Y
if (previousTouch) {
const deltaX = clientX - previousTouch.clientX;
const deltaY = clientY - previousTouch.clientY;
if ((deltaX !== 0 || deltaY !== 0)) { // don't send if both zeros
// send mouse move event
socket.send({dx: deltaX * deltaScale, dy: deltaY * deltaScale})
}
console.log(deltaX, deltaY);
}
// Update the previous touch position for the next event
previousTouch = { clientX, clientY };
}
socket.on("connect", () => touchpad.addEventListener("touchmove", onTouchMove))
</script>
</body>
</html>