How to control the perspective of a unity game using python？

Question

I want to simulate mouse movement through Python, so that the perspective of the Unity game rotates accordingly.

I used pydirectinput.moveTo() to move mouse. Used pydirectinput.keyDown() and pydirectinput.keyUp() to input key.

It can work in 《Borderlands 2》. I can move forward and backward and turn the camera.

But it can't work in 《Aim Hero》, which is a unity game. I can also move forward and backward. The characters in the game move with my control. But the character's perspective doesn't move and shoot.

I execute the command to move the mouse, and the mouse actually does move. However, the mouse will move out of the game's window, and the in-game perspective does not move with it.

Initial suspicion is the difference between Unity games and DirectX games.

This is my code:

import pydirectinput
import pyautogui
import time


def func1():
    # ------------this can work in borderlands 2 and Aim Hero(unity)
    time.sleep(4)
    pydirectinput.keyDown('w')
    time.sleep(1)
    pydirectinput.keyUp('w')
    time.sleep(1)
    pydirectinput.keyDown('d')
    time.sleep(0.5)
    pydirectinput.keyUp('d')
    time.sleep(1)
    # ------------
    
    # ------------this all can't work in Aim Hero(unity)
    pos = pyautogui.position()
    pydirectinput.moveTo(pos.x + 100, pos.y) # can't work in borderlands 2
    # pos = pyautogui.position()
    pydirectinput.moveTo(pos.x + 200, pos.y) # can work in borderlands 2
    time.sleep(0.5)
    pydirectinput.click()
    # -------------

def func2():
    time.sleep(4)
    # in borderlands 2:
    # If the command to move the mouse is executed once, the in-game camera does not move.
    # If the command to move the mouse is executed n times, the in-game camera will move n-1 times

    # in Aim Hero(Unity Game):
    # The mouse keeps moving and eventually moves out of the game window.
    # But the in-game perspective has never changed.

    for i in range(2):
        pos = pyautogui.position()
        pydirectinput.moveTo(pos.x + 10, pos.y)
        # pydirectinput.click()
        print(pos, '\n')
        time.sleep(0.1)


if __name__ == '__main__':
    print("Start!\n")
    func1()
    # func2()
    print("Done.")

Answer 1

Can you try

# ------------this all can't work in Aim Hero(unity)
    pos = pyautogui.position()
    pydirectinput.moveTo(900, 600) # estimated center position of screen
    time.sleep(1.5) #'cause this code working slow
    pydirectinput.click()
    pydirectinput.click()
    pydirectinput.click()
    # -------------

instead of

# ------------this all can't work in Aim Hero(unity)
    pos = pyautogui.position()
    pydirectinput.moveTo(pos.x + 100, pos.y) # can't work in borderlands 2
    # pos = pyautogui.position()
    pydirectinput.moveTo(pos.x + 200, pos.y) # can work in borderlands 2
    time.sleep(0.5)
    pydirectinput.click()
    # -------------

EDIT

Okay I found the problem! It's not about Python, it's about Unity. Let me show you a video from my computer. As far as I understand, if you want to use the pydirectinput.moveTo() method in Unity game, you should not lock your cursor. Be careful about your cursor state and use CursorLockMode.None instead of CursorLockMode.Locked. But it wont be effective for real game :( And I don't know any alternative for now