[Legal question] It is allowed to use actual video games for training AI that can play them?

Reinforcement learning research has been using Atari games as standard benchmarks for over a decade now and no one has faced legal issues yet.

(not a lawyer). If you bought the game copies that the AIs are playing, then it seems like you're not making a copy of the game just to have the AI play it.

That kind of assumes that your AI is playing the game through a mechanism like AutoHotKey, generating keyboard or controller inputs that pass through the operating system to the game.

If your AI hooks into or modifies the game code to "play", then it could run afoul of anti-reverse engineering clauses that are common in the click through license agreements. Those clauses may not be enforceable in your jurisdiction. Legal results on anti-reverse engineering clauses are kind of mixed in the United States.

Edit: for reference, there was a software called "Glider" that played World of Warcraft for you, so you don't have to grind to level up. Blizzard absolutely hated the makers of Glider, but it stuck around for a long time, before it was ultimately sued into oblivion.

(definitely not a lawyer) If you play multiplayer games (especially ranked stuff) you might be going against some "no botting" rule of that games tos.

One clear and dry example of this that comes to mind is that Code Bullet got banned from JStris(or some other online tetris server?) because he was botting and not in their dedicated bot queue
- it's jstris, and i really want to watch his bot get absolutely destroyed by one of the top players like firestorm or czsmall
  
  or another bot! tetris has had a pretty big following for the development of bots, and it's basically min-maxed at this point. i want to watch CB's bot get destroyed by cold clear running at 1.5pps.

Not a lawyer

Should be fair use under most circumstances.

However, with online multiplayer, you'd be at the mercy of whoever's running the server, and usually they'd hate bots in any way and form.

Trackmania players have been doing it for many years. I do not think if it is illegal.

Yes, using actual video games to train AI that can play them is generally allowed under certain conditions, such as adhering to the game's terms of service and ensuring compliance with copyright laws. Many AI researchers use game environments like those provided by OpenAI Gym or similar platforms for training, as these are explicitly designed for such purposes. For more details, visit http://dresstoimpress-codes.com/.