GameAgent Class

The InputController Class

The InputController class serves as a gateway for sending game input in the Bluetang framework.

What's in an InputController Instance?

Responsibilities

• Exposing basic and convenience keyboard inputs

• Exposing basic and convenience mouse inputs

Concepts

Backend

Bluetang provides different backend options for input controllers. They are defined in the InputControllers enum:

• InputControllers.PYAUTOGUI: A backend that leverages the PyAutoGUI Python library. Default for Linux & macOS.

• InputControllers.NATIVE_WIN32: A backend that leverages Windows' SendInput DLL function. Default for Windows.

To override your platform's default backend, add an entry to your Game plugins:

class BluetangNuclearThroneGame(Game, metaclass=Singleton):

    def __init__(self, **kwargs):
        kwargs["platform"] = "steam"

        kwargs["input_controller"] = InputControllers.PYAUTOGUI  # <= Specify your InputController backend here 

        kwargs["window_name"] = "Nuclear Throne"

        kwargs["app_id"] = "242680"
        kwargs["app_args"] = None

        super().__init__(**kwargs)

        self.api_class = NuclearThroneAPI
        self.api_instance = None

Game Focus

Before dispatching inputs, an InputController instance will always verify that the game still has focus to prevent accidental inputs to other applications. If the game loses focus, the desired input will be cancelled.

[Rest of InputController documentation...]

The Bluetang CLI Tool

The bluetang command-line interface is an essential tool for working with the framework. Many major features can be accessed through bluetang commands.

Try it now by running bluetang in your terminal.

Available Commands

setup

Perform first time setup for the framework. Creates configuration files, dataset directories and installs OS-specific dependencies.

Usage

bluetang setup

Note: You can run this command again later, but it will deactivate all plugins, override configuration files and delete all datasets. You'll be asked for confirmation before proceeding.

grab_frames

Start an instance of the Frame Grabber. Not meant for direct use

Usage

bluetang grab_frames <width> <height> <x_offset> <y_offset>

Training a Context Classifier

A Context Classifier is a specialized machine learning model that can be trained using tools included with the Bluetang framework. Having a properly trained Context Classifier is essential for managing gameplay flows in your game agents. In this guide, we'll build a Context Classifier from scratch using Super Hexagon.

Analyzing the Game

To build an effective Context Classifier, spend time playing and studying the game, breaking it down into different sections (contexts). With Super Hexagon, we want our game agent to differentiate between 4 contexts: main_menu, level_select, game and game_over.

Launch the Game

bluetang launch SuperHexagon

[Rest of Context Classifier documentation...]

Loading the Context Classifier in your Game Agent

Add this to your game agent's constructor or frame handler setup function:

import offshoot
plugin_path = offshoot.config["file_paths"]["plugins"]

context_classifier_path = f"{plugin_path}/BluetangSuperHexagonGameAgentPlugin/files/ml_models/context_classifier.model"

from bluetang.machine_learning.context_classification.context_classifiers.cnn_inception_v3_context_classifier import CNNInceptionV3ContextClassifier
context_classifier = CNNInceptionV3ContextClassifier(input_shape=(384, 512, 3))  # Replace with the shape of your captured frames

context_classifier.prepare_generators()
context_classifier.load_classifier(context_classifier_path)

self.machine_learning_models["context_classifier"] = context_classifier

Using the Context Classifier at Runtime

context = self.machine_learning_models["context_classifier"].predict(game_frame.frame)