The Real Alpha

The last three weeks have been very productive. Most of the work went into performance improvements: startup time, memory usage during startup and runtime, CPU and GPU usage during search, and overall configuration processing efficiency have all been improved. In some areas, performance is up by several dozen percent.

Just as importantly, several new subsystems were added that will help me identify and fix problem areas more effectively going forward.

At the moment, there are still two noticeable issues: startup memory usage (usually around 1 to 2 GB, though it drops to about 500 MB after some time) and overall UI responsiveness. The causes of both are understood, but there is no quick and painless fix, so we will be addressing them gradually over the next six months.

New Input Type — Grandma Input

Nothing special, just another input method. It is only useful if the other methods are not working for some reason.

Alpha Testing for New Subsystems

Three new options have been added in Settings: Property Caching, Filesystem Storage, and File References. All of them are intended to improve performance in different ways.

These options are still being tested, so if you decide to enable them, be prepared for possible issues. If you prefer, you can also wait until these systems are officially released and enabled by default. Your participation and feedback would be very valuable.

Bugfixes and Improvements

• Fixed a rare crash during import (ReplaceId NullReferenceException).
• [UI] The secondary panel with logs and variables has been completely rewritten.
• [UI] All windows and overlays now open faster.
• [UI] The import dialog can now also perform synchronization.
• [Synchronization] Fixed a bug that caused the app to crash if synchronization was added inside a folder that was already being synchronized.
• [Scripts] Many improvements were made to the scripting subsystem — it now runs faster and uses memory more efficiently.
• [Scripts] The ImageSharp library has been added to the SDK. In some cases it is more convenient than OpenCV, and future APIs will likely be built around it.
• Improved the precision of all waits and delays; they are now accurate within 1 ms.

New Release!

Here’s a summary of the changes made to the app over the last 4 months.

A lot of work went into this release: improved C# scripting (for Triggers/Actions and Overlays), an upgraded Send Sequence system (which now fully replaces Send Input in terms of functionality), a better configuration subsystem, and much faster, more efficient Image Effects that can now downscale images for better performance.

To support the needs of the new and rapidly growing machine learning community, I also developed a separate app to make it easier to create and train Yolo models, which are fully supported by the ML Search feature. In my experience, this is much faster than preparing data manually, even though the app is still in an early alpha stage. For more information, see: Getting Started | EyeAuras Wiki.

Transition to a new UI technology

One of the main ongoing initiatives is the move to a new UI technology. This change is intended to solve current responsiveness issues in the software, such as switching between auras or creating new triggers/actions.

Although this is a slow transition and may take another 6–12 months, the expected result should bring major improvements. You can already see this in the new Import/Export and Bindings dialogs, which now use this technology. Over the coming months, more parts of the app will be replaced and rewritten one by one (Aura Tree, I’m looking at you).

What’s next?

Performance: The main focus of the next update is performance. I’ve completely redesigned the configuration system; it is currently in testing and should make app startup and overall operation much faster, especially on lower-end PCs. As mentioned above, I’ll also continue rewriting slower parts of the app, such as the Aura Tree, to improve the overall user experience.

Behavior Trees: Currently in development and testing, Behavior Trees are the next major feature to look forward to. They fundamentally change how you work with the app. They also use the new UI technology, so they are much more responsive :)

Stay tuned for updates, and thank you for your continued support and for using the software!

New features and improvements

• Send Sequence 2.0: Combines all action types into an improved input sequencing system.
• New C# Scripting engine: Now available in Triggers/Actions (alpha stage), expanding capabilities with new and improved APIs
• Bindings improvements: The Bindings editor has been rewritten for better performance and usability. It is much, much, much more responsive and flexible.
• Improved scripting with Visual Studio/Rider compatibility: Export and save your code as a C# project for more advanced development workflows.
• Import/Export dialog redesign: More responsive, more detailed, and more user-friendly. The Sync dialog is expected to be updated in one of the upcoming releases.

Performance and usability improvements

• Network messaging improvements: Infrastructure changes for better performance and reduced latency.
• Overlay UI and general UI performance: Significant changes for better performance and responsiveness.
• General performance improvements: Focused on improving overall user experience and software responsiveness.
• Aura Tree responsiveness: Interface improvements for smoother interaction with the Aura Tree.

Other improvements

• Image scaling in the Recalculate Image effect: Now supports scaling on both the X and Y axes. You can now also downscale images using effects to improve performance, and the app will still be able to calculate the correct coordinates of the detected image and click it. This works for ALL trigger types: ML, Color, and Image.
• New Image Effects: Added color range replacement and noise removal effects for better image analysis.
• ML Search improvements: Significant performance improvements and new features in ML Search. This is especially relevant for scripting, since scripts unlock the full potential of ML.

Bug fixes

• [Send Sequence/Input/Text]: Fixed a crash issue.
• [Volume Trigger]: Temporarily disabled due to conflicts.
• [Image Trigger]: Added new features for working with image templates.
• [C# Code Editor]: Performance improvements and future plans to move to the Monaco editor.
• [Bindings]: Multiple UI/UX changes for a better user experience.
• [Send Input]: Fixed issues with driver installation scripts.
• [WebUI Overlay]: Fixed rare crashes related to UI controls.
• [Bug Fix]: Fixed a window restore issue after minimizing.
• [Image/Color/Text Search]: Fixed various issues, including region selection and the FPS selection UI.
• [Overlays]: Fixed offset issues and crashes in certain scenarios.
• [Dependency Graph/Viewer]: Fixed display issues.
• [Network Message]: Fixed a crash caused by a server/client version mismatch.
• [Text Search]: Fixed a crash related to Tesseract memory access.

For more detailed information, please refer to the individual update changelogs available on the website.

Bug Fixes / Improvements

• [Send Sequence/Input/Text]: Fixed the crash "Attempt to delete something that was already deleted..."

Completely redesigned Import/Export dialogs

They’re now more responsive, more informative, easier to use, and hopefully less error-prone :)

P.S. The sync dialog is not ready yet — it will be included in a separate set of changes related to the publishing/sync system.

New C# scripting engine is now available in Triggers/Actions — alpha

Over the next few days I’ll publish more details, but in short: the C# engine that was already used in WebUI to take advantage of Blazor has now been ported and is also available in C# actions and triggers.

This means that everything you could previously use in WebUI is now also available in triggers/actions, including:

• access to new and improved APIs such as Interface IAuraTreeScriptingApi | EyeAuras Docs, Interface IVariablesScriptingApi | EyeAuras Docs, Interface IAuraAccessor | EyeAuras Docs, and more
  • These APIs are intended to remain compatible in future versions, and I’ll make every effort to keep them that way. If I ever need to introduce breaking changes, a dedicated compatibility layer will be added on top of the old APIs to avoid breaking existing code.
  • The plan is to let scripts inject a set of APIs, such as SendInputScriptingApi, to provide easy access to image analysis, input sending, and all other functionality currently available in EyeAuras.
  • The trigger/action/overlay APIs may still change for now, since I don’t yet have a good compatibility-layer solution for them.
• using multiple files to organize your source code more cleanly
• referencing other C#-based auras to access classes and methods implemented there — for example, you can create one aura with all your helper functions and reuse it across projects
• exporting a csproj and editing the code in Visual Studio / JetBrains Rider
• live project editing through Monitor, although this is less useful than it is for WebUI

Autocomplete is currently disabled. I haven’t decided yet whether to switch directly to the Monaco editor or improve the existing text editor first. I’ll share an update as soon as possible.

One of the biggest advantages of this change is that now, with a unified scripting system, any improvement will instantly apply everywhere — WebUI / C# actions / triggers and Behavior Trees.

P.S. Behavior Trees will most likely be released in December, and they’ll be based on the same engine. Stay tuned — it’s going to be very, very cool, I promise :)

New API documentation website

The new website hosting the EyeAuras API documentation is now available at https://docs.eyeauras.net.

This will be especially useful for users who use EyeAuras as a scripting platform. However, the Wiki remains the main resource for non-developers.

At the moment, the documentation is still missing a demos and examples section, which is an essential part of any documentation site. In December, linqse and I will be focusing on improving that area. In addition, most public interfaces and classes now include XML comments.

If any important information is missing, please let us know.

Improved error reporting

Understanding exactly what went wrong is critical when problems occur. Our new error reporting system collects all errors within an Action/Trigger or across the entire Aura.

These errors are hierarchical, meaning Aura-level errors include all errors from their actions/triggers, similar to folders. Users can also mark errors as read whenever they want.

Performance improvement — property caching (alpha)

A new feature has been introduced to significantly reduce disk reads/writes and slightly lower CPU usage. This feature is still being tested and can be enabled in settings via the Property Caching option.

Bug fixes / improvements

• [Volume Trigger]: Temporarily disabled due to conflicts with a subsystem.
• Safe Mode: Improved the mechanism that detects incorrect application exits.
• [Image Trigger]: Added the ability to copy the image sample to the clipboard and paste from it.
• [C# Code Editor]: Performance improvements make the editor more responsive. We still plan to eventually migrate to the Monaco editor.

CVATAAT Improvements (also known as YoloEase)

A collection of fixes and quality-of-life improvements. The biggest change is support for working with multiple projects at the same time — you can now save, load, and share configuration files, and train multiple models.

Oh, and yes — I renamed it to better reflect what the tool actually does.

Fixes and improvements

• Improved startup time, especially noticeable if you use scripts
• Added logging configuration options to minimize performance impact
• Multiple UX/UI improvements in the binding editor

Binding Improvements

Bindings | EyeAuras Wiki is a mechanism that lets you connect multiple action/trigger properties together. For example, you can take the coordinates of a detected image from Image Search and move the mouse there using Send Sequence.

I completely rewrote the Bindings Editor, where all of this is configured. It is now faster, more responsive, and generally easier to use.

CVATAAT Improvements

There are a LOT of different improvements and fixes here: performance, UX, and bug fixes. Listing every single one would not be very useful, so it is better to try it yourself: Getting Started | EyeAuras Wiki

I know there are already some successful use cases, so the technology does work, and training through CVATAAT is definitely easier than doing everything manually.

Bug Fixes / Improvements

• Fixed a memory leak that occurred in some cases when image effects were used

Send Sequence 2.0

Welcome to the improved Send Sequence action. It now combines all Send action types into one: it can send clicks, type text, move the mouse, and click the mouse, including support for links and anchors.

What’s all the fuss about?

For about 1.5 years, there were 3 separate Send action types: Send Input, Send Text, and Send Sequence.

Each one covered its own specific set of features:

• Send Input was mainly focused on clicking somewhere, such as an image location, anchors, and similar targets.
• Send Text supported several text input methods, such as typing character by character or using the clipboard.
• Send Sequence was always focused on “playing back” longer input sequences, but without the precision of Send Input.

That is now a thing of the past. The redesigned version has the same capabilities as all previous Send action types, and it is much easier to extend with new features and new blocks with any kind of functionality, for example:

• selecting a coordinate
• performing math operations
• using the result to move the mouse

What are the benefits?

The main benefit is that building input sequences is now much easier. For example, if you want to click a specific image, you no longer need to alternate between Send Sequence and Send Input.

I also plan to focus on improving Send Sequence going forward, making it more flexible instead of splitting time across 3 different Send action types.

What should I do with the old Send Input and Send Text actions?

From now on, Send Text and Send Input are effectively in maintenance mode. I will not be adding new options to them or actively working on them anymore.

At some point in the future, they will be fully deprecated and automatically converted into Send Sequence.

You can convert them manually at any time by clicking the Convert to Send Sequence button shown at the top of each Send action. For now, the conversion does not remove the original action, so you can verify that the new one behaves exactly as expected.

Improved image/color/text search: borderless mode

As the title suggests, I added a new option to all of these triggers that allows capturing any window without its border.

In addition, all actions that work by linking/anchoring to the position of a found image will take the missing border into account and adjust coordinates accordingly.

This makes tracking objects in applications that are not running in fullscreen mode easier than ever.

Improved color search

A new option has been added that allows searching for specific pixel blocks of a given color. There is a limited set of supported block sizes, such as 2x2, 4x4, and so on, and the code has been optimized for better overall performance.

Based on my tests, 4x4 blocks give the best results.

Keep in mind that under the hood, this mechanism is very similar to how image search works, so in some scenarios it may still be better to stick with image search. In other cases, though, this new color search mode really shines.

And of course, you can configure anchors/links to the found blocks as well.

Watch the video here

CVATAAT improvements

A fairly detailed documentation page has been added, and it should answer most of your configuration-related questions.

The tool is working and can already be very useful even with its current feature set.

Added support for training with Google Colab

Originally, the tool only allowed training models locally on your own PC using your own hardware. That has obvious limitations: you need to actually have the hardware.

Cloud training makes it possible to use enterprise-grade GPUs such as the A100 for faster and more efficient training in some cases.

It does come with its own limitations, but it can be very useful in many situations. Support is still somewhat limited for now—for example, auto-annotation is not yet available in this mode—but I will continue expanding it.

Bugfixes / Improvements

• [Changelog] Fixed a bug that caused changelogs to be displayed incorrectly, with the text getting “mixed” together
• [Offset, Anchors] Fixed a bug that caused crashes in some cases when adjusting Offset/Anchors in image/text/color search

Bug Fixes / Improvements

• [Overlays] Fixed how offsets work and resolved a crash that could occur in some cases when updating anchors/offsets.
• [Graph / Dependency Viewer] Fixed an issue that caused the graph to display incorrectly due to a missing JS file.

Improve Your Bot with Machine Learning-Based Image Analysis

Machine learning—especially object detection, classification, and segmentation—can dramatically expand what game bots are capable of. Imagine bots that can accurately identify in-game items, enemies, and terrain, or classify player behavior to make smarter and more effective decisions. The hard part is getting started: there is often a ton of information to sort through, and setup can be complicated.

Meet the CVAT auto-annotation tool — your new go-to tool for training ML models.

CVATAAT is here to make that process easier. Built for people who are new to botting, it helps you prepare and train your game data with minimal hassle. It’s free, runs on your own computer, and is focused specifically on gaming use cases. Just feed it your game images, label them, and CVATAAT does the heavy lifting by quickly training and delivering optimized models that you can then use in EyeAuras with the ML-Search trigger.

This is still an early alpha, so things may go wrong. Please report any issues so we can make the model preparation process as simple as possible together. I genuinely believe this is the future, and with this tool I want to clear some of the obstacles out of the way.

Bugfixes and Improvements

• [SendSequence] Added support for mouse wheel up/down. For now, this is available only in SendSequence.
• [Dependencies Viewer/Graph] Now uses a different method to render auras/dependencies. It should be much more responsive now.