Usable Insecurity: Bug Preservation via Social Persistence

01-15-2024 | The product security take on edit lock bypass in FromSoftware's "Armored Core 6".

If everyone could make effective use of security practices and services, we'd all be better off. In terms of society's collective defensive posture. "Usable Security" examines human factors solutions toward that goal.

But what about the inverse: how might human factors enable system misuse?

We'll examine a light-hearted, real-world example of "usable insecurity": generating an invalid game state in "Armored Core 6", then making remediation more costly and complicated for the developer. Any player can escalate persistence for this harmless (no piracy/cheating/data-theft) bug using only the game's GUI. And the deployed defense has palatable tradeoffs.

While this post explores product security topics, it is not a deeply-technical exploit writeup. No reverse engineering, asset tampering, runtime instrumentation, social harm, or other potential EULA violations. We're interested in how "power users" can leverage a combo of customization bug and social sharing feature. Without technical skill or any knowledge of software internals.

With that air-tight disclaimer out of the way: let's control cosmetic state in a 3rd-person mech action game. Words I never anticipated typing on this blog - and you probably didn't anticipate reading today.

What's the broader context here, for people that don't play video games?

Armored Core 6: Fires of Rubicon (AC6) is a AAA game by FromSoftware, a critically-acclaimed studio. It marks both the revival and first commercial mega-success of a Japanese mecha action-simulation game series - which debuted in 1997. AC6 won the coveted title of "Best Action Game" at the 2023 Game Awards (think Oscars, but gaming).

At the time of this writing AC6 has sold over 2.8 million copies. The fun bug we're dissecting below affects a multi-million person userbase! A surprising scale in context of an industry-leading product. With potential lessons, for bugs in more consequential systems.

Understanding the User Experience

AC6's core gameplay loop is centered around micro-optimizing character "builds". Builds are combinations of individual combat robot parts, like legs or guns, that give the player an upper-hand in single-player missions and ranked multi-player matches. Build customization injects a stat-heavy strategy layer into what is otherwise a 3rd-person action game.

AC6 includes an absurdly deep cosmetic customization system for builds - a logo editor/creator is bundled-in. So a "build" is a player configuration with 2 highly-tunable components:

1. Robot parts (e.g. functional configuration)
2. Colors/logos (e.g. cosmetic configuration)

Players modify settings for these components using nested menus and, for parts, statistics dashboards.

Unlike part tuning, color/logo customization is purely aesthetic - it grants no competitive advantage in gameplay. It's also free of micro-transaction monetization (old-school, pro-consumer design).

Attack: Bypassing the Edit Lock to Generate Invalid Configs

AC6 players can choose part + color combos freely, with one major exception: official, developer-created character builds are "edit locked". Modifying any part in an official build requires removing the lock - resetting all special colors to default. The developers' security goal is:

• Bind config options: Make official color-schemes exclusive to official builds. This is a "good state" for builds.

The attacker's exploit goal and motivation are:

• Maximum config control: Remove official build restrictions. For both greater freedom and otherwise impossible builds. We can think of illegally-generated builds as "bad states".

So every build/configuration (some object serialized to local disk for persistence between player logins) has two good/valid states:

1. Locked - Official colors + official part set.
2. Unlocked - Custom colors + official part set or custom parts.

How are good states maintained when a [potentially nefarious] player drives all settings menus? With "locking" logic - a series of internal checks and UI warning banners - the game's menu system attempts to ensure a build never reaches a bad state:

3. Invalid - Official colors + custom parts.

We can visualize the locked and invalid states. The dotted-line box represents the edit lock's binding, solid gray lines represent color-to-part mappings, and the solid red line (right-hand diagram) is an invalid mapping (lock broken, thus binding broken):

So to create an invalid build, the player needs to bypass the edit lock. Turns out this is trivial! The AC6 community, including Reddit users and YouTube channels, found a simple GUI-only bypass just 2 weeks after the game's August 2023 launch. Steps:

1. Load an edit-locked official build: AC DESIGN > AC DATA > PRESET > (select official build) > LOAD > YES.

State: locked - parts cannot be edited without losing custom color scheme.

2. Remove the lock from the PAINT menu: AC DESIGN > PAINT > (note "An edit lock is in place for this data..." warning) > YES (remove lock, notice reset color scheme).

State: [intermediate/transitioning].

3. Exit the PAINT menu without saving changes: (press platform-specific back button) > Discard Edits and Quit.

State: invalid (lock bypassed) - keeps official color scheme, sans logo decals, but parts are now fully editable.

That's all it takes. Driving the menus to an unlocked edit state (YES remove lock) and undoing the corresponding color reset (PAINT > Discard Edits and Quit). Binding broken.

Note invalid builds enable otherwise impossible creations. For example, high headlight brightness values can be inherited from the bypassed official build - these values would be fixed and much lower otherwise.

Now that we understand the bug, let's look at a way to persist its results.

Battle: Patch Delivery vs. Social Replication

For a user, builds are labor-intensive outputs. The player could've spent hours tweaking parameters for a single build file. So the data is always saved locally.

Here's the kicker: we can replicate build data across other player's geographically-distributed systems. Fully opt-in for all parties. AC6 has "share codes":

• Player A uploads their build and gets a pseudo-random identifier (the "code").

• Player A transmits the code using some out-of-band channel (e.g. private DM, public forum post, etc).

• Player B redeems the code (non-exclusive) to download a local copy of player A's build data.

Built-in social sharing that works exactly like you'd expect - solid value add. You might've already guessed where this is going: sharing invalid builds. Not a terribly novel idea. Yet there are at least two big-picture scenarios worth table-topping out.

But first: opt-in is worth re-iterating - this design makes invalid builds user-controlled data. The bug is not "worm-able" to unsuspecting clients, copying invalid builds is a player choice. That's best case. Explicit user authorization is a critical last line of defense, for many software systems.

Now when player B redeems a code the shared build is re-locked on download. Player B must locally repeat the bypass to gain full edit control. So social sharing doesn't propagate the bypassed state itself, only the resulting invalid build. The existence of lock-on-download logic implies the edit lock is intended to bind configuration for shared builds - not just official builds.

We omitted this nuance earlier, for simplicity, but the ramification is important: the edit lock has the same failure mode for two distinct usecases, it fails to protect both developer-generated and user-generated content.

With those caveats out of the way, we'll examine our two persistence scenarios.

1. Data Rollback (Anti-patch)

As of patch version 1.05, the edit lock bypass hasn't been fixed. FromSoftware may well intentionally leave it in, we can't be certain. Maybe it could even be made an official achievement for good sport.

But suppose the developer opts to patch. The reasonable fix is to stop creation of new invalid/bypassed builds without deleting existing save data. So players don't experience sudden remote wipe of their meticulous customization work (e.g. data loss).

Then post-patch users (e.g. those running the latest release) can't create an invalid build locally. But they can download one created previously, via share code. That means fully-patched clients can roll back their game data to a "bad" state (see figure 3 above), which should have only been reachable in an unpatched client. And that bad state won't be fixed by future patches.

One way to prevent this scenario, without data loss, is selective blocking of share codes: disallow any which IDs an invalid build. For strong enforcement, blocking needs to be done server-side and on upload.

Let's imagine, for the sake of experiment, that FromSoftware chooses the nuclear option - server-side block all relevant social share codes and remote wipe all client-local invalid builds (forced user data loss). Single goal: no survivors.

2. Distributed Persistence (Anti-wipe)

When a player transmits a share code, they can do so on a broadcast channel - post it publicly for anyone to download. Broadcast sharing means attack asymmetry: a single share ID has the potential to replicate build data across N clients (figure 4).

Proof-of-Concept: G1DKZZV8GEUC is a PS5 share code for the "G13 Raven" official build (sans decals) - created using the above edit lock bypass. This build was an exclusive pre-order bonus, and can't be acquired otherwise. You can re-bypass and save a local copy for offline use :)

In the general case, as N gets higher the probability of a complete (all affected clients) remote delete gets lower. Maybe there's some legal/process/risk hurdle to data deletion. Maybe client connectivity is intermittent. Maybe offline, whole-system backups have already been made (think enterprise environment). Or maybe users intentionally keep systems offline to retain bad state (a common practice for game consoles with known-vulnerable firmware, for homebrew XOR piracy reasons). In every scenario, it's challenging to strongly guarantee deletion of N copies across N nodes as N gets huge (maybe multi-million).

Now AC6 is a high-margin product, both in unit sales (e.g. physical/digital game purchases) and platform subscription revenue (e.g. PS5 multiplayer SaaS). So it's unsurprising that the game shipped with anti-abuse countermeasures on day one.

Defense: Forced Remote Wipe on Connect

While poking at edge cases for this post, I came across build-error-detection logic and a corresponding prompt UI for forced remote wipe (no ability to opt-out if client is network-connected). To the best of my knowledge, this "kill switch" is only used for data either willingly deleted by the owner (e.g. original removed) or forcibly taken down by the developer (e.g. an inappropriate custom logo is reported, a content moderation team takes action).

This means FromSoftware already has production-tested capability for remote wipe after data hits disk, but not selective blocking beforehand. Although simple and likely effective in practice, remote wipe has at least two significant drawbacks:

• Fallible - The client stores an invalid config locally and must connect to game servers to remediate. Locally-generated invalid builds are fully functional as long as the game is played offline. So the theoretical worst case exposure time is indefinitely long.

  • Especially on PC, where native code and network traffic can be instrumented by technical users. Software-only, client-side security controls are limited to "best effort" guarantees. For some applications, "best effort" isn't "good enough".

• Destructive - Remote build wipe is forced data loss. For any user who redeemed a share code, not just the user who knowingly bypassed the lock. The cost of defense includes widespread "collateral damage".

  • For many software services, data loss risks a permanent reputation hit and significant customer turnover.

But let's not miss the forest for the trees. The FromSoft team shipped a smash hit, cross-platform game with few stability issues or technical glitches. They've implemented a capability for emergency removal of objectionable content and have thus far managed to prevent multiplayer cheating. While the security industry all-too-often learns from embarrassing public failures, this post examines finer details in what is an unequivocal success - by any measure.

Any now-harmless, historical examples of usable insecurity in games?

YouTube channel "Modern Vintage Gamer" announced a 23 year-old zero-day for bypassing Sony PS1 game-disk DRM. A movie-license game, released October 2000, included code to "hot swap" game discs. That's a common feature for multi-disk games of the CD era. To continue game story/progression while loading new assets from additional storage media.

Except this big-budget game had only 1 disc when it shipped.

The player enters a disk-swap debug mode if a special "cheat code" is manually input (think "backdoor"). Enable this mode, swap the current/official disk for any home-made copy of game B, and you've now launched game B. No signature validation, no region lock. Complete, usable DRM unlock for clients that won't be patched.

Takeaway

We explored a low-stakes but realistic "usable insecurity" case study. No cheating or piracy involved. Fully opt-in persistence. We saw how any non-technical user could create and preserve an invalid state, via interaction of two factors:

1. Bug (edit lock bypass) - GUI-only flow to create a local, invalid configuration.

2. Feature (social sharing) - Official mechanism to sync configurations across clients.

Nothing but respect for FromSoftware - have been an AC fan since 3 back on the PS2. This time it was all fun and [literally] games! Anything worth extrapolating?

Complex systems are difficult to reason about. Business logic often has exponential branching paths, e.g. "state explosion". So all real-world products ship with some non-zero number of bugs.

Most systems can be patched after bugs are discovered. But patching is non-trivial for complex systems and/or large populations. And when the technical barrier to system subversion is low, any client can become uncooperative. Sometimes coordinated entities can systematically degrade patching capability by increasing its cost. Leveraging unexpected bug-bug, bug-feature, or even feature-feature composition.

So it's best to fix bugs - especially the subset that are exploitable vulnerabilities - before shipping. Instead of attempting to put out the fires [of Rubicon]. Doubly true for "usable vulnerabilities", those reliable for low-skill attackers. Prevention isn't always possible but tends to minimize tradeoffs when it is.

Thus - assuming an architecture/design process already succeeded - we should invest in:

• Automated and Manual Testing - Maybe unit tests asserting on view-models returned from functions could have caught the AC6 bug. When automated tests are infeasible, both exploratory (e.g. pre-merge code review) and structured (e.g. pre-release QA testing) manual effort is exceptionally valuable. Sure - detecting certain flaws, like memory-safety violations, is best left to tools. But in the general case human smarts can find defects automation won't. At scale, repeatable processes can harness those smarts.

• Formal Verification - Limited, but important, classes of problems can be defined out of existence. Maybe a menu design based on Rust's type-state pattern could enable provably correct lock state. If invalid states could cause damage to property or loss of human life, it's wise to guide implementation with a formal model of the intended system and/or use machine-checked annotations to validate functional correctness of the code itself. Unlike testing, verification can prove the absence of certain bugs (within some bounds, e.g. micro-architectural attacks typically out-of-scope).

Security matters when the next breach might be "game over". Good luck out there.

02-22-2024 update: 1.06 patch notes, released 02-05-2024, include "Fixed a bug with AC data uploaded to the game server, where if the data was loaded and then deleted from the game server, the loaded data would be reverted to default paint and decals". We previously triggered this bug to capture the "kill switch" UI screenshot linked in discussion of defense. Given this fix, assume the kill switch is only for forced removal. Post text remains unedited, both share codes are still active, the focal bypass still works in 1.06!

Read a free technical book! I'm fulfilling a lifelong dream and writing a book. It's about developing secure and robust systems software. Although a work-in-progress, the book is freely available online (no paywalls or obligations): https://highassurance.rs/