Controller button icons within text

[Reese Rivers]

As we've been talking on the Discord server, I'd like to propose here a suggestion for using controller button icons based on the player's input method. @InfoTeddy tells me that the input layer can tell the game which controller is in use, allowing the possibility to display e.g. Xbox buttons for an Xbox controller, Nintendo buttons for a Switch controller, etc. Usage of generic button icons could be possible as well.

These are some sprites I've drawn to demonstrate. The icons' base size are 10×10 pixels. The generic Start/Select icons are double-wide (20×10 pixels) as per @Dav999-v's suggestion.

Here's how they could look in action, along with guides to show how they would be aligned within Latin, Japanese and Chinese text:

(This is assuming 8x8, 8x12 and 12x12 font sizes respectively — I don't know if these sizes will be used in the end for the latter two, though it does seem likely.)

One issue that the Japanese and Chinese fonts don't have is that the 10x10 icons may be too tall to comfortably display on text with multiple lines (eg dialogue boxes).

In terms of implementation, I think it could be efficient to encode each button value as an internal code (e.g. Please press $B_USE now) and replacing the code with either text (e.g. ENTER, ACTION) or for buttons, the appropriate number of spaces for each font (1/2 for Latin and Chinese, 2/3 for Japanese), then rendering the appropriate button in that space. Even nicer if it can do it on the fly based on the last-used input method (like how Deltarune does it).

Please forgive me if this is too much to dump here, but I thought it was worth sharing.

[Ethan Lee]

The action set system can indeed identify specific hardware - however, the layer system actually makes it so you just get glyphs and not necessarily the product/vendor ID. This could be a good excuse to provide both bitmaps and descriptions though, so we can use the customized images here.

The text code is exactly the right thing to do, so definitely move forward with this idea - we can still support it even before action sets are integrated; we can just check which device is in use and hardcode the device/glyph dictionary for now.

[Dav999]

So it would need some thought as well about how these would (on a technical level) work with fonts, the text renderer, textboxes, etc. VVVVVV mostly already accounts for the possibility that not all characters have the same width, thanks to graphics.len, but there would be some annoying problems with actually doing so, that I can't list off the top of my head. So it helps if buttons can be drawn on top of fixed numbers of spaces.

So, just thinking out loud. As mentioned, the text would need to include some kind of token to indicate where the button goes (like that $B_USE). Maybe it would be simpler to restrict it a little, so translators can't change which button is displayed, but it's defined by the code when a certain button is expected in a certain string and which button it should be (like providing translators with Please press %s now which is already kinda common, or maybe Please press @ now).

And then the printing function (or some kind of special button-displaying wrapper for it) is told to expect this %s or @, and replaces it with the appropriate number of (non-breaking) spaces based on some options set in the language's metadata, so regular text rendering can take over without issues - while keeping in mind at which coordinates it encountered that token so the button image could be drawn on top after printing the text? And when drawing the image it also keeps in mind the language metadata, so it knows how to center the image according to the space given.

I dunno, we'll probably find a good way!

[Reese Rivers]

For reference, and so it doesn't get lost in the sands of time (aka the VVVVVV Discord code channel), here's an updated spritesheet with icons for analogue stick buttons, the Steam Deck layout, and Nintendo Joy-Con (which is now supported by Steam).


(to clarify the layout: first three rows are Nintendo/PlayStation/Xbox obv, and fourth and fifth are "generic" icons for unknown controllers, plus L4/R4/L5/R5 from the Steam Deck, which also reuses icons from all three console sets)
(I also edited this message a second time to make some of the icons look nicer)

[Reese Rivers]

However after talking it out a bit more with @Dav999-v an 8x8 button set would probably work just as well, if not better, since it wouldn't necessitate as much weird layout considerations. Buttons aren't as hard to represent in that space as I initially thought, so here's what I came up with:

Here's a 12x12 version as well, likely suitable for Chinese and possibly Korean:

Lemme know your thoughts!
Oh, and one more edit to clarify which glyphs are meant for which controller type:

• Switch Pro Controller: Non-coloured ABXY, +/-, L/R (not the "round" ones), ZL/ZR, L/R stick buttons
• Switch Joy-Con: "Generic" action buttons, +/-, L/R/ZL/ZR, "generic" stick button, SL/SR
• PlayStation: Circle/Cross/Square/Triangle, Start/Option, L1/R1/L2/R2/L3/R3
• Xbox: Coloured ABXY, Menu/View, LB/RB/LT/RT, L/R stick buttons (same as Switch)
• Steam Deck: Non-coloured ABXY, Menu/View, L/R 1 through 5 (incl thumbsticks)
• "Generic": Generic action buttons, plaintext "START/SELECT" instead of icons, rounded L/R (on top row to the right) and L2/R2, L/R stick buttons

The "?" button is a placeholder and probably not needed. These could all be encoded at codepoints U+EB00 through U+EB2F.

[Dav999]

I think everything's going to fall into place nicely! Some more thoughts on the technical side of fonts, text rendering and textboxes (especially since VFormat is now a thing):

We'll have different icon sheets to accommodate the different font sizes that we'll need to support, so that the button icon always fits inside the font size (ranging from the 8x8 font we already have, to 12x12 for CJK). All sheets support the same buttons, they're just at a different scale.

All buttons in the spritesheet would have a corresponding "name" (a #define or enum key) like BUTTON_TRIANGLE, BUTTON_PLAIN_A, BUTTON_COLORED_A, etc. They'd be font characters, assigned to private-use-area codepoints, starting at U+EB00 (where the B was chosen for Button), so if those unicode characters were to appear in a text, they'd show up as those colored icons.

We still need to convert actions (flip, interact, open menu, etc) to specific buttons for your controller, so there'd also be an enum (or list of #defines) for every action. So for example, ACTION_FLIP, ACTION_INTERACT, etc. There would then need to be a function that converts an action into an appropriate button string (like "ENTER") or icon (the U+EBxx codepoints from earlier).

That function would primarily be called inside VFormat (in call_with_button). You could then simply do something like this:

vformat_alloc("Press {button} to activate terminal", "button:but", BUTTON_INTERACT);

With varying results depending on your input method:

• "Press ENTER to activate terminal" (if you're on a keyboard and you haven't switched your interaction key)
• "Press E to activate terminal" (if you switched your interaction key)
• "Press <U+EB02> to activate terminal" (non-colored X icon because you're on the Steam Deck/Switch/etc)
• "Press <U+EB13> to activate terminal" (triangle icon because you have a PlayStation controller)

And so on!

Reese and I also experimented with modifying the current text renderer to prototype the icons in the game itself. The text renderer would need a big overhaul anyway to support CJK fonts, but in Graphics::print_char, you can use BlitSurfaceStandard instead of BlitSurfaceColoured to display a character as its original colors (something would need to be passed as an argument to know which characters to allow to be colored and which not to, and you definitely need to check if ct.colour is not 0xFF000000, otherwise the black text outlines break)

There's two possible solutions here, and I'm not sure which one is the best yet:

• The controller icons are separate from the fonts, so they're neatly categorized and reused for different fonts
• The controller icons are just hardcoded into each individual font, with a per-character-range property set in font.xml to determine that these need to be colored

The first one might be less flexible and require more code changes (including centering 8x8 icons into a possible 8x12 font for example), the second one might be a bit easier to implement once the fonts are made but things would be a bit less neatly categorized. But the second one also has another implication that will definitely be well-appreciated in custom levels: you'd be able to define any glyphs you like as basically a colored emoji, and have text boxes with multiple colors in them at the same time, and also custom graphics which you could just "type" into your level as roomtext which shows up in front of the player. Maybe another option would be to do it the first way (separate font and icon files) but to implement the icons as a fully-functional fallback font, so we'd have both the neat organization and lack of reuse, as well as the flexibility. But we'll have to see what the best solution will look like when implementing it.

[Reese Rivers]

Here are updated button icon sets as requested by Dav:
8×8

10×10

12×12

And here are the glyph identifier keys, also requested by Dav:

NINTENDO_DECK_A // Note that for the Deck, the icons are same as Nintendo but the layout is the same as Xbox
NINTENDO_DECK_B
NINTENDO_DECK_X
NINTENDO_DECK_Y
NINTENDO_PLUS
NINTENDO_MINUS
NINTENDO_L
NINTENDO_R
NINTENDO_ZL
NINTENDO_ZR
NINTENDO_XBOX_LSTICK
NINTENDO_XBOX_RSTICK
NINTENDO_SL
NINTENDO_SR
GENERIC_L
GENERIC_R

PLAYSTATION_CIRCLE
PLAYSTATION_CROSS
PLAYSTATION_TRIANGLE
PLAYSTATION_SQUARE
PLAYSTATION_START
PLAYSTATION_OPTIONS
PLAYSTATION_DECK_L1
PLAYSTATION_DECK_R1
PLAYSTATION_DECK_L2
PLAYSTATION_DECK_R2
PLAYSTATION_DECK_L3
PLAYSTATION_DECK_R3
DECK_L4
DECK_R4
DECK_L5
DECK_R5

XBOX_B
XBOX_A
XBOX_Y
XBOX_X
XBOX_DECK_VIEW
XBOX_DECK_MENU
XBOX_LB
XBOX_RB
XBOX_LT
XBOX_RT
NINTENDO_GENERIC_ACTIONRIGHT
NINTENDO_GENERIC_ACTIONDOWN
NINTENDO_GENERIC_ACTIONUP
NINTENDO_GENERIC_ACTIONLEFT
GENERIC_STICK
GENERIC_UNKNOWN

[Misa Elizabeth Kai]

According to Dav, this can be closed as all sub-tasks have been completed.