Xcode 15 String catalog support

[Pavan]

Hi,

As apple announced a new feature in WWDC23 about the String catalog, Is there a plan to support that feature?

https://developer.apple.com/videos/play/wwdc2023/10155/

[Zev Eisenberg]

Doesn't the string catalog generate code for Swift and Obj-C already? What would you want SwiftGen to do that Xcode isn't doing already?

[Pavan]

@ZevEisenberg It's not basically Xcode that translates strings catalog into Localizable.strings during compilation time, but the issue with SwiftGen now is it expects Localizable.strings file to be expected in the beginning to generate swift code

[Zev Eisenberg]

Oh, so Xcode isn't doing code gen of constants for localizable strings into type-checked Swift and Obj-C like it is for color and images in asset catalogs? I haven't dug much into the string side of things yet.

[Pavan]

@ZevEisenberg Yes that's right, it will be great if SwiftGen supports string catalogs too

[Wolfgang Lutz]

Could this be solved by a macro that reads the string catalog during compilation?

[David Jennes]

Yes we'll need to add support for it. Is there a clear file format definition from Apple somewhere? We'll need to write down the requirements, and hopefully we can reuse our parsers from strings/stringsdict parsers.

Note: Macros have nothing to do with this, and won't help either because they can't read files, they only work with the existing swift code/syntax/….

[Gong Zhang]

The WWDC session video wwdc2023-10155 mentioned the String Catalog (*.xcstrings) is a simple JSON file.

As JSON files under the hood, they should also be easily diffable in source control.

Although I can't find any document about the format in details, the format is indeed pretty simple.

{
  "sourceLanguage" : "en",
  "strings" : {
    "Hello, world!" : { // <== key
      "localizations" : {
        "zh-Hans" : { // <== locale
          "stringUnit" : {
            "state" : "translated",
            "value" : "你好，世界" // <== value
          }
        }
      }
    }
  },
  "version" : "1.0"
}

[Wolfgang Lutz]

@gongzhang Sadly, it is way more complex than this.

so far, I got

import Foundation
import PlaygroundSupport

struct StringUnit: Decodable {
    let state: String
    let value: String
}

struct Variation: Decodable {
    let stringUnit: StringUnit
}

struct XCStrings: Decodable {
    let sourceLanguage: String
    let strings: [String: TranslationEntry]
}

struct TranslationEntry: Decodable {
    let comment: String?
    let extractionState: String
    let localizations: [String: Localization]?
}

struct Localization: Decodable {
    let stringUnit: StringUnit?
    let variations: Variations?
}

struct Variations: Decodable {
    let plural: PluralVariation?
    let device: DeviceVariation?
}

struct PluralVariation: Decodable {
    let one: Variation?
    let other: Variation?
    let zero: Variation?
}

struct DeviceVariation: Decodable {
    let ipod: Variation?
}

let path = Bundle.main.paths(forResourcesOfType: "xcstrings", inDirectory: nil).first!
let url = URL(filePath: path)
let data = try! Data(contentsOf: url)
let jsonData = try! JSONDecoder().decode(
    XCStrings.self,
    from: data
)

dump(jsonData)

to parse a part of the xcstrings files, but that are not all the possibilities, yet.

[Patryk Średziński]

Do we get any more further progress in this case?
Apple does generate it for Colors and Assets, but not for StringCatalogs.

I have found out that Apple is not providing the generator because the code is the source of truth. Whenever you add a string key in the code then it'll be automatically generated in the asset catalog.

But they support a backward compatibility, which means during compilation time they generate .strings and .stringdics file.
Can't find it yet, but still, would be nice to get this working in SwiftGen

[Andrii Moisol]

Hello, any progress here?

[Andrii Moisol]

Found .strings and .stringdicts file in compiled binary, but they seems to be in different format (maybe binary).

Localizable.strings:

Localizable.stringsdict:

[Wolfgang Lutz]

Found .strings and .stringdicts file in compiled binary, but they seems to be in different format (maybe binary).

Localizable.strings:

Localizable.stringsdict:

But does a xcstrings file with the same strings produce the same binaries?

[Zev Eisenberg]

That's a binary plist file! bplist at the start gives it away. You can rename it to have a .plist extension and open it in Xcode. plutil can read and write them, as well as JSONSerialization in Foundation.

[Andrii Moisol]

That's a binary plist file! bplist at the start gives it away. You can rename it to have a .plist extension and open it in Xcode. plutil can read and write them, as well as JSONSerialization in Foundation.

So, technically, SwiftGen can be used this way?

[Andrii Moisol]

As I see, apps that do not use string catalogs, also use bplist for .strings and .stringsdict files in compiled app

[Kondamon]

It is not necessarily needed but to create an enum out of the xcstring files you can use the built in JSON parser of SwiftGen. Credits @neilkachu
This has the advantage to easily reuse existing keys and not to change the code base when you already have implemented L10n.

It works this way:
xcstrings (json) -> strings
strings -> swift

TODO:

• Support multiple xcstring files (tables)
• Support all cases when using variations (plurals, devices, etc.)
• Wait for Swift Package Manager support

Use this as config file

json:
  inputs: Project/Assets/Localization
  filter: .+\.xcstrings$
  outputs:
    templatePath: ./utils/en-strings.stencil
    output: Project/Generated/generated-en.strings
strings:
  inputs: Project/Generated/generated-en.strings
  outputs:
    templatePath: ./utils/l21strings.stencil
    output: Project/Generated/Strings.swift

en-strings.stencil:

// Temporary generated en files for enum Generation
// Generated using SwiftGen — https://github.com/SwiftGen/SwiftGen
// Source: https://github.com/SwiftGen/SwiftGen/discussions/1071#discussioncomment-7209188

{% if files %}
{% macro fileBlock file %}
  {% call documentBlock file file.document %}
{% endmacro %}

{# process the file #}
{% macro documentBlock file document %}

  {% if document.metadata.type == "Dictionary" %}
  {% for key,value in document.metadata.properties %}
    {% call propertyBlock key value document.data %}
  {% endfor %}
 {% endif %}
{% endmacro %}

{# process the root dictionary #}
{% macro propertyBlock key metadata data %}
  {% set propertyName %}{{key|swiftIdentifier:"pretty"|lowerFirstWord|escapeReservedKeywords}}{% endset %}
  {% if propertyName == "strings" %}
        {% for propertyKey in data[key] %}
        {% set propertyValue %}{{data[key][propertyKey].localizations.en.stringUnit.value}}{% endset %}
        {% set propertyPluralValue %}{{data[key][propertyKey].localizations.en.variations.plural.other.stringUnit.value}}{% endset %}
        {% if propertyValue %}
"{{propertyKey}}" = "{{propertyValue}}";
        {% endif %}
        {% if propertyPluralValue %}
"{{propertyKey}}" = "{{propertyPluralValue}}";
        {% endif %}
      {% endfor %}
    {% else %}
    {% endif %}
{% endmacro %}

{% if files.count > 1 or param.forceFileNameEnum %}
{% for file in files %}
  {% filter indent:2," ",true %}{% call fileBlock file %}{% endfilter %}
{% endfor %}
{% else %}
{% call fileBlock files.first %}
{% endif %}

{% else %}
// No xcstring files found.
{% endif %}

l21strings.stencil:

// swiftlint:disable all
// Generated using SwiftGen — https://github.com/SwiftGen/SwiftGen

{% if tables.count > 0 %}
{% set accessModifier %}{% if param.publicAccess %}public{% else %}internal{% endif %}{% endset %}
import Foundation

// swiftlint:disable superfluous_disable_command file_length implicit_return prefer_self_in_static_references

// MARK: - Strings

{% macro parametersBlock types %}
  {%- for type in types -%}
    {%- if type == "String" -%}
    _ p{{forloop.counter}}: Any
    {%- else -%}
    _ p{{forloop.counter}}: {{type}}
    {%- endif -%}
    {{ ", " if not forloop.last }}
  {%- endfor -%}
{% endmacro %}
{% macro argumentsBlock types %}
  {%- for type in types -%}
    {%- if type == "String" -%}
    String(describing: p{{forloop.counter}})
    {%- elif type == "UnsafeRawPointer" -%}
    Int(bitPattern: p{{forloop.counter}})
    {%- else -%}
    p{{forloop.counter}}
    {%- endif -%}
    {{ ", " if not forloop.last }}
  {%- endfor -%}
{% endmacro %}
{% macro recursiveBlock table item %}
  {% for string in item.strings %}
  {% if not param.noComments %}
  {% for line in string.comment|default:string.translation|split:"\n" %}
  /// {{line}}
  {% endfor %}
  {% endif %}
  {% set translation string.translation|replace:'"','\"'|replace:'  ','\t' %}
  {% if string.types %}
  {{accessModifier}} static func {{string.name|swiftIdentifier:"pretty"|lowerFirstWord|escapeReservedKeywords}}({% call parametersBlock string.types %}) -> String {
    return {{enumName}}.tr("{{string.key}}", {%+ call argumentsBlock string.types %}, fallback: "{{translation}}")
  }
  {% elif param.lookupFunction %}
  {{accessModifier}} static var {{string.name|swiftIdentifier:"pretty"|lowerFirstWord|escapeReservedKeywords}}: String { return {{enumName}}.tr("{{string.key}}", fallback: "{{translation}}") }
  {% else %}
  {{accessModifier}} static let {{string.name|swiftIdentifier:"pretty"|lowerFirstWord|escapeReservedKeywords}} = {{enumName}}.tr("{{string.key}}", fallback: "{{translation}}")
  {% endif %}
  {% endfor %}
  {% for child in item.children %}
  {{accessModifier}} enum {{child.name|swiftIdentifier:"pretty"|escapeReservedKeywords}} {
    {% filter indent:2," ",true %}{% call recursiveBlock table child %}{% endfilter %}
  }
  {% endfor %}
{% endmacro %}
// swiftlint:disable explicit_type_interface function_parameter_count identifier_name line_length
// swiftlint:disable nesting type_body_length type_name vertical_whitespace_opening_braces
{% set enumName %}{{param.enumName|default:"L10n"}}{% endset %}
{{accessModifier}} enum {{enumName}} {
  {% if tables.count > 1 or param.forceFileNameEnum %}
  {% for table in tables %}
  {{accessModifier}} enum {{table.name|swiftIdentifier:"pretty"|escapeReservedKeywords}} {
    {% filter indent:2," ",true %}{% call recursiveBlock table.name table.levels %}{% endfilter %}
  }
  {% endfor %}
  {% else %}
  {% call recursiveBlock tables.first.name tables.first.levels %}
  {% endif %}
}
// swiftlint:enable explicit_type_interface function_parameter_count identifier_name line_length
// swiftlint:enable nesting type_body_length type_name vertical_whitespace_opening_braces

// MARK: - Implementation Details

extension {{enumName}} {
  private static func tr(_ key: String, _ args: CVarArg..., fallback value: String) -> String {
    {% if param.lookupFunction %}
    let format = {{ param.lookupFunction }}(key, table, value)
    {% else %}
    let format = {{param.bundle|default:"BundleToken.bundle"}}.localizedString(forKey: key, value: value, table: nil)
    {% endif %}
    return String(format: format, locale: Locale.current, arguments: args)
  }
}
{% if not param.bundle and not param.lookupFunction %}

// swiftlint:disable convenience_type
private final class BundleToken {
  static let bundle: Bundle = {
    #if SWIFT_PACKAGE
    return Bundle.module
    #else
    return Bundle(for: BundleToken.self)
    #endif
  }()
}
// swiftlint:enable convenience_type
{% endif %}
{% else %}
// No string found
{% endif %}

[lican]

@Kondamon
Thanks for your solution.But I have a problem with the generated-en.strings file. I need to save the escape characters in the string. How to fix it?

"Authorized rule" = "Authorized: "Priority to accounts with more than 5000 followers, View rules."";
 // Expected output
"Authorized rule" = "Authorized: \"Priority to accounts with more than 5000 followers, View rules.\"";