How to stop auto generation of id to resolve "DynamoDB only supports precision up to 38 digits" error

[Sheshank Kodam]

How to stop auto generation of id? I have autogenerate set to false but ecto still generates a random id and dynamo gives the following error.

2019-04-05 13:35:58.274 [debug] Request BODY: "{\"TableName\":\"Address\",\"Item\":{\"sk\":{\"S\":\"metadata\"},\"pk\":{\"S\":\"b_truck1\"},,\"id\":{\"N\":\"273197026155706542671679141313272484742\"}},\"ExpressionAttributeNames\":{\"#sk\":\"sk\",\"#pk\":\"pk\"},\"ConditionExpression\":\"attribute_not_exists(#pk) and attribute_not_exists(#sk)\"}"
** (ExAws.Error) ExAws Request Error! {"ValidationException", "DynamoDB only supports precision up to 38 digits"}
    (ecto_adapters_dynamodb) lib/ecto_adapters_dynamodb.ex:1164: Ecto.Adapters.DynamoDB.handle_error!/2
    (ecto_adapters_dynamodb) lib/ecto_adapters_dynamodb.ex:512: Ecto.Adapters.DynamoDB.insert/6
    (ecto) lib/ecto/repo/schema.ex:547: Ecto.Repo.Schema.apply/4
    (ecto) lib/ecto/repo/schema.ex:213: anonymous fn/14 in Ecto.Repo.Schema.do_insert/4
    priv/dynamo_db_repo/seed.exs:27: (file)
    (elixir) lib/code.ex:767: Code.require_file/2
    (mix) lib/mix/tasks/run.ex:147: Mix.Tasks.Run.run/5
    (mix) lib/mix/tasks/run.ex:86: Mix.Tasks.Run.run/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (mix) lib/mix/cli.ex:79: Mix.CLI.run_task/2
    (elixir) lib/code.ex:767: Code.require_file/2

Address table

defmodule Figment.DynamoDbRepo.Migrations.AddAddressTable do
  use Ecto.Migration

  @primary_key {:id, :binary_id, autogenerate: false}

  def change do
    create table(:Address, primary_key: false) do
      add :pk,   :string, primary_key: true  # primary composite key
      add :sk,   :string, range_key: true    # primary composite key
      timestamps()
    end
  end
end

Schema

defmodule Figment.Schema.Address do
  use Ecto.Schema

  schema "address" do
    field :pk,           :string
    field :sk,           :string
  end

  def changeset(address, params \\ %{}) do
    address
    |> Ecto.Changeset.cast(params, [:pk, :sk])
    |> Ecto.Changeset.validate_required([:pk, :sk])
  end
end

Query

{:ok, device2_md} = Repo.insert(Address.changeset(%Address{
  pk: "address2",
  sk: "metadata"}))

[Gilad Barkan]

What we do is add our own record ID to the changeset before inserting it.

[Sheshank Kodam]

Some random id?

[franko-franicevich]

Yes - Since it’s DynanoDB, a key/value store, you MUST have a unique ID field populated in some fashion. So, best thing to do is either generate something suitably random, or use another property of the data that is guaranteed to be unique. If it’s random, make sure it’s large, and really random. (ie, use crypto libraries to generate)

…

On 8/04/2019, at 3:36 PM, Sheshank Kodam ***@***.***> wrote: Some random id? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACgHi0ZZa7P4Q3GlnyEg24zAEjkK-6Chks5ve5pagaJpZM4cfm2i>.

[Sheshank Kodam]

As a workaround, I changed the schema to use pk as the primary key.

defmodule Figment.Schema.Address do
  @primary_key {:pk, :binary_id, autogenerate: true}
  @foreign_key_type :binary_id
  use Ecto.Schema

  schema "address" do
    field :sk,           :string
  end

  def changeset(address, params \\ %{}) do
    address
    |> Ecto.Changeset.cast(params, [:sk])
    |> Ecto.Changeset.validate_required([:pk, :sk])
  end
end

For us 'pk' is unique. So it works