error when set json schema with "date-time"
NiuBlibing opened this issue · comments
If set the json schema with "date-time" type, it will failed when crating pipeline.
json schema:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "email_gateway_log",
"type": "object",
"properties": {
"@timestamp": {
"type": "string",
"format": "date-time"
}
},
"required": []
}
error log:
2023-11-03T08:38:19.927345Z ERROR arroyo_compiler_service: Failed to compile: status: Unimplemented, message: "Failed to compile job: warning: some crates are on edition 2021 which defaults to `resolver = \"2\"`, but virtual workspaces default to `resolver = \"1\"`\nnote: to keep the current resolver, specify `workspace.resolver = \"1\"` in the workspace root's manifest\nnote: to use the edition 2021 resolver, specify `workspace.resolver = \"2\"` in the workspace root's manifest\nwarning: not within a suitable 'git' worktree!\nwarning: VERGEN_GIT_BRANCH set to default\nwarning: VERGEN_GIT_COMMIT_AUTHOR_EMAIL set to default\nwarning: VERGEN_GIT_COMMIT_AUTHOR_NAME set to default\nwarning: VERGEN_GIT_COMMIT_COUNT set to default\nwarning: VERGEN_GIT_COMMIT_DATE set to default\nwarning: VERGEN_GIT_COMMIT_MESSAGE set to default\nwarning: VERGEN_GIT_COMMIT_TIMESTAMP set to default\nwarning: VERGEN_GIT_DESCRIBE set to default\nwarning: VERGEN_GIT_SHA overidden\n Compiling types v1.0.0 (/opt/arroyo/build/pipeline/types)\n Compiling udfs v1.0.0 (/opt/arroyo/build/pipeline/udfs)\nwarning: use of deprecated method `chrono::DateTime::<Tz>::timestamp_nanos`: use `timestamp_nanos_opt()` instead\n --> /opt/arroyo/src/arroyo-worker/src/formats.rs:330:27\n |\n330 | Ok(from_nanos(raw.timestamp_nanos() as u128))\n | ^^^^^^^^^^^^^^^\n |\n = note: `#[warn(deprecated)]` on by default\n\nwarning: use of deprecated method `chrono::DateTime::<Tz>::timestamp_nanos`: use `timestamp_nanos_opt()` instead\n --> /opt/arroyo/src/arroyo-worker/src/formats.rs:358:41\n |\n358 | Ok(raw.map(|raw| from_nanos(raw.timestamp_nanos() as u128)))\n | ^^^^^^^^^^^^^^^\n\nwarning: `arroyo-worker` (lib) generated 2 warnings\nwarning: unused imports: `Duration`, `SystemTime`\n --> udfs/src/lib.rs:1:17\n |\n1 | use std::time::{SystemTime, Duration};\n | ^^^^^^^^^^ ^^^^^^^^\n |\n = note: `#[warn(unused_imports)]` on by default\n\nwarning: unused import: `std :: time :: SystemTime`\n --> types/src/lib.rs:1:5\n |\n1 | use std :: time :: SystemTime ;\n | ^^^^^^^^^^^^^^^^^^^^^^^^^\n |\n = note: `#[warn(unused_imports)]` on by default\n\nwarning: unused import: `std::str::FromStr`\n --> udfs/src/lib.rs:2:5\n |\n2 | use std::str::FromStr;\n | ^^^^^^^^^^^^^^^^^\n\nwarning: `types` (lib) generated 1 warning (run `cargo fix --lib -p types` to apply 1 suggestion)\nwarning: `udfs` (lib) generated 2 warnings (run `cargo fix --lib -p udfs` to apply 2 suggestions)\n Compiling pipeline v1.0.0 (/opt/arroyo/build/pipeline/pipeline)\nerror[E0425]: cannot find function `deserialize_rfc3339_datetime_opt` in crate `arroyo_worker`\n --> pipeline/src/main.rs:356:36\n |\n356 | #[serde(deserialize_with = \"arroyo_worker::deserialize_rfc3339_datetime_opt\")]\n | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not found in `arroyo_worker`\n\nFor more information about this error, try `rustc --explain E0425`.\nerror: could not compile `pipeline` (bin \"pipeline\") due to previous error\n", details: [], metadata: MetadataMap { headers: {} }
2023-11-03T08:38:19.928033Z ERROR arroyo_controller::states: fatal state error job_id="job_2BfsNbQC6z" state="Compiling" error_message="Compilation failed for this query. See the controller logs for more details." error="compilation request failed: Failed to compile job: warning: some crates are on edition 2021 which defaults to `resolver = \"2\"`, but virtual workspaces default to `resolver = \"1\"`\nnote: to keep the current resolver, specify `workspace.resolver = \"1\"` in the workspace root's manifest\nnote: to use the edition 2021 resolver, specify `workspace.resolver = \"2\"` in the workspace root's manifest\nwarning: not within a suitable 'git' worktree!\nwarning: VERGEN_GIT_BRANCH set to default\nwarning: VERGEN_GIT_COMMIT_AUTHOR_EMAIL set to default\nwarning: VERGEN_GIT_COMMIT_AUTHOR_NAME set to default\nwarning: VERGEN_GIT_COMMIT_COUNT set to default\nwarning: VERGEN_GIT_COMMIT_DATE set to default\nwarning: VERGEN_GIT_COMMIT_MESSAGE set to default\nwarning: VERGEN_GIT_COMMIT_TIMESTAMP set to default\nwarning: VERGEN_GIT_DESCRIBE set to default\nwarning: VERGEN_GIT_SHA overidden\n Compiling types v1.0.0 (/opt/arroyo/build/pipeline/types)\n Compiling udfs v1.0.0 (/opt/arroyo/build/pipeline/udfs)\nwarning: use of deprecated method `chrono::DateTime::<Tz>::timestamp_nanos`: use `timestamp_nanos_opt()` instead\n --> /opt/arroyo/src/arroyo-worker/src/formats.rs:330:27\n |\n330 | Ok(from_nanos(raw.timestamp_nanos() as u128))\n | ^^^^^^^^^^^^^^^\n |\n = note: `#[warn(deprecated)]` on by default\n\nwarning: use of deprecated method `chrono::DateTime::<Tz>::timestamp_nanos`: use `timestamp_nanos_opt()` instead\n --> /opt/arroyo/src/arroyo-worker/src/formats.rs:358:41\n |\n358 | Ok(raw.map(|raw| from_nanos(raw.timestamp_nanos() as u128)))\n | ^^^^^^^^^^^^^^^\n\nwarning: `arroyo-worker` (lib) generated 2 warnings\nwarning: unused imports: `Duration`, `SystemTime`\n --> udfs/src/lib.rs:1:17\n |\n1 | use std::time::{SystemTime, Duration};\n | ^^^^^^^^^^ ^^^^^^^^\n |\n = note: `#[warn(unused_imports)]` on by default\n\nwarning: unused import: `std :: time :: SystemTime`\n --> types/src/lib.rs:1:5\n |\n1 | use std :: time :: SystemTime ;\n | ^^^^^^^^^^^^^^^^^^^^^^^^^\n |\n = note: `#[warn(unused_imports)]` on by default\n\nwarning: unused import: `std::str::FromStr`\n --> udfs/src/lib.rs:2:5\n |\n2 | use std::str::FromStr;\n | ^^^^^^^^^^^^^^^^^\n\nwarning: `types` (lib) generated 1 warning (run `cargo fix --lib -p types` to apply 1 suggestion)\nwarning: `udfs` (lib) generated 2 warnings (run `cargo fix --lib -p udfs` to apply 2 suggestions)\n Compiling pipeline v1.0.0 (/opt/arroyo/build/pipeline/pipeline)\nerror[E0425]: cannot find function `deserialize_rfc3339_datetime_opt` in crate `arroyo_worker`\n --> pipeline/src/main.rs:356:36\n |\n356 | #[serde(deserialize_with = \"arroyo_worker::deserialize_rfc3339_datetime_opt\")]\n | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not found in `arroyo_worker`\n\nFor more information about this error, try `rustc --explain E0425`.\nerror: could not compile `pipeline` (bin \"pipeline\") due to previous error\n"
2023-11-03T08:38:19.929218Z INFO arroyo_controller::states: finished state machine job_id="job_2BfsNbQC6z"
Thanks for reporting the issue — we have a fix coming in #408
This should be fixed with #408 which will be released in 0.8.