logstash-plugins / logstash-input-s3

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Docs] Document workaround when using s3 private link endpoints with us-east-1

robbavey opened this issue · comments

As reported in #234, when using private link endpoints with us-east-1, the plugin will fail to read from the specified bucket, producing the following error:

  Error: Failed to open TCP connection to <our_bucket>.<our_vpc_endpoint_id>.s3.vpce.amazonaws.com:443 (initialize: name or service not known)
  Exception: Seahorse::Client::NetworkingError
  Stack: uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:943:in `block in connect'
org/jruby/ext/timeout/Timeout.java:114:in `timeout'
org/jruby/ext/timeout/Timeout.java:90:in `timeout'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/n

This is due to a problem fixed in aws/aws-sdk-ruby#2483, but not backport to v2 of the AWS-SDK, where us-east-1 will be stripped from the url when using the IAD regional endpoint plugin (which is included by default).

This should be able to be circumvented by setting s3_us_east_1_regional_endpoint=regional in the aws config file.