S3 generate-presigned-url Request Parameters are ignored

[Chris]

Howdy!

At a high level, the issue is that you cannot supply all the properties used by GeneratePresignedUrlRequest via Amazonica's API. So things like setting ACL policies isn't currently possible.

(:require [amazonica.aws.s3 :as s3])

(def creds {:secret-key "foo" :access-key "bar"})

(s3/generate-presigned-url
    creds
    {:bucket-name          "mybucket"
     :method               "PUT"
     :expires              10000
     :key                  "my-key"
     :request-parameters {:x-amz-acl "public-read"}})

^ This silently drops the Request Parameters.

If I fall back to the Java SDK and set these things by hand, all works as expected

(let [request (-> (GeneratePresignedUrlRequest. bucket-name key)
                    (.withMethod method)
                    (.withExpiration expires))]
    (doseq [[key val] request-parameters]
      (.addRequestParameter request key val))
    (.generatePresignedUrl s3client request))

This is against Amazonica version 0.3.136

Fwiw, after a buncha noodilng around with a debugger, it seems the root of the problem is two fold:

1. amazonica.core/create-request-bean assumes all *Request style beans take a single argument (The relevant S3 methods fall outside of this pattern by taking two arguments)
2. amazonica.core/matches? expects the reflected methods to match the property name fairly closely (e.g. :key -> setKey()), but s3 is again the odd man out and uses a singular form of the property name (addRequestParameter(String key, String value))

Thanks for this library btw! 👍 It has been a good Clojure learning experience stepping through it ^_^

[Michael Cohen]

Unfortunately this is one of the corner cases where the underlying Java client deviates from the JavaBeans convention. 0.3.138 is on clojars and should fix this. Note that the expiry key is expiration and takes a java.util.Date or millis since the epoch.

[Kenneth Kalmer]

We've spotted a regression in when setting :response-headers as per #302. This is on 0.3.142:

ERROR clojure.lang.PersistentArrayMap cannot be cast to com.amazonaws.services.s3.model.ResponseHeaderOverrides
java.lang.ClassCastException: clojure.lang.PersistentArrayMap cannot be cast to com.amazonaws.services.s3.model.ResponseHeaderOverrides
        at amazonica.aws.s3$eval29997$fn__30037.invoke(s3.clj:223)
        at amazonica.core$coerce_value.invokeStatic(core.clj:397)
        at amazonica.core$coerce_value.invoke(core.clj:385)
        at amazonica.core$create_request_bean.invokeStatic(core.clj:654)
        at amazonica.core$create_request_bean.invoke(core.clj:646)
        at amazonica.core$prepare_args.invokeStatic(core.clj:786)
        at amazonica.core$prepare_args.invoke(core.clj:765)
        at amazonica.core$fn_call.invokeStatic(core.clj:871)
        at amazonica.core$fn_call.doInvoke(core.clj:864)
        at clojure.lang.RestFn.invoke(RestFn.java:442)
        at amazonica.core$intern_function$fn__29912.doInvoke(core.clj:1031)
        at clojure.lang.RestFn.invoke(RestFn.java:421)
       ....

The snippet of code based on #302 is:

    (let [creds {:access-key "open" 
                 :secret-key "sesame" 
                 :endpoint "us-east-1"}
          item {:bucket-name      "the-bucket"
                :key              key
                :method           "GET"
                :response-headers {:content-disposition "attachment"}
                :expiration       (-> 6 t/hours t/from-now)}
          url  (s3/generate-presigned-url creds item)]

      (.toString ^java.net.URL url))

t is clj-time, and we moved the value of :expiration into the item map where it was the third parameter to generate-presigned-url.

[Gary Chen]

We've spotted a regression in when setting :response-headers as per #302. This is on 0.3.142:

ERROR clojure.lang.PersistentArrayMap cannot be cast to com.amazonaws.services.s3.model.ResponseHeaderOverrides
java.lang.ClassCastException: clojure.lang.PersistentArrayMap cannot be cast to com.amazonaws.services.s3.model.ResponseHeaderOverrides
        at amazonica.aws.s3$eval29997$fn__30037.invoke(s3.clj:223)
        at amazonica.core$coerce_value.invokeStatic(core.clj:397)
        at amazonica.core$coerce_value.invoke(core.clj:385)
        at amazonica.core$create_request_bean.invokeStatic(core.clj:654)
        at amazonica.core$create_request_bean.invoke(core.clj:646)
        at amazonica.core$prepare_args.invokeStatic(core.clj:786)
        at amazonica.core$prepare_args.invoke(core.clj:765)
        at amazonica.core$fn_call.invokeStatic(core.clj:871)
        at amazonica.core$fn_call.doInvoke(core.clj:864)
        at clojure.lang.RestFn.invoke(RestFn.java:442)
        at amazonica.core$intern_function$fn__29912.doInvoke(core.clj:1031)
        at clojure.lang.RestFn.invoke(RestFn.java:421)
       ....

The snippet of code based on #302 is:

    (let [creds {:access-key "open" 
                 :secret-key "sesame" 
                 :endpoint "us-east-1"}
          item {:bucket-name      "the-bucket"
                :key              key
                :method           "GET"
                :response-headers {:content-disposition "attachment"}
                :expiration       (-> 6 t/hours t/from-now)}
          url  (s3/generate-presigned-url creds item)]

      (.toString ^java.net.URL url))

t is clj-time, and we moved the value of :expiration into the item map where it was the third parameter to generate-presigned-url.

Still having this issue on 0.3.164. The workaround I found is to import ResponseHeaderOverrides from amazon sdk and set the response headers.

(ns my-name-space
  (:require [amazonica.aws.s3               :as s3])
  (:import [com.amazonaws.services.s3.model ResponseHeaderOverrides]))

 
     (let [creds {:access-key "open" 
                  :secret-key "sesame" 
                  :endpoint "us-east-1"}
           response-headers (doto (ResponseHeaderOverrides.)
                               (.setContentDisposition "inline")
                               (.setContentType "text/html"))
           item {:bucket-name      "the-bucket"
                 :key              key
                 :method           "GET"
                 :response-headers response-headers
                 :expiration       (-> 6 t/hours t/from-now)}
           url  (s3/generate-presigned-url creds item)]
 
       (.toString ^java.net.URL url))