lin-credible / req

Simple Go HTTP client with Black Magic

Req

Simple Go HTTP client with Black Magic (Less code and More efficiency).

News

Brand-New version v3 is released, which is completely rewritten, bringing revolutionary innovations and many superpowers, try and enjoy :)

If you want to use the older version, check it out on v1 branch.

v2 is a transitional version, due to some breaking changes were introduced during optmize user experience

Table of Contents

• Features
• Get Started
• Debugging - Dump/Log/Trace
• Quick HTTP Test
• HTTP2 and HTTP1
• URL Path and Query Parameter
• Form Data
• Header and Cookie
• Body and Marshal/Unmarshal
• Custom Certificates
• Basic Auth and Bearer Token
• Download and Upload
• Auto-Decode
• Request and Response Middleware
• Redirect Policy
• Proxy
• Unix Socket
• Retry
• TODO List
• License

Features

• Simple and chainable methods for both client-level and request-level settings, and the request-level setting takes precedence if both are set.
• Powerful and convenient debug utilites, including debug logs, performance traces, and even dump the complete request and response content (see Debugging - Dump/Log/Trace).
• Easy making HTTP test with code instead of tools like curl or postman, req provide global wrapper methods and MustXXX to test API with minimal code (see Quick HTTP Test).
• Works fine with both HTTP/2 and HTTP/1.1, which HTTP/2 is preferred by default if server support, and you can also force HTTP/1.1 if you want (see HTTP2 and HTTP1).
• Detect the charset of response body and decode it to utf-8 automatically to avoid garbled characters by default (see Auto-Decode).
• Automatic marshal and unmarshal for JSON and XML content type and fully customizable (see Body and Marshal/Unmarshal).
• Exportable Transport, easy to integrate with existing http.Client, debug APIs with minimal code change.
• Easy Download and Upload.
• Easy set header, cookie, path parameter, query parameter, form data, basic auth, bearer token for both client and request level.
• Easy set timeout, proxy, certs, redirect policy, cookie jar, compression, keepalives etc for client.
• Support middleware before request sent and after got response (see Request and Response Middleware).

Get Started

Install

go get github.com/imroc/req/v3

Import

import "github.com/imroc/req/v3"

Basic Usage

// For test, you can create and send a request with the global default
// client, use DevMode to see all details, try and suprise :)
req.DevMode()
req.Get("https://api.github.com/users/imroc")

// Create and send a request with the custom client and settings
client := req.C(). // Use C() to create a client
    SetUserAgent("my-custom-client"). // Chainable client settings
    SetTimeout(5 * time.Second).
    DevMode()
resp, err := client.R(). // Use R() to create a request
    SetHeader("Accept", "application/vnd.github.v3+json"). // Chainable request settings
    SetPathParam("username", "imroc").
    SetQueryParam("page", "1").
    SetResult(&result).
    Get("https://api.github.com/users/{username}/repos")

Videos

• Get Started With Req (English, Youtube)
• 快速上手 req (Chinese, BiliBili)

API Reference

Checkout Quick API Reference for a brief and categorized list of the core APIs, for a more detailed and complete list of APIs, please refer to GoDoc.

Examples

Checkout more examples below or runnable examples in the examples directory.

Debugging - Dump/Log/Trace

Dump the Content

// Enable dump at client level, which will dump for all requests,
// including all content of request and response and output
// to stdout by default.
client := req.C().EnableDumpAll()
client.R().Get("https://httpbin.org/get")

/* Output
:authority: httpbin.org
:method: GET
:path: /get
:scheme: https
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36
accept-encoding: gzip

:status: 200
date: Wed, 26 Jan 2022 06:39:20 GMT
content-type: application/json
content-length: 372
server: gunicorn/19.9.0
access-control-allow-origin: *
access-control-allow-credentials: true

{
  "args": {},
  "headers": {
    "Accept-Encoding": "gzip",
    "Host": "httpbin.org",
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36",
    "X-Amzn-Trace-Id": "Root=1-61f0ec98-5958c02662de26e458b7672b"
  },
  "origin": "103.7.29.30",
  "url": "https://httpbin.org/get"
}
*/
	
// Customize dump settings with predefined and convenient settings at client level. 
client.EnableDumpAllWithoutBody(). // Only dump the header of request and response
    EnableDumpAllAsync(). // Dump asynchronously to improve performance
    EnableDumpAllToFile("reqdump.log") // Dump to file without printing it out
// Send request to see the content that have been dumped	
client.R().Get(url) 

// Enable dump with fully customized settings at client level.
opt := &req.DumpOptions{
            Output:         os.Stdout,
            RequestHeader:  true,
            ResponseBody:   true,
            RequestBody:    false,
            ResponseHeader: false,
            Async:          false,
        }
client.SetCommonDumpOptions(opt).EnableDumpAll()
client.R().Get("https://httpbin.org/get")

// Change settings dynamiclly
opt.ResponseBody = false
client.R().Get("https://httpbin.org/get")

// You can also enable dump at request level, which will not override client-level dumping,
// dump to the internal buffer and will not print it out by default, you can call `Response.Dump()`
// to get the dump result and print only if you want to, typically used in production, only record
// the content of the request when the request is abnormal to help us troubleshoot problems.
resp, err := client.R().EnableDump().SetBody("test body").Post("https://httpbin.org/post")
if err != nil {
    fmt.Println("err:", err)
	if resp.Dump() != "" {
        fmt.Println("raw content:")
        fmt.Println(resp.Dump())
    }
    return
}
if !resp.IsSuccess() { // Status code not beetween 200 and 299
    fmt.Println("bad status:", resp.Status)
    fmt.Println("raw content:")
    fmt.Println(resp.Dump())
    return
}

// Similarly, also support to customize dump settings with the predefined and convenient settings at request level.
resp, err = client.R().EnableDumpWithoutRequest().SetBody("test body").Post("https://httpbin.org/post")
// ...
resp, err = client.R().SetDumpOptions(opt).EnableDump().SetBody("test body").Post("https://httpbin.org/post")

Enable DebugLog for Deeper Insights

// Logging is enabled by default, but only output the warning and error message.
// Use `EnableDebugLog` to enable debug level logging.
client := req.C().EnableDebugLog()
client.R().Get("http://baidu.com/s?wd=req")
/* Output
2022/01/26 15:46:29.279368 DEBUG [req] GET http://baidu.com/s?wd=req
2022/01/26 15:46:29.469653 DEBUG [req] charset iso-8859-1 detected in Content-Type, auto-decode to utf-8
2022/01/26 15:46:29.469713 DEBUG [req] <redirect> GET http://www.baidu.com/s?wd=req
...
*/

// SetLogger with nil to disable all log
client.SetLogger(nil)

// Or customize the logger with your own implementation.
client.SetLogger(logger)

Enable Trace to Analyze Performance

// Enable trace at request level
client := req.C()
resp, err := client.R().EnableTrace().Get("https://api.github.com/users/imroc")
if err != nil {
	log.Fatal(err)
}
trace := resp.TraceInfo() // Use `resp.Request.TraceInfo()` to avoid unnecessary struct copy in production.
fmt.Println(trace.Blame()) // Print out exactly where the http request is slowing down.
fmt.Println("----------")
fmt.Println(trace) // Print details

/* Output
the request total time is 2.562416041s, and costs 1.289082208s from connection ready to server respond frist byte
--------
TotalTime         : 2.562416041s
DNSLookupTime     : 445.246375ms
TCPConnectTime    : 428.458µs
TLSHandshakeTime  : 825.888208ms
FirstResponseTime : 1.289082208s
ResponseTime      : 1.712375ms
IsConnReused:     : false
RemoteAddr        : 98.126.155.187:443
*/

// Enable trace at client level
client.EnableTraceAll()
resp, err = client.R().Get(url)
// ...

DevMode

If you want to enable all debug features (dump, debug log and tracing), just call DevMode():

client := req.C().DevMode()
client.R().Get("https://imroc.cc")

Quick HTTP Test

Test with Global Wrapper Methods

req wrap methods of both Client and Request with global methods, which is delegated to the default client behind the scenes, so you can just treat the package name req as a Client or Request to test quickly without create one explicitly.

// Call the global methods just like the Client's method,
// so you can treat package name `req` as a Client, and
// you don't need to create any client explicitly.
req.SetTimeout(5 * time.Second).
	SetCommonBasicAuth("imroc", "123456").
	SetCommonHeader("Accept", "text/xml").
	SetUserAgent("my api client").
	DevMode()

// Call the global method just like the Request's method,
// which will create request automatically using the default
// client, so you can treat package name `req` as a Request,
// and you don't need to create any request and client explicitly.
req.SetQueryParam("page", "2").
	SetHeader("Accept", "application/json"). // Override client level settings at request level.
	Get("https://httpbin.org/get")

Test with MustXXX

Use MustXXX to ignore error handling during test, make it possible to complete a complex test with just one line of code:

fmt.Println(req.DevMode().R().MustGet("https://imroc.cc").TraceInfo())

HTTP2 and HTTP1

Req works fine both with HTTP/2 and HTTP/1.1, HTTP/2 is preferred by default if server support, which is negotiated by TLS handshake.

You can force using HTTP/1.1 if you want.

client := req.C().EnableForceHTTP1().EnableDumpAllWithoutBody()
client.R().MustGet("https://httpbin.org/get")
/* Output
GET /get HTTP/1.1
Host: httpbin.org
User-Agent: req/v3 (https://github.com/imroc/req)
Accept-Encoding: gzip

HTTP/1.1 200 OK
Date: Tue, 08 Feb 2022 02:30:18 GMT
Content-Type: application/json
Content-Length: 289
Connection: keep-alive
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
*/

And also you can force using HTTP/2 if you want, will return error if server does not support:

client := req.C().EnableForceHTTP2()
client.R().MustGet("https://baidu.com")
/* Output
panic: Get "https://baidu.com": server does not support http2, you can use http/1.1 which is supported
*/

URL Path and Query Parameter

Path Parameter

Use SetPathParam or SetPathParams to replace variable in the url path:

client := req.C().DevMode()

client.R().
    SetPathParam("owner", "imroc"). // Set a path param, which will replace the vairable in url path
    SetPathParams(map[string]string{ // Set multiple path params at once 
        "repo": "req",
        "path": "README.md",
    }).Get("https://api.github.com/repos/{owner}/{repo}/contents/{path}") // path parameter will replace path variable in the url
/* Output
2022/01/23 14:43:59.114592 DEBUG [req] GET https://api.github.com/repos/imroc/req/contents/README.md
...
*/

// You can also set the common PathParam for every request on client
client.SetCommonPathParam(k1, v1).SetCommonPathParams(pathParams)
	
resp1, err := client.Get(url1)
...

resp2, err := client.Get(url2)
...

Query Parameter

Use SetQueryParam, SetQueryParams or SetQueryString to append url query parameter:

client := req.C().DevMode()

// Set query parameter at request level.
client.R().
    SetQueryParam("a", "a"). // Set a query param, which will be encoded as query parameter in url
    SetQueryParams(map[string]string{ // Set multiple query params at once 
        "b": "b",
        "c": "c",
    }).SetQueryString("d=d&e=e"). // Set query params as a raw query string
    Get("https://api.github.com/repos/imroc/req/contents/README.md?x=x")
/* Output
2022/01/23 14:43:59.114592 DEBUG [req] GET https://api.github.com/repos/imroc/req/contents/README.md?x=x&a=a&b=b&c=c&d=d&e=e
...
*/

// You can also set the query parameter at client level.
client.SetCommonQueryParam(k, v).
    SetCommonQueryParams(queryParams).
    SetCommonQueryString(queryString).
	
resp1, err := client.Get(url1)
...
resp2, err := client.Get(url2)
...

// Add query parameter with multiple values at request level.
client.R().AddQueryParam("key", "value1").AddQueryParam("key", "value2").Get("https://httpbin.org/get")
/* Output
2022/02/05 08:49:26.260780 DEBUG [req] GET https://httpbin.org/get?key=value1&key=value2
...
 */


// Multiple values also supported at client level.
client.AddCommonQueryParam("key", "value1").AddCommonQueryParam("key", "value2")

Form Data

client := req.C().EnableDumpAllWithoutResponse()
client.R().SetFormData(map[string]string{
    "username": "imroc",
    "blog":     "https://imroc.cc",
}).Post("https://httpbin.org/post")
/* Output
:authority: httpbin.org
:method: POST
:path: /post
:scheme: https
content-type: application/x-www-form-urlencoded
accept-encoding: gzip
user-agent: req/v2 (https://github.com/imroc/req)

blog=https%3A%2F%2Fimroc.cc&username=imroc
*/

// Multi value form data
v := url.Values{
    "multi": []string{"a", "b", "c"},
}
client.R().SetFormDataFromValues(v).Post("https://httpbin.org/post")
/* Output
:authority: httpbin.org
:method: POST
:path: /post
:scheme: https
content-type: application/x-www-form-urlencoded
accept-encoding: gzip
user-agent: req/v2 (https://github.com/imroc/req)

multi=a&multi=b&multi=c
*/

// You can also set form data in client level
client.SetCommonFormData(m)
client.SetCommonFormDataFromValues(v)

GET, HEAD, and OPTIONS requests ignores form data by default

Header and Cookie

Set Header

// Let's dump the header to see what's going on
client := req.C().EnableForceHTTP1().EnableDumpAllWithoutResponse() 

// Send a request with multiple headers and cookies
client.R().
    SetHeader("Accept", "application/json"). // Set one header
    SetHeaders(map[string]string{ // Set multiple headers at once 
        "My-Custom-Header": "My Custom Value",
        "User":             "imroc",
    }).Get("https://httpbin.org/get")

/* Output
GET /get HTTP/1.1
Host: httpbin.org
User-Agent: req/v3 (https://github.com/imroc/req)
Accept: application/json
My-Custom-Header: My Custom Value
User: imroc
Accept-Encoding: gzip
*/

// You can also set the common header and cookie for every request on client.
client.SetCommonHeader(header).SetCommonHeaders(headers)

resp1, err := client.R().Get(url1)
...
resp2, err := client.R().Get(url2)
...

Set Cookie

// Let's dump the header to see what's going on
client := req.C().EnableForceHTTP1().EnableDumpAllWithoutResponse() 

// Send a request with multiple headers and cookies
client.R().
    SetCookies(
		&http.Cookie{
            Name:     "testcookie1",
            Value:    "testcookie1 value",
            Path:     "/",
            Domain:   "baidu.com",
            MaxAge:   36000,
            HttpOnly: false,
            Secure:   true,
        },
        &http.Cookie{
            Name:     "testcookie2",
            Value:    "testcookie2 value",
            Path:     "/",
            Domain:   "baidu.com",
            MaxAge:   36000,
            HttpOnly: false,
            Secure:   true,
        },
    ).Get("https://httpbin.org/get")

/* Output
GET /get HTTP/1.1
Host: httpbin.org
User-Agent: req/v3 (https://github.com/imroc/req)
Cookie: testcookie1="testcookie1 value"; testcookie2="testcookie2 value"
Accept-Encoding: gzip
*/

// You can also set the common cookie for every request on client.
client.SetCommonCookies(cookie1, cookie2, cookie3)

resp1, err := client.R().Get(url1)
...
resp2, err := client.R().Get(url2)

You can also customize the CookieJar:

// Set your own http.CookieJar implementation
client.SetCookieJar(jar)

// Set to nil to disable CookieJar
client.SetCookieJar(nil)

Body and Marshal/Unmarshal

Request Body

// Create a client that dump request
client := req.C().EnableDumpAllWithoutResponse()
// SetBody accepts string, []byte, io.Reader, use type assertion to
// determine the data type of body automatically. 
client.R().SetBody("test").Post("https://httpbin.org/post")
/* Output
:authority: httpbin.org
:method: POST
:path: /post
:scheme: https
accept-encoding: gzip
user-agent: req/v2 (https://github.com/imroc/req)

test
*/

// If it cannot determine, like map and struct, then it will wait
// and marshal to JSON or XML automatically according to the `Content-Type`
// header that have been set before or after, default to json if not set.
type User struct {
    Name  string `json:"name"`
    Email string `json:"email"`
}
user := &User{Name: "imroc", Email: "[e-mail]"}
client.R().SetBody(user).Post("https://httpbin.org/post")
/* Output
:authority: httpbin.org
:method: POST
:path: /post
:scheme: https
content-type: application/json; charset=utf-8
accept-encoding: gzip
user-agent: req/v2 (https://github.com/imroc/req)

{"name":"imroc","email":"[e-mail]"}
*/


// You can use more specific methods to avoid type assertions and improves performance,
client.R().SetBodyJsonString(`{"username": "imroc"}`).Post("https://httpbin.org/post")
/*
:authority: httpbin.org
:method: POST
:path: /post
:scheme: https
content-type: application/json; charset=utf-8
accept-encoding: gzip
user-agent: req/v2 (https://github.com/imroc/req)

{"username": "imroc"}
*/

// Marshal body and set `Content-Type` automatically without any guess
cient.R().SetBodyXmlMarshal(user).Post("https://httpbin.org/post")
/* Output
:authority: httpbin.org
:method: POST
:path: /post
:scheme: https
content-type: text/xml; charset=utf-8
accept-encoding: gzip
user-agent: req/v2 (https://github.com/imroc/req)

<User><Name>imroc</Name><Email>[e-mail]</Email></User>
*/

Response Body

// Define success body struct
type User struct {
    Name string `json:"name"`
    Blog string `json:"blog"`
}
// Define error body struct
type ErrorMessage struct {
    Message string `json:"message"`
}
// Create a client and dump body to see details
client := req.C().EnableDumpAllWithoutHeader()

// Send a request and unmarshal result automatically according to
// response `Content-Type`
user := &User{}
errMsg := &ErrorMessage{}
resp, err := client.R().
	SetResult(user). // Set success result
	SetError(errMsg). // Set error result
	Get("https://api.github.com/users/imroc")
if err != nil {
    log.Fatal(err)
}
fmt.Println("----------")

if resp.IsSuccess() { // status `code >= 200 and <= 299` is considered as success
	// Must have been marshaled to user if no error returned before
    fmt.Printf("%s's blog is %s\n", user.Name, user.Blog)
} else if resp.IsError() { // status `code >= 400` is considered as error
	// Must have been marshaled to errMsg if no error returned before
    fmt.Println("got error:", errMsg.Message) 
} else {
    log.Fatal("unknown http status:", resp.Status)
}
/* Output
{"login":"imroc","id":7448852,"node_id":"MDQ6VXNlcjc0NDg4NTI=","avatar_url":"https://avatars.githubusercontent.com/u/7448852?v=4","gravatar_id":"","url":"https://api.github.com/users/imroc","html_url":"https://github.com/imroc","followers_url":"https://api.github.com/users/imroc/followers","following_url":"https://api.github.com/users/imroc/following{/other_user}","gists_url":"https://api.github.com/users/imroc/gists{/gist_id}","starred_url":"https://api.github.com/users/imroc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/imroc/subscriptions","organizations_url":"https://api.github.com/users/imroc/orgs","repos_url":"https://api.github.com/users/imroc/repos","events_url":"https://api.github.com/users/imroc/events{/privacy}","received_events_url":"https://api.github.com/users/imroc/received_events","type":"User","site_admin":false,"name":"roc","company":"Tencent","blog":"https://imroc.cc","location":"China","email":null,"hireable":true,"bio":"I'm roc","twitter_username":"imrocchan","public_repos":129,"public_gists":0,"followers":362,"following":151,"created_at":"2014-04-30T10:50:46Z","updated_at":"2022-01-24T23:32:53Z"}
----------
roc's blog is https://imroc.cc
*/

// Or you can also unmarshal response later
if resp.IsSuccess() {
    err = resp.Unmarshal(user)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("%s's blog is %s\n", user.Name, user.Blog)
} else {
    fmt.Println("bad response:", resp)
}

// Also, you can get the raw response and Unmarshal by yourself
yaml.Unmarshal(resp.Bytes())

Customize JSON&XML Marshal/Unmarshal

// Example of registering json-iterator
import jsoniter "github.com/json-iterator/go"

json := jsoniter.ConfigCompatibleWithStandardLibrary

client := req.C().
	SetJsonMarshal(json.Marshal).
	SetJsonUnmarshal(json.Unmarshal)

// Similarly, XML functions can also be customized
client.SetXmlMarshal(xmlMarshalFunc).SetXmlUnmarshal(xmlUnmarshalFunc)

Disable Auto-Read Response Body

Response body will be read into memory if it's not a download request by default, you can disable it if you want (normally you don't need to do this).

client.DisableAutoReadResponse()

resp, err := client.R().Get(url)
if err != nil {
	log.Fatal(err)
}
io.Copy(dst, resp.Body)

Custom Certificates

client := req.R()

// Set root cert and client cert from file path
client.SetRootCertsFromFile("/path/to/root/certs/pemFile1.pem", "/path/to/root/certs/pemFile2.pem", "/path/to/root/certs/pemFile3.pem"). // Set root cert from one or more pem files
    SetCertFromFile("/path/to/client/certs/client.pem", "/path/to/client/certs/client.key") // Set client cert and key cert file
	
// You can also set root cert from string
client.SetRootCertFromString("-----BEGIN CERTIFICATE-----XXXXXX-----END CERTIFICATE-----")

// And set client cert with 
cert1, err := tls.LoadX509KeyPair("/path/to/client/certs/client.pem", "/path/to/client/certs/client.key")
if err != nil {
    log.Fatalf("ERROR client certificate: %s", err)
}
// ...

// you can add more certs if you want
client.SetCerts(cert1, cert2, cert3) 

Basic Auth and Bearer Token

client := req.C()

// Set basic auth for all request
client.SetCommonBasicAuth("imroc", "123456")

// Set bearer token for all request
client.SetCommonBearerAuthToken("MDc0ZTg5YmU4Yzc5MjAzZGJjM2ZiMzkz")

// Set basic auth for a request, will override client's basic auth setting.
client.R().SetBasicAuth("myusername", "mypassword").Get("https://api.example.com/profile")

// Set bearer token for a request, will override client's bearer token setting.
client.R().SetBearerToken("NGU1ZWYwZDJhNmZhZmJhODhmMjQ3ZDc4").Get("https://api.example.com/profile")

Download and Upload

Download

// Create a client with default download direcotry
client := req.C().SetOutputDirectory("/path/to/download").EnableDumpAllWithoutResponseBody()

// Download to relative file path, this will be downloaded
// to /path/to/download/test.jpg
client.R().SetOutputFile("test.jpg").Get(url)

// Download to absolute file path, ignore the output directory
// setting from Client
client.R().SetOutputFile("/tmp/test.jpg").Get(url)

// You can also save file to any `io.WriteCloser`
file, err := os.Create("/tmp/test.jpg")
if err != nil {
	fmt.Println(err)
	return
}
client.R().SetOutput(file).Get(url)

Download Callback

You can set DownloadCallback if you want to show download progress:

client := req.C()
client.R().
	SetOutputFile("test.gz").
	SetUploadCallback(func(info req.UploadInfo) {
        fmt.Printf("downloaded %.2f%%\n", float64(info.DownloadedSize)/float64(info.Response.ContentLength)*100.0)
    }).Post("https://exmaple.com/upload")
/* Output
downloaded 17.92%
downloaded 41.77%
downloaded 67.71%
downloaded 98.89%
downloaded 100.00%
*/

1. info.Response.ContentLength could be 0 or -1 when the total size is unknown.
2. DownloadCallback will be invoked at least every 200ms by default, you can customize the minimal invoke interval using SetDownloadCallbackWithInterval.

Multipart Upload

client := req.C().EnableDumpAllWithoutRequestBody() // Request body contains unreadable binary, do not dump

client.R().SetFile("pic", "test.jpg"). // Set form param name and filename
    SetFile("pic", "/path/to/roc.png"). // Multiple files using the same form param name
    SetFiles(map[string]string{ // Set multiple files using map
        "exe": "test.exe",
        "src": "main.go",
    }).
    SetFormData(map[string]string{ // Set form data while uploading
        "name":  "imroc",
        "email": "[e-mail]",
    }).
	SetFromDataFromValues(values). // You can also set form data using `url.Values`
    Post("http://127.0.0.1:8888/upload")

// You can also use io.Reader to upload
avatarImgFile, _ := os.Open("avatar.png")
client.R().SetFileReader("avatar", "avatar.png", avatarImgFile).Post(url)
*/

Upload Callback

You can set UploadCallback if you want to show upload progress:

client := req.C()
client.R().
	SetFile("excel", "test.xlsx").
	SetUploadCallback(func(info req.UploadInfo) {
        fmt.Printf("%q uploaded %.2f%%\n", info.FileName, float64(info.UploadedSize)/float64(info.FileSize)*100.0)
    }).Post("https://exmaple.com/upload")
/* Output
"test.xlsx" uploaded 7.44%
"test.xlsx" uploaded 29.78%
"test.xlsx" uploaded 52.08%
"test.xlsx" uploaded 74.47%
"test.xlsx" uploaded 96.87%
"test.xlsx" uploaded 100.00%
*/

UploadCallback will be invoked at least every 200ms by default, you can customize the minimal invoke interval using SetUploadCallbackWithInterval.

Auto-Decode

Req detect the charset of response body and decode it to utf-8 automatically to avoid garbled characters by default.

Its principle is to detect Content-Type header at first, if it's not the text content type (json, xml, html and so on), req will not try to decode. If it is, then req will try to find the charset information. And req also will try to sniff the body's content to determine the charset if the charset information is not included in the header, if sniffed out and not utf-8, then decode it to utf-8 automatically, and req will not try to decode if the charset is not sure, just leave the body untouched.

You can also disable it if you don't need or care a lot about performance:

client.DisableAutoDecode()

And also you can make some customization:

// Try to auto-detect and decode all content types (some server may return incorrect Content-Type header)
client.SetAutoDecodeAllContentType()

// Only auto-detect and decode content which `Content-Type` header contains "html" or "json"
client.SetAutoDecodeContentType("html", "json")

// Or you can customize the function to determine whether to decode
fn := func(contentType string) bool {
    if regexContentType.MatchString(contentType) {
        return true
    }
    return false
}
client.SetAutoDecodeContentTypeFunc(fn)

Request and Response Middleware

client := req.C()

// Registering Request Middleware
client.OnBeforeRequest(func(c *req.Client, r *req.Request) error {
	// You can access Client and current Request object to do something
	// as you need

    return nil  // return nil if it is success
  })

// Registering Response Middleware
client.OnAfterResponse(func(c *req.Client, r *req.Response) error {
    // You can access Client and current Response object to do something
    // as you need

    return nil  // return nil if it is success
  })

Redirect Policy

client := req.C().EnableDumpAllWithoutResponse()

client.SetRedirectPolicy(
    // Only allow up to 5 redirects
    req.MaxRedirectPolicy(5),
    // Only allow redirect to same domain.
    // e.g. redirect "www.imroc.cc" to "imroc.cc" is allowed, but "google.com" is not
    req.SameDomainRedirectPolicy(),
)

client.SetRedirectPolicy(
    // Only *.google.com/google.com and *.imroc.cc/imroc.cc is allowed to redirect
    req.AllowedDomainRedirectPolicy("google.com", "imroc.cc"),
    // Only allow redirect to same host.
    // e.g. redirect "www.imroc.cc" to "imroc.cc" is not allowed, only "www.imroc.cc" is allowed
    req.SameHostRedirectPolicy(),
)

// All redirect is not allowd
client.SetRedirectPolicy(req.NoRedirectPolicy())

// Or customize the redirect with your own implementation
client.SetRedirectPolicy(func(req *http.Request, via []*http.Request) error {
    // ...
})

Proxy

Req use proxy http.ProxyFromEnvironment by default, which will read the HTTP_PROXY/HTTPS_PROXY/http_proxy/https_proxy environment variable, and setup proxy if environment variable is been set. You can customize it if you need:

// Set proxy from proxy url
client.SetProxyURL("http://myproxy:8080")

// Custmize the proxy function with your own implementation
client.SetProxy(func(request *http.Request) (*url.URL, error) {
    // ...
})

// Disable proxy
client.SetProxy(nil)

Unix Socket

client := req.C()
client.SetUnixSocket("/var/run/custom.sock")
client.SetBaseURL("http://example.local")	

resp, err := client.R().Get("/index.html")

Retry

You can enable retry for all requests at client-level (check the full list of client-level retry settings around here):

client := req.C()

// Enable retry and set the maximum retry count.
client.SetCommonRetryCount(3). 

// Set the retry sleep interval with a commonly used algorithm: capped exponential backoff with jitter (https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/).
client.SetCommonRetryBackoffInterval(1 * time.Second, 5 * time.Second)

// Set the retry to sleep fixed interval of 2 seconds.
client.SetCommonRetryFixedInterval(2 * time.Seconds)

// Set the retry to use a custom retry interval algorithm.
client.SetCommonRetryInterval(func(resp *req.Response, attempt int) time.Duration { 
    // Sleep seconds from "Retry-After" response header if it is present and correct.
	// https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
    if resp.Response != nil {
        if ra := resp.Header.Get("Retry-After"); ra != "" {
            after, err := strconv.Atoi(ra)
            if err == nil {
                return time.Duration(after) * time.Second
            }
        }
    }
    return 2 * time.Second // Otherwise, sleep 2 seconds
})

// Add a retry condition which determines whether the request should retry.
client.AddCommonRetryCondition(func(resp *req.Response, err error) bool { 
    return err != nil
})

// Add another retry condition
client.AddCommonRetryCondition(func(resp *req.Response, err error) bool {
    return resp.StatusCode == http.StatusUnauthorized
})

// Add a retry hook which will be executed before a retry.
client.AddCommonRetryHook(func(resp *req.Response, err error){
    req := resp.Request.RawRequest
    fmt.Println("Retry request:", req.Method, req.URL)
    // Modify request settings in the retry hook.
	resp.Request.SetBearerAuthToken(token)
})

You can also override retry settings at request-level (check the full list of request-level retry settings around here):

client.R().
    SetRetryCount(2).
    SetRetryInterval(intervalFunc).
    AddRetryHook(hookFunc2).
    SetRetryHook(hookFunc1). // Unlike add, set will remove all other retry hooks which is added before at both request and client level.
    AddRetryCondition(conditionFunc2).
    SetRetryCondition(conditionFunc1) // Similarly, this will remove all other retry conditions which is added before at both request and client level.

TODO List

• Wrap more transport settings into client.
• Support h2c.
• Design a logo.
• Support HTTP3.

License

Req released under MIT license, refer LICENSE file.