Download a file to destination with progress

[Mark Turner]

I had started to look into this and thought about opening up a PR, but figured I'd open an issue to begin with in case you had some input.

Would be great to add support in order to download files from Google Drive to a destination as downloading using GetFileData means entire data is in memory.

We can download using:

try await urlSession.download(for: request)

Which returns:

/// - Returns: Downloaded file URL and response. The file will not be removed automatically.

https://developer.apple.com/documentation/foundation/urlsession#2934757

However I'm unsure how to get the download progress.

It seems we may be able to take the approach using AsyncBytes, but this still means entire download living in memory.
https://khanlou.com/2021/10/download-progress-with-awaited-network-tasks/

[Mark Turner]

Ideally would be great to be able to combine the async download method with URLSessionTaskDelegate delegate progress updates, i.e:

public struct GetFileDownload: Sendable {
    // ... (other parts of the struct)

    public static func live(
        auth: Auth,
        keychain: Keychain,
        httpClient: HTTPClient,
        onProgress: @escaping (Double) -> Void // Progress closure
    ) -> GetFileDownload {
        GetFileDownload { params in
            // ... (authorization and request setup)

            let delegate = ProgressDelegate(onProgress: onProgress)
            let (responseData, response) = try await httpClient.download(for: request, delegate: delegate)

            // ... (status code validation and return data)
        }
    }
}

class ProgressDelegate: DownloadProgressDelegate {
    private var totalBytesReceived: Int64 = 0
    private var totalBytesExpected: Int64 = -1
    private let progressClosure: (Double) -> Void

    init(onProgress: @escaping (Double) -> Void) {
        self.progressClosure = onProgress
    }

    func didReceiveData(_ bytesReceived: Int64, totalBytesExpected: Int64) {
        self.totalBytesExpected = totalBytesExpected
        totalBytesReceived += bytesReceived
        let progress = totalBytesExpected > 0 ? Double(totalBytesReceived) / Double(totalBytesExpected) : 0
        DispatchQueue.main.async {
            self.progressClosure(progress)
        }
    }
}

[Mark Turner]

Confirmed - does appear the only URLSessionDataDelegate which is invoked when passing delegate to async download function is the authentication challenge. https://developer.apple.com/documentation/foundation/urlsessiondelegate/1409308-urlsession#discussion

[Dariusz Rybicki]

Thanks for the feedback @markst, it's a nice feature to have. I would start by updating the HTTPClient dependency to support downloads with progress reporting. The live implementation could create URLSessionDownloadTask and observe its progress.

public struct HTTPClient: Sendable {
  public typealias DataForRequest = @Sendable (URLRequest) async throws -> (Data, URLResponse)
  public typealias DownloadForRequest = @Sendable (URLRequest, @escaping OnProgress) async throws -> (URL, URLResponse)
  public typealias OnProgress = @Sendable (Double) -> Void

  public init(
    dataForRequest: @escaping DataForRequest,
    downloadForRequest: @escaping DownloadForRequest
  ) {
    self.dataForRequest = dataForRequest
    self.downloadForRequest = downloadForRequest
  }

  public var dataForRequest: DataForRequest
  public var downloadForRequest: DownloadForRequest

  public func data(for urlRequest: URLRequest) async throws -> (Data, URLResponse) {
    try await dataForRequest(urlRequest)
  }

  public func download(for urlRequest: URLRequest, onProgress: @escaping OnProgress) async throws -> (URL, URLResponse) {
    try await downloadForRequest(urlRequest, onProgress)
  }
}

extension HTTPClient {
  public static func urlSession(_ urlSession: URLSession = .shared) -> HTTPClient {
    HTTPClient(
      dataForRequest: { request in
        try await urlSession.data(for: request)
      },
      downloadForRequest: { request, onProgress in
        var progressObservation: NSKeyValueObservation?
        defer { _ = progressObservation }
        return try await withCheckedThrowingContinuation { continuation in
          let task = urlSession.downloadTask(with: request) { url, response, error in
            if let error {
              continuation.resume(throwing: error)
            } else if let url, let response {
              continuation.resume(returning: (url, response))
            } else {
              continuation.resume(throwing: URLError(.unknown))
            }
          }
          progressObservation = task.progress.observe(\.fractionCompleted) { progress, _ in
            onProgress(progress.fractionCompleted)
          }
          task.resume()
        }
      }
    )
  }
}

This should report progress during download, and return URL once completed (or throw an error on failure). It's just a draft (I didn't test it), but it should give you a starting point. We can add a new DownloadFile dependency to the client, that uses the above HTTPClient.downloadForRequest function.

I would appreciate it if you open a new pull request, and I will try my best to help!