How to track progress of multiple simultaneous downloads with AFNetworking?

Question

I am using AFNetworking to download files that my app uses for a sync solution. At certain times, the app downloads a series of files as a batch unit. Following this example, I run the batch like this:

NSURL *baseURL = <NSURL with the base of my server>;
AFHTTPRequestOperationManager *manager = [[AFHTTPRequestOperationManager alloc] initWithBaseURL:baseURL];

// as per: https://stackoverflow.com/a/19883392/353137
dispatch_group_t group = dispatch_group_create();

for (NSDictionary *changeSet in changeSets) {
    dispatch_group_enter(group);

    AFHTTPRequestOperation *operation =
    [manager
     POST:@"download"
     parameters: <my download parameters>
     success:^(AFHTTPRequestOperation *operation, id responseObject) {
         // handle download success...
         // ...
         dispatch_group_leave(group);

     }
     failure:^(AFHTTPRequestOperation *operation, NSError *error) {
         // handle failure...
         // ...
         dispatch_group_leave(group);
     }];
    [operation start];
}
// Here we wait for all the requests to finish
dispatch_group_notify(group, dispatch_get_main_queue(), ^{
    // run code when all files are downloaded
});

This works well for the batch downloads. However, I want to display to the user an MBProgressHUD which shows them how the downloads are coming along.

AFNetworking provides a callback method

[operation setDownloadProgressBlock:^(NSUInteger bytesRead, long long totalBytesRead, long long totalBytesExpectedToRead) {
}];

... which lets you update a progress meter pretty easily, simply by setting the progress to totalBytesRead / totalBytesExpectedToRead. But when you have multiple downloads going simultaneously that is hard to keep track of on a total basis.

I have considered having an NSMutableDictionary with a key for each HTTP operation, with this general format:

NSMutableArray *downloadProgress = [NSMutableArray arrayWithArray:@{
   @"DownloadID1" : @{ @"totalBytesRead" : @0, @"totalBytesExpected" : @100000},
   @"DownloadID2" : @{ @"totalBytesRead" : @0, @"totalBytesExpected" : @200000}
}];

As each operation's download progresses, I can update the totalBytesRead for that specific operation in the central NSMutableDictionary -- and then total up all the totalBytesRead and totalBytesExpected' to come up with the total for the whole batched operation. However, AFNetworking's progress callback methoddownloadProgressBlock, defined as^(NSUInteger bytesRead, long long totalBytesRead, long long totalBytesExpectedToRead){}does not include the specific operation as a callback block variable (as opposed to thesuccessandfailure` callbacks, which do contain the specific operation as a variable, making it accessible). Which makes it impossible, as far as I can tell, to determine which operation specifically is making the callback.

Any suggestions on how to track the progress of multipole simultaneous downloads using AFNetworking?

Answer 1

when starting the operations, you could safe each operation, downloadID and the values for totalbytesRead and totalBytesExpected together in a NSDictionary and all dicts to your downloadProgressArray.

Then when the callback methods is invoked, loop through your array and compare the calling operation with the operation in each dict. this way you should be able to identify the operation.

Answer 2

If your block is inlined, you can access the operation directly but the compiler might warn you of the circular referencing. You can work around by declaring a weak reference and use it inside the block:

__weak AFHTTPRequestOperation weakOp = operation;
[operation setDownloadProgressBlock:^(NSUInteger bytesRead, long long totalBytesRead, long long totalBytesExpectedToRead) {
    NSURL* url = weakOp.request.URL; // sender operation's URL
}];

Actually, you can access anything inside the block, but you need to understand block to go for that. In general, any variable referred in the block is copied at the time the block created i.e. the time that the line got executed. It means my weakOp in the block will refer to the value of the weakOp variable when the setDownloadProgressBlock line got executed. You can think it like what would each variable you refer in the block would be if your block got executed immediately.

Answer 3

I just did something very similar (uploaded a bunch of files instead of downloading)

Here's an easy way to solve it. Lets say you are downloading maximum of 10 files in one batch.

__block int random=-1;
    [operation setUploadProgressBlock:^(NSUInteger bytesWritten, long long totalBytesWritten, long long totalBytesExpectedToWrite) {

        if (random == -1) // This chunk of code just makes sure your random number between 0 to 10 is not repetitive
        {
            random = arc4random() % 10;
            if(![[[self myArray]objectAtIndex:random]isEqualToString:@"0"])
            {
                while (![[[self myArray]objectAtIndex:random]isEqualToString:@"0"])
                {
                    random = arc4random() % 10;
                }
            }
            [DataManager sharedDataManager].total += totalBytesExpectedToWrite;
        }

        [[self myArray] replaceObjectAtIndex:random withObject:[NSString stringWithFormat:@"%lu",(unsigned long)totalBytesWritten]];

Then you calculate it like this:

NSNumber * sum = [[self myArray] valueForKeyPath:@"@sum.self"];
float percentDone;
percentDone = ((float)((int)[sum floatValue]) / (float)((int)[DataManager sharedDataManager].total));

[self array] will look like this:

array: (
    0,
    444840, // <-- will keep increasing until download is finished
    0,
    0,
    0,
    442144, // <-- will keep increasing until download is finished
    0,
    0,
    0,
    451580  // <-- will keep increasing until download is finished
)

Answer 4

Blocks are made just to makes these things easyer ;-)

HERE you can find an example project. Simply push the + button and insert the direct URL for a file to download. There is no error checking and no URL redirection so insert only direct URLs.
For the relevant part look in these methods of the DownloadViewController:

- (UITableViewCell *)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath
- (void)alertView:(UIAlertView *)alertView clickedButtonAtIndex:(NSInteger)buttonIndex

Here the explanation:

When you pass a variable to a block, the block makes a copy of the variables passed from outside. Because our variable is simply an object pointers (a memory address), the pointer is copied inside the block, and since the default storage is __strong the reference is maintained until the block is destroyed.

It means that you can pass to the block a direct reference to your progress view (I use an UIProgressView since I've never used MBProgressHUD):

UIProgressView *progressView = // create a reference to the view for this specific operation

[operation setDownloadProgressBlock:^(NSUInteger bytesRead, long long totalBytesRead, long long totalBytesExpectedToRead) {

    progressView.progress = // do calculation to update the view. If the callback is on a background thread, remember to add a dispatch on the main queue
}];

Doing this, every operation will have a reference to its own progressView. The reference is conserved and the progress view updated until the progress block exists.

Answer 5

You must be downloading some zip file , video file etc. Make a model of that file to download containing fields like (id, url, image , type , etc...)

Create a NSOperationQueue and set maxConcurrentOperationCount according to your requirement make public method. (in a singleton class)

- (void)downloadfileModel:(ModelClass *)model {

    NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
    NSString *path = [[paths objectAtIndex:0] stringByAppendingPathComponent:[NSString stringWithFormat:@"%@.zip",model.iD]];


    NSURLRequest *request = [NSURLRequest requestWithURL:[NSURL URLWithString:model.url]];
    operation = [[AFDownloadRequestOperation alloc] initWithRequest:request targetPath:path shouldResume:YES];

    operation.outputStream = [NSOutputStream outputStreamToFileAtPath:path append:NO];

    [operation setUserInfo:[NSDictionary dictionaryWithObject:model forKey:@"model"]];
    ////// Saving model into operation dictionary

    [operation setProgressiveDownloadProgressBlock:^(AFDownloadRequestOperation *operation, NSInteger bytesRead, long long totalBytesRead, long long totalBytesExpected, long long totalBytesReadForFile, long long totalBytesExpectedToReadForFile) {

  ////// Sending Notification 
  ////// Try to send notification only on significant download          
                        totalBytesRead = ((totalBytesRead *100)/totalBytesExpectedToReadForFile);
                                [[NSNotificationCenter defaultCenter] postNotificationName:DOWNLOAD_PROGRESS object:model userInfo:[NSDictionary dictionaryWithObject:[NSString stringWithFormat:@"%lld",totalBytesRead] forKey:@"progress"]];
/// Sending progress notification with model object of operation

                    }

    }];

[[self downloadQueue] addOperation:operation];  // adding operation to queue

}

Invoke this method for multiple times with different models for multiple downloads.

Observe that notification on controller where you show the download progress. (possibly tableView Controller). Show all downloading operations list in this class

For showing progress Observe the Notification and fetch the model object from notification and get the file id from notification and Find that id in your Table View and Update that particular cell with the progress.