Process a list with a loop, taking 100 elements each time and automatically less than 100 at the end of the list

Question

Is there a way to use a loop that takes the first 100 items in a big list, does something with them, then the next 100 etc but when it is nearing the end it automatically shortens the "100" step to the items remaining.

Currently I have to use two if loops:

for (int i = 0; i < listLength; i = i + 100)
{
    if (i + 100 < listLength)
    {
        //Does its thing with a bigList.GetRange(i, 100)
    }
    else
    {
        //Does the same thing with bigList.GetRange(i, listLength - i)
    }
}

Is there a better way of doing this? If not I will at least make the "thing" a function so the code does not have to be copied twice.

Answer 1

You can make use of LINQ Skip and Take and your code will be cleaner.

for (int i = 0; i < listLength; i=i+100)
{
    var items = bigList.Skip(i).Take(100); 
    // Do something with 100 or remaining items
}

Note: If the items are less than 100 Take would give you the remaining ones.

Answer 2

In dotnet 6 you can use chunk:

//Child collections will be comprised of 10 elements each.
IEnumerable<int[]> sublists = numbers.Chunk(10); 

https://exceptionnotfound.net/bite-size-dotnet-6-chunk-in-linq/

There is also a reference to use a group by to do this, which is quite an interesting solution for older versions of the framework: Split a collection into `n` parts with LINQ?

Answer 3

You can keep an explicit variable for the end point:

for (int i = 0, j; i < listLength; i = j)
{
    j = Math.min(listLength, i + 100);
    // do your thing with bigList.GetRange(i, j)
}

Answer 4

List<int> list = null;
int amount_of_hundreds = Math.Floor(list.Count/100);
int remaining_number = list.Count - (amount_of_hundreds * 100);

for(int i = 0; i < amount_of_hundreds; ++i)
    {
    for(int j = 0; j < 100; ++j)
        {
        int item = list[(i * 100) + j];
        // do what you want with item
        }
    }

 for(int i = 0; i < remaining_number; ++i)
    {
    int item = list[(amount_of_hundreds * 100) + i];
    // do what you want with item
    }

Answer 5

you can try below approach also:

        int i = 1;
        foreach (item x in bigList)
        {

            batchOperation.Insert(x); //replace it with your action; create the batch
            i++;
            if (i >100)
            {
                table.ExecuteBatch(batchOperation); //execute the batch
                batchOperation.Clear();
                i = 1; // re-initialize 
            }

        }

        if (batchOperation.Count >= 1 )
        {
            table.ExecuteBatch(batchOperation); //do this for the residue items 
        }

Answer 6

This is my solution, before all, i skip the last rows (rows - skiplast) after i get top of the array.

    [TestMethod]
    public void SkipTake()
    {
        var rows = 100;
        var skip = 1;
        var skiplast = 95;
        var result = Enumerable.Range(1, rows).Take(rows - skiplast).Skip(skip - 1 == 0 ? 1 : skip).ToList();
    }

Answer 7

I didn't like any of the answers listed, so I made my own extension:

public static class IEnumerableExtensions
{
    public static IEnumerable<IEnumerable<T>> MakeGroupsOf<T>(this IEnumerable<T> source, int count)
    {
        var grouping = new List<T>();
        foreach (var item in source)
        {
            grouping.Add(item);
            if(grouping.Count == count)
            {
                yield return grouping;
                grouping = new List<T>();
            }
        }

        if (grouping.Count != 0)
        {
            yield return grouping;
        }
    }
}

Then you can use it:

foreach(var group in allItems.MakeGroupsOf(100))
{
    // Do something
}

Answer 8

You can split one big List to many small lists by limit, like this:

var limit = 100;
foreach (var smallList in bigList.Select((x, i) => new { Index = i, Value = x })
                                 .GroupBy(x => x.Index / limit)
                                 .Select(x => x.Select(v => v.Value).ToList())
                                 .ToList())
{
    Console.WriteLine($"{smallList.Count}");
    Console.WriteLine(String.Join("\r\n", smallList));
}