A nice improvement in .NET is the introduction of LINQ, in .NET 4.5. With that, working with data was simplified a lot and, when I go to a language that doesn’t have something like it, I feel lost (having to deal with for and foreach became painful for me :-).

The features available in LINQ made my code more synthetic and readable, but sometimes, there was something that wasn’t easily attained with the default features. Microsoft heard that and introduced new features, many of them I was expecting since a long time. These new features  came with no huge announcements, but, nevertheless, they are very nice improvements.

Index and Range parameters

Index and Ranges were introduced in C#8, they can ease a lot when you must get a subrange of an array or list:

var arr = Enumerable.Range(1,100).ToArray();
Console.WriteLine($"[{string.Join(",",arr[10..15])}]");
Console.WriteLine(arr[^1]);
var list = new List<int>(arr);
Console.WriteLine(list[^1]);
Console.WriteLine(list[^5]);

When you run this code, you will get something like this:

[11,12,13,14,15]
100
100
96

This is nice, but when you wanted to work with Linq, you were not able to use indexes and ranges. Until now. .NET 6 allows using Ranges and Indices in Linq queries. This is very nice, because when you wanted a subrange of an IEnumerable, you should do something like:

list.Skip(10).Take(5)

Now, you can do the same with:

list.Take(10..15)

Or take the last 10 elements with

list.Take(^10..)

You can also take a single element with ElementAt using Indices. To get the last element in the list, you can use:

list.ElementAt(^1)

Chunking

One thing that is very common is to divide our data in chunks, in order to present it in pieces, so the user does not have to scroll long lists of information. Until now, you had to program that by hand, which could lead to errors. With the new Chunk method, you can split your data in chunks. For example, if you want to split the data in blocks of seven elements, you can do:

var chunked = list.Chunk(7);

With this code, you will obtain something like

[1,2,3,4,5,6,7]
[8,9,10,11,12,13,14]
[15,16,17,18,19,20,21]
[22,23,24,25,26,27,28]
[29,30,31,32,33,34,35]
[36,37,38,39,40,41,42]
[43,44,45,46,47,48,49]
[50,51,52,53,54,55,56]
[57,58,59,60,61,62,63]
[64,65,66,67,68,69,70]
[71,72,73,74,75,76,77]
[78,79,80,81,82,83,84]
[85,86,87,88,89,90,91]
[92,93,94,95,96,97,98]
[99,100]

And you can get one chunk with

chunked.ElementAt(3)
chunked.ElementAt(^1)

Zipping

Sometimes you want to combine three Enumerables into one. Combining Enumerables is done using the Zip method, which allowed to combine only two items at once. .NET 6 introduced the possibility of zipping three sequences at once (if you want more sequences, you must chain Zip functions). For example, if you have these three enumerables:

var list1 = Enumerable.Range(1,100).Select(i => $"ID {i}").ToList();
var list2 = Enumerable.Range(1,100).Select(i => $"Name {i}").ToList();
var list3 = Enumerable.Range(1,100).Select(i => $"Address {i}").ToList();

You can combine it into an IEnumerable of tuples with three elements each with:

var zipped = list1.Zip(list2,list3);

One note, here: in .NET 5 you could use a function to zip two sequences int another one and generate anything else than a tuple. .NET 6 didn’t change that and, if you want to zip the three IEnumerables into an IEnumerable of a class, for example, you must still do something like this:

var zipped1 = list1.Zip(list2, (l1, l2) => new { ID = l1, Name = l2 })
    .Zip(list3, (l1, l2) => new { ID = l1.ID, Name = l1.Name, Address = l2 });

DistinctBy, ExceptBy, UnionBy, InterceptBy

One thing that I use a lot is the distinct operator, to get unique values in a sequence. Until now, when I had a class and wanted to get distinct values in a class by some field, and I’m not interested in the other fields, I had to do something like:

public record Person(string Name, int Age);
var people = new List
{
    new Person("John", 30 ),
    new Person("Peter", 40),
    new Person("Mary", 20 ),
    new Person("Jane", 30 ),
    new Person("Larry", 50),
    new Person("Anne", 50 ),
    new Person("Paul", 20),
};
var distinctByAge = people.GroupBy(p => p.Age).Select(g => g.Key);

That worked fine, but lacked clarity – the intent was not explicit and it was hard to understand – Why this GroupBy is there ?

In .NET 6, the DistinctBy comes to solve that. Now, you can use something like this to get the distinct values :

var distinctAges = people.DistinctBy(p => p.Age).Select(p => p.Age);

Now the intent is clear and the code is easier to follow.

You can also use ExceptBy, to filter a sequence depending on another, like in

var excludedAges = new List<int> {30,40};
var people1 = people.ExceptBy(excludedAges, p => p.Age);

One note, here. Due to the way ExceptBy is coded (it uses a HashSet), it will only add the first duplicate element in the result. In our code, it should show:

Person { Name = Mary, Age = 20 }
Person { Name = Larry, Age = 50 }
Person { Name = Anne, Age = 50 }
Person { Name = Paul, Age = 20 }

But it only shows:

Person { Name = Mary, Age = 20 }
Person { Name = Larry, Age = 50 }

If you want all items that don’t match the excluded ages, you should still go with:

var people2 = people.Where(p => !excludedAges.Contains(p.Age));

If you want to join two sequences, removing duplicates between them, you can use the UnionBy method. This code joins the two lists into another, removing the duplicates:

var people3 = new List<Person>
{
    new Person("John", 20 ),
    new Person("Peter", 25),
    new Person("Paul", 20 ),
    new Person("Ringo", 22 ),
    new Person("George", 23),
    new Person("Anne", 50 ),
    new Person("Mark", 20),
};
var people4 = people.UnionBy(people3, p => p.Name);

If you want the have the names present in both lists, you can use the IntersectBy method:

var includedAges = new List<int> {30,40};
var people5 = people.IntersectBy(includedAges, p => p.Age);

In the same way of the ExceptBy, the duplicates are not included. If you want to include them, you should use:

var people6 = people.Where(p => includedAges.Contains(p.Age));

MaxBy and MinBy

When using the methods Max and Min, the sequences should implement the IComparable interface, so they could be compared and the maximum and minimum evaluated. That posed a problem, especially if the class you wanted to compare didn’t implement the IComparable interface. Now, with MinBy and MaxBy you don’t have to use the IComparable and can use something like:

var minByAge = people.MinBy(p => p.Age);
var maxByAge = people.MaxBy(p => p.Age);

This code won’t show all the elements with minimum age. To get that, you should use something like

var minAge = people.Select(p => p.Age).Min();
var allMinByAge = people.Where(p => p.Age == minAge);
var maxAge = people.Select(p => p.Age).Max();
var allMaxByAge = people.Where(p => p.Age == maxAge);

FirstOrDefault, LastOrDefault, SingleOrDefault with a default parameter

These three functions returned Default(T) if the element was not found or the list was empty. This could pose a problem or extra checks if the element was not found. Now, we can set a default value when the element is not found and, in this case, we don’t have to deal with null checks:

var firstOrDefault = people.FirstOrDefault(p => p.Age == 25,new Person("Unknown",25));
var lastOrDefault = people.LastOrDefault(p => p.Age == 25,new Person("Unknown",25));
var singleOrDefault = people.SingleOrDefault(p => p.Age == 25,new Person("Unknown",25));

In all the three cases, the code will return a Person with name Unknown and Age = 25

TryGetNonEnumeratedCount

When you have an IEnumerable and you use the Count() method, it will enumerate the collection, even if it has another method to get the count in another way, thus penalizing the performance. For that, .NET 6 implemented the TryGetNonEnumeratedCount method to try to use another method to get the count, if available. This function will return true if a faster method was available, or false, if not. That way, you can take an action to use something more performant and avoid multiple enumerations. For example:

IEnumerable<Person> people7 = people;
Console.WriteLine(people7.TryGetNonEnumeratedCount(out int count));

Will return true, because The List implements the Count property to get the count. When  you have an IEnumerable, result of a Linq operation, like in

var people6 = people.Where(p => includedAges.Contains(p.Age));
Console.WriteLine(people6.TryGetNonEnumeratedCount(out int count1));

It will return false and the count1 variable will have the actual count of the sequence.

Conclusion

As you can see, there are several improvements to Linq in .NET 6. They were not huge improvements, but brought some ease to the development. I’m sure that I will use them a lot.

The sample code for this article is at https://github.com/bsonnino/LinqImprovements

As an MVP, I sometimes receive licenses to software from the vendors for my usage. Some of them become indispensable for me and I feel in the obligation to write a review (yes, it’s a biased review, as I really like the tool and use it on a daily basis :-)) as a way to say thank you!

One of these tools is Linqpad (https://www.linqpad.net/). It’s a simple tool, with a small footprint, but I have used it in so many ways that I find it incredible. There is a free version that has a lot of features to start, but I really recommend the paid version (if you have the $95 to spend, the Premium edition has even a debugger to debug your snippets).

Introduction

Once you open Linqpad, you will see a simple desktop like this:

At first, the name of the tool may indicate that this is a notepad for linq queries, but it’s much more than that! If you take a look at the Samples pane, you can see that there’s even an Interactive Regex Evaluator.

A closer look at that pane shows that you are not tied to C#: you can also use F# there. In fact, there is a full F# tutorial there. If you open the Language combo, you can see that you can use also VB or SQL queries.

My first usages in Linqpad were to learn Linq (the name is Linqpad, no?). At the beginning, Linq seems a little bit daunting, with all those extension methods and lambdas. So, I started to try some Linq queries, making them more difficult as my knowledge was improving. In Linqpad, you have three flavors of code: Expressions, where you have a single expression evaluated; Statements, where you have some statements evaluated and Program, where you can have a full program run in Linqpad (I use this when I want to run a console program and don’t want to open Visual Studio and create a new project).

In the Expression mode, you can enter a single expression, like this:

from i in Enumerable.Range(1,1000)
  where i % 2 == 0
  select i

If you run it, you will see the result in the Results pane:

As you can see, all the results are there, there is no need to open a console window or anything else. And, what’s better, you can export the results to Excel, Word or HTML. You can also use the other Linq format, the functional one:

Enumerable.Range(1,1000).Where(i => i %2 == 0)

After that, you can start tweaking your code and clicking on the Run button and observing the results. If you have the paid version, you also have Intellisense in the code, so you can check the syntax.

For example, to get the sum of the squares of the even numbers, we can do something like this:

If we have something more complicated than a single expression, we can run it using the C# statements. For example, to get all methods and parameters of the methods in the Directory class, we can use these statements:

var methodInfos = typeof(Directory).GetMethods(BindingFlags.Public | 
  BindingFlags.Static);

methodInfos.Select(m => new 
{
  m.Name, 
  Parameters = m.GetParameters() 
}).Dump();

You may have noticed something different in the code above: the Dump method. Linqpad adds this method to dump the values to the results pane. It is very powerful, you don’t need to know the type of the object, all the properties are shown there:

And you are not limited to old C#, you can also use C#7 features and even async programming. For example, this code (based on https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/async/walkthrough-accessing-the-web-by-using-async-and-await) will download asynchronously some pages from the web and will display their sizes:

async Task Main()
{
	await SumPageSizesAsync().Dump();
}

private async Task<List<string>> SumPageSizesAsync()
{
	var results = new List<string>();
	// Declare an HttpClient object and increase the buffer size. The
	// default buffer size is 65,536.
	HttpClient client =
		new HttpClient() { MaxResponseContentBufferSize = 1000000 };

	// Make a list of web addresses.
	List<string> urlList = SetUpURLList();

	var total = 0;

	foreach (var url in urlList)
	{
		// GetByteArrayAsync returns a task. At completion, the task
		// produces a byte array.
		byte[] urlContents = await client.GetByteArrayAsync(url);

		// The following two lines can replace the previous assignment statement.
		//Task<byte[]> getContentsTask = client.GetByteArrayAsync(url);
		//byte[] urlContents = await getContentsTask;

		results.Add(DisplayResults(url, urlContents));

		// Update the total.
		total += urlContents.Length;
	}

	// Display the total count for all of the websites.
	results.Add(
		$"\r\n\r\nTotal bytes returned:  {total}\r\n");
	return results;
}

private List<string> SetUpURLList()
{
	List<string> urls = new List<string>
			{
				"https://msdn.microsoft.com/library/windows/apps/br211380.aspx",
				"https://msdn.microsoft.com",
				"https://msdn.microsoft.com/library/hh290136.aspx",
				"https://msdn.microsoft.com/library/ee256749.aspx",
				"https://msdn.microsoft.com/library/hh290138.aspx",
				"https://msdn.microsoft.com/library/hh290140.aspx",
				"https://msdn.microsoft.com/library/dd470362.aspx",
				"https://msdn.microsoft.com/library/aa578028.aspx",
				"https://msdn.microsoft.com/library/ms404677.aspx",
				"https://msdn.microsoft.com/library/ff730837.aspx"
			};
	return urls;
}

private string DisplayResults(string url, byte[] content)
{
	// Display the length of each website. The string format
	// is designed to be used with a monospaced font, such as
	// Lucida Console or Global Monospace.
	var bytes = content.Length;
	// Strip off the "https://".
	var displayURL = url.Replace("https://", "");
	return $"\n{displayURL,-58} {bytes,8}";
}

When you run it, you will see something like this:

And you are not tied to the default C# libraries. If you have the Developer or Premium versions, you can download and use NuGet packages in your queries. For example in this previous article, I’ve shown how to use the Microsoft.SqlServer.TransactSql.ScriptDom package to parse your Sql Server code. You don’t even need to open Visual Studio for that. Just put this code in the Linqpad window:

static void Main()
{
	using (var con = new SqlConnection("Server=.;Database=WideWorldImporters;Trusted_Connection=True;"))
	{
		con.Open();
		var procTexts = GetStoredProcedures(con)
		  .Select(n => new { ProcName = n, Tree = ParseSql(GetProcText(con, n)) })
		  .Dump();
	}
}

private static List<string> GetStoredProcedures(SqlConnection con)
{
	using (SqlCommand sqlCommand = new SqlCommand("select s.name+'.'+p.name as name from sys.procedures p " +
	  "inner join sys.schemas s on p.schema_id = s.schema_id order by name", con))
	{
		using (DataTable procs = new DataTable())
		{
			procs.Load(sqlCommand.ExecuteReader());
			return procs.Rows.OfType<DataRow>().Select(r => r.Field<String>("name")).ToList();
		}
	}
}

private static string GetProcText(SqlConnection con, string procName)
{
	using (SqlCommand sqlCommand = new SqlCommand("sys.sp_helpText", con)
	{
		CommandType = CommandType.StoredProcedure
	})
	{
		sqlCommand.Parameters.AddWithValue("@objname", procName);
		using (var proc = new DataTable())
		{
			try
			{
				proc.Load(sqlCommand.ExecuteReader());
				return string.Join("", proc.Rows.OfType<DataRow>().Select(r => r.Field<string>("Text")));
			}
			catch (SqlException)
			{
				return null;
			}
		}
	}
}

private static (TSqlFragment sqlTree, IList<ParseError> errors) ParseSql(string procText)
{
	var parser = new TSql150Parser(true);
	using (var textReader = new StringReader(procText))
	{
		var sqlTree = parser.Parse(textReader, out var errors);

		return (sqlTree, errors);
	}
}

You will see some missing references. Just press F4 and it will open the following screen:

Click the Add NuGet button and add the Microsoft.SqlServer.TransactSql.ScriptDom package, then run the program. You will see something like this:

You can even click on the ScriptTokenStream result, to see the list of tokens in the procedure:

You can also simplify the query by using the connections available in Linqpad. Just go to the connections pane, add a new connection and point it to the WorldWideImporters database. Then select the connection in the connections combo and use this code:

void Main()
{
	ExecuteQuery<string>("select s.name+'.'+p.name as name from sys.procedures p " +
	  "inner join sys.schemas s on p.schema_id = s.schema_id order by name")
		  .Select(n => new 
		    { 
			  ProcName = n, 
			  Tree = ParseSql(ExecuteQuery<string>("exec sys.sp_helpText @objname={0}",n).FirstOrDefault()) 
			})
		  .Dump();
}

private static (TSqlFragment sqlTree, IList<ParseError> errors) ParseSql(string procText)
{
	var parser = new TSql150Parser(true);
	using (var textReader = new StringReader(procText))
	{
		var sqlTree = parser.Parse(textReader, out var errors);

		return (sqlTree, errors);
	}
}

You will see the same results. As you can see, you don’t even need to open the connection and create the command to run it. You can run your queries against your databases the same way you would do with any data. And if you are a SQL guy, you can run your queries directly using the SQL language. And, if you are brave and want to learn F#, you have here a really nice tool to learn.

Conclusions

At first, the size and appearance of Linqpad may fool you, but it’s a very nice tool to work, saving you a lot of time to try and debug your code. If you have some code snipped that you want to test and improve, this is the tool to use. And, one feature that I didn’t mention that’s invaluable when you are optimizing your code is the timing feature. After the execution of each query, Linqpad shows the execution time, so you can know how long did it take to execute it.