Category Archives: 11374

Dangers of BEGIN and END

I’ve presented this material at three conferences recently, so it’s about time I wrote a blog post on it…

As programmers, we love modularisation – even in the SQL space. We make stored procedures, views, and functions to encapsulate our code. This improves maintainability, simplifies the development experience, and is generally useful.

But there’s a time when it’s a bad thing for SQL Server.

There’s an amazing component of SQL Server called the Query Optimizer (I always want to write Optimiser, but I’m assuming it’s a proper noun and putting up with the US spelling). When we write queries in T-SQL, it’s the Query Optimizer that works out how to actually run the query. It works out what indexes can be used to improve performance, what order tables (well, indexes and heaps) should be accessed, how to perform the joins, and so on. I find that a rough appreciation of the power of the Query Optimizer can really help query writers.

For example, the Query Optimizer will translate a correlated sub-query in the SELECT clause into a LEFT OUTER JOIN, so that you don’t have to. It will also work out when joins can be rendered pointless and thereby removed from the plan altogether. If you let these principles help you in your query design, you can see significant benefits. It also helps you write queries that are easier to maintain, as there’s little point in trying to be clever by writing a query in a different way if the Query Optimizer will handle it in the same way as before.

If you use a view in another query, the definition of the view is used in the query as if you had written it with a sub-query. A view is simply that – a stored sub-query. They are sometimes referred to as ‘virtual tables’, but I disagree. They are stored sub-queries. Sure, the analogy falls down when you start considering indexed views, but on the whole, a view should be seen as a stored sub-query. The Query Optimizer takes the view definition, applies it in the second query, simplifies it where possible, and works out the best way of executing it. If you’re only interested in a couple of columns out of the view, the Query Optimizer has an opportunity to take that into consideration.

Stored procedures are different. You can’t use a stored procedure in an outer query. The closest you can get to this is to use OPENROWSET to consume the results of a stored procedure in an outer query, but still the whole procedure runs. After all, it’s a procedure. A set of T-SQL commands, not a set of queries. I see the clue to this as the BEGIN and END that stored procedures generally use. I like stored procedures, but I do get frustrated if they’re returning more information than I need, since I have no way of letting the system know that maybe it doesn’t need to do as much work.

Functions are in between, and come in two varieties. A function can be inline, or it can be procedural. I don’t think you find this differentiation in many places – and normally people talk about this particular drawback as being associated with Scalar Functions as compared to Table-Valued Functions, but the problem is actually one of simplification.

An inline function must be a table-valued function at this point in time. It takes the form:

CREATE FUNCTION dbo.fnFunctionName(<paramlist>) RETURNS TABLE AS
( SELECT …. );

It is always this form, with a sub-query enclosed in a RETURN statement. It can return many columns and many rows, but the definition of the table is implied by the SELECT clause. This is essentially a view that can take parameters.

The other form is one that involves BEGIN and END. Scalar functions (unfortunately) require this (but hopefully one day will not).

CREATE FUNCTION dbo.fnFunctionName(<paramlist>) RETURNS int AS
RETURN ( … )

As the RETURN statement is enclosed between a BEGIN and END, it can be preceded by other statements, used in working out what value should be returned.

Table-valued functions can use BEGIN and END, when multiple lines are required to calculate the rows in the table being returned.

CREATE FUNCTION dbo.fnFunctionName(<paramlist>) RETURNS @table TABLE (<fields>) AS


In this kind of function, the table variable is populated with data, and returned to the outer query when the RETURN command is reached.

But when the Query Optimizer comes across a procedural function, it cannot simplify it out and executes the function in a different context.

The execution plan will report that the cost of running the function is zero. But it’s lying. The way to see the impact of the function is to look in SQL Profiler, where you’ll see potentially many calls to the function, as it needs to work out the result for each different set of parameters it’s passed. The pain can be quite great, and you will never have noticed if you just look at the Execution Plans.

The moral of the story is to make sure that your functions are able to be simplified out by the Query Optimizer. Use inline table-valued functions even in place of scalar functions. You can always hook into them using CROSS/OUTER APPLY in your FROM clause, or even use them in your SELECT clause (not “SELECT Claws” – that would make it related to my company LobsterPot Solutions, and “SELECT Claus” is just a bit Christmassy) using a construct like SELECT (SELECT field FROM dbo.fnMyTVF(someParam)) …

Consider the Query Optimizer your friend. Study Execution Plans well to look at how the Query Optimizer is simplifying your query. And stay away from BEGIN and END if possible.

Plane old trouble

Speaking at two SQL conferences in the last two months (SQL Down Under in New South Wales, and SQLBits V in Old South Wales), I’ve had some flights to do. This isn’t normally a big deal, but both times I managed to have some stress getting home.

Firstly, I should point out that both conferences were really good. Very different to each other – SQL Down Under was held at a university campus in a country town, SQLBits was in a 5-star hotel with conference centre – but both great events. There’s something about having a conference with a dedicated technology that makes it special. At TechEd you brush shoulders with people who have very different areas of expertise, but at a dedicated SQL conference, you end up having a lot in common with just about everyone.

At SQL Down Under I got to catch up with many people from around Australia that I see only a couple of times a year. Friends that I know from previous trips to Wagga, or from user groups I’ve visited, TechEd, even the occasional class I’ve taught. The content is always good, and it’s great to see people honing their skills in presenting. This year one of the highlights was seeing John Walker present for the first time.

At SQLBits, I got to meet many people for the first time (first time I’ve done a SQL conference in the UK). I got to see old friends like Simon, Jamie, Tony & Darren again, and meet people like Chris, Chris, Allan, James & Martin (of course there are many more names I could list). I had never heard any of these guys present before, so I tried to get around to as many sessions as I could. I was disappointed that the sessions I was giving clashed with Brent’s, but I was pleased that I could meet him for the first time.

Coming home from Wagga, I had to meet a flight taking me from Melbourne to Adelaide. I had allowed plenty of time to make the transfer, but when the flight out of Wagga was well over an hour and a half late, I knew I couldn’t make it. There was a fair crowd of SQL people at the airport, so we were joking about different tactics that could be used to help me make the connection. The flights were with different carriers, so apart from letting me check in for theSDC14510 second flight on their computer, there was nothing the Wagga staff could help with (they were very nice and helpful though, let me use their printer and everything). When I got to Melbourne, it turned out that the flight I was booked on had been cancelled, and my ticket transferred to a later flight, which I managed to catch. Home later than expected, but crisis avoided somehow…

Not so lucky on the way home from the UK. My flight to Australia stopped at Bangkok on the way, and as I got off, the crew were saying that we had an hour and a half. I got back to the gate in about an hour-fifteen, only to be told that I was too late. Apparently the 90 minutes was from the wheels touching down to the wheels taking off again, and we only had about 30 minutes in which to get back to the gate (bearing in mind that at Bangkok airport you need to wander down from the gate to a security area, and get re-admitted to the Departure area, before returning back to the same place you got off the plane in the first place). 24 hours later I got on a flight to Australia, but not before a stressful night trying to work out how best to get a replacement ticket, considering that nowhere in Bangkok was open for the first 16 hours I was accidentally in Thailand.

It hasn’t put me off the idea of travelling to conferences. Everything that happens gives me a story to tell, and I guess these last couple of months have just given me more stories than I expected. If you’re into SQL, and there’s a SQL conference near you, you should really try to get to it. Just pray that you have a better time getting home than I did.

StreamInsight talk coming up at SQLBits

My talk on StreamInsight is up next. I’ll try to blog more about that later. For now, I want to mention more about SQLBits itself. This is by far the largest SQL-only conference I’ve attended (I haven’t been to SQL-PASS yet), and it’s great to be involved.

Yesterday I had an all-day seminar about the new items for Developers in SQL 2008. It was a good time – the delegates responded very positively, and many of them have caught up with me since.

But for me, the conference is being a great way of catching up with (and meeting for the first time) a bunch of SQL people that I rarely see. I’ve met people that lived only a few miles from where I grew up, and people that read my blog (Hi!), discovered people who have connections to Adelaide, and even found that my Adelaide friend Martin Cairney (who is also here) has a strange connection to Donald Farmer (of Microsoft), that their parents shared a back fence or something… Now Trevor Dwyer tells me a colleague of his knows me from somewhere… the world is very small here.

My StreamInsight talk will be interesting I hope. I have some stuff to show off, and I plan to involve the audience a little as well. If you’re at SQLBits and feel like being involved in an interactive session, then definitely come along. I want to hear from people in the audience who have dabbled with StreamInsight and also other vendors’ Complex Event Processing offerings. This is a brand new technology from Microsoft, and there will be a large range of adoption levels in the room.

SQLBits V, in Old South Wales

I recently gave a talk in New South Wales, so now I’m going to give one in Old South Wales. In Newport, to be precise.

As I’ve written before, I’ve been a big fan of the SQLBits conferences that is run by many UK-based friends of mine. Unfortunately for them, they had a presenter pull out recently, and unfortunately for them, I’m going to fill in.

Weather-wise, it’ll be a nice change from the scorching weather we’ve had in Adelaide recently. We’re setting new records for days over 30C here, the streak which will be broken on Monday if the temperature drops to 28C, before climbing again after that. I’ll be going to temperatures which are more like 40F than 40C. I think I’ll be the one wearing two jumpers and a coat.

I’ll be involved in all three days of the conference, doing a full day of SQL 2008 for Developers on the Thursday, and hour-long sessions on the next two, on the topics of StreamInsight and Query Simplification respectively.

It’s a great opportunity to be involved, and I’m sure it’ll be a good time. There are several tracks, and the quality is bound to be high. I’m planning to attend sessions by friends, lots of people I’ve never heard present before.

If you’re going to be in the UK on Nov 19-21, make sure you get along, and say hi! I’ll also be receiving delivery of my (signed) copies of the SQL Server MVP Deep Dives book, which is going to be good too.

More SQL Conferences coming up, including SQL Bits and SQL Down Under

I know I won’t be there, as I’m a million miles away in Australia, but being from the UK myself, I always have an interest in the UK SQL community and in particular, events like SQL Bits.

This is the fifth SQL Bits conference, and they keep getting larger and larger. I’ve heard it’s now the largest SQL-focussed event in Europe. It’s going to be in South Wales (that’s OLD South Wales, not New South Wales), in November. I’m sure the area is lovely, good beaches ‘n all that… but considering it’s late November in Wales, I think you’ll be going for the SQL content, not the scenery.

Of course, if you are in New South Wales, then you ought to be thinking slightly earlier, in particular, the second weekend in October. The third SQL Code Camp is being held in Wagga, with many regular speakers (like myself) and quite a few new ones too.

These two events are clearly the significant SQL events in the last quarter of the year. I’m sure no-one cares about SQL PASS, after all. (I do wish I was going to this one, but I won’t be. I plan to go one year, but I was in the US that week last year, and I don’t plan to be away from home for two birthdays in a row. Maybe next year. It is the biggest SQL event in the world, with great speakers from everywhere, including many good friends of mine.)

No matter where you are in the world, there are SQL events that you should be going to. Professional development is really important for your career, and you shouldn’t neglect it. That being said, make sure you find me at TechEd Australia.