SQL Server

TechEd 2005 - Presentations Available Online

Marcus Mac Innes — Thu, 28 Jul 2005 11:56:00 GMT

Following on from my recent post about this year's TechEd Europe. You can now watch some of Kimberly and Rafal's presentations online at http://www.microsoft.com/uk/technet/itsshowtime/.

TechEd 2005 – What’s The Message This Year?

Marcus Mac Innes — Sun, 10 Jul 2005 19:02:00 GMT

Once again, Microsoft pulled off its annual TechEd conference without a hitch. If you’ve never been to TechEd, then it would be difficult to imagine the sheer scale of the event. The conference centre becomes a mini city with 6000 commuters making their way from session room to session room. Between sessions, there is a great buzz of in depth conversations taking place about the previous session content’s validity or relevance to a particular organisation. These conversations as it turns out are often of more benefit than the session content itself!

It’s been an interesting year in the technology community, a year in which Service Oriented Architecture (SOA) appears to have gone mainstream. Last year at the conference, when audiences were asked about how may of them were currently building SOA based applications, I recall a figure of about 15% raising their hands, this year that same question received a 60% vote. This increase is entirely in keeping with what I am seeing across the international communities that I am involved with. It seems that since last year, the majority of new projects are SOA based, which means that Microsoft’s relentless marketing of the SOA strategy is working… But is this right?

There were an abundance of SOA tracks again at this years TechEd. These architecture and “connected systems” tracks had that familiar feel to them, a few words changed here and there but more or less the same slides as last year. Even Clemens Vasters had little to add over the previous year and only got away with it because of his charismatic stage presence. So what’s going on? Why are we hearing the same story over and over again? My guess is that with only 60% of the audience raised their hands… there’s still another 40% to be converted!

Now this begs the big question which dominated most of the offline conversations, is SOA right for every project? There were no sessions on 3-tier, n-tier, client / server or stand alone application architectures. Are these architectures wrong? Should every application no matter how large or small be SOA?

My response is a definite “no” (for the moment). As I’ve said many times before, moving to SOA represents a mind shift much like the shift developers needed to make when they embraced Object Oriented (OO) programming back in the late 80s, early 90s. If we asked the same question in 1990, “should every application be object oriented?” I bet we would have exactly the same response as the question we are asking today about SOA, “No”. OO after all was viewed as too complex for small applications and posed a performance problem on the then slow computers. But if we asked this OO question today, everyone would agree that save a few exceptions, most applications should be built using object oriented methods.

Gregor Hohpe of ThoughtWorks and author of Enterprise Integration Patterns, gave an interesting talk on “Event Driven Architecture, ARC311” in which he gave some practical example of appropriate patterns and their implementations for building and testing event driven architectures. These concepts and implementations are very relevant to building SOA applications and hopefully we shall see more of these advanced talks in the future.

One of the reasons why SOA is not a one size fits all today is that there is a fundamental lack of knowledge, competence and agreement amongst technologists as to exactly how these applications should be built. There is a high barrier to entry in terms of upfront education, an engineering overhead in terms of a increase in design and implementation complexity and to top it all off, a definite runtime performance penalty. While the benefits of a successfully completed SOA project are clear, the investment and risks involved are greater and the business decision mandates caution and careful internal assessments before automatically jumping on board the SOA train.

Regarding education, Microsoft is doing it’s best to pump the message home, telling architects and developers that SOA is the future, teaching them to think about service boundaries, asynchronous programming, reliable versioned messaging… this is the TechEd message. It was the same message last year and people went home filled with new ideas, backed up with memorised example diagrams on how systems could hang together in a new connected, interoperable, scalable and most importantly re-usable way.

I’ve lost count of the number of people I’ve encountered who have run into big trouble trying to implement applications in SOA. The most recent conversation I had was with a US company who were looking for advice on how best to implement an atomic transaction across a service boundary. They explained how they had built an application comprising a number of services and were now having problems ensuring consistency across each of the service databases. If you are reading this thinking “yes, I’m having that problem too”, then hopefully you are in the early stages of your agile development lifecycle!!! Because you are going to need to start again. Requiring an atomic transaction across a service boundary is a clear indication that your services are not divided correctly, your foundations are wrong. This is where SOA projects are running over budget, into difficulty or simply failing. It all boils down to SOA implementation as the classic expression goes “the devil is in the detail”.

Ron Jacobs of Microsoft’s Patterns and Practices group gave a great talk on “Dealing with Data in Service-Oriented Architecture, ARC308” which expands on where Pat Helland left off before he moved to Amazon earlier this year. (Pat if you are reading this… you were missed this year!). This talk deals with the problem of understanding service boundaries from the perspective of segmenting the data within a given domain and deriving your boundaries accordingly. Service orientation is all about “data ownership” and if this message is not crystal clear, then don’t even attempt anything in SOA until this penny drops. (Remember how you needed to understand objects before you could write anything in OO). I would have liked to see a bigger drive from Microsoft in this area, because at then end of the day not every organisation has the skills of a competent and experienced Enterprise Architect on board who fully understands these concepts. Smaller organisations are trying to build SOA applications too and need to understand this cornerstone of the design principal.

The second stumbling block for building SOA or connected systems, is tool support. Those guys who after leaving Amsterdam last year and headed off to build new SOA applications quickly realised that in a non SOA world, they were living in the lap of luxury with respect to the quantity and quality of the tools that were available to help build applications. SOA is new and good tools are few and far between, meaning that you need to do a great deal of extra work (by hand) in order to make things work. Visual Studio 2003 doesn’t provide a great deal of help because once again the devil is in the detail. “[WebMethod]” gave out the entirely wrong message, facilitating ease of construction while negating design. Thankfully, this is something Microsoft is willing to admit and as such had invited my friend Christian Weyer to speak and participate in a number of sessions at this year’s event. Christian who is part of the ThinkTecture team was one of the first to fill the tool support gap with his product WS-ContractFirst which provides a quick and easy way to generate “correct” service interfaces and WSDL code. I believe that Christian also mentioned that the next version of his tool would support WSE 3.0, which will be a great addition.

Furthering the need for better tool support, we are now on the verge of the release of Visual Studio Team System. The more I look at this new suite, the more I realise why it has been so long in the making. The product is simply massive and will present a significant jump in tool support for all .NET technologists from each discipline. I’m not going to go on about this, other than to say that I sincerely believe this release will mark the beginning of a new era of development, the Software Factory.

Thirdly, SOA is currently lacking infrastructure support. Projects soon run into trouble when they realise the complexity of working in a connected (sometimes disconnected) world. Gone are the atomic COM+ transactions and in come the reliably delivered messages… But where is the reliable delivery infrastructure? Indigo will obviously address a number of the infrastructure issues and I would hazard a guess that in the mean time 80% of SOA applications as build using MSMQ as a reliable transport. It seems odd therefore that Microsoft have never released an official MSMQ transport for their Web Services Extension framework (WSE)… You’ve have to rely on third party open source code, mmm.

Benjamin Mitchell, jumps to the rescue and presents a session (CTS350) on WSE 3.0 for which Microsoft have just released a CTP of WS-ReliableMessaging. I’m not sure about the support on this release; I assume it carries a “use at your own risk” label.

In all, TechEd this year could have concentrated a bit more on expanding on last year’s theory with more talk on implementation. Fabriq, EDRA, Proseware, in fact all the implementation references were noticeably missing this year from session titles. Have Microsoft had the same challenges concerning the correct implementation of SOA as the rest of the .NET community? It seems so.

So when I asked earlier about whether SOA was right for every project, I gave a “no” (for the moment) answer. In time, I believe with the right tools, agreement on standards, supporting infrastructure, more groundswell in developer education and experience regarding implementation, I can’t see why it wouldn’t be just as easy to build applications based on SOA architectures.

SQL Server 2005 was also big on the hit list this year. The June CTP is out and appears with improved UI and tools. Kimberly L Tripp is to SQL Server as Clemens Vasters is to Connected Systems. This no nonsense girl has been there and done that, she travels the world fixing problems and as a result just knows what issues we are facing and are likely to face. Her presentation skills are unbeatable and I enjoyed every single moment of each of her talks. Apart from now having the CLR in the database (which I personally believe is of marginal benefit) the new 2005 version seems to have addressed a number of operational issues which enable greatly increased availability. Kim’s demo of online recovery was ingenious created using a tiny 4 port USB hub connected to her laptop. Inserted into the hub were four 128MB USB keys onto which she had partitioned a 1.4 million row database table. While pulling one of the keys out of the hub clearly simulated the complete failure of the drive containing the online data, it didn’t compare to Andy Lees keynote demonstration of clustering in action as he took an actual sledge hammer to a live server on stage in order to simulate a server failure… Kim’s demo however was a little more close to home as she brought the audience through the entire process of a full manual recovery without taking the database or indeed the failed table offline. This is the kind of demo you will be glad to have seen one day…

Testing is also something Microsoft has woken up to and this year’s TechEd saw a number of core personalities, including Roy Osherove of TeamAgile fame making appearances at panel discussions and giving talks on Test strategies. Visual Studio 2005 obviously addresses the need for increased tool support for testing with its introduction of unit testing and load testing capabilities and these were shown off in a number of interesting talks.

Rafal Lukawiecki of Project Botticelli is always a must see with his inspiring security talks. This guy will probably win the best speaker prize again this year…

So to round up TechEd 2005, non-Microsoft people were by far the best speakers in terms of content and presentation skills. If Kimberly L Tripp, Rafal Lukawiecki or Clemens Vasters are around over the coming year and you have an opportunity to attend one of their talks, you’re guaranteed to be inspired.

Also, for the Irish amongst us… be careful (very careful) of Clare and Rob’s generous hospitality, Microsoft style, which ensures that the TechEd experience continues well into the night! These guys really know how to put on the best show!

…and thanks Microsoft for my new phone!

ASP.Net 2.0 Roadshow in Dublin

Marcus Mac Innes — Mon, 07 Mar 2005 10:14:00 GMT

Following my talk on Securing ASP.Net Applications at last Friday’s Dublin leg of Microsoft’s ASP.Net 2.0 European Roadshow, there were a couple of requests for the slides and code samples. They can be downloaded from here.

Talking with developers afterwards in the bar, it was really interesting to see how many of them had heard of SQL Injection and XSS attacks but had not actually realised how vulnerable an application and the other applications which share the same database can be. Most had never actually seen how an attack would be carried out.

I demonstrated how security on a web site can be compromised by taking advantage of holes within the application code. These holes can be discovered by using a series of probes which disclose whether or not vulnerability exists.

The demonstration attacked the site’s Login page to discover vulnerabilities. These were then exploited to bypass the application security altogether. We were then able to take control of the database for this and other applications which share the database, create our own login accounts with administrative privileges and access the site’s customer’s credit card information.

I also demonstrated using a Cross Site Scripting attack how the Forms Authentication cookie could be stolen and silently sent to an attacker’s website to be stored for later use. Once the attacker is in position of this cookie, they would be able to login to the original web site using someone else’s authentication token.

The key message of the talk was to ensure that all user input is validated before any other processing is done. This together with strict use of secure coding standards would have disabled any attacker’s opportunities.

For more information on writing secure code, please feel free to contact me.

And many thanks to Microsoft for asking me to speak at this great event with the ASP.NET team!

SQL Server 2005 - Where's the Database Diagram?

Marcus Mac Innes — Wed, 22 Sep 2004 08:57:00 GMT

I finally got a chance to install Microsoft SQL Server 2005 Beta 2 (formerly Yukon) the other day so that I could do some forward compatibility tests on the new XML data type for a project we're currently working on.

I fired up the new Query Analyser (now simply called 'new Query' in the Management Studio) and ran the create scripts for a relatively large database.

First port of call, let's see if they've enhanced (fixed) database diagramming, especially the printing options as I always find when working with large database schemas that it's a great help to print out the entire schema and stick it up on the wall.

Maybe Microsoft's Product Unit Manager for SQL Server Tools Euan Garden is playing a big joke on everyone or maybe he's just being very very clever here with Beta 2 because while looking for the database diagram he got me to experience nearly every nook and cranny of the entire application, learned a lot but found no sign of a diagram. Then Euan got to me do a full search of the new help system... very nice but still no mention of a diagram.

Google finally came to the rescue and I found this info on Marc LaFleur's blog.

The 'Diagram' is GONE...

[UPDATE: It appears that all the negative feedback regarding the removal of diagramming in SQL Server 2005 has paid off. The VS Data team have come to their senses and have blogged about the diagram's return here.]

Guid Primary Keys are Revolutionised in SQL Server 2005

Marcus Mac Innes — Tue, 14 Sep 2004 00:35:00 GMT

Jimmy Nilsson blogs about the new NEWSEQUENTIALID() function in SQL Server 2005 (Yukon) which provides an incremental GUID generator and hence the long overdue answer to uniqueidentifier as a primary key. While the importance of this may well be lost on non database oriented people, this function alone will revolutionise our choice of table primary keys.

Traditionally GUID generation produces essentially random numbers which cannot realistically be used as primary keys since new rows are inserted into the table at random positions leading to extremely slow inserts as the table grows to even a moderate size. Inserting into the middle of a table with clustered index, rather than appending to the end can potentially cause the database to have to move large portions of the data to accommodate space for the insert. This can be very slow.

Jimmy wrote an article back in 2002 which provided a reasonable solution to the problem, but unfortunately one that I was not personally a fan of. I preferred to maintain INT primary keys for both space and performance reasons and provided a GUID as a secondary key whenever there was a requirement to publish an identifier outside of a service boundary. I think this also fits with Pat Helland's thoughts on data when he talks about Data on the Outside vs. Data on the Inside because the INT keys are private to the internal workings of the service while the GUID provide the necessary external unique identifiers.

All this (as Jimmy points out) will however be in the past with the introduction of NEWSEQUENTIALID(). Let's hope they provide an equivalent function in the CLR!

They'll have to... won't they?

Passing an Array to a Stored Procedures

Marcus Mac Innes — Fri, 09 Apr 2004 09:18:00 GMT

Take a look at this very interesting article by Erland Sommarskog which gives a number of detailed descriptions along with performance measurements on how to pass an array to a stored procedure. This is extremely useful for creating a dynamic WHERE clause.

[Update: updated article link...]

SQL Server Desktop Engine 2000 Tools

Marcus Mac Innes — Tue, 16 Mar 2004 09:40:00 GMT

For all those small SQL Server 2000 installations that make use of Desktop Edition (MSDE), there are now a number of tools available from Microsoft which provide essential assistance in both development, deployment and administration. These include:

MSDE 2000 Deployment Toolkit : A plug-in to Visual Studio that automates the process of creating an integrated package with both the application and the MSDE database.
SQL Server Web Data Administrator : Enables you to easily manage your SQL Server data, wherever you are. Using its built-in features, you can do the following from a Web browser:

Create and edit databases in SQL Server 2000 or Microsoft SQL Server 2000 Desktop Engine (MSDE 2000)
Perform ad-hoc queries against databases and save them to your file system
Export and import database schema and data
Manage users and roles
View, create and edit stored procedures

SQL Server 2000 Security Tools : Used to scan instances of SQL Server 2000 and SQL Server Desktop Engine (MSDE) 2000. The tools help detect instances vulnerable to the "Slammer" worm, and then apply updates to the affected files.
Microsoft SQL Server 2000 Best Practices Analyzer : A database management tool that lets you verify the implementation of common Best Practices. These Best Practices typically relate to the usage and administration aspects of SQL Server databases and ensure that your SQL Servers are managed and operated well.