Simple.Data 2.0

Today I started work on the next version of Simple.Data. Even though there hasn’t been an actual, official release of 1.0 yet. That’s something I hope to rectify in the next week or so, by fixing the few remaining issues and hoping the documentation is sufficient, although not exhaustive.

The initial work on 2.0 involves addressing some of the issues that have appeared as a result of a code-base growing organically from something that started out much less ambitious. These changes will clean the code up to provide a better base to build on, and hopefully to make it easier to write adapters for a wider range of back-end data platforms. I’ll also be building a RavenDB adapter, not because RavenDB’s own API needs improving upon, but because I’d like RavenDB to be one of the platforms that can be swapped in to an application built using Simple.Data.

Once that’s done, there are a few headline features for the 2.0 release:

Async support

Async’s strong point is non-blocking I/O operations, so getting it into Simple.Data is a no-brainer. I’ve got to work out the details, but you should be able to await everything.

Batch operations

You’ll be able to create a batch of operations and execute it with a single connect-run-disconnect process. This will also support adding arbitrary Actions and Funcs into the chain of operations to be run between database calls, using previously-returned data.

Metadata exposed

You’ll be able to access all the database schema information from the underlying database, without worrying about differing INFORMATION_SCHEMA or sys.* or what-have-you. Where appropriate, you’ll also be able to connect to a server and get database information too.

Plug-ins

There are lots of things that could be added to Simple.Data that would probably stop it being quite so Simple, so I’m going to add various hooks and extension points to allow either me or other people to add this stuff. Adapter authors will be able to add custom “methods” to the Database or Table objects, and you’ll be able to check whether those methods are available at runtime.

I might also refactor out some of the more esoteric functionality that Simple.Data currently has into plug-ins. We’ll see.

Better WithStuff

The With operations on queries will be getting much more powerful, allowing complex object graphs to be populated in a single call. I’m also looking at allowing object graphs to be saved with a single call, if that’s actually possible.

Oh, and IntelliSense

It’s not possible to glom onto the built-in IntelliSense in Visual Studio and augment it. Believe me, I tried. All you can do is completely replace it. Which, obviously, is a nightmare. But with the upcoming release of Roslyn (I am assured there will be a new preview release shortly after VS2013’s full release), I suspect replacing IntelliSense is going to be a lot easier.

It’s still a sufficiently big undertaking that I’m not going to just build it for Simple.Data. My plan is to create an open source IntelliSense replacement that is designed from line one to be extensible by anyone who wants to, and then to build a Simple.Data extension for it. I’m hoping I can make the engine for that extension modular enough to be able to create ReSharper and CodeRush completion plug-ins, too.

If you’re interested in contributing to that project, particularly on the UI/UX side of things, I’d be grateful for any help.

.NET 4.5

The biggest other change is that I’m pretty sure 2.0 is going to be .NET 4.5 (and up) only. With all the async/await stuff, maintaining compatibility with 4.0 will likely require too much time and effort, and will probably compromise the code and performance.

Contributors

If you’re the author of an adapter or provider for Simple.Data and you don’t want to continue to maintain that package through the 2.0 release, please let me know and we can bring it into the central fold. You’ll continue to be credited as the original author for the life of the package.

RFC

So, if you’ve got any comments or anything about any of that, please leave them below or reach out to me on Twitter.

Hidden Complexity

This coming Monday at Skills Matter in London, I’ll be giving my new talk, Hidden Complexity: Inside Simple.Data and Simple.Web for the first time. In October, I’ll be presenting it at Dev Day in Krakow and Leetspeak in Malmo as well.

Simple.Data and Simple.Web are like the proverbial duck: on the surface, everything is clean and calm and simple, but underneath, there’s a whole lot going on. There’s some runtime code generation using System.Linq.Expressions; in the case of Simple.Data, there’s a lot of dynamic stuff going on; in Simple.Web, I’ve built a whole asynchronous pipeline on top of Tasks; and there are some downright abuses of the C# language in there for good measure.

In the course of writing these things, I’ve learned a hell of a lot, sometimes from trial and error, and sometimes from gurus like Jon Skeet (mainly abuses of C#) and Bill Wagner (mainly code generation), so this talk is my attempt to share some of that acquired knowledge. It might not be that useful in day-to-day programming – I certainly don’t use most of it on a regular basis – but I hope at the very least it will be an interesting and entertaining look at how fun code can get when you step off the well-trodden paths and into the backwoods of C# and .NET programming.

I’m on the look-out for more opportunities to present this one, so if you run a user group or have an event planned where this might fit in, please get in touch.

Simple.Data 0.18 / 1.0.0-rc1

Just published the latest Simple.Data build to NuGet with both the above versions (the 0.x versions get patched more often, generally the –pre packages are updated with a minor version bump, like 0.17 to 0.18). We’re really nearly at 1.0 now, just fixing the last few issues and getting the docs finished (Dan Maharry is doing a bang-up job on those).

New feature

I’ve had a couple of feature requests that have been outstanding for a very, very long time now: the ability to specify a command timeout; and a way to do “identity inserts” in SQL Server (that’s where you override a table’s identity column specification and supply your own values).

Both these features are kind of specific to ADO, so I didn’t want to implement them in a way that affected other adapters like MongoDB or OData. At the same time, I wanted to allow those adapters to have their own customisation options. I also didn’t want to make enormous changes to the code to support these features: if a command timeout were added as an optional parameter to the various “methods”, I’d have had to go through adding code in dozens of places.

So here how you want to insert a record into a SQL Server table, with an identity override, and a command timeout of 1 second:

db.WithOptions(new AdoOptions(commandTimeout: 1, identityInsert: true))
    .Users.Insert(Id: 42, Name: "Marvin", Occupation: "Android");

db.Users.Insert(Name: "Eddie", Occupation: "Shipboard Computer");

Let me explain what happens there.

When you call WithOptions, a shadow copy of the underlying Database (or Transaction) object is created, with a shadow copy of the Adapter, and the options are set on that copy. So the first call to Users.Insert happens on that shadow copy, and the second call happens on the original, which doesn’t have those options set.

Somebody told me they thought having the options at that point in the call chain was clunky, but there are a couple of reasons for it being there; one technical, and one API design.

The technical reason is that putting it anywhere else is either difficult or impossible. I thought about putting it on the table object instead of the database, but that wouldn’t work for stored procedures, which are a common case for needing to set a longer timeout period. And it can’t go at the end of the call chain, because most methods return directly, so you can’t chain anything onto them.

The design reason is that this way allows you to create a reusable database object with options set, which strikes me as being potentially very useful. Maybe you want all commands executed against the database to have a shorter timeout than the default 30 seconds. In that case, you only need to set the options once, on a database object held in a field in a type, and then all your calls happen with those settings. Also, I can envisage other future uses for this Options-based approach, other options that might be database-wide, so putting the WithOptions call there leaves maximum scope for those future options.

Fixes

I fixed a bunch of small-to-medium problems in this release, too. There was an issue with the way I was scanning assemblies and using MEF to compose adapters and providers which is now fixed. I’ve also fixed a problem with the magic casting in medium trust environments, and a handful of problems people reported on GitHub. There are still a couple of issues outstanding, which will get fixed in a 0.18.x patch soon, and then I sincerely hope there’ll be a short RC2 period before the long-awaiting 1.0 final release. Not least because there’s a 1.5 that I really want to get on to, adding async support.

I’d like to thank everyone who has helped with this release, either with pull requests, or bug reports, diagnostic efforts and patience, or in one instance somebody who gave an excellent answer to a really obscure question on Stack Overflow. I can’t call out everybody, but you are all appreciated.