NuGet Release of KindOfMagic

Happy to announce that latest version 1.2.1 of KindOfMagic MSBuild task is now available as NuGet package.

I’ve got several requests to publish KindOfMagic on NuGet, but due to lack of time wasn’t able to accomplish this task before. But now it is finally happen. I dig into PowerShell scripts to find out how to transform .csproj files to include KindOfMagic.targets. Fortunately, it wasn’t too difficult.

So all is needed now to install KindOfMagic for your project is just to execute the following command in NuGet Package Manager Console:

Install-Package KindOfMagic

Have fun and Merry Christmas!

Portable Usage Scenario of Lex.DB

Preface

One of the beauties of .NET framework is that it works the same way across all Microsoft platforms including PCs, tablets, gaming consoles, phones and browsers. However there are still some major differences between CLR and BCL versions to make the same code truly cross-platform.

For example, WinRT platform (Windows Store Apps) lacks FileStream class and, to accommodate COM reflection, turns CLR reflection along with messed up namespaces upside down. Windows Phone 7 BCL lacks HashSet and several Linq.Expression classes. Silverlight 5 lacks slim versions of synchronization primitives like ReaderWriterLockSlim, Stopwatch and BufferedStream classes.

It is not always that lacking features are just missing. It could be that the certain version of platform offers some other ways of doing the same things. Theoretically, if we stay in subset of .NET which is common between all these platforms, we could produce truly cross-platform assembly. We just need to use only those methods and classes which are available across all target platforms we’d like to address.

This technique is already available (built-in in VS2012) and called Portable Class Library. By selecting different combinations of target platforms (.NET 4.0-4.5, SL 4-5, WP 7-7.5-8, .NET for Windows Store Apps, Xbox 360) we limit our API set to lowest common denominator of those target platforms. The produced assembly is compatible with all target platforms on binary level, so no recompilation required.

Isn’t this a dream?

But practical experience proves that portable API set is too limited to be useful. The more-or-less complex library like Lex.DB is just impossible to make 100% portable across all target platforms: no files to read/write data, no thread synchronization and no expressions to make Lex.DB fast. Consider all issues and there is no Lex.DB.

What about abstracting some platform-dependent parts from Lex.DB into interfaces and implementing them in platform-specific assemblies? Nice idea, but what will we end up with? We’ll get a portable assembly that requires platform-dependent sidekick anyway. Right now we have just one platform-dependent all-in-one assembly, so why bother?

Wait; there should be some way to make portable database access. Yes, there is. Abstracting data access logic of your app itself.

Portable Way

The idea is simple. We put everything what could be portable in our portable Common Logic assembly. Some components have to interact with platform specific code. To make them portable with need to break the dependency by abstracting interaction via “service” interface.

  • Data Entities – data classes to read/write in Lex.DB
  • View Models – non-visual models of UI in your app
  • Business Logic to work with data entities and view models
  • Resources – text localization and other
  • Interfaces to abstract underlying platform features (IMessageBox, IDialog, IApplication)
  • IDataAccessService interface to abstract data access
  • Service Locator or IoC (Inversion-Of-Control) singleton to obtain platform-specific service implementations

Platform-specific application references platform-specific Lex.DB and portable Common Logic assembly, gaining access to all classes above. During initialization phase our application sets the Service Locator or IoC-container up with platform-specific implementations of these services to allow Common Logic interact with the platform.

portable

Working example

I wrote an article to Nokia Developer wiki, in details explaining this technique. The example provided is a Contacts application, that works on all MS platforms: WinRT, Windows Phone 8, Silverlight 5, .NET (WPF). 10000 records are loaded in a split second. Check it out.

In the example I use my Visual Studio plugin KindOfMagic, which makes implementation of INotifyPropertyChanged objects completely effortless. This plugin saved me lot of nerves and time. Check it also out.

 

That’s it for today.

Supported types in Lex.DB

(previous post)

Lex.DB supports following data member CLR types of out of box:

  • string (System.String)
  • byte (System.Byte)
  • int (System.Int32)
  • long (System.Int64)
  • double (System.Double)
  • float (System.Single)
  • decimal (System.Decimal)
  • bool (System.Boolean)
  • System.DateTime
  • System.DateTimeOffset
  • System.TimeSpan
  • System.Guid
  • Enums based on int, long and byte types

In addition to types listed above, Lex.DB supports derivative nullable types:

  • byte?
  • int?
  • long?
  • double?
  • float?
  • decimal?
  • bool?
  • DateTime?
  • DateTimeOffset?
  • TimeSpan?
  • Guid?
  • Nullable Enums

as well as collection based types like:

  • List<T>
  • HashSet<T>
  • T[] (Arrays)
  • Dictionary<TKey, TValue>

where T, TKey, TValue are supported non-array types.

Serialization of custom types

Knowing that the combination of types above could never be enough, Lex.DB will soon allow extension of database type system (starting from next release).

However, there is a catch:

  • Each new registered type must have unique identifier inside your data universe, collision will corrupt the data
  • Custom types have to be registered before any Lex.DB instance initialization
  • Once written in database, types must ALWAYS be registered (even if your app has evolved and there are no data members of this type anymore), otherwise data will become unreadable
  • Registration type identifier (short) < 1000 are reserved for internal and future use

Let’s define a custom type we’d like to support:

public struct Point
{
  public float X { get; set; }
  public float Y { get; set; }
}

To register this type with Lex.DB we need to provide serialization extension. It will look like this:

public static class CustomTypesSerializers
{
  public static Point ReadPoint(DataReader reader)
  {
     return new Point { X = reader.ReadDouble(), Y = reader.ReadDouble() };
  }

  public static void WritePoint(DataWriter writer, Point value)
  {
    writer.Write(value.X);
    writer.Write(value.Y);
  }
}

Please notice, that we read the data in the same order as we write it into the stream. Read and Write methods are public and static. They have naming pattern like ReadTYPENAME and WriteTYPENAME, where TYPENAME is name of your custom type without namespace.

DataReader and DataWriter are subclasses of BinaryReader and BinaryWriter accordingly, so all reading and writing methods are available to you.

Now here is how to register it. This method must be called somewhere inside initialization section of your app. I would recommend put it in static constructor of your Application class. It guarantees that it will be called only once and before instance constructor of your app.

Serialization.Extender.RegisterType<Point, CustomTypesSerializers>(1000);

That’s it for today! Stay tuned.

(to be continued)

Indexing in Lex.DB

(previous post)

In this post I will explain, how to improve data access in Lex.DB using built-in indexing feature.

To keep the lib compact, Lex.DB provides very simple, yet powerful indexing:

  • all non-PK indexes are non-unique
  • non-calculated indexes support composite keys based on up to 3 data members
  • calculated indexes (indexes based on calculated data members) are supported
  • all indexes (including PK index) are stored together in the index file of the table

Let’s take an example for the previos blog post. Again our demo class looks like this:

public class Contact 
{   
  public int Id { get; set; }   
  public string FirstName { get; set; }   
  public string LastName { get; set; }   
  public DateTime? BirthDate { get; set; }   
  public string FullName { get { return FirstName + " " + LastName; } } 
}

To define an index we’ll modify our mapping declaration like this:

db.Map<Contact>()
  .Automap(i => i.Id)
  .WithIndex("LastName", i => i.LastName);

The name of the index must be unique for the table. It will be used later for querying.

We can also chain several index definitions together like this:

db.Map<Contact>()
  .Automap(i => i.Id)
  .WithIndex("LastName", i => i.LastName)
  .WithIndex("FullName", i => i.FullName);

Now we have two independent non-unique indexes for LastName and FullName properties. Notice that FullName is calculated one.

Let’s query the table using our freshly defined indexes.

var list1 = db.Table<Contact>().LoadAll("FullName", "Scott Guthrie");
var list2 = db.Table<Contact>().LoadAll("LastName", "Papa");

So simple is that.

continue…

Introduction to Lex.DB

(previous post)

Let’s start with simple example. We have a data class to store locally.

Every piece of data has to have a primary key. The type of primary key is up to you. What is important, the PK type T has to implement IComparable<T>.

Lex.DB maintains the Red-Black-Tree index of primary key values in memory. The index is loaded on first use, so memory is allocated on demand. Moreover, if Lex.DB detects that index is updated by another process, the index will be reloaded. During read/write operations, Lex.DB locks index file for read/write access accordingly. This approach maintains the DB consistency all the time.

So our demo class looks like this:

public class Contact 
{ 
  public int Id { get; set; } 
  public string FirstName { get; set; } 
  public string LastName { get; set; } 
  public DateTime? BirthDate { get; set; } 
  public string FullName { get { return FirstName + " " + LastName; } } 
}

The Contact class defines several properties, some are calculated and read only. Lex.DB does not require data members to be properties, public fields are also supported. However, if you like your class to be UI-bindable, data members have to be properties.

Now that we have our data class, we may actually start with database. Create a test project of your preferred platform (.NET 4, .NET 4.5, Silverlight 5, Windows Phone 8.0, Windows Store App) and add a reference to the NuGet package named “lex.db”. At the moment of writing this post, Lex.DB is in Prerelease version yet, so include Prerelease packages in search dialog to locate it.

Alternatively, from NuGet Package Manager Console you may run the following command to install Lex.DB:

install-package lex.db -pre

Then add a “using Lex.Db;” in the header of your C# file.

To initialize Lex.DB database, we need to name and initialize it first.

// demo here is a name of the folder, where all data will be stored.
var db = new DbInstance("demo");

Next step is to define our data to store. During this phase we produce highly optimized mapping code to blazingly fast read/write our data classes.

db.Map<Contact>().Automap(i => i.Id);

Automap method uses reflection to inspect and produce mappings for all public properties and fields. It is also possible to manually enlist all properties and fields to be serializable in our data storage like this:

db.Map<Contact>()
  .Key(i => i.Id)
  .Map(i => i.FirstName)
  .Map(i => i.LastName)
  .Map(i => i.BirthDate);

The approach above provides more flexibility, but it has to be kept in sync in case we extend our Contact class with more data members.

After we’ve finished with declaration of our data classes, following method initializes database instance:

db.Initialize();

Now we are ready to rock-n-roll… Just store DbInstance variable for all your data access needs somewhere.

To save data:

db.Save(new Contact { Id = 1, FirstName = "Lex", LastName = "Lavnikov" },
        new Contact { Id = 2, FirstName = "John", LastName = "Doe" },
        new Contact { Id = 3, FirstName = "Scott", LastName = "Guthrie" },
        new Contact { Id = 4, FirstName = "John", LastName = "Papa" });

To load all data:

var list = db.LoadAll<Contact>();

To load certain instance by key:

var contact = db.LoadByKey<Contact>(1);

An the end, don’t forget to release all resources allocated:

db.Dispose();

That’s it for intro.

continue…

Announcing Lex.DB

After several years writing Line-of-Business (LOB) .NET applications, I started to really like disconnected app architecture.

The idea is to load all needed data at once, keep it locally, minimize access time and server communication.

Good examples of such architecture are various mail clients (Outlook, iPhone mail app) as well as feed readers (Twitter and Facebook apps).

After primary data block is received, the app needs a short-lived connection to the server to synchronize changes: download new and updated records, list of deleted records, upload pending changes and, if needed, resolve optimistic locking concurrency issues.

My LOB app needed all these features, so my quest to find a good library for local data storage begun.

First, I made a list of criterias to search for viable solution:

  • concurrent write access (in case user starts several instances of my app)
  • compatibility with older/newer table structures (new field is introduced, old version of my app should continue to work)
  • be really fast (slow app could be a bummer, remember original FB client for iPhone)
  • support indexing
  • support upsert/merge operation
  • be written in pure C# without p/invokes to be portable at least across .Net processor platforms (x86/x64/ARM) and runtimes (.Net, Silverlight, Windows Phone, WinRT). I don’t want to rewrite my app from scratch every time Microsoft introduces new platform.
  • support of isolated storage for Silverlight app without elevated privileges
  • support POCO without extensive use of Attributes (since they are pain to apply on tool-generated classes)
  • have reasonable compact storage
  • be easy to use and understand

After evaluating all of these requirements above, I made following conclusions:

  • SQLite is quite good, but it’s native, platform dependent dll. What means you have to anchor your code to specific processor architecture. If you publish your Windows 8 app, you have to submit 3 versions (x86/x64/ARM) every time you update your app. Consider also testing 3 times more. And if you want crossplatform Xaml app for Mac & Windows (I mean good old Silverlight), you can totally forget SQLite.
  • SQLite C# port. Too complex to understand, trust and thus use in production. Maybe in my previous life.
  • Sterling. This one I liked a lot, some nice workarounds to avoid implementation of full-blown Linq provider especially, but performance and storage were unacceptably slow. Knowing that Silverlight does quota-check on every isolated storage file IO, database access was implemented absolutely wrong technically.
  • some other commercial solutions. Fall short in different areas (attributes, performance, complexity, concurrent access etc).

So what do brave people do, if something is just not there? Right, gather knowledge, compile and innovate.

So Lex.DB was born…

continue…