Serializing large calendars using DDay.iCal

Calendars have become pervasive these days. Almost all business applications need some form of support for calendars and scheduling, and iCalendar has become the defacto standard for calendar interchangeability.

Even though we have a very nice calendar support in applications, clients also want to view their schedule in third party calendar applications like Google Calendar, Outlook or on their own intranet. Since most of these applications support iCalendar, supporting many other applications is possible with one format.

DDay.iCal by Douglas Day provides excellent support for iCalendar in .NET applications. DDay.iCal is RFC5545 compliant, and has an easy to use API that is described in more detail in the code project article. While this worked wonderfully in initial development and testing, we encountered problems while serializing large calendars with the library. It requires you to build the complete calendar in memory before serializing it to disk. For large calendars, this can consume a lot of memory.

The following function shows how you would serialize 100,000 events using the default DDay.iCal serializer.

[csharp]
public static void TestDefaultSerialization()
{
Console.WriteLine("Serializing using default DDay serializer");

DateTime startTime = DateTime.Now;

using (var iCal = new iCalendar())
{
iCal.AddLocalTimeZone();
iCal.AddProperty("X-WR-CALNAME", "CalendarName");

using (var writer = new FileStream("test.ical", FileMode.OpenOrCreate))
{
for (int count = 0; count < 100000; count++)
{
var evnt = new Event { Summary = "Event " + count };
iCal.Events.Add(evnt);
}
var serializer = new iCalendarSerializer();
serializer.Serialize(iCal, writer, Encoding.UTF8);
}
}
Console.WriteLine("Done: " + (DateTime.Now - startTime));
}
[/csharp]

TestDefaultSerialization first builds an iCalendar with 100,000 events in memory, and then serializes it using the iCalendarSerializer. As can be seen in the following Process Explorer usage graphs, large calendar can consume a considerable amount of memory. In this example, 100,000 simple events consumed almost 2 gigs of ram.

After spending some time in the DDay serialization code, I wrote a new DDayCalendarWriter class. Unlike iCalendarSerializer, which starts serializing the calendar to disk after it is completely built in memory, DDayCalendarWriter can serialize calendar children as they are created. You don’t need to fully populate the calendar with all the children before beginning to write them to disk. Instead, you create an iCalendar object, set all the required properties, and pass it to DDayCalendarWriter‘s constructor. Then, use DDayCalendarWriter‘s Write method to add children to the calendar. This way, calendar children can be garbage collected as soon as they are serialized to disk, and you can virtually create as large a calendar as your hard disk allows.

The following function shows DDayCalendarWriter in action.

[csharp]
public static void TestDDayCalendarWriter()
{
Console.WriteLine("Serializing using DDayCalendarWriter");

DateTime startTime = DateTime.Now;

using (var iCal = new iCalendar())
{
iCal.AddLocalTimeZone();
iCal.AddProperty("X-WR-CALNAME", "CalendarName");

using (var stream = new StreamWriter("test2.ical"))
{
using (var writer = new DDayCalendarWriter(iCal, stream))
{
for (int count = 0; count < 100000; count++)
{
var evnt = new Event {Summary = "Event " + count};
writer.Write(evnt);
}
}
}
}
Console.WriteLine("Done: " + (DateTime.Now - startTime));
}
[/csharp]

The following graph shows DDayCalendarWriter‘s memory and IO usage, while serializing the same 100,000 events.

As an added bonus, DDayCalendarWriter is also faster than DDay’s default iCalendarSerializer. On my dual core laptop, iCalendarSerializer took 1 min and 5 seconds to serialize 100,000 events, while DDayCalendarWriter took 44 seconds.

Checkout the complete source here: https://github.com/blinemedical/dday-utilities.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s