Wednesday, April 15, 2009

UTC DateTime in code

Some of us remember the big disaster with the Mars climate orbiter - due to a simple unit conversion error, hundred's of millions of dollars quickly evaporated in the Mars thin atmosphere. Compilers were designed to catch such errors early, but one must help compiler to find such errors.

Enter the world where Time is no longer an entity by itself, but always tied to a location. In our code, we try to keep all the DateTime values in Coordinated Universal Time (UTC - Temps Universel Coordonné) so that location is just a user interface setting, and has no affect on the internal program state. The problem is that .Net's DateTime may contain both units - the local time, the UTC time, and most troublesome, the Unspecified time.

By accident, one may pass a local or unspecified time into the code where UTC is expected, causing incorrect trading behavior in financial applications, errors in the stored data, and many other, more subtle errors. One may always call ToUniversalTime() method or validate input to have Kind=UTC, but both approaches have their limitations:
  • ToUniversalTime will work fine if the value was either UTC or Local, but will treat Unspecified values as Local. Now imagine DateTime was stored in the database, which returns all values as unspecified. Being proper developers, we stored UTC value, got back unspecified, performed the conversion... The result is the garbled data.
  • Parameter validation also has issues - programmer must never forget to validate parameters on ALL incoming calls, and, more importantly - compiler will not help us! An exception will be thrown only during execution... Mars mission again?
The proposed solution is very simple, yet gives coders the needed type safety for the UTC timestamps: a new struct to store only the UTC time.

The new UtcDateTime value has a single long value of Ticks, and is implicitly convertible to DateTime, so it can always be passed to any method requiring DateTime. On top of it, it implements most of the methods DateTime has, so it is a drop-in replacement. Yet, when method expects a UtcDateTime as a parameter, one must make an effort to convert the DateTime to UtcDateTime by calling an explicit cast or constructor (will validate DateTime.Kind == UTC), hence spending the time thinking what format the original value was in.

Ideally, the entire solution should use UtcDateTime for type safety, converting only when reading values from the database table (Unspecified to UtcDateTime) or external calls.

Here's the source of the UtcDateTime. Some optimizations and cleanup are still pending.



using System;
using System.Runtime.InteropServices;
using System.Runtime.Serialization;

namespace NYurik.Types
{
///
/// This is a replacement of the object that ensures type safety
/// when dealing with the Coordinated Univeral Time (UTC). Internally, the value is
/// stored as a long tick count.
///

///
/// Use this struct instead of the to store
/// the date as a UTC value in a 1-byte-packed structures.
/// DateTime may not be used in serialization due to different packing on
/// 32bit and 64bit architectures.
///

[Serializable, StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct UtcDateTime : IComparable, IFormattable, IConvertible, ISerializable, IComparable,
IEquatable
{
///
/// Same as except as UTC kind
///

public static readonly UtcDateTime MaxValue =
(UtcDateTime) DateTime.SpecifyKind(DateTime.MaxValue, DateTimeKind.Utc);

///
/// Same as except as UTC kind
///

public static readonly UtcDateTime MinValue =
(UtcDateTime) DateTime.SpecifyKind(DateTime.MinValue, DateTimeKind.Utc);

private readonly long _value;

public UtcDateTime(long ticks)
{
_value = ticks;
}

public UtcDateTime(int year, int month, int day)
: this(new DateTime(year, month, day, 0, 0, 0, DateTimeKind.Utc))
{
}

public UtcDateTime(DateTime value)
{
if (value.Kind == DateTimeKind.Utc)
_value = value.Ticks;
else
throw new ArgumentOutOfRangeException(
"value", value,
"DateTime must be in UTC\n" +
"You may either use value.ToUniversalTime() or DateTime.SpecifyKind(value, DateTimeKind.Utc)to convert.");
}

public static UtcDateTime Now
{
get { return (UtcDateTime) DateTime.UtcNow; }
}

public static UtcDateTime Today
{
get { return (UtcDateTime) DateTime.UtcNow.Date; }
}

#region Properties

public int Day
{
get { return ((DateTime) this).Day; }
}

public int DayOfYear
{
get { return ((DateTime) this).DayOfYear; }
}

public int Hour
{
get { return ((DateTime) this).Hour; }
}

public DateTimeKind Kind
{
get { return DateTimeKind.Utc; }
}

public int Millisecond
{
get { return ((DateTime) this).Millisecond; }
}

public int Minute
{
get { return ((DateTime) this).Minute; }
}

public int Month
{
get { return ((DateTime) this).Month; }
}

public int Second
{
get { return ((DateTime) this).Second; }
}

public long Ticks
{
get { return _value; }
}

public TimeSpan TimeOfDay
{
get { return ((DateTime) this).TimeOfDay; }
}

public int Year
{
get { return ((DateTime) this).Year; }
}

public UtcDateTime Date
{
get { return (UtcDateTime) ((DateTime) this).Date; }
}

public DayOfWeek DayOfWeek
{
get { return ((DateTime) this).DayOfWeek; }
}

#endregion

#region IComparable Members

public int CompareTo(object value)
{
if (value is UtcDateTime)
return CompareTo((UtcDateTime) value);
throw new ArgumentException("UtcDateTime is not comparable with " + value.GetType());
}

#endregion

#region IComparable Members

public int CompareTo(UtcDateTime other)
{
return _value.CompareTo(other._value);
}

#endregion

#region IConvertible Members

bool IConvertible.ToBoolean(IFormatProvider provider)
{
return ((IConvertible) ((DateTime) this)).ToBoolean(provider);
}

char IConvertible.ToChar(IFormatProvider provider)
{
return ((IConvertible) ((DateTime) this)).ToChar(provider);
}

sbyte IConvertible.ToSByte(IFormatProvider provider)
{
return ((IConvertible) ((DateTime) this)).ToSByte(provider);
}

byte IConvertible.ToByte(IFormatProvider provider)
{
return ((IConvertible) ((DateTime) this)).ToByte(provider);
}

short IConvertible.ToInt16(IFormatProvider provider)
{
return ((IConvertible) ((DateTime) this)).ToInt16(provider);
}

ushort IConvertible.ToUInt16(IFormatProvider provider)
{
return ((IConvertible) ((DateTime) this)).ToUInt16(provider);
}

int IConvertible.ToInt32(IFormatProvider provider)
{
return ((IConvertible) ((DateTime) this)).ToInt32(provider);
}

uint IConvertible.ToUInt32(IFormatProvider provider)
{
return ((IConvertible) ((DateTime) this)).ToUInt32(provider);
}

long IConvertible.ToInt64(IFormatProvider provider)
{
return ((IConvertible) ((DateTime) this)).ToInt64(provider);
}

ulong IConvertible.ToUInt64(IFormatProvider provider)
{
return ((IConvertible) ((DateTime) this)).ToUInt64(provider);
}

float IConvertible.ToSingle(IFormatProvider provider)
{
return ((IConvertible) ((DateTime) this)).ToSingle(provider);
}

double IConvertible.ToDouble(IFormatProvider provider)
{
return ((IConvertible) ((DateTime) this)).ToDouble(provider);
}

decimal IConvertible.ToDecimal(IFormatProvider provider)
{
return ((IConvertible) ((DateTime) this)).ToDecimal(provider);
}

DateTime IConvertible.ToDateTime(IFormatProvider provider)
{
return ((IConvertible) ((DateTime) this)).ToDateTime(provider);
}

object IConvertible.ToType(Type conversionType, IFormatProvider provider)
{
return ((IConvertible) ((DateTime) this)).ToType(conversionType, provider);
}

public TypeCode GetTypeCode()
{
return ((DateTime) this).GetTypeCode();
}

public string ToString(IFormatProvider provider)
{
return ((DateTime) this).ToString(provider);
}

#endregion

#region IFormattable Members

public string ToString(string format, IFormatProvider formatProvider)
{
return ((DateTime) this).ToString(format, formatProvider);
}

#endregion

#region ISerializable Members

public void GetObjectData(SerializationInfo info, StreamingContext context)
{
((ISerializable) ((DateTime) this)).GetObjectData(info, context);
}

#endregion

#region DateTime Arithmetic

public UtcDateTime Add(TimeSpan value)
{
return (UtcDateTime) ((DateTime) this).Add(value);
}

public UtcDateTime AddDays(double value)
{
return (UtcDateTime) ((DateTime) this).AddDays(value);
}

public UtcDateTime AddHours(double value)
{
return (UtcDateTime) ((DateTime) this).AddHours(value);
}

public UtcDateTime AddMilliseconds(double value)
{
return (UtcDateTime) ((DateTime) this).AddMilliseconds(value);
}

public UtcDateTime AddMinutes(double value)
{
return (UtcDateTime) ((DateTime) this).AddMinutes(value);
}

public UtcDateTime AddMonths(int months)
{
return (UtcDateTime) ((DateTime) this).AddMonths(months);
}

public UtcDateTime AddSeconds(double value)
{
return (UtcDateTime) ((DateTime) this).AddSeconds(value);
}

public UtcDateTime AddTicks(long value)
{
return (UtcDateTime) ((DateTime) this).AddTicks(value);
}

public UtcDateTime AddYears(int value)
{
return (UtcDateTime) ((DateTime) this).AddYears(value);
}

public TimeSpan Subtract(UtcDateTime value)
{
return ((DateTime) this).Subtract(value);
}

public UtcDateTime Subtract(TimeSpan value)
{
return (UtcDateTime) ((DateTime) this).Subtract(value);
}

public DateTime ToLocalTime()
{
return ((DateTime) this).ToLocalTime();
}

public UtcDateTime ToUniversalTime()
{
return this;
}

#endregion

#region Equality

public int CompareTo(DateTime value)
{
return ((DateTime) this).CompareTo(value);
}

public bool Equals(UtcDateTime other)
{
return other._value == _value;
}

public override bool Equals(object obj)
{
if (ReferenceEquals(null, obj)) return false;
if (obj.GetType() != typeof (UtcDateTime)) return false;
return Equals((UtcDateTime) obj);
}

public override int GetHashCode()
{
return _value.GetHashCode();
}

#endregion

#region Formatting

public string[] GetDateTimeFormats()
{
return ((DateTime) this).GetDateTimeFormats();
}

public string[] GetDateTimeFormats(IFormatProvider provider)
{
return ((DateTime) this).GetDateTimeFormats(provider);
}

public string[] GetDateTimeFormats(char format)
{
return ((DateTime) this).GetDateTimeFormats(format);
}

public string[] GetDateTimeFormats(char format, IFormatProvider provider)
{
return ((DateTime) this).GetDateTimeFormats(format, provider);
}

public string ToLongDateString()
{
return ((DateTime) this).ToLongDateString();
}

public string ToLongTimeString()
{
return ((DateTime) this).ToLongTimeString();
}

public string ToShortDateString()
{
return ((DateTime) this).ToShortDateString();
}

public string ToShortTimeString()
{
return ((DateTime) this).ToShortTimeString();
}

public override string ToString()
{
return ((DateTime) this).ToString();
}

public string ToString(string format)
{
return ((DateTime) this).ToString(format);
}

#endregion

#region Operators

public static UtcDateTime operator +(UtcDateTime d, TimeSpan t)
{
return (UtcDateTime) ((DateTime) d + t);
}

public static bool operator ==(UtcDateTime left, UtcDateTime right)
{
return left.Equals(right);
}

public static explicit operator UtcDateTime(DateTime value)
{
return new UtcDateTime(value);
}

public static bool operator >(UtcDateTime t1, UtcDateTime t2)
{
return t1._value > t2._value;
}

public static bool operator >=(UtcDateTime t1, UtcDateTime t2)
{
return t1._value >= t2._value;
}

public static implicit operator DateTime(UtcDateTime value)
{
return new DateTime(value._value, DateTimeKind.Utc);
}

public static bool operator !=(UtcDateTime left, UtcDateTime right)
{
return !left.Equals(right);
}

public static bool operator <(UtcDateTime t1, UtcDateTime t2)
{
return t1._value < t2._value;
}

public static bool operator <=(UtcDateTime t1, UtcDateTime t2)
{
return t1._value <= t2._value;
}

public static TimeSpan operator -(UtcDateTime d1, UtcDateTime d2)
{
return d1 - (DateTime) d2;
}

public static UtcDateTime operator -(UtcDateTime d, TimeSpan t)
{
return (UtcDateTime) ((DateTime) d - t);
}

#endregion
}
}

Thursday, March 20, 2008

Mediawiki on .NET - not just yet

Task of the day - run MediaWiki, the engine behind Wikipedia using .NET Phalanger project. Easier said then done.

Result so far: PHP5 is not supported by Phalanger, latest build can't even process "require()" PHP call. Waiting for the newer release...

Software
  • IIS with ASP.NET enabled
  • Visual Studio 2008 pro
  • Phalanger
  • MediaWiki source code
  • MySQL
Issues
  • The latest stable Phalanger build - March 2008 official release - is broken. It fails on
    require_once( "$preIP/includes/WebStart.php" );
    Unfortunately, Phalanger team does not have nightly builds setup yet, and building it is not as simple as I was hoping. IMHO, continuous integration is the key for this kinds of projects.
  • Beta 4 (2007) does not support VS2008, and does not implement any of the PHP5 features. I don't know if the final release has PHP5 support, but b4 definitely does not support the new XML DOM objects. I guess it should be possible to use older MediaWiki release for PHP4, but I haven't tried that yet.
Version-updated configuration file i used (taken from this forum post)

<configuration>

<phpNet>
<classLibrary>
<add assembly="PhpNetMySql, Version=2.0.0.0, Culture=neutral, PublicKeyToken=2771987119c16a03" section="mysql" />
<add assembly="php_curl.mng, Version=2.0.0.0, Culture=neutral, PublicKeyToken=4ef6ed87c53048a3" section="curl" />
<add assembly="php_exif.mng, Version=2.0.0.0, Culture=neutral, PublicKeyToken=4ef6ed87c53048a3" section="exif" />
<add assembly="php_gd2.mng, Version=2.0.0.0, Culture=neutral, PublicKeyToken=4ef6ed87c53048a3" section="gd2" />
<add assembly="php_domxml.mng, Version=2.0.0.0, Culture=neutral, PublicKeyToken=4ef6ed87c53048a3" section="domxml" />
<add assembly="php_xml.mng, Version=2.0.0.0, Culture=neutral, PublicKeyToken=4ef6ed87c53048a3" section="xml" />
<add assembly="php_mbstring.mng, Version=2.0.0.0, Culture=neutral, PublicKeyToken=4ef6ed87c53048a3" section="mbstring" />
</classLibrary>

<compiler>
<!-- Whether to compile scripts in debug mode. -->
<set name="Debug" value="true" />
</compiler>

<error-control>
<set name="ReportErrors" phpName="error_reporting">
<remove value="Warning,Notice,Strict"/>
</set>
</error-control>

<globalization>
<!--
Encoding used for converting source files to Unicode and for run-time binary data conversions.
The value should be one of the identifiers of the code-page supported by Windows OS.
These include e.g. values "Latin1", "Latin2", "Windows-1250" etc
An empty value means the default code-page used by OS used.
Option has application scope and cannot be changed in application sub-directories.
-->
<set name="PageEncoding" value="Windows-1250" />
</globalization>
</phpNet>

<system.web>
<!-- Encoding used by web server to encode output data sent to the client. -->
<globalization responseEncoding="Windows-1250" />
</system.web>

</configuration>

Wednesday, March 19, 2008

Building AnkhSVN

Random impulses create random results... This time the impulse was to build AnkhSVN. After talking on IRC #ankhsvn channel, jeremyw and other developers (thanks!!!) gave me some info to get up and running directly from Visual Studio.

Required software (in parenthesis - the versions I used)
Steps to get up and running
  • Download AnkhSVN source code from http://ankhsvn.open.collab.net/svn/ankhsvn/trunk User name "guest", no password.
  • Open src\AnkhSvn.2008.sln
  • Make Ankh.Package a startup project.
  • Right-click Ankh.Package and click "Properties".
  • Choose the "Debug" tab on the left.
  • Under "Start Action", select "start external application" and choose your devenv.exe.
  • For "Start Options", use "/rootSuffix Exp /RANU" without the quotes
  • Now can Run AnkhSVN by hitting F5
Enjoy!




As for the old package.py, it apparently no longer works nor is supported.
Here I will just keep my original notes, ignore...


* Python (2.5.2 - msi)
* Python for Windows Extensions (pywin32-210.win32-py2.5.exe)
* Perl (ActivePerl 5.10.0.1002)

* Made sure the path to python is set in the PATH env variable (c:\python25\)
* Run Visual Studio 2008 Command Prompt
* Executed the build with "package.py -d 2008" command from c:\projects\ankhsvn dir.

Kept getting this weird error from python:
Staging APR, APR-UTIL and APR-ICONV...
Traceback (most recent call last):
File "C:\projects\ankhsvn\package.py", line 1298, in
build_berkeley_db()
File "C:\projects\ankhsvn\package.py", line 257, in build_berkeley_db
print "Building Berkeley DB..."
IOError: [Errno 9] Bad file descriptor

Apparently, console output descriptor was getting corrupted somehow, so as a temp workaround I commented out all the print statements (ugly, I know): "print " --> "pass #print "

Rerunning it produced yet another error:
Traceback (most recent call last):
File "C:\projects\ankhsvn\package.py", line 1298, in
build_berkeley_db()
File "C:\projects\ankhsvn\package.py", line 264, in build_berkeley_db
convert_dsw_to_sln(bdb_dsw_file)
File "C:\projects\ankhsvn\package.py", line 638, in convert_dsw_to_sln
vsdte.ExecuteCommand("File.SaveAll")
File "", line 3, in ExecuteCommand
File "C:\Python25\lib\site-packages\win32com\client\dynamic.py", line 258, in _ApplyTypes_
result = self._oleobj_.InvokeTypes(*(dispid, LCID, wFlags, retType, argTypes) + args)
pywintypes.com_error: (-2147418111, 'Call was rejected by callee.', None, None)
After getting stuck here I found out that package.py is now obsolete and should not be used :)))