Are we validating design decisions with metrics and facts?

One key factor that we should always consider is what actions have we taken to make sure that our design decisions are in line with the prioritized quality attributes of the product to develop.

A good way to grasp this is with a simple example.

The argument here is on how we give factual support to our decisions, and not on discussing the technical merits of the decisions of the following two samples.

Lets suppose that we have a routine that needs to do some stuff with a collection of numeric integer values, and performance is the main concern (the topmost prioritized quality attribute).

Should we use an array as collection? Or should we use a Generics collection, like say List?

We will compare two routines whose source code is as similar as it can be, except for the actual type of the collection holding the numeric integer values.

Code Sample 1 (array-based collection)

 class Program

 {

 public const int TOP_VALUE =
1000;

 

 static void Main()

 {

 DateTime
start = DateTime.Now;

 

 int[]
List = new int[TOP_VALUE];

 

 for
(int i = 0; i < TOP_VALUE; i++)

 {

 

 List[i] = i;

 }

 

 for
(int i = 0; i < TOP_VALUE; i++)

 {

 

 int
j = (int) List[i];

 

 Console.WriteLine(“{0}: {1}”, i, j);

 }

 

 DateTime
end = DateTime.Now;

 

 Console.WriteLine(“Array Sample; TOP VALUE: {0}; Elapsed Time:
{1}”
, TOP_VALUE, end.Subtract(start).ToString());

 

 Console.Read();

 

 

 }

 }

 

Code Sample 2 (Generics -based collection)

 class Program

 {

 public const int TOP_VALUE =
1000;

 

 static void Main()

 {

 DateTime
start = DateTime.Now;

 

 List<int> list = new List<int>();

 

 for
(int i = 0; i < TOP_VALUE; i++)

 {

 

 list.Add(i);

 }

 

 for
(int i = 0; i < TOP_VALUE; i++)

 {

 

 int
j = list[i];

 

 Console.WriteLine(“{0}: {1}”, i, j);

 }

 

 DateTime
end = DateTime.Now;

 

 Console.WriteLine(“List<> Sample; TOP VALUE: {0}; Elapsed Time:
{1}”
, TOP_VALUE, end.Subtract(start).ToString());

 

 Console.Read();

 

 

 }

 }

 

The first sample, the one using the array-based collection, includes an explicit type cast to force unboxing into the picture.

Instead of giving way to assumptions and scenario analysis, we will use each one of the two routines to take some measures and calculate some simple performance metrics.

By changing the value of the constant, we will compare the performance of the same two routines with different sizes of the collections, from 1000 items all the way up to 10000000 items, in ten-fold increments.

We will also take three measures for each size of the collections, and then use the average value of each set of three measurements.

The performance metric we will measure is execution time (elapsed time of execution in seconds). You may see that the source code of the two routines already includes code to determine the elapsed time of execution.

The metrics capture for the two routines is as follows:

 To all practical purposes, both routines show just about the same elapsed time of execution for the same range of collection sizes, that is, from 1000 items to 10000000 items, which means that they experience and offer the same performance!

With this simple factual support, we can make a strong argument regarding performance without having to resort to assumptions or scenarios.

Lets bear in mind that, if we were to argue with just assumptions in a court of law, the other side would spit hearsay to the presiding judge.

Once we have metrics and facts to support whatever design decisions we could make, we better have some good explanations regarding what may be happening under the hood.

Thats a good subject for an upcoming post.

See ya!

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s